HM.APPLICATIONS Updated: 10 December 2001 (listing of application areas and applications for which HumanMarkup is designed) ------------ INTRODUCTION ------------- This document describes the different application areas and specific applications for which the HumanMarkup specifications are designed and from which the HumanMarkup effort has requested requirements and related information for inclusion in its work. Further, the domains are listed for which HumanMarkup is designed to be useful. --------- DOMAINS [In HumanMarkup usage, (not RDF), domains refers to the largest conceptual containers of information relevant to HumanMarkup in terms of the means and types of communication and documentation for use in digital information systems. For our purposes, one can easily think of domains as media, although media more exactly refers to actual protocols and technologies rather than the information transmitted using those media.] --------- Audio [Audio includes all signal protocols that encode information derived from or meant to produce sound waves; this explicitly includes voice information and natural language processing from speech/voice to text and from text to speech/voice.] Chronemic [Chronemic includes information that is in any way time-dependent, such as animation, music, synchronization of various media domains, stimulus and response, Haptic [research shows, and pointing and forced feedback devices show, that digitally encodable information based on the Haptic sense of touch is and will continue to develop as a domain for the input and output of digital information.] Olfactory [Olfactory, while not widely supported by current digital information systems will inevitably be developed because the technology to produce synthetic odors, aromas, fragrances and other descriptions of the aerosol materials for input and output of digitally encodable information based on the olfactory sense of smell.] Multimedia [Multimedia includes all domains expressly and explicitly encoded to combine Audio, Haptic, Olfactory, Text and Video information in any combination.] Text [Text includes all alphanumeric information, which subsumes data, and includes all written representations of language, which subsumes all ideographic or symbolic forms of writing.] Video [Video includes all forms of digitally encoded visual information using 2D and 3D display with and without animation, whether recorded, programmed or presented live in real time, or as close as the digital information systems can approach to real time.] ------------- DOMAIN DATA [Classification of data within domains. The first entry below is meant to demonstrate how data can be presented in a form which is suitable as either a full description ready for data or as a complete search parameter. Further work in liaison with the OASIS Topic Maps Published Subjects Technical Committee and other Topic Maps Standards groups. After the initial entry, an abbreviated form of entry is usually used to save time and is meant as an indication of data areas and not a full description, except where it seems appropriate to fully delineate an area as a further example of full utility. Note: Once listed in one context, many items such as types of reports which can apply to numerous fields such as Business and Government and Medical are not repeated since this document is aiming at a basic comprehensiveness, and not exhaustiveness, which is, in any event, ultimately impossible as long as humanity continues evolving. As an aid to quick indentification, we have put the key descriptions of domain data in boldface. ] ------------- Classification System:Legal Entity:Human Profile:Name:John Doe:Human Physical Description Markup Language:Height:Unit:Metric:Datatype:Float:? Classification System:Business:Reports:Calendar Notifications: Classification System:Business:Reports:Conference Reports: Classification System:Business:Reports:Contact Evaluations: Classification System:Business:Reports:Contracts: Classification System:Business:Reports:Human Resources:Performance Evaluations: Classification System:Business:Reports:Meeting Minutes: Classification System:Business:Reports:Human Resources: Classification System:Business:Reports:Negotiation Updates: Classification System:Education:Records:Grades: Classification System:Entertainment:Genre Languages: Classification System:Entertainment:Reports:Schedules: Classification System:Government Records:Federal:Executive Branch:Cabinet Department:Justice:Reports:Total Number, Type, Outcome, Organized Crime Prosecutions, 1958:Date Requested:00/00/0000:Requested By:John Doe:Date Generated:00/00/0000 Classification System:InstantMessaging:Internet:Time Initiated:00:00:00 PST:Initiator:Jane Doe:Contactee:John Doe:Logged Transcript Unavailable:Time Terminated:00:00:00 PST Classification System:Medical Diagnoses:Service Type:Individual General Practitioner:John Doe M.D.:Patient:Jane Doe:Scheduled Appointment Time:00:00:00 PST:Human Physical Description Language:Result:Authorization Required Classification System:Publication:Journalism:Media:Broadcast Television:National Daily News Report:Scheduled Start Time:00:00:00 PST:Feature Subject:Jetliner Crash: Classification System:Publication:Journalism:Media:Print:Daily Newspaper:DoeTown Metropolitan:Restaurant Reviews: Classification System:Publication:Default:Library of Congress Listing: Note: The entire domain, if you will, of domain data is subject to numerous overlapping classifications which will, hopefully, be harmonized consistently within the overall classifications which will make up the "Semantic Web," and will also hopefully be enhanced by inclusion within other indepedent classification and referral-by-inference systems such as, "Opencyc." -------------------------------------- APPLICATION AREAS [We define this category as tasks in which HumanMarkup can significantly improve performance according to criteria we will attempt to explain in each case, so that we don't give the impression that fidelity of communication is the only goal of HumanMarkup, but, rather, that fidelity of communication has many aspects from accurancy to timeliness. It should be understood at the outset that the Human Markup Language is not itself an application, nor is it ready once it is accepted as a standard markup language specification, to immediately create applications. It must be combined with other programming languages to create middleware applications which use the specified elements of the schemata of the Human Markup Language to perform their functionality. -------------------------------------- Agents [Automated software programs constructed to perform various tasks without human supervision such as retrieving information and storing it for delivery by request, when prescribed by criteria set by users or at specified times.] Agents:Humanoid Representations [Animated simulacrums of individuals, based on actual or fictitious individuals using Human Physical Description Language, such as a Personalized Instant Messaging Answering Agents, Corporate SpokesAgents, Retail Sales Agents, etc.] Monograph: Agents can be all or part AI, partly or wholy based on a specific human individual's information; based on Cultural Modules, or modified by such to cater to specific interaction purposes, HumanMarkup will enable (intelligent) agents to possess and dynamically alter their depiction of internal emotional states in appropriate ways in order to "humanize" or "anthropomorphize" the computing environment. These representations include what is thought of as "Avatars" in popular parlance, but are not constrained to such manifestations. "Avatars" are commonly thought of as 3D Humanoid Representations which interact with 3D environments and other "Avatars" or non-Humanoid Agents in Scenario-, or Context-Bound situations, which might also be constructed more readily using HumanMarkup "Genre Languages." This manifestation of Humanoid Representations are best described in the Novel "Snow Crash" by Neal Stephenson, Bantam Books, 1992. Agent Applications: Personal Avatars; Personal Preference Representation; Automatic HumanML Web Crawlers; Personalized Automatic E-mail Managers; AI systems [Artificial Intelligence means intelligence created by artifice or artificial means, by which we mean non-biological, at least at this point in time, 12/2001. In the not-too-distant future it is possible that semi-artificial biological systems or semi-biologicial artificial systems may be included in an expanded definition of artificial in this context by which the term may come to mean human-created, non-human intelligence. Artificial Intelligence Systems is perhaps best summed up by a sentence from the website of the MIT Artificial Intelligence Laboratory (http://www.ai.mit.edu): "Our goal is to understand the nature of intelligence and to engineer systems that exhibit intelligence." The simplest definition of intelligence is the capacity to learn, reason and understand. It is instructive that in the same statement on the MIT Artificial Intelligence Laboratory Website, the following sentence also occurs: "Our intellectual goal is to understand how the human mind works." Monograph: For our purposes we will be engaging the field of Artificial Intelligence on two levels. First we need to provide for the requirements of a human-centered display metaphor for the presence of Artificial Intelligence-Based Agents, as Agents are defined above. Second, we need to use the research from Artificial Intelligence studies because, as is shown in the MIT Artificial Intelligence Laboratory statement, studies into the creation of Artificial Intelligence will help inform us more thoroughly about human intelligence. The following set of questions also comes from the MIT Artificial Intelligence Laboratory Website, in the Messsage from the Director and they are included to show how the process of looking for answers to these questions needs to be included in our studies as well. "Where does the mind reside? What is the nature of memory? What are the roles of emotions? What sort of representations does the brain use? What does our visual system compute? How did evolution shape us? How do we learn? What is consciousness?" AI Applications: Automatic Adaptive Language Translation (An application designed for the automatic translation of text, sign, speech, voice, video, gesture, etc. from one language, or domain, to another. An application of this kind allow for implementing or accounting for HumanMarkup tags for emotional content, facial gesture, body gesture, etc while also incorporating as many communication media related to the context of the message as needed or called for by an application user. AI Techniques for use in Automatic Adapative Language Translation: Actors; Augmented Transition Networks(ATN) including Cascaded ATNs(CATN); Automatic Associated Database Referencing; Associative Memories ; Bayesian Inference ; Bayesian Networks ; Beam Search, Bidirectional Search; Case Frames ; Case Grammars ,; Categorial Grammars ; Chart Parsing; Classification ; Competitive Networks ; Complexity Measure ; Conceptual Clustering ; Conceptual Dependency; Conceptual Graphs ; Constituent Likelihood Grammar ; Context-free Grammar ; Context-sensitive Grammar ; Dependency Grammar ; Deterministic Parsing ; Discrimination Learning; Discourse Representation Theory ; Distributional Analysis; Dynamic Programming; Dynamic Time Warping; Earley1s Algorithm ; Explanation-Based Learning; Feature Structures ; Focussing (Version Space); Formant and Formant Synthesis ; Frames ; Frame (Schema) ; Functional Data Model ; Functional Unification Grammer (FUG) ; Generalized Phrase Structure Grammar ; Head-driven Phrase Structure Grammar; Heterarchy; Hidden Markov Models; Interlingua; Interval Calculus; IS-A Hierarchy; Island Parsing; Lexical Access (Dictionary Lookup); Linear Predictive Coding Logic Grammar; Meta-level Inference; Meta-rules; Montague Semantics; Morphographemics; Multi-actor System ( Multi-agent system ); Multi-pulse LPC (Modified Linear Predicitive Analysis, Synthesis of speech); Object-oriented Programming; Partitioned Semantic Net; Pattern Directed Retrieval/Invocation; Pattern Matching; Predictive Parsing (Expectation-based Parsing); Preference Semantics; Production Rule System (Production System,If/Then Rules,Situation/Action Rules) Relational Database Query Formulation; Rhetorical Structure Theory; Script; Self-organizing Feature Maps; Semantic Grammar; Semantic Head-Driven generation; Semantic Networks; Semantic Primitives; Semantic Syntax; Symbolic Marker-passing; Template Matching; Temporal Logic; Top-down Parsing; Transfer; Tree-Adjoining Grammars(TAG); Unification Grammars(UG); Universal Graph Representation; Unsupervised Learning; Vocoder Representation; Vowel Quadrilateral; Annotations [The practice of annotating documents preceded computer science, in the formalism of Annotated Bibliography for instance, however, for our purposes we are referring to the practice of adding specific descriptive comments before or after blocks of computer software programming code; for the purpose making the code more readable by humans; for the purpose of documenting how the the code is intended to function; for the purpose of making debugging or copying portions of code easier. The same principle can be extended into greater usefulness through the Human Markup Language, by making it possible to add HumanMarkup tags to existing documents in any human language and to such documents in progress.] Monograph: Annotation of texts or other communications forms can provide meta information for the purpose of Interpretation. Annotations will take on a greater significance when long individual anecdotal documents or collections of documents can be marked up in a way to make the marked up information useful, both for finding specific types of information within documents and for turning such information into data such as the number of instances certain kinds of information are contained within a population of documents. This is a significant change in the way Annotation works, and can be the basis for a number of applications in fields as disparate Social Services and the auditing of circulation numbers by extended criteria currently incapable of determining more than the total population exposed to media content such as periodicals and broadcast radio and television programs. Despite the fact that we will be defining more specific application areas, mark up languages are by their very nature annotational so all of our applications could be thought of as being Annotational Applications, but we will attempt to cleave to more immediate and direct applications of traditional annotation. Annotation Applications: Bibliographies; Business Records; Court Records; Educational Records; Employment Records; Government Records; Medical Records; Business:Documentation Integration [By subjecting the entire documentational knowledge base of a business to HumanMarkup Preparation, in addition to the specific functionality of such instruments as purchase orders, inventory reports, equipment requests, sales forms, etc., a larger and more integrated approach to business documentation is possible. It can be used for the following specific purposes, but it is not limited to such, and should be understood to extend beyond the common perception that HumanMarkup deals mostly or exclusively with emotional content and context. That context is, however, one of the most significant, and neglected, areas of information processing, which has a long history of detrimental effects that can reliably be attributed to that neglect as symptom of a larger societal neglect, perhaps best exemplified by the connotation in public perception of what it means when a nervous joke is made about "going postal." HumanMarkup in Business Documentation Integration can be used to identify human emotional content or possible emotional content in documents, making it clear how much, or how little, influence emotional content has within the document or document population. This kind of evaluation can be accomplished by various kinds of analysis, such as determining the percentage of emotional content in total information, rating the level of intensity of emotions portrayed, described or utilized in the context of arguments or proofs within the document, or as significant absences in arguments or proofs within the document. This can determine the amount or level of emotional content in claims made within a document or document population. It will likewise be possible to identify by the same process, emotional human content and purposefully and intentionally embed this annotation markup within documents in order to make the relationship of emotional content to conclusions or claims or as characterizations of information clear, exlicit and unequivocal.] Business:Documentation Integration Applications: Multi-Purpose Corporate Portals (improved through HumanMarkup); Business-wide Document Production, Classification, Backup, Availability and Archiving; Communications:Dialogue Systems [Two-party communications can be given means for identifying emotional content and cultural content and other kinds of unjiquely human contextual content. These identifications can be marked up after the fact or indicated by individuals while engaged through various means, as simple as the emoticons in text chat use today, known as smiley faces, or as complex as the use of similar icons in combination with color coding if a standard is agreed upon using colored, highlighted text with gradient differentials of hue, shade or saturation to indicate increasing or decreasing intensities. In Voice-Over-IP, various kinds of audio effects can be added to speech and/or interpretations of vocal qualities can be automatically added to indicate importance attached by speakers, or degrees of intensity of emotional content.] Communications:Dialogue Systems Applications: Text Chat; Voice-Over-IP; E-mail; Transcription Services; Messenger Agent Services; Communications:Multi-User or Conferencing Systems [Simultaneous Multimedia Integration Language(SMIL) interoperability with HumanMarkup will enable a new kind of video conferencing and multi-user, multi-purpose environment. Many of the features of streaming media currently in use can and will be combined with various presentation technologies that will allow closed or open Virtual Private Network- or Mbone- based meetings and conferences conducted through the Internet, which can be enhanced through the use of Human Markup specifications employed in all of the protocols so combined.] Communications:Multi-User or Conferencing Systems Applications: Niche Conferencing Solutions; Interactive Entertainment:Pay-per-Request Live Concerts; Communications:Single-User Pull-Personalization Web Content [By enabling greater depth of individual information for personalization of preferences, dormant Pull-Based applications can be offered end users of web services and technologies.] Communications:Single-User Pull-Personalization Web Content Applications: Niche-specific subscription-based multimedia webzines; Niche-specific subscription-based e-mail newsletters; Communications:Single-User Push-Personalization Web Content [the number of ways in which publicly-provided personal information can be used is beyond the scope of this document and, like many such seemingly obivous commercially-based uses of HumanMarkup-based information collections, it does not immediately present itself until attention is drawn to it. This phenomenon is likely to be widespread as HumanMarkup begins making inroads into the commercial sphere. Of course, non-profit, public utilization is not only possible, but likely also, once the potential begins to be proven.] Communications:Single-User Push-Personalization Web Content Applications: Marketing Research from compensated HumanMarkup-based population profiles; Niche-specific Direct Marketing based on increasingly focused HumanMarkup-based Market Research Conflict Resolution/Prevention Systems [This kind of multi-disciplinary system becomes more workable with HumanMarkup because a more complete range of Human contextual information can be made available, most obviously by the inclusion of culturalModules, but, perhaps more importantly, using such information to define a perhaps-unconscious or subconscious predisposition to intentionality within the context of potential or actual conflicts on micro to macro scales. Intentions are often misunderstood and lead to unresolved conflicts. Such intentions are misunderstood perhaps even more so by the parties harboring unacknowledged intentions or intentions that are so culturally ingrained that they are not recognized as the basis for consistently misunderstanding other cultures and individuals who do not share the same predisposing cultural backgrounds. In short, what this implies on behalf of the use of HumanMarkup is the ability to take more factors into account when potential or actual conflict arises whether the social situation involves as few as two individuals disputing the disposition of a tree which belongs to one individuals but is blocking the other individual's view; or neighborhoods within a city competing for municipal services or disputing jurisdictional authority; or ethnic populations within a geographical entity competing for political, social, historical, religious or economic power and influence. It is important to say that HumanMarkup will only provide more information on these factors, but cannot in and of itself resolve or prevent conflicts. It can only improve the felicity of information and perhaps prime the pump for improved civic, regional, national and international planning.] Conflict Resolution/Prevention Systems Applications: Mediation Services:Information Gathering; Mediation Services:Report Generation; Debate/Discussion [As currently practiced through email lists, a large amount of contextual information simply cannot be transmitted easily. With the advent of Voice XML and Voice-Over-IP, HumanMarkup will extend its primary function to include this area. A new kind of understanding among Humans may evolve. We can't know beforehand how it will work or what, exactly it will look like. However, we do know some parameters. Where HumanML tags can add contextual information, be it cultural or emotional, greater meaning can be transmitted and with greater information there exists the possibility of greater understanding.] Debate/Discussion Applications: Real-Time Font, Size, Color and Iconographic Text Display; Pre-Expression HumanMarkup Display of Participant Predisposition (explicit depiction of characteristics including individual's personal and cultural intentions, assumptions, motivations, and arguments) HumanMarkup-based Databases [The entire field of databases is due for a major change, of which HumanMarkup will be a part. This is due to various factors, such as a huge amount of Multi-Media content which can and needs to be cataloged in a way that makes it accessible for quick repurposing and quick checking for Intellectual Property Rights attached to individual units, and different kinds of information which do not fit either into traditional Relational Database formats or Object Databases. The entire issue of Topic Maps, and the Semantic Web as an inference-based reference system requires this change. How it will come about remains to be seen, but we can formalize our HumanMarkup portion of the information/knowledge base mix. HumanMarkup will allow for several different kinds of information to be made accessible more easily through Database Management techniques by the use of markup tags which can be separately documented and accessed so that entire documents, much less populations of relatied documents, need not be transferred across the web for including requested types of information. In addition, such information can be numerically evaluated in a number of ways through simple statistics or larger transforms which can draw quantifiable conclusions from a base of anecdotal data. Another set of issues to consider is how relational databases, for example the x, y, z coordinate position databases for the relative positions of bodypart segment vertices displaying a current state or state-change of an interactive 3D Animated Humanoid are going to require close coordination with object databases for semi-automatic culturally-based behavioral reactions like a facial gesture of wincing in embarassment or offense when someone misuses a cultural taboo in public. These interactions are the kinds which is required for believable human agents or bots, in say, an online 3D retail caravansary of trade goods from Morocco, even if the representation originates in elsewhere These are three specific new kinds of databases or combinations of databases and descriptions of the kinds of database management applications that will be developed using HumanMarkup.] HumanMarkup-based Database Applications: Document Analysis; Document Population Analysis; Topic Map Search Refinement; Cultural Reference Repository Search and Retrieval; Human Phsyical Characteristics Desicription Markup Language-based Applications Area [This area, like Artificial Intelligence, represents a large body of information which deserves its own document, and, unlike AI, we are going to separate it off into its own Documentation Structure as early as this first working draft of the HM.applications document. The first document in this proposed structure will be the forthcoming "Preamble to the Proposed Charter to Establish the Human Physical Characteristics Description Markup Language as a subset of the Human Markup Language." Please check the OASIS HumanMarkup TC website for details and updates.] Law [See Business Documentation Integration; the same principles which will be seen in Business Documentation Integration applies to Law. All other textual modifications, and database management techniques that HumanMarkup makes possible, apply in Law. However, in Law, the very basis of the practice is anecdotal and operates on the basis of precedent and previous practices, so many new kinds of multi-media evidence and reporting will be seen through the use of HumanMarkup.] Law Applications: HumanMarkup-based XML-Voice Interrogations; HumanMarkup Legal Records Enchancement; Messaging [See: Debate/Discussion; the same defining remarks characterize Messaging as characterize Debate/Discussion.] Messaging Applications: General Messaging Environment Enhancement; Short Message Service Enhancement; HumanML Web Browser Enhancement; HumanML Message Routing; Multimedia [See Communications:Multi-User or Conferencing Systems because the description of this applications area is suitable for Multimeda as well. This might also be termed New Media, except that most of the media pre-existed the digital information systems which HumanMarkup is specifically designed to enhance. However, to avoid confusion, since what is new media today will be definitionally old media in as many years as it takes for newer media to supplant existing media and media combinations, the term Multimedia is preferred simply because it will not become outdated, and can always be updated.] Multimedia Applications: Multimedia Art; Multimedia Art:Comics; Multimedia Art:Poetics Multimedia Art:Theatrics; Multimedia Art:Theatrics:Playwriting; Multimedia Art:Theatrics:Stage Direction; Multimedia Conferencing; Multimedia Education; Multimedia Entertainment; Multimedia Presentations; Music [Music is the predominant art form in the domain of audio, and offers a wide range of possible applications due to its long history of study and documentation. It is almost as well studied as Language. Music has many cultural contexts that need to be documented and codified. it is closely identified in all cultures with emotional states, but those associations are very different from culture to culture.] Music Applications: Automatic Mood Music Generation; Automatic Mood Music Text Accompaniment; Profiles [There will be a plenitude of profiling applications, the details of which come to mind all too well in the context of the time period in which this is written, December, 2001. The ramifications for legal identification in the context of digital information systems are enormous, and have been dealt with superficially elsewhere. HumanMarkup can add greater depth when needed by various applications.] Profile Applications: Psychological Profiles; Marketing Profiles; Automatic Form-filling; Agent Profiles; Psychology/Psychiatry[This arena is best dealt with by the professionals who practice it. We can list a number of potential applications that will be enhanced or enabled by HumanMarkup.] Psychology/Psychiatry Applications: Psychiatric Case Report Generation; Case Report Population Statistical Study Generation: Online Psychotherapy; Role Playing Games [See Agents. This will serve for many areas, from entertainment to training] HumanMarkup Enhanced/Enabled Search Engines [In essence, this refers to a more fully realized Semantic Web than we can address in a document as limited in scope as this one. Suffice it to say that this, in lockstep, so to speak, with Topic Maps, and new Database Management Solutions, will be a very powerful force in the future digital envirnoment.] Sound Design/Voice [This area might also simply be called audio, except that we have made a separate category for Music, which it deserves as much as Linguistics does in other contexts than applications areas. Sound Design and Voice XML, will fill out the audio domain with sound effects that can be married to HumanMarkup tags by application authors, and conferencing technologies which stream audio files will also be capable of accepting HumanMarkup modification.] Sound Design/Voice Applications: Humanmarkup-based Automatic Natural Language Audio Interpretation/Translation Systems; Humanmarkup-based Live- or Scripted-Speech Accompaniment/Enhancement; Humanmarkup-based Machine-Generated Natural Language Audio Delivery Systems; Humanmarkup-based Text-to-Speech-to-Text Conversion Natural Language Audio Delivery Systems with Emotional Fidelity;Humanmarkup-based Voice-Over-IP Accompaniment/Enhancement; Virtual Reality [See Agents, Artificial Intelligence and Human Physical Characteristics Description Markup Language. These areas inform this category and the entirety of the combined areas is much too large for this document. Following the precepts of Interoperability upon which OASIS is based, work in this area will be largely involved with the Virtual Reality Modeling Language specifications as developed by the Web 3D Consortium in its VRML97 ISO Specification: International Standard ISO/IEC 14772-1:1997, the proposed: Amendment 1: Final Proposed Draft Amendment, the XML-based proposed new version: X3D International Standard ISO/IEC xxxxx:200x--November 25, 2001 and the associated specification most directly related to HumanMarkup: proposed The Humanoid Animation Specification International Standard ISO/IEC xxxxx-1:200x. The urls for all of these resources and associated documentation will be included in the forthcoming "Preamble to the Proposed Charter to Establish the Human Physical Characteristics Description Markup Language as a subset of the Human Markup Language."]