Character and setting are important parts of the stories we tell to ourselves, the ways we struggle to understand events around us. The power of an embodied character as computer-interface does not lie in its parsing of natural-language, or the accuracy of its user-models - all of these features of an intelligent interface can be expressed in other, safer, more conventional ways. An animated character, however, brings into play factors about human nature previous interface-metaphors have rarely exploited: the subtle cues about emotional state and intention expressed through body language in our day-to-day life.

Many researchers have examined ways of creating believable animation and coherent characters, which can display emotion, motivation or internal state. [Bates94][Perlin96][Blumberg95][Laurel91] This project, however, seeks meaningful stimuli for those same emotions and reactions.

This paper describes a set of experiments in using the emotional and behavioural state of a character to convey meaningful information using subtle, continuous and non-intrusive means. Rather than traditional dialog-boxes, full-text messages, or discrete and indistinct attempts at communication, we are creating emotionally charged mappings between events and behaviours.

"…the characters can then be used much like an actor is used in a production. They can…both take direction from an outside force and perform reasonable actions autonomously when not directed" [Galyean95]

The goal is not to build entirely intelligent and self-contained systems. Rather, the intelligence, modeling capabilities and expert knowledge of future AI systems is simulated, and filtered through an interface designed to communicate in new and more effective ways. For example, a system-warning for closing an unsaved document called "Untitled" and, say, erasing a hard disk, should "feel" emotionally distinct to the user. Regardless of the intention of the user, the system should realize one is a dramatically more meaningful event, and react accordingly.

A character and the environment can deliver this kind of message, imply additional information in a manner requiring less time and attention than traditional interface elements.

Three working examples of these ideas will be discussed in the remainder of this paper, as well as directions for continued work in this area.

Current experiments use the beta-released Microsoft Agent ™ Active-X control for animation and character appearance. The Active-X control is a windowless animated Genie, which can be instructed to perform distinct animations through a C++, Visual Basic, VBScript or Javascript interface. Instructions include such commands as "GlanceLeft", "MoveTo(x,y)", "Approval", and "Sad". In this project, the genie is controlled through dynamically generated Javascript (via Perl 5.0 CGI and shell scripts, served by a Netscape Communications server on an HP9000/750). Further control is provided through local Visual Basic programs. The character runs on an Intel Pentium/155 under Windows NT 4.0

The first project created filters to react to incoming email-messages according to a given priority. A flat-text file was written, containing email addresses and associated "priority" ratings. High-volume lists, for example, were given low-priority, while close friends or the author’s professor received higher rankings. The list simulated the work of an intelligent user-modeling agent, which might be able to glean such rankings by observing user-actions over a period of time, or through communicating with other user-agents in the same workgroups or net-communities.

When email is received at the user’s Post-Office server, it is stored for delivery as usual. A Perl script scans for incoming messages, determines behaviours according to hard-coded reactions to the highest-ranked message in the queue, and generates Javascript instructions. These instructions are sent via HTTP to the Active-X genie on the user’s machine, which reacts accordingly.

Figure 1: Genie and Email

The reactions include such behaviours as simply glancing at the icon of the user’s email-reader for low priority messages, or pointing more vigorously for higher rated email. Top priority messages cause the character to wave its arms to attract attention, move across the desktop to a position next to the email-reader application, and stare at the icon.

At any time, clicking on the agent itself reveals a list of the waiting email it is reacting to.

Difficulties may arise as the agent gets more sophisticated responses to a larger number of stimuli – gaze-direction is a fairly arbitrary reaction to the specific domain of email handling. However, as the perceived importance of the message increases, the ambiguity of the character’s response decreases, allowing useful information to be conveyed without interrupting the user’s original tasks.

Maitre-D is a web-site based browsing agent being developed by the author. Through off-line textual analysis of the pages associated with a site, Matire-D creates relations between groups of pages. As a visitor browses the site, Maitre-D notices which page the user is on and produces rated recommendations of related URLs, regardless of whether they are linked from the current page or not.

An animated character can work in concert with Maitre-D, as the interface to the intelligence. The Active-X character is spawned by a second browser-window, which also contains embedded Javascript functions to track events in the first, user-controlled window. Whenever the user moves to a new page, the new URL is reported back to Maitre-D, which runs server-side. Browser history, keywords which describe the new page, and fields simulating the input of a personal user-modeling agent are then used to produce a group of recommended "next pages" for the user to browse. The strength of Maitre-D’s highest recommendation is used to create behaviours in the client-side character.

For example, low-rated recommendations produce no visible reaction in the character. If the user wishes to view the agent’s suggested pages, they can click on the character directly for a plain-text, ordered listing. Better matches can cause the character to look around, nod approval, or wave arms to attract attention. Again, clicking produces a list of recommendations, and selecting from the list sends the browser to the new page.

Figure 2: Genie and Maire-D

The look and feel of the desktop environment provides a rich domain for subtly crafting informative emotional states. By treating the computer as a "setting", we can leverage a huge body of experience in stagecraft, lighting-design and cinema to communicate with the user. The goal is to inspire a general feeling indicative of the state of the system, rather than conveying specific information.

The character can also react to the environment, allowing behavioural modes that don’t directly address the user, yet still serve a purpose. A-life techniques can thus be used to create a reactive character that responds to the stimuli of the computer system. The agent’s goals and desires can express valuable information implicitly, simply through its actions and behaviours.

This area is still being examined, with one attempt linking window/frame border sizes to web-download progress (or other duration-based gauges). A Visual Basic program polls an instance of a web-browser for progress information, and adjusts system-preferences accordingly.

A second program modifies desktop wallpaper according to system performance or network traffic, pushing up towards shades of red to show intense activity, and remaining in darker shades during normal conditions. Tinting the colour-scheme towards green could signify viral infection, fragmented disks or corrupted segments. Monitor keystoning could signify low diskspace.

Again, the intention is not to replace discrete communications, or convey complete information. However, through continuous styles of interaction, general state can by implied, and the user can take appropriate actions to determine specific problems and solutions.

The first priority will be to build a better, more useful animation/character system. The Microsoft Active-X control proved useful for rapid prototyping, but it doesn’t support the multiple simultaneous directives (eg: Glance Left AND look Sad) necessary for more complex behaviours. Also, it has a lot of built in "idle" states with movements and behaviours which have no meaning whatsoever, despite the tendencies of users to attribute importance to them (eg "Why is he bored? Am I boring him?").

Furthermore, the animation-agent is too obtrusive. A character should be able to carry on its activities with many levels of subtlety, attracting the user for explicit communication or going about its business in the periphery of attention, as needed.

Internally meaningful reactions (ie: those not specifically addressing the user, but part of the character’s environment and role) should be explored further. If the animated agent is to be understood as an avatar of the computer’s state, it should be continuously "displaying" information through general activity, rather than only animating to inform the user of an discrete event. For example, creatures might map network security levels to nervousness and agitation, or processor load as animation-speed or visual jitter.

The project will continue to explore these areas using new and more flexible animation system, as well generating more meaningful mappings, in the months to come.

Bates, Joe "The Role of Emotion in Believable Agents", Communications of the ACM, Special Issue on Agents, July 1994

Blumburg, Bruce, and Galyean, Tinsley "Multi-level Direction of Autonomous Creatures" Proceedings of ACM SIGGRAPH 1995

Galyean, Tinsley "Narrative Guidance of Interactivity" MIT PhD Thesis, 1995

Keller, E.F. "Reflections on Gender and Science" Yale University Press (New Haven, CT 1985)

Laurel, Brenda "Computers as Theatre" Addison-Wesley (New York, NY 1991)