The Improvisational Theater Space is an interactive stage where human actors can
perform accompanied by virtual actors. Virtual actors are modeled as
animated "Media Creatures" that are behavior-based automous software agents.
It uses real time computer vision,
speech recognition and and speech anlaysis to sense the perfomer's actions on stage.
We used Artifical Life programming methods and behavior-based
design to avoid rigid scripting of user and content interaction.
The main result of this
work is the construction of animated media creatures endowed with intentionality
and autonomous behaviors. Media Creatures allow content to be active and to present
itself to the user by dynamically adapting to the context of the interaction. We used
Media Creatures to create an engaging Improvisational Theater Space where the
user/performer is engaged in an improvisational dialogue with a typographic actor.
The Improvisational Theater Space is an interactive stage where human actors can
perform accompanied by virtual actors. This space was created following the model of
the street theater, the mimes world and the improvisational theater in
general [Johnstone79].
We wanted to create an interactive stage where human and digital actors meet to
generate an emergent story through their interaction. An important constraint was to
bypass strict scripts of the encounter. We thought of an Improvisational Theater Space not
just as a performance space but also as a playground for participatory theater,
interactive storytelling, or museum exploration.
By participatory theater we mean a situation where a user first watches a performer
interacting with a virtual actor and then is offered to take the place of the human
performer and experience an emergent story from a subjective point of view. In this
case it is important to avoid the need for the user to memorize lines of text based on a
one time observation of the performance. The user only needs to roughly grasp the
situation and the interaction modality in order to be able to participate. This
augmented storytelling environment responds to the teller's gestures, words, and tone
of voice by enriching the presentation with sound, text, images, or graphics.
All of these scenarios need to satisfy the following necessary conditions for the
interactive experience to be truly engaging and immersive:
To satisfy the first condition, we created Theater Space as an improvisational stage. A set
of possible situations is set in advance and the human actor is given the choice to
change the order of the actions and the mood in which scenes are played.
In order to have believable characters, we modeled the virtual actors as Media Creatures.
Media Creatures are autonomous agents with behaviors, sensors and goals (internal
motivations). A media creature knows whether its content is text, image, a movie clip,
sound or graphics and acts accordingly. It also has a notion of its role and ``mood'', i.e.
it can express basic emotions like happy, sad, angry, scared etc.
Media Creatures are modeled according to Blumberg's tool kit for behavior-based
animation of autonomous creatures [Blumberg95]. The choreographic component of
a media creature specifies its behavior according to the context of the interaction with
the performer. As opposed to scripted animation that imposes a pre-defined sequence
of actions to a virtual actor, behavior-based animation, defines
a tree of actions that are driven either by the internal motivation of the creature or by
the external actions of the user/performer or by a combination of both. Hence the
model of the interaction between the human and the digital actor consists of a nearly
infinite tree of possibilities rather than a linear sequence of consecutive actions.
The autonomous structure of the Media Creatures allows them to exhibit a behavior
that "makes sense" to the user even when he/she is not "doing the
right thing". In addition the behavior system and sensors of the
creatures
enable them to understand
situations while interacting with the user/performer and to act as improvisational
theater performers.
We use the term Media Creatures or Media
Actors or Media Tellers as synonymous, according to the particular context in which they
are interacting with the user/performer.
Finally for the third condition to be met the interactive stage
uses remote sensing technology to interpret the actions of the
user/performer. This work is premised on the assumption that immersive spaces require
natural, wireless interaction so that the user/performer is not encumbered with wires or
sensors. Most current virtual environments use bulky head-mounted displays, data
gloves and body suits with multiple cables. While such systems can be extremely
accurate they limit severely the freedom of movement of the user. This results in an
unnatural man-machine interaction and it is the first obstacle to creating a truly
engaging environment. The interactive Theater Space described in this document use
sensing devices which are embedded in the environment, endowing the latter with
perceptual intelligence. As people use primarily vision and audition to communicate
with each other, these interactive spaces use real time computer vision and audition
technology as their source of perceptual information.
In the past, the idea of creating performances that involve both humans actors and inanimate
objects as characters can be tracked back to the beginning of the century. The
avangard Italian Futurists movement inspired a new form of theater called Synthetic
Theater [Kirby71]. The theater of the Bauhaus also experimented with a non-verbal,
dance-inspired theater
of objects with living actors stylized as geometric, often cubic shapes, resembling
modern marionettes and automatons [Schlemmer71].
Although futurist performances did not have great success with the public they can be
seen as early tentatives of staging human actors together with objects in a theater of
situations that required participation from the public.
We believe that the use of Media Creatures/Actors enhances possibilities for
communication in the performance art. For example
although cinema has used text in the past as a means to communicate the character's
inner thoughts instead of voice over - examples include Woody Allen's Annie Hall and
Peter Greenway's Prospero's Book - no similar effects have been used in theater to our
knowledge. The video accompanying this paper shows an example of use of a Text Actor
- modeled by a Media Creatures - that interacts with a human actor by playing the role
of his "alter ego".
Some authors ( [Pinhanez96]) have envisioned a computerized
theater that relies on the knowledge of the
script to drive a computerized stage.
The main drawback of this approach is that it relies on the director
and the actor to rigidly follow a script for the system to be able to work. For instance it
is not uncommon in theater that both the actors and the director change the script
either during rehersals or even right before or during the final
performance [Brook95].
In our view this type of rule based system will not be able to compensate for
human errors or be responsive when some non planned ``magic'' between the actors
happens on stage. It tends to force human interpreters to rigidly follow a predefined
track and therefore empoverishes the quality of the performance.
In summary we believe that behavior-based media actors are a promising approach to
interactive theater for three main reasons:
1. Behavior-based vs script based theater has room for improvisation, both in
the case of the improvisational or street theater in general or for classical scripted
theater that the director and the actors need to interpret and therefore modify.
2. The system is tolerant to human error and actually encourages actors to enrich
or change the performance according to the reaction of the audience.
3. The system can scale from a performace space to an entertainment space.
Behavior-based theater can allow for user participation either during or after the
performance without requiring the new users to learn all the script in advance.