Glorianna Davenport
Stefan Agamanolis, Brian Bradley, Flavia Sparacino *
Interactive Cinema Group, MIT Media Laboratory
keywords: sustainable story environment, society of audience, distributed storytelling, interactive narrative, transcultural, ecological, dream submission, dream processing
Digital media and networked, two-way communication channels are rapidly transforming our access to knowledge, our inventions, and our artistic messages. In the past, artistic practice focused on the construction of fixed, static "expressive objects" which stood as intermediaries between the artist and her audience. Today, technology allows us to make computer-assisted artworks "with a sense of themselves." These works incorporate behaviors and real-time responses patterned after those of living things: creatures, communities, and ecosystems.
In this paper, we present the theory, process, and progress of a new collaborative work -- "the Dream Machine" -- currently under development by the Interactive Cinema Group at the MIT Media Lab. The Dream Machine is a highly-distributed interactive narrative designed to engage a large, widely-dispersed society of audience. It uses the techniques of cinema, theater, and architectural space design to improvisationally craft a playful, lyrical, emergent story experience in close collaboration with its audience of "co-actors." The Dream Machine's presence simultaneously spans several venues in both the real and the story world. It is accessible through the correlated, distributed environments of the World Wide Web, live-performance Public Spaces, and pager networks.
Three major questions lie at the heart of our exploration:
The Dream Machine project is designed to enhance the communication between and among people as they shape, personalize, and navigate their way through information-rich environments and dynamically adaptive, emergent stories. It presents its audience of co-actors with multiple interconnected options for interaction, including:
The Dream Machine project focuses on our awareness of information flow as intervention, and on our interpretation and use of information as adjustment. These forms of engagement are used to generate transformation both the individual and the collective level.
Background: What is art and how do we know it?
Throughout history and across cultures, the artist has provided a timely understanding of the experiences, the longings, and the imaginings of her culture. When an art object or performance is released into the world, the probing forces of public scrutiny, recognition, and empathy reveal meaning and reinterpret it for a larger audience. In this way, the artist's personal energy takes from and is reabsorbed into the emergent wellspring of culture.
Cultural need and technological opportunity periodically converge, enabling artists to appropriate radically new materials and techniques for incorporation into the art object. Occasionally, these opportunities have catalyzed the invention of fundamental, revolutionary forms: the novel (in the late 17th century; photography (in the 19th century); high-rise architecture and cinema (in the 20th century). At other times, new techniques have provided milestones for cultural shift: Abstract Expressionism, "happenings," rock & roll.
Today, we are using digital technology to reinvent the art object. At the heart of this technology is the concept of "System." Input and output devices serve as surrogate extensions of our senses, providing connectivity to the world. Computational engines bear some correspondence to our own brains and nervous systems. Synthetic memory devices approximate our own capabilities for remembrance and recall. When hooked together into a system, the human-like (or at least, humanizable) attributes of these devices beg the digital artist to create systemic worlds and scenarios which invite audiences to actively participate in the co-construction of narrative meaning. The audience's engagement becomes active and conversational, rather than passively receptive; narrative reverie transforms into meaningful, consequent involvement.
Large-scale computer networks can interconnect a widely dispersed society of audience -- on the individual as well as the collective level -- across vast gulfs of distance, time, and culture. The networked fusion of a systemic work to its audience steers the dialectic of the artistic community away from concerns about how active process disrupts the pure abstraction of expressive principle, towards a concern about who controls the experience itself.
Once an author abandons the notion of total control over her work and instead engages her audience in a process of co-construction, we discover that an even more fundamental question looms large: can the art object become organic, adaptive, and generative? No matter how freely we engage in reordering a collection of fixed, pre-made parts, the combinatorics are finite, and the set of useful recombinations is significantly smaller than that. Only when the seed ideas operate in a bottom-up fashion -- as embryonic nuclei in the midst of semi-autonomous behavioral characters and generative state engines -- can we launch a digital narrative which challenges our fundamental assumptions about closure, objectness, identity, and temporal construction.
As we enter the 21st century, we must view knowledge and our human situation as organic parts of a larger system. Using the computational medium, we can more closely simulate the human engagement in complex systems which tout no final authority or answer but which recognize co-construction as a valuable process, full of uncertainty and discovery.
Why now?
Each fall at MIT, I teach a course called "Workshop in Elastic Movie Time." In the 1997 session *, the convergence of desire and technological opportunity presented us with a set of fundamental challenges: How can we create a graphically beautiful and consequent narrative world which stretches across several venues, both real and synthetic? How can we imbue this world with enough robust autonomy to accommodate interactions with a human audience of collaborators, co-conspirators, and co-actors? As the Dream Machine project formed and emerged from our imaginations, the world became a fanciful geography, layered with the dreams of a global audience, filled with dream images submitted by that audience, and populated by cross-cultural characters who dynamically manipulate and behaviorally respond to information (both real and imagined) about themselves and the audience.
In this paper, we present several beginning fragments of this new, ongoing narrative project. Our approach to constructing this work is collaborative, growth-oriented, and evolutionary. Major research questions include: Who orchestrates the narrative or story response -- the designated human leader or a machine surrogate which can act and react convincingly, without hesitation? How does this systemic approach affect closure?
Because every aspect of our dream world is constructed from digital bits, it is flexible, extensible, highly responsive, and dynamically reconfigurable. The system grows and develops organically through use. First, a small collection of individual narrative nuclei must be developed. Then, because the audience is invited to contribute to and extend the narrative as they engage with it, these nuclei grow and interweave into a larger system of interconnected narrative -- a sort of relational and causal macrame, as opposed to simple causal chaining.
In order for the Dream Machine experience to work as narrative, we have prepared several flexible story scenarios, story settings, and semi-autonomous characters (the Cannon Woman, the Dream Doctor, Mme. Zelda, the Gossip Girls, the Twin Reporters), each with their own associated personalities and behaviors. These initial narrative nuclei will give shape to the larger whole; as we launch them into a network capable of communication among diverse and widely-distributed parts -- algorithmic, imagistic, and human -- we expect to witness the unfurling of an almost chaotic story experience in which random interaction, consensus, parallelism, and transmutation are all equally valid energies. Our digital narrative will reflect stochastic rhythms -- the rhythm of a process which, as it approaches the articulation of reality, pauses, regroups to absorb energy in the proximity, and then advances as a fresh perception.
First- and second-generation authors will continue to create narrative nuclei and embed their governing vision in the behavioral attributes of particular parts. We do not seek to author everything that will be said. As parts and characters come into being, they are only partially realized, embryonic expressions still in search of mass. Through them, we seek to create a situation and a pool into which we can peer, observe the actions of contemporaries, perform, discover our own emotional connectedness... We believe that these narrative nuclei have the capacity to mature and to shape the complex organism of the Dream Machine, whose eventual nature can not yet be predicted.
Why Dreams?
-- Arthur Miller, Big Sur and the Oranges of Hieronymous Bosch
Throughout the centuries, artists have created dreamlike representations to express that which is fleeting, confusing, deliciously sensual, humorously profound. At a time when digital imaging and networking have become widespread and robust, we seek to create a Dream Machine -- a very distributed story for the millennium. The Dream Machine takes its cues from the emotionally and visually rich language of dreams, as well as from the phenomena of an electronically connected, global society of audience. While we cannot predict with certainty what will draw the society of audience into a fictional narrative, the universal and pan-cultural experience of dreams (which Freud described as "the royal road to the subconscious") seems a propitious place to start.
DreamWorld is not a destination for somber go-getters of the Information Age. The experience of traversing DreamWorld will resemble a walk through a pleasure garden, where each setting and encounter delights and surprises us. In this world, our actions have narrative consequence, both immediately and in the long term. Characters can lead us down a path, but they can also mis-lead. Juxtaposed audiovisual images -- simultaneously literal and abstract -- can engage our emotions and cause us to work backwards from the realization of visualized fragments to the nuclei of thought.
Each encounter with the Dream Machine is designed to generate a story experience at some layer of granularity, from brief encounters to longer, episodic structurings. Over the past eight months, this group of authors have worked collectively (and individually) on several narrative nuclei which form the basis of this world. Each component has its source in some particularly absorbing, obsessive, or inexplicably recurrent image; each is the articulation of a particular insight tumbled into thematic existence by contact with the external world at a propitious moment. These robust, evocative, adaptive interactive narratives may choose to offer the amenities of short- and long-term story closure, but they are also well-suited to reflect the intrusive uncertainty of reality and the complex, precarious nature of existence.
A Very Distributed Story Environment
One fundamental idea driving the Dream Machine's design is our desire to create a very distributed story system in which a shared narrative engine simultaneously spans and interconnects several manifestations of the story world: live-performance Public Spaces, situated in architectural venues; the World Wide Web; and a mobile, highly distributed network of pagers, which provide tiny, highly mobile alphanumeric windows into the story. Such a world would allow the audience to participate via their actual physical presence in a place, or through their "virtual presence," facilitated by devices. The media format of the narrative varies according to the venue and the participants. Interactions which occur in one venue affect all venues; thus, story and personal transformations occur in response to interactions which are both known and unknown, seen and unseen, individual and collective.
The Public Spaces provide opportunities for accidental as well as intentional immersion in visually dynamic dreamscapes, for meetings with other participants, and for encounters with real and synthetic story-world characters. They integrate large-scale video projection, image and audio processing, remote audience-sensing technology, and networked communications to create "the Edge of DreamWorld" -- places where passers-by have the opportunity to peer inside of the story world and meet its inhabitants without actually entering it. Audience activity within these spaces is fed back to the narrative system, affecting the story state and launching messages to and from distributed participants.
The Web Presence provides a flexible, networked World Wide Web
interface to the story world. Here, Java-animated settings and character representations freely intermingle with photorealistic images, sounds, movies, moving text, and other media types to create an interactive story environment. In this space, short- to long-term story episodes and co-constructionist activities are the normal modes of engagement. Here, one can expect accidental or deliberate encounters with other audience members, fictional characters, and the stuff dreams are made of.
The Pager Network provides highly mobile, bi-directional contact with the story world. It carries a mixture of human-to-human and story-to-human messaging and story fragments which can reach the audience in any place, at any time.
The Play:
Beginning players normally lack clear knowledge or expectations about how to alter the state of the dreamscape or its encompassing narrative: they begin as simple wayfarers -- "innocence agents." As their knowledge of the world grows, they gradually adopt deeper, more assertive roles which actively generate the story's progress. Once a player accepts a role, she is expected to perform certain functions and to actively seek rewards within the narrative. As Guides, Seers, Goddesses, or Demi-Gods, players are endowed with special powers which allow them to manipulate the rules of interaction and transformation. As Citizens or Character Surrogates, the player's powers are human-scale, comparable to the human characters which populate DreamWorld. Through their access to maps, signs, knowledge, transformational resources, and each other, the players affect strategies and create narrative transformations according to an iterative process of intervention and adjustment.
Iconic Imagery, Maps, and Transformations:
Narratives can be built around iconographies of place, time, character, action, situation, theme, emotion, scale, and value transformation. One key aspect of this project is the development of a cast of transcultural characters which convey a sense of a diverse global story community and allow viewers to experience the artforms and idiosyncrasies of many different traditions. Often, a particular iconography is culture-specific and does not translate well across cultures; therefore, we have designed our Dreamscapes to connect as fully as possible with diverse iconographies which have achieved global (or at least large-scale) recognition. It is not impossible to imagine that under certain circumstances the Minotaur in the maze might be transformed into Santa Claus collecting his reindeer at the North Pole.
Maps and plans of the narrative space (both physical and conceptual) are essential to the content-maker, and they also establish a world view for the participants. The Dream Machine offers many types of maps -- both physical and metaphysical -- to assist the narrative traveler. Architectural plans, road maps, symphonic scores, Buddhist mandala, and weather maps are just a few of the representative forms which lead to greater understanding and insight.
Narratives often rely on rich, multilayered iconographies which symbolize a value system or a set of interrelationships. Examples include: ancient mythology, the Zodiac, the Circus. Fundamental aspects of a universe are frequently folded into an iconography of character; from Shiva to Scrooge, from the Red Queen to the Tin Man, participants can "read" narrative intent and acquire expectations from a well-designed representation. Similarly, an iconography can compress the rules and values of a universe into a single location or setting. It can be used to denote specific resources of power, such as money (Monopoly) or player type and rank (chess). Stylistic iconographic types are often invented to indicate narrative state: examples include cinematic transitions and musical themes or refrains. In interactive productions, iconography can be used to flag potential moments of transformation.
Improvisational Performance and Information Ecologies --
The Dream Machine is about the fantasy of escape; as the narrative enters our consciousness (via a pager message, or an encounter with a foreign character in a hallway, or in a rainstorm on the World Wide Web), it begs us to launch into a voyage of discovery between what we call the real world and places on the horizon that speak to us of freedom or adventure. Ruled by ecological cycles of information and disinformation, the DreamWorld exhibits the poetics of transformation in many forms. It unfurls over time with the active participation of the audience, who engage in parallel activities of voyeurism and improvisational performance. By inviting the audience to actively engage in imagining and revisioning (particularly through interactive story involvement, the submission of dreams, and the trading of dream imagery for experience), we empower the audience to amplify, qualify, and even invent aspects of their journey forward.
The supreme delight of a dream lies in its transformative power. When the personality liquefies, so to speak, as it does so deliciously in dream, and the very nature of one's being is alchemized, when form and substance, time and space, become yielding and elastic, responsive and obedient to one's slightest wish, he who awakens from his dream knows beyond all doubt that the imperishable soul which he calls his own is but a vehicle of this endless element of change.
Central Metaphors of the Dream Machine
When traveling through DreamLand, one discovers unexpected centers of activity. Each of these beckons us, the audience, to improvise action: none more than the virtual Circus. When we enter this artificial reality1, it perceives us and projects our image into a computer-graphic Big Top with its performance ring, a chair, and a cannon. As we explore this space, we discover a physical relationship between our real and virtual selves. We raise a hand, and we find ourselves holding an umbrella. Perhaps it will rain cats and dogs soon. We lower our arm, and the umbrella disappears. As we relax into the chair on the set, our mirror-self relaxes into a virtual chair of exotic appearance. Should we lean forward to think, a virtual gramophone appears to musically accompany our thoughts.
These transformations initiate us into our improvisational performance role. Much of our research concerns how to inform the audience of their magical powers. When elephants enter the space, we want the audience to act as if they were the elephant trainer; when we raise a hand, the elephants sit down. In another act, the audience is paired with a virtual clown. Can the clown induce the audience to fire a woman out of the cannon? Boom! The Cannon Lady flies out of the cannon. She flies entirely out of this part of DreamWorld, swooshing into the WWW site or onto the Edge, creating turmoil in either case. Can we be propelled to stir the surface of certainty, to be fired with the Cannon Lady into another reality?
******
The Edge of DreamWorld places us just outside the story space, yet connects us to it. The presentation of images and sounds affects the activities of passing crowds; in turn, the crowd activity (such as their speed, trajectory, proximity, and size) affects the behavior of characters and objects on the screen. As we bustle down a long corridor, we are caught in the gaze of an exotic Indian dancer. She happily begins to apply make-up in preparation for a traditional dance. If the viewer leaves too soon, the dancer glares angrily at him, turns her back, and retreats into DreamLand. If the viewer lingers, he is rewarded with a beautiful dance. In contrast with other parts of the Dream Machine, interactions in these public spaces is by nature more subtle and passive. Instead of a direct interaction with narrative elements, the movements and gyrations of passers-by affect the moods and behavior of characters, the "weather" inside of DreamLand, and the flow of story information in other parts of the system. At the Edge, juxtapositions and transformations are generated by dynamic, computer-assisted poetic perception combined with the audience flow through sensor-laden architectural spaces -- do we consider this a chance operation?
***
In the DreamWorld narrative environment, current information is Water. It has a natural ecological cycle similar to the Earth's water cycle. Clouds of information form high in the atmosphere; as they travel, they occasionally release information and story fragments in the form of rain or snow. This precipitation informs whatever characters or objects it touches, affecting their mood, emotions, situation, or state of knowledge. Fallen information gathers in puddles and pools. It flows in streams into larger rivers, which may be swum in or fished. Eventually, it flows into a vast ocean of accumulated information, where the cycle begins again.
The Wind is another important element in this information world. It flows quickly across a broad expanse of features in the landscape, picking up and carrying particles of informational soil and water, and redepositing them in other places. Wind provides a mechanism for browsing and making connections among widely-dispersed pieces. Certain vigorous characters, such as a Text Tornado, can read messages from the dust.
History is embedded in the rocks and soil. It is solidified into stratified layers, where Depth represents Time: you can dig down into the past. The flow of wind and water can erode the rocks and soil, revealing previously buried features. There is also a plate tectonics of History. Vast clusterings of history congealed into rocks can move and collide beneath the surface, out of sight. When these plates collide, they can force huge mountains of History or Concept to appear on the surface, in plain sight. When they move apart, vast chasms form. New activity occurs within the split and attempts to fill or bridge the chasm.
For a more personally directed search of History, you can plant a Query as if it was a seed. This seed grows a root system which penetrates the historical layers of soil. Fed by rain and the flow of underground streams, the above-surface growth represents the appearance of new relevant information. Branches grow and thicken; leaves and green growth tips provide a quick visual measure of recent activity. Especially interesting "hits" are highlighted by flowering. You may plant your own private Information Garden, or you may enjoy the Public Gardens. A walk through a Pleasure Garden may actually trick you into doing useful work.
Conclusion:
The Dream Machine project is an experimental narrative for the new Millennium. It draws upon and extends the storytelling traditions and methodologies of literature, performance, and cinematic narrative to create a new, living, organic form -- the very distributed, emergent story. Based upon a highly distributed, networked architecture, connected to remote sensors, and steered by a sophisticated computational engine, the Dream Machine senses and dynamically responds to information flows, intentionality, and occurrences in both the real and the story world. It is designed to simultaneously play out on a variety of devices, each with its own characteristic powers, limitations, and bandwidths. It actively engages its global society of audience in the process of co-construction of meaning.
Unlike stories from an earlier time, our narrative is about dynamic process in the post-mechanical age. Even as we celebrate machines, we also mock them -- launched by human intervention, the Cannon Lady flies from the circus cannon, creating dust storms where she comes to earth in DreamLand. As we generously borrow from the past, we are wedded to a future in which all things familiar can be transformed by the human eye, mind, and hand.
* End Note: Contributors from the 1997 workshop:
Faculty: Glorianna Davenport, Ron MacNeil, Carl Malamud, Joe Paradiso, Larry Friedlander.
Students: Stefan Agamanolis, Aseem Agarwala, Freedom Baird, Barbara Barry, Brian Bradley, Kevin Brooks, Ledia Carroll, Chloe Chao, Peter Cho, Pascal Chesnais, Jeremy Gealle, Karrie Karahalios, Alexander Kim, Tara Rosenberger, Arjan Schutte, Flavia Sparacino, Joanna Stone, Phillip Tiongson, Hannes Vilhjalmsson, Russ Windman, Craig Wisneski, Yuh-Shioh Wong, Jeanne Yu.