1001 Electronic Story Nights: Interactivity and the Language of Storytelling

Glorianna Davenport

This conference focuses on interactivity. I have worked with interactive cinematic projects since 1980. In this talk, I will discuss some of my current thinking about "the language of interactivity," and show you some of the recent work we have been doing at the MIT Media Lab. I am not concerned that you understand every detail about the inner workings of these pieces -- some of them will be available out in the lobby later for your examination. In discussing these examples, I will emphasize general features and concepts. If any of you have burning questions during this "show and tell," wave your hand around and I'll try to take the occasional question.

We're here to celebrate change and the new opportunities for expression which are arising out of new, enabling technologies. As we witness the evolution of these technologies, we see that is that they are moving us toward systems which can learn. Several ancillary technologies are also especially important, in that they measure and can manipulate user input. For example, new sensor technology will help a system sense the presence and activities of an audience without requiring them to actually handle an input device. One type of sensor responds to the presence of the small electrical currents which typically circulate through the human body. This type of input can provide a feedback loop amongst an audience, story materials, and a sequencing or storytelling engine.

In this slide we see Associate Professor Pattie Maes, whose research focuses on computational agents, sitting in the middle of a virtual space chatting with a dog. The dog, Silas, is an autonomous agent; he has been programmed in 3D to exhibit high level autonomous behaviour: Silas feels hungry, Silas searches for food; Silas drinks some water, Silas needs to pee, identifies an upright architectural feature, and takes a leak. We will revisit this creation of Ph.D. candidate Bruce Blumberg later, in the context of a particular story.[1]

For the moment, we use this example of an interaction between the dog, the virtual world, and Pattie to discuss a vision to which we must aspire when discussing the "language of interactivity". Ron Evans, a native American storyteller, clarifies this vision in a story he tells about the chief of an African tribe and a missionary who visits with them from time to time. On one such visit, the missionary brought along a television as a gift to the chief of the tribe. When he arrived at the village, he presented the gift in a ceremony which was accompanied with appropriate pomp and circumstance. The gift generated great excitement in the village. Every night, the chief turned on the television set and the whole village stood around it and watched the stories that were coming out of the set. (We need not concern ourselves with the source of electricity, and we can imagine that they were able to tune into a transmission from a well-positioned satellite.) Several days later, the missionary bid farewell to the chief and continued on his journey. Six months later, the missionary returned to the village, where he discovered the television set was nowhere to be seen. The people of the village no longer gathered around the television each evening; instead, they gathered around their tribal storyteller. The missionary, somewhat baffled and hurt, went to visit the chief. With agitated voice and gesture, he asked the chief what had become of the television. The chief calmly replied, "I listen to my storyteller; he tells many stories." The missionary pressed the point, "But the television set, it too has many stories." The chief nodded wisely and responded, "Ah, but my storyteller knows me." [2]

Until the machine can understand story and synthesize new story elements, effective personalization will require close attention on the part of the story's author. To achieve personalized delivery, the human storyteller must invent a fabric rich enough to accommodate many pathways through a particular story space. These pathways reflect the personal interests and attention of diverse audiences. As Michael [Hill] has already mentioned, publishers often fear that the diversity of extensible storytelling greatly escalates the task of production. Like many cinematic producers before them, these publishers may trade-off timely completion against expressive invention.

Yesterday, John Collette hosted my visit to the Australian Film, Television and Radio School -- I've heard about this school for many years, and found the facilities quite remarkable! However, what I most appreciated was John's enthusiasm concerning the new media. No chance for the old media to grow stale with this level of energy! John dreams of bringing all kinds of computers into the school. As we talked about digital production, John and I discussed his reservations about building this new medium on top of an older, well-established medium. Michael [Hill] has made reference to this as well. Many years ago, I named my group Interactive Cinema. At the time, the low bandwidth media -- text, still picture, and sound -- were being rapidly assimilated into the computational language. My concern at that time, which continues today, was how to bring this computational approach -- which enables personalization -- into the high bandwidth arena of cinematic storytelling.

How do we work in this new medium, which seems so different? This medium is participatory and democratic; as makers, we must respond to these attributes. The medium supports distributed connectivity, which changes the demographics and the very experience of audience. As John and I considered the topic for today's discussion, I was inspired to reformulate the story which Steven Hawking tells in the opening of A Brief History of Time. As Hawking tells it:

A well-known scientist (some say it was Bertrand Russel) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy. At the end of the lecture, a little old lady at the back of the room got up and said: "What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise." The scientist gave a superior smile before replying, "What is the tortoise standing on?" "You are very clever, young man, very clever," said the old lady. "But it's turtles, turtles, turtles all the way down!" [3]

In response to John's anxiety about mixing the new and the old, I suggest a that the old lady was not far off. As digital storytelling emerges, we will discover that we are not dealing with galaxies of bits, but rather that we stand on the shoulders of great storytellers through the ages, and that from today forward it will be story, story, story all the way down!

Last year, this conference concerned itself with Narrative. I believe that we must understand the premise of narrative before we can examine the "language of interactivity." From an historical perspective, major advances in civilization may be attributed to the need and desire of human beings to share stories. Spoken language was perhaps the first, truly revolutionary advance of civilization. To begin, any particular set of utterances must have been shared with only a few people who were situated a single, very small geographic location. Language flourished with the "glory that was Rome." By decree of the Roman emperors, all inhabitants of conquered lands were required to adopt Latin as their official language. As common language spread, travelers were able to share stories which contained valuable news -- the location of a drought, war, disease. The content of these stories enabled a limited capability for prediction and was often critical to the survival of the audience. Today, spoken language remains the most powerful force in securing and extending community.

While spoken language helped build community, spoken communication lacked durability and extensibility. Without a common language, how did the traveler share his message with the community into which he had wandered? Some stories were communicated by drawing on walls; other stories were shared by means of pictorially expressive gestures combined with what was probably some unfamiliar sound sets. Pictograms often lasted longer than the spoken word: generation after generation could visit the rock cave and view the durable images, which carried meaning served some special need. Slowly, pictorial signs were codified into symbols. These symbols, eventually, were formalized into an alphabet system. The abstraction of language provided a collection of signs which could be recombined, according to rules of adjacency, into groups of letters representing spoken words -- an interlinking of thought, vision, and sound. With the invention of written language, verbal communication gained a permanence and an interoperability. Written language enabled the poets, priests, scribes, and others who acquired the facility to write down their stories. Permanence insured an enabling access to myths and moral tales which was not limited to generational or parochial continuities.

It took thousands of years for the written language to be embodied in a mechanical device for the purpose of making many copies. Movable type was first invented by the Chinese over one thousand years ago, but it was the Europeans, not the Chinese, who applied the technology of movable type to the printing press, allowing the printer's design of a written artifact to be reproduced and distributed in lots of hundreds or thousands of copies. Centuries later, the word processor placed the capability of the typesetter onto the desktop of offices and homes.

A 19th-century invention, photography, introduced a new paradigm for replication. In the case of the photograph, the image-making engine -- the camera -- is itself a mechanical device. The human mind, eye, and body work together to propose and capture the intended image. The photograph mirrors the world, not as we perceive it, but as it has been composed, framed for posterity. Our rereading of the image is governed by its composition, its form, and the size of the display. Generally speaking, the size of the display governs our immersion in the image.

The printing press changed the way in which the storyteller perceived her audience. As soon as many copies could be printed, the idea of the "mass" audience took hold. The press-owning entrepreneur had to grapple with the notion of distribution channels. As the commercial enterprise of print grew, the client of the press was often a commercial or political entity, rather than an individual reader: advertising increasingly subsidized the cost of newspaper printing, and soon, the expensive private newsletter soon gave way to the "penny press," affordable to all. However, distribution of identical copies of news on a daily basis required stylistic and formal coherence in the production process. The invention of radio deepened the rift between authors and audience. Whether equated to copies or to transmission, stories for the "mass" audience required a distribution channel which was generally owned by someone other than the person making the media content.

As the "mass media" of print, radio and television grew and took hold, the telephone was an innovation which stood apart. Invented in the late 18th century, it took 40 years for the telephone to become recognized as a household technology. In its infancy, nobody believed how person-to-person conversations over shorter or longer distances would revolutionize the world. The thrust of a personalized medium appearing amidst the mass media will be truly understood only in the future. For the moment, using ourselves as subjects, we occasionally and unscientifically have discerned moments -- times of political stress or in the creation of media idols -- when the worlds of mass media and personal communication came into a profound (and profitable) symbiosis.

Interactivity brings pressure to bear on the channel. In order to make the telephone efficient, switches had to be automated. Today, we must design into our media objects a feedback-channel which can elicit and make use of signals from the audience -- these signals from the audience will control the "automated switches" of future storytelling systems to some extent. In the case of early video-on-demand trials, the back-channel allowed each viewer to request a particular movie from a fixed menu of choices. In this case, the back-channel was minimal. The wider the bandwidth of the back-channel, and the more distributed the system, the more the audience can contribute to the program. Again, it was interesting to talk to John Collette yesterday, because John does not believe that interactivity is about information. I can buy that, because I believe that successful interactivity is really about story. However, I believe, that this is an issue of semantics and emphasis. Ultimately, interactivity engages us in the assembly and construction of story, but in the process the information bits -- whether these are the program proper or a trail of user activity -- are central to the endeavor.

Even as we are trying to understand it, the digital universe is changing. Distinguishing information from story will become more and more difficult as the lines between storage, program, content, medium, and interface become more transparent. Take, for example, the problem of text and interactivity. Today, a limited form of hypertext has taken off. The "hot link" brings us to more information. Often, this results in our getting lost and forgetting why we were reading to begin with. This raises two problems: one is the issue of effective presentation, and the other is the problem of the memory trace ("Where have we been, where are we going, and who has been here before us?"). Text poses a difficult problem because there is no standard temporal dimension to text. We are used to the page format, a 2-dimensional expanse of text; however, reading large amounts of text on an electronic screen is not a particularly pleasant experience. Over the years at the Media Laboratory, we have dreamed of a flexible surface for electronic text, one that you could carry with you in your back pocket, hold however you choose, and read at your leisure. The goal of reusable, flexible digital "paper" has recently been taken up by Joe Jacobson, a young physicist on the faculty of the Media Laboratory [4]. Current progress includes the invention of an ink substrate which supports a heat-reversible process, where particles turn from black to white or white to black at certain temperatures. We imagine a future in which you can stop at a kiosk on your way to work and down-load the next section of the news onto your own personal "super paper." This is the sort of radical project which challenges many of our underlying assumptions about the electronic interface.

Before delving in to the relationship of story to interactivity, we need to examine the notion of interactivity itself. What does interactivity really mean? The fact that I can take my piece of "super paper" and recycle it, fill it with news which is meaningful to me, poses a physical as well as an electronic model of interactivity. At the physical level, I pull the sheet of "super paper" from my back pocket; I introduce it to the system; I retrieve it from the system; I hold it up and scan its contents. These interactions with the object provide a practical method for achieving my goal, catching up with this morning's news. However, interactive in this context also includes the way in which electronic sub-systems interact: the information and the layout can be affected by my personalized user profile. Once the paper can track our eye movements, we can go even further in personalizing the selection of material and its layout. In addition, in a networked electronic world, the "super paper" can actively link together a larger society of audience, networked communities which share observations with other people and with programs.