Wearable Performance

Flavia Sparacino, Alex Pentland and Glorianna Davenport
MIT Media Lab

Abstract

Wearable computers offer the street performer powerful tools with which to create innovative experiences for the audience. As wearable technology moves computation from the desktop onto the user's body, it provides an invitation to free performance from the indoor stage, bringing a new adaptive richness to the mobile world of street theater. We find it therefore compelling to explore several application contexts in which wearable technology enhances, extends and creates new examples of street performance. In this paper we offer a taxonomy of different genres of street performance and we describe how the wearable computer transforms, augments and enriches the traditional form in an original and witty manner.
Keywords: wearable computers, augmented performance, street theater.


Introduction

Wearable computers are transforming our technological landscape by reshaping the heavy, bulky desktop computer into a lightweight, portable device that is accessible to the user at any time. Although the computational power of wearable computers is certainly not equivalent to that of some high-end desktop computers, the portability and set of functionalities will nevertheless determine a migration of the computational engine from the house or the lab onto the user itself. An analog to this tranformation can be found in the transition from the drama played in the theater building to the street theater. Street and outdoor performance has a long historical tradition. However its recent form is motivated by the need to bring performance art to the people rather than people to the theater. We have found it therefore natural to try and merge the world of the street performers with the one of the wearable computer and to explore synergies between them. Based on the observation that many street performers are actually skilled craftsmen of their own props and that some have good technological skills or are at least attracted by the potential offered by technology [1], we have investigated how some street performers could benefit from the use of an affordable wearable computer. We have found that wearable computing can contribute to street performance in three ways: 1. It can reduce the amount of ``stuff'' that the performer needs to carry around by creating ``virtual props'' or virtual no-weight musical instruments. 2. It can augment and enrich the performance by adding digital actors that collaborate with the performer in the piece. 3. It can allow for new types of street performances that were not possible before the introduction and spread of wearable computers. In this paper we describe three different types of wearable-augmented performances. A different version of a wearable computer was designed for each so as to satisfy the artistic intent that motivates them. We then draw conclusions based on our past experience with creating technologically augmented performances [3,4].

Wearable Street Performance: Taxonomy and Technology

Bim Mason has carried out an extensive study of street performers [1]. He has defined five categories that group performers according to their motivation and artistic intent. There are: Entertainers, Animators, Provocateurs, Communicators and Performing Artists. We would like to provide a short description of these categories and use them to classify the three types of wearable street performers presented in this paper. Entertainers are defined as those performers with the simple aim of pleasing the audience, either by making them laugh or by impressing them with skills such as juggling, acrobatics or magic. In contrast, Animators play games with the audience. They use audience interaction not just for part of the show but as the main act itself. Provocateurs are more concerned with loosening-up society as a whole. They ask questions of society by going to the limits of conventionally acceptable behavior. Communicators see themselves as educators who feel they have something to teach to the rest of society or a message to pass on. Finally, Performing Artists are mainly interested in showing an artistic work, and their own personal view of art, focusing more on form rather than content. Our work in wearable performance shows examples of how some of these street artists' work is transformed by the new technology, according to the above mentioned taxonomy.


The Performing Artist: The Augmented Mime

The idea is to create a performance that the public is able to enjoy both as is and also as as an augmented performance. The augmented performance allows to operate a semantic tranformation of story fragments acted by the mime, through the use of the added computer graphics objects. In the augmented reality display water is turned into fire, simple inanimate objects become dangerously animated, and yawning generates expressive digital typography. The mime is wearing a small wearable computer [2] in his backpack. A flat panel display is connected to the wearable for the audience to use. Any member of the audience can take the display and hold it against the performer. The panel will act as an augmented performance loupe, showing digital props or graphically augmented objects that collaborate with the performer. This is done by attaching a very small camera the other side of the display such that when the viewer holds the display against the performer's body the camera is also taking a wide angle video image of him. Through this artwork we are interested in exploring how point-of-view tranforms our perception of reality. The "semantic lens" carried by the mime offers to members of the audience a new, transformed interpretation of the story told by the mime.


The Communicator: The Networked News Teller

The Networked News Teller is a street performer that carries a wearable computer with a ``private eye''. The wearable runs a program that shows a constant update of news in the private eye of the performer. This program is set to explore a fixed set of news providers' web pages. It then reconstructs a page that reports the same news from the different point of views of the different news providers. After having chosen a topic, the News Teller reads updated information about it through his private eye and interrogates the passerbys about their opinion on the subject matter. She can then ``perform the news'' in the street based on her interaction with the public and the information appearing on her wearable. The performance consists in a re-interpretation or enactment of the different point of views expressed by the public interviewed in the street and the information published by the press. This type of street performance becomes particularly interesting when the news being reported is one that creates expectation and clustering of opposite opinions of the public. Good examples are presidential elections, or waiting for the verdict of the court in a trial. A performer who interprets artistically and ``discusses'' with the audience on the street themes that have great relevance and impact for our society, represents a social figure of stature.


The Entertainer: The One-Man Orchestra

Many of us are familiar with street musicians that are strapped with wires and carry a large number of musical instruments that can be triggered by different parts of their body. Usually they carry drums and cymbals that receive input from the feet, a mouth organ fixed in front of their mouth, and a guitar. All this gear is usually heavy to carry around and encumbering while playing. In addition the musical mapping between the musical instruments and the body parts cannot change during the performance. The One-Man Orchestra needs to carry only a lightweight wearable computer in his backpack. The computer is connected to five accelerometers placed on his head, hands and feet. The sensory information is processed in real time and a midi output is sent to a synthetiser. Loudspeakers and amplifiers can be worn or placed nearby. Music is generated by body movements by creating a significant mapping between body parts and musical instruments as well as between movements and musical notes. This mapping can dynamically change during the performance according to the artistic intent of the musician. Sound generated from the surrounding environment or speech coming from the audience can be easily sampled and integrated into the performance.


Conclusions

We have found it compelling to explore a variety of application contexts of wearable technology to street performance to create a richer or even a new experience for the viewer. By customizing a networked, multimedia computer that can be worn as clothing or is built into the performer's clothes, we can offer to the street performer new and powerful tools for gathering audiences. We hope that the creation of a community of wearable augmented performers with a set of experiences and needs will also serve as a push towards future improvements of this new technology.


Acknowledgments

We would like to thank Thad Starner for his suggestions, technical wisdom and enthousiasm for this work, and also the many "vismoders" that have contributed to different aspects of this research.


References

[1] B. Mason, Street theatre and other outdoor performance, Routledge Publisher, New York, 1992.
[2] T. Starner, S. Mann, B. Rhodes, J. Levine, J. Healey, D. Kirsch, R.W. Picard and A. Pentland, "Augmented Reality Through Wearable Computing," To appear in Presence, Special Issue on Augmented Reality, 1997.
[3] F. Sparacino, DirectIVE: Choreographing Media for Interactive Virtual Environments, Masters Thesis, MIT Media Lab, October 1996.
[4] C. Wren, F. Sparacino, A. Azarbayejani, T. Darrell, T. Starner, A. Kotani, C. M. Chao, M. Hlavac, K. Russell and A. Pentland, "Perceptive Spaces for Performance and Entertainment: Untethered Interaction using Computer Vision and Audition", Applied Artificial Intelligence Journal, Special Issue on Entertainment and AI/Alife, Vol. 11, Number 4, June 1997.
Flavia Sparacino
Last modified: Wed Oct 14 01:29:57 EDT 1998