IBM Systems Journal, vol. 39, issues 3-4, pg. 479 - 510.
November 2000.

The future of artistic and expressive communication in the varied forms of film, theater, dance, and narrative tends toward a blend of real and imaginary worlds in which moving images, graphics, and text cooperate with humans and among themselves in the transmission of a message. We have developed a "media actors" software architecture used in conjunction with real-time computer-vision-based body tracking and gesture recognition techniques to choreograph digital media together with human performers or museum visitors. We endow media objects with coordinated perceptual intelligence, behaviors, personality, and intentionality. Such media actors are able to engage the public in an encounter with virtual characters that express themselves through one or more of these agents. We show applications to dance, theater, and the circus, which augment the traditional performance stage with images, video, music, and text, and are able to respond to movement and gesture in believable, aesthetical, and expressive manners. We also describe applications to interactive museum exhibit design that exploit the media actors' perceptual abilities while they interact with the public.

Read the paper in PDF form.