Kevin Brooks, Ph.D.
Motorola Human Interface Labs
Published in Design Management Journal, Spring 2001
As our computer experiences become more complex, we are constantly looking for ways to manage this growing complexity. Software developers weave increasingly more features into already bloated applications because… well, because they can, and because competition with each other demands that they do so. Human interface designers, as members of development teams, would like to temper ever exploding feature sets with smarter interface tools to make life easier for the computer using masses. Examples of this range from simple floating tool palettes in Adobe Photoshop, to complex intelligent interfaces and interface agents which are slowly making their way out of the academic labs and into the commercial marketplace. The dilemma is that easy software sells better than difficult software, feature rich software sells better than feature poor software, and the two best selling categories, easy and feature rich, seem to contradict each other.
Some of the new solutions to this dilemma of increasing complexity are as old as the hills. Humans have been expressing themselves and interacting with each other for millions of years. There really is nothing new under the sun, just stuff we forgot. As designers and consumers of digital experiences, we can find answers to digital interface complexity in both the physical and the familiar. We can look to the methods and metaphors we as humans have always relied on. We can use the softer elements of our humanity to design the harder mechanisms of our technology. It requires, however, a careful choreography.
When I was a film student, I viewed many films which were described as fine examples of the film production craft. More than once we were shown a black and white "experimental" short film produced by the National Film Board of Canada, where two brightly dressed classical ballet dancers glided through an otherwise entirely black frame. It was a stark and simple scene showing elegant choreography and coordination between a male and female dancer. Directed by Norman McLaren in 1968, Pas De Deux (pronounced pa - de - deu and literally meaning dance of two) is an award winning film. Not because of the amazingly beautiful camerawork, musical accompaniment or dancing skill, although all those features are certainly notable, but instead because McLaren chose to accentuate the film by violating our established notions of human body movement and our rigid sense of time, with the use of surreal multi and after-image visuals effects. As the dancers moved, their ghostly images would multiply, both leading and following the main image of their bodies, in a series of duplicate likenesses. These optically produced effects brought into focus the title of the piece in a brilliant way. While one might assume that the pas de deux referred to the two dancers, the real dance was between the audience's expectation of dance itself and their changing conception of spatial and temporal representations of movement. Seeing the multiple body images made one think differently about moving in time. The special effects technology of the film blended with the art form.
Similarly, there is a pas de deux between digital technology and design. From its earliest military beginnings in the 1940's and 50's, to its rebirth with the personal computer in the 1980's, to its re-rebirth with the commercialization of the internet and the exponential evolutionary growth of computer architecture in the 1990's, digital computer technology continues to be a dancer whose movements touch all of our lives. In the western world, business, entertainment and even family communication is increasingly dependent upon personal computers, cell phones, personal digital assistants and the internet. We are starting to see our world as simply a collection of nodes: callable, email-able and ping-able. This technological influence is spreading, breaking through the formerly restrictive boundaries of justification for technology's presence — namely population density and to some extent, market value. Cell phones can now call and be called from what used to be known as remote landscapes. When Motorola's Iridium satellite system was operational for public use, it was possible to place a call with the nearest cell tower hundreds of miles and several local cultures and dialects away. Remote is not longer remote.
Let us consider the telephone, a device which has become so common over the past century that few people are aware of the underlying technologies which make it work and the technological changes it has undergone. What we have been aware of is how the phone has changed on the outside — its changes in interface. We were aware of the change from party line to private line, we were aware of the change from rotary to touch tone, we are aware of the option to roam cordlessly within our homes while on the phone, and we are aware of the opportunity to roam wirelessly through the streets and highways while on the phone. In all that time and throughout all of that underlying technological evolution, the only substantive interface changes have been from the party line hand crank, to the rotary dial, to the touch tone buttons. Sure, any of us now can place a call while merging into 65 mph traffic, but the way we place that call basically has not changed since the introduction of touch tone almost 40 years ago.
But let us now think beyond the telephone to the personal computer, and then beyond the PC of today, which really struggles to provide any sense of personal-ness at all. The PC of today is more of an impersonal device, little more personal than its room size mega-heat generating predecessors. It is personal only in the sense that we can own it and to some limited extent dictate how parts of the interface appear, but that's it! For humans to truly be able to dance with digital technology, we need to design technologies that provide experiences which bridge cultures, languages, currencies and ideologies. Smart means different things to different people. Tool means different things to different people. Therefore, the possible permutations for Smart Tool are mind boggling. How do we approach this issue? What universal tools, methods, or approaches do we have that span such wide landscapes?
Perhaps it is in our notion of the word personal that holds us back. Instead of personal digital technology, what we need is intimate digital technology. Intimacy includes a closeness, a collaboration toward a common goal, a persistent awareness and profound knowledge of the other. More importantly, intimacy includes a common story. We can tell the story of our intimate friendships. We can tell the story behind our intimate objects. Many of us have stories behind our intimate apparel. To be intimate goes beyond ownership — it means fit, function, acceptance and definition. People define themselves in part by the intimacies in their lives. Why not work toward intimate computing design? A digital experience founded on stories.
This is possible because stories and storytelling are largely about two things: culture and structure. It is impossible to have story without at least a small amount of both culture and structure, two complex and intertwined elements. As storytellers and authors Norma Livo and Sandra Rietz put it, "Story structure is not an accidental or idle invention, but the profound product of a culture's evolved perceptions of the way the universe works." Stories represent a means of approaching design, be it digital or otherwise, in a way which incorporates and integrates human experience, expression, thought and feelings. When you ask for someone's story you get it all, bundled together in a tight package. Inside that package are the reasons why they do things and those reasons hold the keys for designing tools.
The use of stories, story collection and story creation is nothing new to the development and design process in business. The typical business uses of story can be categorized into two types, which for the sake of argument are admittedly simplified. These two types are: Design/Marketing Stories and Engineering Stories. Design/Marketing stories are stories which are typically collected at the front end of the product design process and is the most common use of stories for designers. Design and/or marketing teams will seek out an appropriate cultural cross section of the population and either ask them a series of questions about their lives or follow them around for some period of time observing their behavior. The thinking is that if a set of people who fall into a particular marketing segment which the business wants to reach have a clear need for a particular device, service or method of using a device, then that need helps create a clear design direction and R & D mandate.
An engineering story is one which happens further along in the research and development process. Engineering stories typically have a technological foundation and are used to wrap illustrative context around technical ideas. When some sort of technological device or method has been created or is fairly far along in the technical thinking process, stories are created, perhaps as text, perhaps in storyboard form, perhaps as visionary videos, which demonstrate the ideas behind the new device or method. These stories are created to justify the project's further support within the company and are often shown outside of the company to impress customers, potential customers and the public at-large as an example of foresight and innovative thinking.
Each of these two types of stories has its place. Each can support well targeted design and development efforts and can be used to inspire designers and engineers alike. But what if the devices or technologies we are telling the stories about knew their own stories? What if the technology could tell its own story? What if the technology knew some of its owners stories?
This may sound like the ramblings of a lunatic. Why would I want my cell phone to have the ability to tell my story? Good question. Actually, this is not about cell phones. Instead think about a device that one would carry around that supports multiple functionalities: mobile phone, appointments, address book, text messenger, note taker/recorder, music player/recorder, health monitor, news & weather monitor, location finder, shopping assistant, wallet, home security monitor, dictionary, language translator, emergency beacon, etc. The list goes on and on. Such a device could be integral to our lives. As digital electronics get smaller, as power requirements diminish, and as power sources become smaller and more technically sophisticated (i.e. alcohol based fuel cells), more functionality will fit into smaller packages. While it is not likely that anyone would want their cell phone "knowing" about their life, it might be handy for a device which can arrange travel, negotiate prices for shopped items, and has some limited access to personal health information to be able to deduce: Because of the injury last month, a later flight would be better if a bulkhead seat can be guaranteed. The recently purchased lumbar support travel pillow will fit in a coach seat.
This is an example of a simple scale-up of current technology. One might even call it an intertwining of technologies and story. Trees that grow up from saplings right next to one another sometimes intertwine themselves around each other. Sometimes as they grow they merge into one tree. Other times they somehow avoid merging and remain two distinct but intertwined entities. This last example is more of an intertwining, the seams are still visible.
Going further, what if a device which could perform all those functions mentioned earlier and more, could store dates, events, names, activities and other notable details of one's life as they become available during its normal usage? And what if this device did not store these details in a database, but in a storybase? We could define a storybase as a storage system based on the work of one or multiple narrative theorists over the last century like Vladimir Propp , Edward Branigan or Seymour Chatman to name a few, and applied with the technical creativity of artificial intelligence scientists like Pattie Maes , Roger Schank or Selmer Bringsjord . In other words, what if a device we carried around learned pieces of our lives in such a way as to make it possible to resequence the pieces into many useful constructs as needed? Just as stories can be told and retold differently many times while maintaining their basic truth, so too can our lives be represented in many ways using the same set of details. If we can start to see the use of digital technology as telling a story of life — daily, weekly, monthly, yearly life — then we can start to design with the required sensitivities to go beyond handsets, keyboards and other older interfaces, which while being functional, lash us to a conservative way of thinking about interface and computing.
We know how to simply bear functional but cumbersome technology, though few of us can say we like it. We know how to simply live with inconvenient technology. Instead we could dance with technology; leading, following, gliding, creating together. Designer and Professor Hiroshi Ishii of the MIT Media Lab knows this well. <http://tangible.media.mit.edu/>
Prof. Ishii's Tangible Media Group researches and designs beautiful devices which blur the line between bits and atoms. Their goal is for the two to blend together seamlessly. One of their projects called musicBottles, is a good example of this and is based on a story about Ishii's mother. In Japan, Professor Ishii mother knew nothing about computers and would not think to use one. But what she did do, know a lot about and was very comfortable with was cooking. She knew how to go about her kitchen collecting bottles of spices and other ingredients and she knew something about how ingredients blend together to create a desired taste. With this in mind Prof. Ishii created glass bottles which hold information in audio form. To get at this information all one has to do is pull the stopper off the bottle. To hear the weather forecast, open the weather bottle to hear the sound of rain or the sound of a clear day represented by singing birds. The bottles were further adapted to hold music, thus giving them their name. Each of the three bottles in the set held the jazz performance of a single musical instrument. Opening and closing bottles added and subtracted instruments from the performance. Most recently Prof. Ishii's student Ali Mazalek further adapted the bottles to hold stories. The bottles held genies who would tell their own stories, but whose stories were affected by the presence or absence of the other genies.
Professor Ishii and his students seek the dance, the pas de deux of digital technology and natural tangible human oriented design. We are learning to do this at Motorola as well. It is a challenging dance to choreograph. It requires diverse skills and unique vision. But to get to that next step where we are not just lugging devices around with us, but utilizing and coordinating with digital partners, we need such vision. Choreographing this dance requires an ergonomics not just of body, but also of mind and spirit. It requires thinking about the activities of our lives as intimate stories, not as a set of functions we routinely perform, but as a structured way we view the world. If we can embrace our humanity as technologists and choreograph our technology as designers, then none of us will ever need to fear the advance of technology, but simply learn the dance.
References
Branigan, E. (1992). Narrative Comprehension and Film. New York, New York, Routledge.
Bringsjord, S. (1998). Chess Is Too Easy. MIT's Technology Review. 101: 23-28.
Brooks, K. (1999). Metalinear Cinematic Narrative: Theory, Process, and Tool. Media Laboratory. Cambridge, MIT: 218.
Chatman, S. (1978). Story and Discourse - Narrative Structure in Fiction and Film. Ithaca and London, Cornell University Press.
Flynn, M. J. (2000). Technology -> History -> Chronolog: Touch Tone Telephones - 1964, AT&T Labs, Bell Laboratories. 2001. <http://www.att.nl/technology/history/chronolog/64touch.html>
Livo, N. J. and S. A. Rietz (1986). Storytelling - Process & Practice. Littleton, Colorado, Libraries Unlimited, Inc.
Maes, P. (1992). Behavior-Based Artificial Intelligence. Second Animat Conference on Adaptive Behavior, Hawaii.
Propp, V. (1968). Morphology of the Folktale. Austin, Texas, University of Texas Press.
Schank, R., R. Bareiss, et al. (1992). Agents in the Story Archive, Northwestern University.