Originally published in Veredas, Ano 3, No. 32, 1998, Rio de Janeiro, Brazil, pp. 12-15, and Blimp - Film Magazine, N. 40, Graz, Austria, 1999, pp. 49-54.

Beyond the Screen: New Directions in Interactive Art

Eduardo Kac

From the room size computers of the 1940s to the desktop, laptop, palmtop, and wearable computers of the present, human interaction with this powerful and pervasive calculating machine has changed. When in the 1960s computers started to become capable of producing and manipulating images, computer graphics became a prominent research topic among engineers. Likewise, computers started to attract the attention of experimental visual artists all over the world.

Surprisingly, at times the work produced by engineers achieved strong visual and cultural impact. This is exemplified by the Japanese team called Computer Technique Group, from Tokyo. In the late sixties they produced classics such as "Running Cola is Africa," a black-and-white graphic morphing sequence showing the transformation of a runner into a Coca-Cola bottle which then morphed into the map of Africa [1].

Working against the background of Pop, conceptualism, and kinetics in the 1960s, many innovative artists abandoned the tactile appeal of the analog realm and ventured into the unknown domain of computer graphics. Classic examples include the work of the Americans John Whitney [2] and Charles Csuri [3], the Brazilian Waldemar Cordeiro [4], the Hungarian Vera Molnar [5], and the German Manfred Mohr [6]. Many artists working with computers at the time explored algorithms that generated multiple forms of abstract or Constructivist art. Others created figurative images that were charged poetically through specific graphic procedures (e.g.,warping, morphing, zooming). Cordeiro's work is particularly distinct in this context because the artist, living under the worst phase of the Brazilian military dictatorship, produced computer images that were rich in personal, emotional, or subtle political content.

Computer graphics in art continued to flourish in the 1970s and 1980s, as new algorithms were developed and digital images started to acquire color, rich shading, and photographic qualities [7]. Computers were gradually becaming incorporated in interactive art installations, as exemplified by historical exhibitions such as "Software," curated by Jack Burnham in 1970 for the Jewish Museum in New York [8]. Computer graphics were prominent in videos and films in the 1980s, and even television commercials started to feature digital animation regularly. The launch of the Macintosh computer in 1984 and the graphic software industry that followed it made computer imaging accessible to a larger number of artists. Consequently, the creation of still images presented new challenges to a younger generation of artists, who enjoyed unprecedented creative freedom. As the new frontier of computer graphics became a stable industry and an established artistic practice, experimental artists in the 1990s started to push the digital image into new areas of imagination and experience. The works discussed below reveal some of the most fascinating works created today in the areas of virtual reality, interactive performance, avatars, telepresence, and artificial life.

Inside the image

Since the late 1980s the term virtual reality has been used and abused in scholarly journals and popular magazines alike, often taken to mean different things for different purposes. When first developed by Ivan Sutherland in the late 1960s, the technology of virtual reality was intended to enable scientific visualization of three-dimensional data in real time through the use of head-mounted stereoscopic electronic displays. Because the technology has grown less expensive over the last ten years, it has catapulted from research labs into myriad applications, such as education, military training, medicine, and gaming. True to its origins, the concept refers to a visual space that can be seen as such by the viewer and in which this viewer can navigate in three dimensions in real time. If the viewer perceives the space through a stereoscopic device, he or she has the sensation of being immersed in the space. For the viewer to have a seamless experience, the computer must be powerful enough to calculate every subtle change in point of view in real time.

In 1995 the Canadian artist Char Davies, working with designers and programmers, created "Osmose" [9], a virtual-reality immersive artwork that invited viewers to move through synthetic infinite worlds. In this work Davies, who lives in Montreal, presented a unique interface to what she calls the ëimmersantí (the person immersed in the virtual world). In the form of a vest, this interface provided real-time motion tracking based on breathing and balance. This meant that viewers could inhale to rise and exhale to descend and could move forward or backward in the virtual space by leaning forward or backward in the physical world. Viewers navigated a complex world made of natural forms, such as trees, and synthetic elements, such as three-dimensional Cartesian wireframe grids filled with diaphanous substances.

"The public installation of Osmose," explained Davies, "included large-scale stereoscopic video and audio projection of imagery and interactive sound transmitted in real-time from the point-of-view of the "immersant". This projection enabled an audience, wearing polarizing glasses, to witness each immersive journey as it unfolded. Although immersion took place in a private area, a translucent screen equal in size to the video screen enabled the audience to observe the body gestures of the immersant as a poetic shadow-silhouette."

Her most recent work, entitled "Éphémère" (ephemeral in French), was also created with a team of designers and programmers and premiered in 1998 at the National Gallery of Canada in Ottawa. Whereas in "Osmose" the immersant could move through a forested glade populated by static objects, in "Éphémère" every object is in a state of flux. Organized in three levels, this new work also makes use of organic and natural metaphors, except that this time an analogy is suggested between nature and the human body. As in "Osmose," "…Éphémère" uses the breathing and balance vest interface to propel the viewer in space, makes creative use of three-dimension sound, and can only be fully experienced with a virtual- reality headset. As viewers try to make the most of the allotted 15-minute time slots, their sense of time might get warped. The digital image becomes a navigational space, and they might get lost exploring new worlds.

Graphics As Body Interface

With the intention of presenting a more dystopian view of virtual reality, the Barcelona artist Marcel.li Antunez Roca created an interactive performance that is at once delirious and frightening. Entitled "Epizoo" [11], it was first presented in Mexico in 1994 and since then has been seen in more than 55 cities. The work was seen in Rio de Janeiro in the context of the theater festival Rio Cena Contempor‚nea, last October. I saw the piece in a small theater in Helsinki in 1996, sitting on the stage in a circle of approximately 50 people. As the audience waited for the artist's entrance, I started to notice the apparatus up front: an exoskeleton of sorts, a small camera attached to a glove, speakers, and a large projection screen raised above the performer's small designated area of action. I also noticed some computer equipment, on one side of the room.

Marcel.li solemnly entered the stage wearing a robe. He positioned himself up front, at the center of the designated area, and disrobed. With the help of an assistant he donned the pneumatic exoskeleton, creating on-stage the image of a cyborg, mixture of man and machine. This apparatus pressed metal components against several parts of the artist's body, such as his chest, ears, mouth, nose, and buttocks. At the top of his head, a large Bunsen burner suggested that a flame was also going to be part of the show. The large amount of plastic tubing (necessary for the functioning of the pneumatic exoskeleton) that surrounded the artist suggested that his movements would be hindered.

As soon as the music started, one of Marcel.li's assistants sat at the computer and started to click on images, which danced about on the large screen above the artist's head. When the assistant clicked on the images, we noticed that the hinged metal parts of the exoskeleton also started to move, and the clicking sounds were very noticeable. The metal components moved the artist's selected body parts in funny, if not scary, ways. As the person at the computer activated the artist's body, moving its parts in a peculiar choreography, it also became clear that the limited mobility of the artist was also significant in evoking the dangers of technologies of control. His body was besieged.

The digital images seen on the screen, a mixture of stills and animations often including the artist's own likeness, functioned perfectly as an interface to his body. At once humorous in their treatment and terrifying in their content, they portrayed scenes of torture and violence, transforming body parts in combinatory and disposable elements. The artist stood under the screen, and turned around regularly to reveal all possible viewing angles. With his glovecam, he added a few additional points of view by raising and swinging his hand. Real-time editing enabled the audience to see a combination of the digital interface and the live video.

As the artist's body was manipulated through the interface, the audience saw his mouth and nose being stretched open, his ears flopped forward and backward, his chest and buttocks being raised up and down. Midway through the performance, the audience was invited to seize control of the multimedia interface and assume control of Marcel.li's body. Many did, and the spectacle of cold and detached manipulation of hot and sweaty human flesh through a clean and dry digital interface continued. The whole performance lasted for about 30 minutes. It culminated with a large flame shooting up from the artist's head and helped to form a conclusive critical view of the man-machine interface.

Avatars and Databases

While the body in question in "Epizoo" is made up of on flesh and bones, the virtual bodies in "Bodies© INCorporated" [12] are built of pixels, wireframes, and textures. "Bodies© INCorporated" is a web-based work-in-progress by the California artist Victoria Vesna, under development in collaboration with artists, musicians, companies, and programmers. The basic premise of the site, which first went online in 1996, is that webviewers become active in a mock corporate structure, and as they acquire shares, they can order digital bodies of their choice. The project employs VRML (Virtual Reality Markup Language) to create a three-dimensional representation of the new body in a database, which can be seen by other participants. VRML transforms the web from a two-dimensional space similar to print into a three-dimensional navigable environment. While avatars can exist in a two-dimensional space, as evidenced by the popular Palace, three-dimensionality quite literally opens up new worlds for the avatar experience. While many participants emailed Vesna and asked her to create an area with live avatar-based chat rooms, to enable these bodies to be displayed in an active social environment, the author explained that this is not her intention. She is exploring what she refers to as "database aesthetics", enabling Web users to create, access and modify a complex database that critiques the conversion of the Internet from a social space to a marketplace.

Many denizens of chat rooms and other social areas of the Net often assume multiple identities in their exchanges with other anonymous participants, concealing or forging distinct traits such as gender, age, and race. Exploring the nuances of interaction on the Net, Victoria sees "Bodies© INCorporated" as an investigation into social psychology and group dynamics in a corporate context. After announcing the creation of the site and the availability of digital bodies to be created on demand, Victoria was overwhelmed with responses. Orders came in for males, females, and hermaphrodites, with sexual preferences ranging from heterosexual to transsexual, from homosexual to bi- and asexual. Most requests were for bodies that represented alter egos, followed by desired sexual partners and, in smaller numbers, significant others. To shift the focus from an exclusively sexual context, textures were added to the bodies to add symbolic value to the otherwise smooth surface of their digital skin. While the majority of requests were for bodies without any texture, many selected from a menu of texture maps that included black rubber, blue plastic, bronze, chocolate, clay, clouds, concrete, glass, lava, pumice, and water.

"Initially, the participant is invited to construct a virtual body out of pre-defined body-parts, textures, and sounds, and gain membership to the larger body-owner community," Victoria explains. "The main elements of the on-line site are three constructed environments (subsidiaries of Bodies© INCorporated), within which different sets of activities occur: LIMBO©INCorporated, a gray, rather nondescript zone, where information about inert bodies that have been put on hold -- bodies whose owners have abandoned or neglected them -- is accessed; NECROPOLIS© INCorporated, a richly textured, baroque atmosphere, where owners can either look at or choose how they wish their bodies to die; and SHOWPLACE!!!© INCorporated, where members can participate in discussion forums, view star/featured bodies of the week, bet in the deadpools, and enter "dead" or "alive" chat sessions."

The creation of digital bodies that can be used to represent an individual may sound to some readers like the exclusive domain of science fiction, but in fact it is a real, growing business. Good examples are companies like Viewpoint Data Labs, which sells three-dimension body models and which sponsors Victoria's project, and Cyberware, which pioneered the market for three-dimensional detailed scans of people and objects. Cyberware's technology, including a whole-body scanner, was used to make popular films like Star Trek IV, The Abyss, Robocop II, Nightmare on Elm Street, Terminator II, The Doors, Neuromancer, Batman II, and Jurassic Park. With the realization of Toy Story in 1995, the first completely computer-animated feature film in the history of motion pictures, it becomes conceivable that a young actor or actress whose body is scanned today could star in a movie long after his or her death. Victoria Vesna knows that a culture obsessed with fitness and shapely bodies finds an acute reflection of itself in the detached and calculated digital incorporations her site provides. Viewers become emotionally attached to their avatars and projected idealized "significant others", giving rise to new questions of identity, information storage and retrieval about physical and virtual bodies, and social interaction in cyberspace.

Servers In The Shade

Avatars form dynamic representations of discrete entities in the network. However, it is also possible to use the Internet and other telematic networks to create a direct link with a real physical space. The California artist and scientist Ken Goldberg is one of the few who have been consistently exploring the unique aesthetic possibilities of telepresence art (the combination of telecommunications and remote action). Some of his previous web-based telepresence works include the "Mercury Project" (1994) and the "TeleGarden" (1995). The first presented viewers with objects buried in the sand. These objects were introduced as archaeologically significant within a fictitious narrative context. The viewer could control an industrial robotic arm to activate an air jet and reveal the buried artifacts. The viewer could also retrieve updated stills to see the results of his action. The second was a small garden with an industrial robotic arm at the center. The arm was controlled via the web and allowed remote participants to plant seeds and water plants. Viewers could also see live pictures of the garden.

In both cases, the digital image was an important component of the work and performed a specific function: it created a visual bridge between viewers on the web and the actual physical space where the apparatuses were located. With a recent piece currently online, in addition to preserving the bridge-like status of the digital image, Goldberg gave it new role. In the "ShadowServer" (1997), rather than observe an image that represents an action, the web participant is given the opportunity to create the image him or herself [13]. In other words, the gap between action and image is decreased, because the action is itself the remote creation of the picture.

Goldberg describes his work: "The apparatus is housed in a lightproof box that contains physical objects, some of which move of their own accord within the apparatus. Viewers can interact with these objects via buttons. Viewers can select any combination of five buttons and then [click on the button] ëCast a Shadowí, which activates a combination of lighting devices and returns a digital snapshot of the resulting shadow. Each combinations of buttons produce different lighting conditions. Certain random combinations will provide clues which lead to a mysterious Sixth button. The Sixth button illuminates hidden secrets in an alcove of the apparatus."

The images created by the viewer through the ShadowServer interface are invariably evocative of Moholy-Nagy's beautiful and mysterious photograms, and even more so of Nathan Lerner's Light Box photograms. A member of the Bauhaus, the historic German art school that profoundly influenced art and design in the twentieth century, Moholy-Nagy coined the term photogram to designate his cameraless photographs produced directly through the contact of objects with photographic paper. Between his first experiments with the photogram in 1922 and his premature death in 1946, the Constructivist master produced approximately 500 photograms that effectively demonstrated his belief that light was an art medium in its own right. Seeking refuge from the rise of Fascism in Europe in the 1930s, Moholy-Nagy immigrated to the United States to found the New Bauhaus in Chicago. Among his students was the now deceased well-known American photographer Nathan Lerner, who in 1938 invented the Light Box [14]. This was a perforated box with lights positioned outside and inside which objects were suspended to create exquisite photograms. Lerner wrote at the time: "I felt that if I could create a virtual world of darkness, which I could then develop into a disciplined world of light, I would be approaching the solution of the problem of controlled selection [of light]." As the light-box experiment acquires a remote and automatic nature in Goldberg's web work, we perceive a distinct historical resonance between the light modulation adventure of the avant-garde photographers and the democratizing gesture of network art. Moholy-Nagy 's pedagogy was based on trying to bring out what he believed to be the creativity inherent in everyone. As anonymous viewers create countless digital photograms on the web, the ShadowServer is an instance in which this vision comes full circle.

Living Pictures

The desire to work between real and digital realms is not exclusive to telepresence art. In their interactive installation entitled "A-Volve," the Austrian Christa Sommerer and the French Laurent Mignonneau created a unique metaphor of artificial life by merging tangible and intangible elements [15]. The work premiered at the international electronic arts festival Ars Electronica, in Linz, Austria, in 1994. In "A-Volve" the European duo, currently residing in Japan, allowed digital images generated in real time by anonymous viewers to acquire life-like behaviors and to interact among themselves in a 15-cm-deep water-filled glass pool measuring 180 x 135 cm. Viewers accustomed to traditional computer animation discovered that these animated organisms were unpredictable in their motions and acquired idiosyncratic behavioral patterns in this real-time interactive environment.

As viewers approached the installation, in addition to the water pool they saw a pedestal with an embedded touch-screen monitor. Asked to draw freely on this monitor with their fingers, viewers improvised and sketched both the profile and the top view of an artificial organism. Moments later they saw this creature emerge from the depths of the water pool and start to swim with its own unique behavior and motion pattern. The creature also interacted with other artificial organisms already in the pool in complex ways, following survival rules that included mating and predatory patterns. Viewers could look into the pool and observe the creatures "in the water" because a projection screen formed the floor of the water pool, and the real-time images were projected upward from a video projector embedded in the base of the water pool. The sensation was further enhanced by the fact that the digital environment in which these creatures dwelled was created with single-point perspective and a dark, fuzzy bottom, which gave the visual impression of a much deeper lagoon.

The title of the piece clearly evokes the idea of artificial evolution, because instead of the expected letter E in evolve we find the A that also prefixes the emerging scientific discipline of ALife, or artifical life. One of the key ideas of this new scientific field is that what we know about life is, of course, based on life on Earth and that life could conceivably take on countless other forms ó many of which we may not be ready to recognize because of our biased terrestrial expectations. What we know is carbon-based life, and even so we are constantly surprised by new discoveries that seem to shatter the comfortable assumptions that so far have served as pillars of the biological sciences. A good example is the recent discovery of thriving colonies of microorganisms living in inhospitable environments, such as inside rocks and the bottom of the sea, where temperatures and toxicity are incredibly high. To explore alternatives to the concept of life as we know it, scientists create algorithms that emulate basic life patterns, such as birth, growth, reproduction, and death, and allow them to interact with one another. This often results in unpredictable emerging behaviors that even more closely resemble complex interactions typical of living carbon-based creatures. Surprises may occur, and thus further the inquiry into an artificial biology.

"A-Volve" brings this concept out of the removed domain of scientific laboratories and gives it a somewhat tangible expression. The piece allows viewers to become participants when they assume responsibility for the creation of these organisms and when they interact with them by moving their hands in the water. If viewers "grab" one of the creatures, they can bring it closer to another one and make them mate. This results in an offspring that soon afterward can be seen wiggling in the water. This situation allowed viewers to interfere even more with the evolutionary path of this digital microcosm and to discover how tenuous the boundaries between real and artificial can be.


The works examined above reveal new directions for interactive art. Undermining the role of the individual image and giving greater emphasis to the dynamic quality of the experience, these pieces challenge the notion that the artwork must be centered on the "author" and that it must be materially stable, as is common in painting and sculpture. Essentially immaterial, with varying degrees of emotional, intellectual, and technical complexity, these electronic art works are seen regularly, but not as often in the same spaces and by the same audiences that form the art market. These and other artists who are developing a new art based on contemporary media are also finding alternative venues to present their work. In some circumstances, like in the case of Victoria Vesna and Ken Goldberg, the Internet is the "natural" digital space to show the work, which can simultaneously reach multiple audiences worldwide. Char Davies and Sommerer & Mignoneu often show their work in museums and Marcel.li Antunez Roca has shown his performance in more than 50 cities in 17 countries. Electronic art is seen regularly in many different venues, in several countries, and in multiple forms. The Guggenheim Museum, in New York, announced in 1998 that it would spend 1 million dollars commissioning and buying digital art [16]. Institutions such as ZKM, in Kahlsrue, Germany, the Ars Eletronica Center, in Linz, Austria, and the InterCommunications Center, in Tokyo, are primarily dedicated to produce, promote, and preserve media art. Other institutions also invest regularly in electronic art exhibitions, conferences, and documentation, such as the Itaú Cultural Center, in São Paulo, Brazil. This growing international interest is a clear indication that electronic art has a lot to tell us about the contemporary experience, about new possibilities for art in a digital society, and about ourselves.


1 - Reichardt, Jasia. Cybernetic Serendipity: The Computer and the arts ( New York: Praeger, 1968), pp. 75-77.

2 - Whitney, John, Digital Harmony: On the Complementarity of Music and Visual Art. Peterborough, NH: McGraw-Hill, 1980.

3- Csuri, Charles. "Computer Graphics and Art," in Tutorial: Computer Graphics, Beatty, J. and Booth, K. (eds), (Silver Spring, MD: IEEE, 1982), pp.558-570. Originally published in 1974 in Proceedings of the IEEE.

4 - Cordeiro, Waldemar. Arteônica (São Paulo: Editora das Américas, 1972); Fabris, Annateresa. "Waldemar Cordeiro: Computer Art Pioneer", Leonardo Volume 30, No. 1 (1997), pp. 27-31.

5 - Molnar, Vera. "Towards Aesthetic Guidelines for Painting With the Aid of a Computer", in Leonardo, Vol. 8, N. 3 (1975), p. 185; "My Mother's Letters : Simulation by Computer', Leonardo, Vol. 28, N. 3 (1995), pp 167-170.

6 - Leavitt, Ruth, Artist and Computer (New York: Harmony, 1976), pp. 92-96.

7 - See Franke, Herbert. Computer Graphics, Computer Art (London: Phaidon, 1971) and Annabel Jankel and Rocky Morton, Creative Computer Graphics (Cambridge: Cambridge Univ. Press, 1984).

8 - Burnham, Judith B. (Coordinator), SOFTWARE - Information Technology: Its New Meaning for Art (New York: Jewish Museum, 1970). See Also" Edward A. Shanken, "The House That Jack Built: Jack Burnham's Concept of "Software" as a Metaphor for Art", in Roy Ascott, ed., Reframing Consciousness: Art and Consciousness in the Post-Biological Era -- Proceedings of the Second International CAiiA Research Conference (Exeter: Intellect, 1999). Forthcoming.

9 - See: Davies, Char. "Osmose: Notes on Being in Immersive Virtual Space". Sixth International Symposium on Electronic Arts Conference Proceedings, Montreal: ISEA’95, 1995, pp. 51-56; Lunenfeld, Peter . "Char Davies." Art + Text, no. 53 (1996), pp. 82-83; Goldberg, Ken. "Virtual Reality in the Age of Telepresence", Convergence, 4 (1), 1998, 33-37.

10 - See Gagnon, Jean . "Dionysus and Reverie: Immersion in Char Davies' Environments." Char Davies: Éphémère, exhibition catalog, (Ottawa: National Gallery of Canada, 1998), n/pp; Mirapaul, Mathew. "An Intense Dose of Virtual Reality." The New York Times Online (7/9/1998); Davies, Char. "Éphémère,: Landscape. Earth, Body and Time in Immersive Virtual Space", in Roy Ascott, ed., Reframing Consciousness: Art and Consciousness in the Post-Biological Era -- Proceedings of the Second International CAiiA Research Conference (Exeter: Intellect, 1999). Forthcoming.

11 - Roca, Marcel.li Antúnez. "Epizoo", in Leonardo, Vol. 29, No. 1, 1996, p. 11; Macrì, Teresa., Il Corpo Postorganico; Sconfinamento della Performance (Genova: Costa & Nolan, 1996), pp. 32-52; Giannetti, Claudia (ed.). Marcel.li Antúnez Roca: Performances, objetos y dibujos (Barcelona: MECAD, 1998).

12 - Vesna, Victoria. "Bodies© INCorporated", in Ippolito, Jean et alli (eds.), Siggraph '96 Visual Proceedings (New York: ACM, 1996), p. 16; Vesna, V. "Under Reconstruction: Architectures of Bodies INCorporated", in Novakov, Anna (ed.), Veiled Histories: The Body, Place and Public Art (New York: Critical Press, 1998), pp. 87-117. See also: http://arts.ucsb.edu/bodiesinc/

13 - Bureaud, Annick. "Review of Shadowserver", Leonardo Electronic Almanac, Volume 5, No. 11, November 1997; October 30, 1997; Mirapaul, Matthew. "Made in the Shade", New York Times Online (October 30, 1997). See also: http://taylor.ieor.berkeley.edu/shadowserver/index.html

14 - Moholy-Nagy,Laszlo. Vision in Motion (Chicago: Paul Theobald, 1947), p. 200.

15 - Sommerer Christa and Mignonneau, Laurent. "Art as a Living System", in Sommerer C. and Mignonneau, L. (Eds.) Art @ Science (Vienna/NewYork: Springer, 1998), pp. 148-161.

16 - Mirapaul, Matthew. "Guggenheim to Add Digital Art to Its Collection", New York Times Online (June 25, 1998).

Back to Kac Web