Animation, Embodiment, and Digital Media

I came across an intriguing traffic light in Spain some years ago. Pedestrians are supposed to cross the road when a little ‘walking’ green man lights up. The green man walks at first and then runs as the light is about to turn red. The traffic light shows a matchstick man animated in a walking cycle. Images produced by a matrix of light-emitting diodes (LEDs) are displayed successively at a certain speed to achieve an illusion of movement. The green man appears to walk, then run because of the varying playback speed, a technique widely employed in creating moving images.

As mentioned in Chapter 1, animated phenomena are technology-mediated presentations of digital objects reminiscent of our everyday experiences of life, including spontaneous reactions to stimuli (e.g., the fight-or-flight response in case of threats), recurrent behaviors (e.g., breathing), gradual changes (e.g., growth), and even quick shape-shifting (e.g., metamorphosis). For instance, an application icon bouncing restlessly in response to a user click demonstrates reactivity. Many screensaver programs on personal computers display animated graphical patterns, which seem to vary and repeat themselves indefinitely. A few video games use gradually extending graphical objects, such as a ‘growing’ pile of dirty dishes awaiting the players, to be an indicator counting down to the end of the game. A user-controlled character in a computer platform game like Super Mario Bros, might intermittently change its appearance. When an observer perceives similar signals in the digital environment, to what degree is one reminded of life or living things? Would one consider the restlessly bouncing application icon to be performing just a mechanical springboard effect? Or is it a funny character attending to a user’s needs? How do you feel when a graphical dog silhouette runs on the computer screen during file transfer? When an iPhone user drags an app icon across the screen and other icons make way for it, does it conjure up the idea of a person making his or her way through a crowd?

Some animated phenomena, such as reaction to stimuli or adaptation to changes, entail interactive dynamic presentation supported by a digital environment, where one can act upon certain digital objects or react to them ‘physically’ via touch-based or motion-sensitive interfaces. For example, users run their finger across a touchpad to move a pointer or to pan the camera across a space. A player of a flight-simulation game on a handheld device with built-in accelerometers can tilt the device to steer the virtual jet or hang glider, dodging obstacles along the way. The audience of the interactive installation Text Rain (Camille Utterback & Romy Achituv, 1999) may move their body parts like their hands or head to see their own projected image catching and lifting falling letters (please see Figure 8.2). In all the above instances, the audiences engage their bodies with the digital environments. In fact, they are not only engaged bodily, but also affectively. The connection between bodily engagement and affection can be explicated by arguments from phenomenology in philosophy. Through repeated engagement our bodies are able to build motor habits, to develop a sense of familiarity with a situation or an object (Russon, 2003, pp. 29–30). Habituation makes us feel at home in a ‘world’, whether it is physical or digital. It frees up our cognitive power to do more sophisticated activities involving self-expression or evaluation. We gain a sense of control and satisfaction this way. Hence, through motor action and habituation we are at ease in a digital environment, which feels familiar and intimate to us. This briefly explains our affective engagement with a particular digital design.

As mentioned in Chapter 3, through repeated bodily engagement our bodies become habituated, automatically and fluently coping with mundane situations without employing high-level cognitive processing. Our attention is then freed up for more sophisticated mental activities like interpretation, remembering, and imagination. Some might have this experience: after traveling between home and a new workplace a few times, we do not need to pay too much attention to finding the way. It is because our body ‘has absorbed’ the path via continuous and simultaneous perception (e.g., recognizing that café around the corner) and bodily sensation (e.g., feeling that unpaved gravel road). We might instead divert our energy to mentally rehearse an upcoming presentation. Typing is another common experience showing the value of habituation. Putting the ten fingers on a keyboard and practicing the keys repeatedly, most of us can acquire the skill to type almost automatically. A typist does not have to explicitly locate the position of a certain key, but he or she can move his or her fingers accordingly while reading her handwritten notes. Hence, one can think about where to eat after work while typing. Similar habitual skills can be developed when using some ‘easy-to-use’ digital applications or interfaces. For instance, the gesture unlock interface of many Android-based smartphones now allows users to unlock the phone by running fingers to draw a pattern preset by the phone owner.

Previous chapters articulated human experience of animated phenomena in terms of perception, action, interpretation, and imagination. Through bodily engagement in lively digital environments, people become aware of liveliness, build habits to interact with the medium transparently, feel at home in the environment, develop a sense of intimacy and pleasantness with it, make sense out of it, and are encouraged to imagine. They are also ready to explore further possibilities. With sophisticated habitual skills, they improvise with the environment, as all natural beings do in their habitat. Birds make use of found branches or leaves from surroundings to build their nests. A chameleon changes its skin colors to camouflage itself. A windsurfer knows how to make use of the wind and his or her weight to drive the board in a certain direction. As Daniel E. Koshland Jr writes in Science, improvisation with ‘environmental challenges’ is one of ‘the seven pillars of life’ (Koshland, 2002). Although Koshland’s improvisation refers to the kind of ‘slow’, long-term, and fundamental adaptation of a being in response to its habitat, which differs from the sense in which I use the term, that is an adaptive, ongoing co-performance between a user and a digital environment, both ideas entail unplanned, unexpected, and unfinished changes, that is, the contingent nature of life.

The corpus of digital media artifacts demonstrating animated phenomena includes works from an array of media types. One of these is the user interface. This kind of artifact, which can be a device or a program, is created to mediate between users and machines, mostly computer-based in today’s context, for effective communication and control. Since the demonstrations of Ivan Sutherland’s Sketchpad (1963) and Douglas Engelbart’s early computer mouse, user interfaces have become one of the indispensable constituents of computer-based systems, giving rise to a new area of study in the computing discipline: human-computer interaction (HCl). The initial objective is to enable users to complete certain tasks such as data processing, information access, content creation, or telecommunication. One of the design criteria is to make interfaces more ‘user-friendly’ or ‘usable’. Following the views of Donald Norman (Norman, 1988), Jakob Nielsen (Nielsen, 1993), and others, these terms generally describe a tool that is easy and rapid to learn, to understand, or to operate. Their ideas have led to the prevalent user-centered approach to interface design. The paradigm emphasizes usability, effectiveness, and efficiency.

People consume digital media in many ways, for various purposes. Some use it to organize resources, access information, or communicate with each other, as discussed in Chapter 6. Apart from these productivity-oriented objectives, others use digital media for enjoyment, amusement, or entertainment. They play video games or have fun with entertainment apps (a new breed of application category that has flourished on smartphones and tablet computers aiming at an engaging and enjoyable experience rather than competitive, goal-specific gaming pleasure). No matter how a piece of digital entertainment engages its players, whether in relaxing or exciting mode, animated phenomena are prominent and pervasive in their corresponding digital environments. This chapter looks at these emerging phenomena of liveliness.

After examining animated phenomena that are pervasive in such domains as user interfaces and digital entertainment, this chapter looks at the phenomenon of liveliness in the realm of art, which has long been a platform for humans to express, to interrogate, and to experiment. Contrasting with the previous types of digital artifacts in terms of creators’ intentions or consumers’ motivations, fine art practice still shares some features with them in animated phenomenal terms. Many works of art, such as mobile sculptures by Alexander Calder, mechanical automata by Jacques de Vaucanson, and others, revolve around the illusion of life, not to mention numerous arthouse animated movies. For an expanded illusion of life, works of digital art are particularly illustrative. By ‘digital art’ I mean the field of work whose discourse processes rely on the use of digital technology. This kind of work usually incorporates computer programs to produce variable and dynamic instances that show autonomous, reactive, transformative, and contingent behaviors. Hence, John Conway’s Game of Life (1970) as mentioned in Chapter 3 belongs to the category by definition. The tiny but distinctive program, as the name tells, simulates the mechanics of evolution on the cellular level and projects an image of life on a two-dimensional plane. The pixels in the grid turn on and off continuously and responsively in a seemingly autonomous fashion, resulting in a diverging pattern similar to the evolution of cells.

In the previous three chapters, we have conducted a close reading of an array of digital media artifacts which belong to the basic-level categories including user interface, video game, and digital art. These three commonly separate groups of artifacts generally contrast with each other in terms of both the creator’s motivation and the consumer’s expectation, but the works included in the corpus resonate in the pursuit of liveliness. As the analyses in previous chapters have shown, the properties of these artifacts with respect to the four qualitative variables exemplify the principles of technological liveliness. They are able to immerse their users in a virtual environment with both primary and secondary liveliness; they entail coupling and/or sustaining patterns of engagement that match users’ bodily motion and perception, resulting in a sense of intimacy; they anchor sensation for immediate conceptual blends yielding embodied concepts of virtual space, followed by elaboration of metaphorical blends; they generate output on the fly and in real time, supporting users to explore diverse possibilities in outcomes. In short, they all rely on animated phenomena to give human users immersive, familiar, embodied, evocative, and divergent kinds of experience. This study is not intended to be an exhaustive survey of all digital media artifacts, and inevitably there are many other salient examples that demonstrate comparable values in those four variables.