New Directions in Third Wave Human-Computer Interaction: Volume 1 - Technologies

Third wave HCI (Human Computer Interaction) proposes an innovative method for framing human computer interactions by putting emphasis on the terms and conditions of the interactive relationship prior to determinations concerning the human subject and its computational object. As promising as this “relational turn” appears to be, there are important theoretical, epistemological, and axiological challenges that remain and need to be addressed. This chapter takes up and investigates a number of these open questions regarding third wave HCI. It begins by briefly reconsidering the three waves or paradigms of HCI research and demonstrating how what appears last in the numbered sequence, the third wave, is actually older and “more original” than it initially appears to be. It then examines the opportunities and challenges of the phenomenological commitment that is operationalized in third wave HCI. And it concludes by identifying and outlining the consequences of this innovation for current and future research efforts.

Artificial intelligence (AI) has recently been highlighted as a design material in the HCI community. This acknowledgement is a call for interaction designers to consider intelligence as a resource for design. While this view is valid and well-grounded, it brings with it a need to better understand how intelligence as a design material can be used in formgiving practices. This chapter seeks to address this need by suggesting a new approach that integrates AI in the designer’s toolkit. This approach considers intelligence as being part of, and expressed through, an object's character, hereby integrating artificial intelligence into a product's form. We describe and discuss this approach by presenting and reflecting on our experiences in a design course where students were asked to give form to intelligent everyday objects in three iterative design cycles. We discuss the implications of our approach and findings within the frame of third wave HCI.

Decisions in designing artificial social interactants to reproduce culturally-specific forms of human sociality evince a range of conceptions of the norms and cognitive processes involved in the human social interactions themselves. Regarding the use of machine learning (ML) in such systems, decisions whether or not to use this approach implicitly presents questions on the nature of the interpersonal adaptation that takes place and indicate a range of conceptions of the values which structure these interactions. In the design of virtual performers of musical free improvisation, several designers assume that the experience of equal partnership between improvisers can only be afforded through deployment of ML in such systems. By contrast, tests of agents not based in ML reveal that human beings experience illusions of “adaptation” in interactions with systems which lack any adaptive capacity. Such results suggest that HCI research with artificial social interactants may be used to raise new questions about the nature of human interaction and interpersonal adaptation in the formation of relationships over time.

In this paper, we examine digital creativity as a collective activity performed through socio-technological networks of agency. We introduce metacreation—the automation of creative tasks with machines—as a domain that is usefully examined from a 3rd wave HCI approach. We discuss four general human-computer interaction activities that commonly appear in metacreation: (1) metagenerating form; (2) searching/finding; (3) helping machines learn; and (4) evaluation/iteration. These are not necessarily specific to metacreation, but nevertheless point to particular design considerations in a metacreative context. Four creative interaction design themes are considered in their relation to metacreation: direct manipulation and real-time control; supporting playful interaction and divergent goals; the programmatic design of behaviours, and; managing distributed creativity. We then identify three paradigms of interaction design for metacreation: operation-based interaction, involving the direct manipulation of generative algorithms; request-based interaction, involving the submission of requests to a system that returns results; and ambient interaction, that involves the operation of autonomous metacreative processes in the background. Our discussion of these suggests possible trends for design: an increasingly complex and modular future for networked human-machine digital creativity; an increasing role for request-based metacreative systems where users specify, rather than construct, outcomes; the increasing role of metacreation in ‘prosumer’ content creation; and, consequently, the reduction of labour involved in creating media. The chapter makes clear, we hope, that metacreative practices present unique challenges and opportunities for interaction design.

Three-dimensional (3D) printing is a novel digital technology that has gathered momentum and public recognition over the past few years. In this chapter, I examine the sociocultural and political dimensions of 3D printing technologies. I begin with an overview of the third wave human-computer interaction (HCI) approach to digital technologies and contributions made by social and cultural theory that are relevant to understanding the broader contexts of 3D printing technologies and how they are represented, discussed and experienced. This is followed by a discussion of the sociotechnical imaginaries that animate speculations about their possibilities and the agential capacities identified by research investigating the lived experiences of those who have tried using these technologies. The chapter ends with some brief reflections on future research directions.

This chapter examines how 3D printing has been framed as a liberatory technology that confers agency to users on the one hand, and an automated system that de-centers the user on the other. These entangled visions, we argue, can be traced to values that are threaded into 3D printing’s DNA. By historically situating the social context of 3D printing, tracing its roots to the CAD/CAM revolution of the 1950s and 1960s, we denaturalize assumptions about the technology’s users, its modes of interaction, and its societal impact, offering third wave HCI new insights for broadening how it considers context and values.

Maker culture, from soldering sensors on an Arduino to 3D printing a prosthetic limb, has established that hobbyist computing is intimately rooted in the physical world. In education, ‘physical computing’ courses have captured this interest, introducing code through its physical interactions. Interpreted more broadly, physical computing sits at the nexus of a number of strands within HCI including tangible interaction, ubiquitous computing, and spatial/mobile systems. Ideas of embodiment and an experiential approach to design are natural frameworks within which to view physical computing and so it is almost tautologically third wave. However, the hidden action of computation in certain kinds of sensor-rich ubicomp and the AI turn in computing calls any simple identification into question. Product design appears to encounter the ‘waves’ in a different order; as its artefacts become more digital, it is having to consider the agency of computing and adopt more analytic approaches in research and design. Physical computing forces us to regard the ‘waves’ less as a teleological progression, and more as complementary approaches addressing different facets of human experience with physically embodied digital technology. Furthermore, it suggests there are new challenges ahead as we seek to find research and design paradigms that use physical objects as part of rich collaborations with active computation.

This chapter draws out three threads from contemporary HCI, design research, and design practice to consider the Internet of Things from a third wave HCI perspective. These stances towards the IoT emphasize an agentic thing in itself as component of a broader system; the network that a system resides inside of as members of a broader social or cultural context; and the role of a system to articulate and maintain sites of contestation around public issues. These perspectives both build on and react to a set of categories that describe how the existing Internet of Things has been approached from a second-wave HCI orientation, as primarily providing opportunities for command and control, making technologies more efficient, and consuming products and services. This third-wave perspective hopes to broaden the conversation around the potentials for networked technologies that operate inside of rich cultural and social context.

In this chapter, we examine inclusive design of technology for people with disabilities in the context of the Third Wave HCI. As technology becomes more integrated into our lives beyond work, there are increasing opportunities for people with disabilities to have new experiences through technology. However, we argue design knowledge and practice in inclusive design has lagged behind the broader HCI field in two different, but related, ways. First, when new technology is released, an implementation lag in designs for access and enablement invariably lead to late adoption of technology for people with disabilities. Secondly, this implementation lag has resulted in a conceptual lag, where to solve these problems the research field remains grounded in HCI methodologies from First and Second Waves. This results in a reliance in checklist style engineering approaches that are unable to properly support user experience design. We explore these ideas in the two examples of the web and digital games, and argue that while we must not supplant previous approaches, we need to decouple the implementation lag from the conceptual lag to change inclusive design research and practice. We argue that we must not only plan for accessibility, but instead adopt pluralistic approaches that recognise the diversity of lived experiences of people with disabilities, and use them to design options for people to customise their own inclusive experiences.

The Autism Too Much Information (TMI) Virtual Reality Experience is a virtual reality (VR) application produced by The National Autistic Society (NAS) as part of an awareness campaign. The design of the application creates a short narrative simulation from a first-person perspective, which conveys aspects of what it may be like for a child on the autistic spectrum to experience a stressful situation precipitated by environments with ‘too much information’. The application is part of a recent trend in VR and 360-degrees video, to create simulations of subjective experience, as a means to generate empathy. Yet the success of such tools depends significantly on how well sound and graphics can be used to communicate such experiences in a meaningful way. In this article, we provide a case study of the Autism TMI Virtual Reality Experience, as a means to unpack design issues for these simulations. Through an expert analysis and pilot study of user experience, we propose three distinct forms of subjective first-person simulation that may be produced in virtual reality. We argue that the Autism TMI Virtual Reality Experience exemplifies the third of these: ‘deep subjectivity’, which may lead to an improved sense of empathy by representing various aspects of multimodal perception and emotion. However, our study also suggests that VR may offer limited benefits over 360-video for generating a sense of empathy.

This chapter investigates the second and third wave HCI design processes for an auditory display design and proposes a multifaceted framework. The purpose of using such a framework in a participatory context is to provide the possibility to create shared design knowledge in sonification and building new sonification designs on top of the prior work. From the early stages of the project involving the domain scientists in the process seemed to be an obvious choice. The process works in the sense that we gathered a diverse set of data analysis problems, solutions, and methods that work for data scientists within a sonification framework.

Colocative display is the technique of sonically articulating the screen area of audiovisual media by dynamically placing and animating the associated sounds in spatial localization to their visual cues. Two such systems have been designed to date, the author’s Pixelphonics system and the Allosphere facility at University of California at Santa Barbara. While both technologies are prototypes, and thus lacking in a rich historical tradition that might inform what film theorist David Bordwell has called historical poetics, Bordwell’s concept of analytical and theoretical poetics can be fruitfully brought to bear to elucidate the general principles for making colocative audiovisual media and applications. The poetics, or ‘principles of making’ colocative media are situated within a discussion of the empirical dimensions of auditory localization, cognition and attentional resources, general audiovisual practices, acoustics and phenomenology. A new concept, that of the soundscene, is introduced to hone in on the particular design affordances of colocative displays. This inquiry blends second and third wave HCI approaches in its hybridization of humanist media theories with cognitivist-attentional usability perspectives.

In the field of language technology, researchers are starting to pay more attention to various interactional aspects of language – a development prompted by a confluence of factors, and one which applies equally to the processing of written and spoken language. Notably, the so-called ‘phatic’ aspects of linguistic communication are coming into focus in this work, where linguistic interaction is increasingly recognized as being fundamentally situated. This development resonates well with the concerns of third wave HCI, which involves a shift in focus from stating the requirements on HCI design primarily in terms of “context-free” information flow, to a view where it is recognized that HCI – just like interaction among humans – is indissolubly embedded in complex, shifting contexts. These – together with the different backgrounds and intentions of interaction participants – shape the interaction in ways which are not readily understandable in terms of rational information exchange, but which are nevertheless central aspects of the interaction, and which therefore must be taken into account in HCI design, including its linguistic aspects, forming the focus of this chapter.

Humanity is at a special time in its relationship with technology where there is an increasing likelihood of artificially replicating characteristics which we thought were in the realm of the distinctly human. Artificial Intelligence and Robotics are making the news increasingly often and replication of body parts is also making progress. This chapter looks at senses which although not exclusively human have a powerful potential to support other higher functions and aspects of human life. Technology has been developing nature-inspired artifacts which resemble somehow their human counterparts with specific practical applications. So far these explorations have been mostly isolated. It is only a matter of time, however, until these become physically and logically connected into a cooperative fashion – in fact Robots typically use them although not always in their full capacity. Such developments will provide machines with interface capacities of a higher order, bringing new powerful tools to solve new problems, whilst raising unexpected scenarios and challenges for our societies. Regardless of whether we want it or not, it seems impossible to stop technological progress in this direction. We assume this development is here to stay. Thus we look at the artificial and human synergies, how interaction with machines is influenced by sense-like interface capabilities – namely “sensorial computing”.