Advances in Longitudinal HCI Research

Longitudinal studies in HCI research have the potential to increase our understanding of how human–technology interactions evolve over time. Potentially, longitudinal studies eliminate learning or novelty effects by considering change through repeated measurements of interaction and use. However, there seems to exist no agreement of how longitudinal HCI study designs are characterized. We conducted an analysis of 106 HCI papers published at the CHI conference from 1982 to 2019 where longitudinal studies were explicitly reported. We analysed these papers using classical longitudinal study metrics, e.g. duration, metrics, methods, change or stability. We illustrate that longitudinal studies in HCI research are highly diverse in terms of duration lasting from few days to several years and different metrics are applied. It appears that the paper contribution type highly influences study design, while only a little more than half of the papers discuss or illustrate change/stability during their studies. We further underline considerations of durations versus saturation, identifying points of measurements and matching contribution types with research questions. Finally, we urge researchers to extend implications presented on perceiving duration as a singular attribute, as well as longitudinal systematic approaches to ‘in situ’ studies and ethnography in HCI.

Within the information systems research, there is a long tradition for longitudinal research, and it plays a significant role in the research literature. In this chapter, we will overview the reasons provided by researchers for when a longitudinal study is appropriate. Longitudinal studies have a particular focus on time and change. Time and change address a concern for understanding the details of human actors’ behaviour and perceptions both as individuals and in social arrangements. This addresses ‘how’ to conduct a longitudinal study and why a deeper level of understanding is beneficial. In this chapter, we will map longitudinal research in information systems from the last two decades. This mapping shows critical distinctions that can be used in designing longitudinal research. The most important difference in longitudinal studies is between variance studies and process studies. Variance studies set the research design before the data collection, treat the change over time as a black box, favour a positivist stance and ask what-questions to see how the input causes the output over time. Process studies have a research design that emerges gradually as the data collection and analysis moves forward, favours an interpretive stance and asks what happens within the process.

The Experience Sampling Method is used to collect participant self-reports over extended observation periods. These self-reports offer a rich insight into the individual lives of study participants by intermittently asking participants a set of questions. However, the longitudinal and repetitive nature of this sampling approach introduces a variety of concerns regarding the data contributed by participants. A decrease in participant interest and motivation may negatively affect study adherence, as well as potentially affecting the reliability of participant data. In this chapter, we reflect on a number of studies that aim to understand better participant performance with Experience Sampling. We discuss the main issues relating to participant data for longitudinal studies and provide hands-on recommendations for researchers to remedy these concerns in their own studies.

In this chapter, we focus on longitudinal first-person research methods in HCI. First-person research involves data collection and experiences from the researcher themselves, as opposed to external users (or participants). We present three projects where longitudinal ‘auto-approaches’ to research and design in HCI were applied, namely one auto-ethnography and two autobiographical designs. These projects help illustrate the benefits and challenges of using these first-person research methods in longitudinal HCI and interaction design research. We conclude the chapter by reflecting on themes and lessons that resonate across the three projects (i.e., range of participation, data collection, time to reflect, concluding).

Automated data collection has a significant role in collecting reliable longitudinal data in human–computer interaction (HCI) studies that involve human participants. While objective data collection can be obtained by and mediated through personal informatics, subjective data is mostly collected through labour-intensive tools. The potential of sensor-embedded everyday objects as subjective data collection tools is underexplored. Hence, in this chapter, we investigate the use of such products for subjective data collection purposes in longitudinal studies. First, we demonstrate current practices on subjective data collection tools and examine the aforementioned research gap. Following that, we discuss the results of three discussion sessions in which we collected insights from six expert researchers on the enablers and barriers of using sensor-embedded everyday objects as subjective data collection tools. We present our insights with use-case scenarios to communicate what possible roles sensor-embedded everyday objects could have in collecting subjective data in future longitudinal HCI studies and discuss how they could be further developed within the field.

This chapter formalizes the traditional randomized experiment as a sequential decision problem in which treatments are allocated to units sequentially to achieve a specific goal. This problem description is known as the multi-armed bandit (MAB) problem and we describe it in detail and relate it to the methodological considerations that arise when designing longitudinal studies in HCI. Subsequently, the chapter reviews multiple treatment allocation policies—attempts to solve the MAB problem—and analyzes their properties. Next, we discuss utility of a sequential perspective on experimentation for various methodological purposes such as early stopping, best arm selection, and powerful testing. We demonstrate how in many cases, and particularly in longitudinal studies, the “intermediate” results of an experiment can be used to improve the experimental design. We close off by discussing several recent software packages that allow readers to implement and analyze sequential experiments.

How can technologies be created that take on a long-term place in people's lives and that coevolve with them over time? What kinds of qualities should designers consider in crafting such kinds of computational things? And, how should we study and evaluate such new technologies through a longer temporal frame? In this chapter, we draw on examples of longitudinal field studies of the Photobox and Olly research products to explore these questions and to detail tensions and techniques that emerged across these two cases. Our findings reveal key tensions that researchers ought to be wary of when conducting longitudinal field studies of slow technology research products and techniques that can be applied to mitigate them.

As self-tracking has evolved from a niche practice to a mass-market phenomenon, it has become possible to track a broad range of activities and vital parameters over years and decades. This creates both new opportunities for long term research and also illustrates some challenges associated with longitudinal research. We establish characteristics of very long-term tracking, based on previous work from diverse areas of Ubicomp, HCI, and health informatics. We identify differences between long- and short-term tracking, and discuss consequences on the tracking process. A model for long-term tracking integrates the specific characteristics and facilitates identifying viewpoints of tracking. Finally, a research agenda suggests major topics for future work, including respecting gaps in data and incorporating secondary data sources.

Ultrasound mid-air haptic feedback is a novel output technology that allows users to experience a sense of touch in mid-air on the unadorned palm and fingers of the hand. Even though a growing body of research has studied various aspects of the UX of mid-air haptics, little is known about what happens to the users’ perception and experience after repeated use. The main reason for this is that today, mid-air haptic technology is not easily integrated in everyday devices (e.g. smartphones) nor widespread, making it difficult for it to be tested outside of a lab environment. This chapter describes the set-up of a longitudinal in-lab study, in which a mixed-method design was used to understand how the hedonic, pragmatic and emotional aspects of the UX of mid-air haptics changed over time. In eight sessions, spread over a five-week period, 31 participants interacted with a gesture-controlled home automation system augmented with mid-air haptic feedback. We report in this chapter on our participant recruitment and retention approach, the mixed-method set-up that was used, and (an excerpt of) the main results. Subsequently, we summarize best practices and propose suggestions for researchers who in the future intend to conduct a multimethod longitudinal study.

Crowdsourcing platforms can be roughly divided into two kinds: the ones that offer simple, short, and unskilled work (microtasking) and those that offer complex, longer tasks, which are difficult to break down and usually involve creativity (macrotasking). Past research has mapped the landscape of microtask crowdsourcing. Little, however, is known about where commercial platforms stand when it comes to creative crowdsourcing. Which types of creative tasks are offered? How are these remunerated? Do all platforms facilitate the same type of creative work? Given the increasing importance that creative crowdsourcing is expected to play in the near future, in this chapter we partially map the current state of this type of online work over time. During a six-month period, and on a daily basis, we collected public data from seven creative crowdsourcing platforms. Our data, covering more than thirteen thousand tasks, show that there are plenty of graphic design tasks but better financial rewards for other types of creative tasks, as well as a trend for creative crowd work platforms to offer longer tasks. Judging from the total rewards in those six months, we can also conclude that creative crowdsourcing will benefit from a shift to dynamic rather than fixed rewards, but also that this type of crowd work is still at an embryonic stage and has growth potential. Finally, our results highlight the need for a platform data watchdog, as well as the need for a more nuanced perspective of creative crowdsourcing, distinguishing between the types of platforms within this genre of online work.