9. How Do Interruptions Affect Productivity?

Before we review what has been learned from interruption experiments, it is worth taking a moment to reflect on the purpose of an experiment. Experiments are designed to test a hypothesis. For example, do people work slower when interrupted compared to when they have not been interrupted? To test this hypothesis, the researcher manipulates a feature of interest (the independent variable), which in our case might be the presence or absence of an interrupting task. The researcher wants to learn whether this manipulation has an effect on an outcome measure (the dependent variable), which in our case might be how quickly a task is completed.

In a typical interruptions experiment, the researcher will ask a participant to work on a contrived task that they have designed. For example, the participant might be asked to use a computer interface to order some tasty donuts [32]. The cover story is provided to give some context to the task that the participant has been asked to work on, and it can be easily adjusted to suit the target domain of the study. For example, naval researchers have asked participants to place orders for the construction of ships [46], and healthcare researchers have asked participants to place orders for prescription medicines [18]. Regardless of the domain, the researcher gives the participant detailed instructions on how to complete the task using the interface and plenty of opportunities to practice it before starting the main part of the experiment.

In the main part of the experiment, participants will be asked to complete a number of tasks (e.g., place ten orders for doughnuts) using the instructed procedure. While the participant is working on this task, the researcher will occasionally interrupt them and ask them to work on a secondary task instead. The secondary task might require the participant to solve some mental arithmetic problems [32] or use a mouse to track a moving cursor on the screen [39]. In these experiments, the arrival of this interrupting task is carefully controlled by the experimenter, and the participant is often given no choice but to switch from the primary task to the interrupting task. This is because the researcher wants to learn whether the interrupting task affects the quality and pace of the work produced on the primary task.

This discussion leads us to consider how we measure the impact of an interruption on task performance. The primary measure that has been used is the time it takes a participant to resume work on the primary task after dealing with an interruption. This time-based measure is referred to in the literature as the resumption lag [4, 45]. The resumption lag measures the time it takes a person to re-engage with a task following an interruption. A longer resumption lag following an interruption reflects a general decrease in productivity: people are taking more time to complete a task, even when the time spent working on the interrupting task is deducted. In this way, the resumption lag is taken to reflect the time that is needlessly “wasted” as a consequence of being interrupted and later having to resume an unfinished task.

Over recent years a number of experiments have been reported that use the resumption lag measure to carefully unpack which features of an interrupting task make it disruptive. Experiments have investigated whether longer interruptions are more disruptive than shorter interruptions—finding that longer interruptions result in longer resumption lags [19, 39]. Studies have also been conducted to learn whether there are better or worse points in a task to be interrupted—shorter interruption lags are found when interruptions occur at natural breakpoints in a task, such as the completion of a subtask [2, 7]. The content of an interrupting task also matters—interruptions that are relevant to the primary task are less disruptive than interruptions that have nothing to do with the primary task [17, 21]. As we will discuss, the resumption lag has been explained by assuming that interruptions interfere with people’s ability to remember what they were doing prior to the interruption.

When a person resumes a task following an interruption, it often matters whether they get it right or make a mistake. Previous research has shown that interruptions increase the likelihood of errors being made on a task, in that important components of the task are either repeated or missed [9, 32, 46]. This finding has been taken as evidence to support the idea that following an interruption people fail to remember what they were doing in a task prior to being interrupted.

It has also been informative to consider whether there is a link between how quickly a task is resumed and the likelihood that an error is made. As discussed, interruption researchers have generally considered a longer resumption lag to be a bad thing— reflecting time needless wasted following an interruption. In contrast, Brumby et al. [9] found that longer resumption lags following an interruption were in fact beneficial in terms of reducing the occurrence of errors. This has important practical implications for the design of systems to encourage more reflective task resumption behavior in situations where interruptions are commonplace. Based on these findings, Brumby et al. developed and tested a post-interruption interface lockout that allowed users to look at the task interface but prohibited actions to be made. This interface lockout led to a significant reduction in resumption errors because it encouraged users to take the time to cognitively re-engage with a task before diving back into it and making a mistake.

A criticism that is often leveled at the kind of interruption experiments that we’ve reviewed is that the controlled setting in which they are conducted bears little resemblance to people’s actual work environments and how they manage the interruptions that they experience at work. In other words, our experiments can lack ecological validity because an important aspect of the phenomena that we are attempting to investigate is missing. This is an important concern because it means that the results of these interruption experiments might be of limited practical value or that they might not be valid at all when taken away from the controlled setting of the lab and applied to an actual work setting.

How might an interruption experiment lack ecological validity? Interruption experiments are often conducted in controlled environments in which the researcher actively works to remove unwanted distractions and interruptions (e.g., participants will be asked to turn off their phone and give their complete attention to the researcher’s task). The reason for this is that the experimenter wants to carefully control the nature and the timing of any interruptions so as to learn how they affect performance. Ironically, this desire for control presents a major threat to the ecological validity of the experiment. This is because most of the everyday interruptions that we experience are not forced but are instead discretionary. For example, an e-mail notification might appear on a screen, but we can choose whether to act on it or ignore it. By using enforced interruptions that participants have to attend to, interruption experiments can fail to capture this important aspect of the phenomena that they are attempting to study in the lab.

To overcome concerns about low ecological validity, Gould et al. [18] has taken an approach that relaxes experimental control over the environment in which participants work to study how naturally occurring interruptions affect performance. To do this, Gould et al. used an online crowdsourcing platform, Amazon’s Mechanical Turk, to host an interruptions experiment. Just like in a regular interruptions experiment, participants were asked to use a browser-based task interface to place orders for prescription medicines. But unlike a traditional lab experiment, participants worked on this task in their regular everyday environment: an office, a coffee shop, or their home. These are naturalistic environments that are filled with everyday interruptions and distractions. In addition, workers on crowdsourcing platforms, like Amazon’s Mechanical Turk, often work on multiple tasks at the same time; the environment is designed to encourage workers to complete as many tasks as possible so as to maximize their pay. This means that a competing (interrupting) task is often present, vying for the participant’s attention.

By running an interruptions experiment on a crowdsourcing platform, Gould et al. [18] found that workers switched to other tasks once every five minutes. This was revealed by window switching events and pauses in progression through the task. These interruptions were not inserted by the experimenter but were naturally occurring and at the discretion of the participant. Interestingly, this rate of interruptions corresponds to that seen in observational studies [16]. While these interruptions tended to be quite brief (around 30 seconds on average), Gould et al. found that they were sufficient to negatively impact performance on the primary task: participants who interrupted more often were considerably slower at completing the task, even after accounting for the time spent not working on the task. We know this only because the primary task interface was under the control of the researchers; this was not a naturalistic observation study. Gould et al.’s study provides a bridge between controlled experiments and observation studies; it provides evidence that the disruptiveness of interruptions can be readily detected out in the field and that it is not an artificial product of the controlled setting used in interruption experiments.

By conducting controlled experiments, researchers have been able to establish that task interruptions take time to recover from and lead to errors. Experiments offer an empirical approach for systematically testing whether the manipulation of an independent variable (e.g., the duration of a task interruption) has an effect on a dependent variable (e.g., the duration of the post-interruption resumption lag). Establishing whether the manipulation of an independent variable has an effect on the dependent variable is of both practical and theoretical value.

In practical terms, knowledge is developed about what makes an interruption disruptive, allowing practical intervention to be developed and tested. For example, Brumby et al. [9] established that when people made faster task resumptions, they were more likely to make an error. Learning about this prompted the development of an interface lockout mechanism that stopped users from resuming a task quickly following an interruption, reducing task errors.

An important characteristic of cognitive models is that they generate an exact prediction (i.e., generate a number) as an outcome (e.g., likelihood of an error), given an input (e.g., time away from the main task), and a formal description of how input is transformed into output (i.e., a computer program that captures theory of the process of forgetting). Other more conceptual theories of interruptions [6] or multitasking [49] also provide insight into human behavior and thought but typically tend to miss at least one of these three components (output, input, or transformation step) or describe them in less formal terms, such that the details that are needed to give an exact prediction are not available.

The value of cognitive models lies in their ability to predict aspects of human behavior and thought in detail. Cognitive modeling aims to unravel human thought by uncovering the details and making those details open for scientific debate [40]. As an example, take the Memory for Goals theory of forgetting [4], which has been applied to explain the results of interruption experiments. The model can be used to make a prediction for how quickly tasks will be resumed after an interruption. To do so, the model uses a mathematical function, derived from psychological theory, to determine how quickly a person will be able to recall what they were doing prior to dealing with an interruption based on the strength of this memory. The value of the model is that it gives a prediction for how quickly someone will resume a task (i.e., the resumption lag). Moreover, the general theory of memory retrieval that underpins this model helps explain why these resumption lags occur (namely, because of forgetting).

Since the inception of the basic Memory for Goals theory, the theory has been refined in many ways. Examples include the prediction of errors due to interruptions [46], the prediction of task switching performance [3], and the prediction of concurrent multitasking performance [7]. The initial modeling effort was crucial in this regard: by specifying a theory (of forgetting) in detail, it allowed researchers to make predictions regarding how memory impacts other settings, which could then be tested. In the end, these new experiments led to further refinements of the theory and to an even broader understanding of the cognitive mechanisms involved in recovering from an interruption.

Although the value of cognitive models lies in the details, this is also its Achilles’ heel. If a model is to be used to make predictions for a new task, then a researcher or practitioner needs to be able to specify those details ahead of time. To then specify those details, they also need to have a detailed understanding of the modeling framework and how these details should be specified within it. This is not feasible for every researcher and practitioner.

Fortunately, building on a long tradition in human-computer interaction research [10], more and more tools are being made to allow for predictions in applied settings, including dynamic settings such as driving [8, 43]. Moreover, in some cases not all details might be needed to make a prediction. For example, based on the mathematical equations behind Memory for Goals theory, recent work by Fong, Hettinger, and Ratwani [15] was able to predict the likelihood that emergency physicians resumed their original task after an interruption on their everyday emergency ward.

One of the main insights to come from modeling work using the Memory for Goals theory is that the longer an interruption, the more likely it is that errors are to occur, including forgetting to resume the task altogether (and for specific cases, the models can give even more specific and exact predictions). Therefore, the implication of this work is that there is value in avoiding being interrupted.

Models can also be used to inform our understanding of discretionary self-interruptions. Previous studies have found that people often choose to interrupt themselves, switching between different activities every few minutes [16, 18]. For example, an information worker who is focusing on a particular work activity will still likely choose to monitor and check their e-mail regularly, switching back and forth between application windows. How often should the person switch between these two different activities?

In our own research, we have used cognitive models to examine how the demands of a task affect the benefit of different switching strategies (i.e., how long to focus on one task before switching back to another task). We studied this in the context of a dual-task experiment in which participants had to control a dynamic task while performing a text-entry task [13, 26, 27]. We used a cognitive model to identify the best possible strategy for dividing attention between these two tasks and then compared this to what people actually chose to do in the experiments. Across several studies, we found that people were very quick at locating the best possible strategy for dividing their time between tasks. We learn from this work that people are actually pretty good at multitasking, when the relative importance of each task is made clear to them. Cognitive modeling was a vital step in this work as it was used to identify the best possible switching strategy; without this, it would not have been possible to objectively benchmark how well people were multitasking.

Cognitive models develop our understanding of why and how interruptions are disruptive. They do this by instantiating theory using mathematical models and simulations. This puts into practice the ideas we have for what is causing an interruption to impact performance. Through this line of research, Memory for Goals has emerged as an important theory. The core idea is that when dealing with an interruption, people forget what it is they were working on. Resuming a task therefore involves remembering what one was doing before the interruption. By casting this as a memory retrieval process, the Memory for Goals theory is able to draw on general theories about the nature of human memory. In practical terms, cognitive models can be used to both explain existing data and make predictions about what will happen in novel situations or settings.

Most in-situ studies of interruptions have been conducted in the workplace. Workplaces can be dynamic places, and interruptions can be triggered from a number of sources involving people (colleagues, phone calls, ambient conversations), and computer and smartphone notifications (e.g. e-mail, social media, text messaging). However, interruptions can also originate from within an individual (e.g., due to mind-wandering, [37]).

Constant interruptions and the consequent fragmentation of work are a way of life for many information workers [12, 33, 38]. By closely monitoring workers in-situ, it was found that people switched activities (conversations, work on computer applications, phone calls) about every three minutes on average. At a less-granular level, when activities were clustered into tasks, or “working spheres,” these were found to be interrupted or switched about every 11 minutes [16]. There is a relationship of length of time on task and interruptions: the longer time spent in a working sphere, the longer is the interrupting event. It has been proposed that when interruptions are used as breaks, then such longer interruptions might be due to replenishing one’s mental resources [47].

In a work environment, observations found that people self-interrupt almost as often as experiencing interruptions by an external source such as a phone call or colleague entering the office [16, 33]. When these field studies were done, more than a decade ago now, most self-interruptions were found to be associated with people initiating in-person interactions. Most external interruptions were also due to verbal-based interruptions from other people rather than due to notification mechanisms from their e-mail or voicemail. In more recent years, social media has become popular in the workplace, and it is likely that the main triggers of self and external interruptions in the present-day workplace may be different.

Interruptions may be beneficial or detrimental. In a workplace diary study, Czerwinski et al. [12] showed how the work context of information workers continuously changes because of interruptions. A study of corporate managers showed that while interruptions can disrupt tasks, managers appreciate the usefulness of interruptions as it provides the opportunity to get useful work-related information [20]. While social media and online micro-breaks may provide numerous benefits in the workplace, field studies have shown that they create challenges due to switching contexts.

Generally, interruptions that disrupt concentration in a task, especially when they occur at a point that is not a natural breaking point for a task, can be detrimental [24]. External interruptions cause information workers to enter into a “chain of distraction” where stages of preparation, diversion, resumption, and recovery take time away from an ongoing task [22]. When notifications from smart phones were turned off for a week, people reported higher levels of attention [31]. A large cost in switching tasks on the computer is that it has been associated with higher stress [34]. Yet, people are able to adjust their work practices to manage constant face-to-face interruptions [42], as well as to manage interruptions from computer-mediated communication [48].

Interruptions in the workplace can also provide benefits. Longer interruptions (or work breaks), such as taking a walk in nature during work hours, have been shown to increase focus and creativity at work [1]. Observational studies have identified that people use a variety of social media and news sites to take breaks to refresh and to stimulate themselves [29]. However, a growing number of workplaces have policies that regulate the use of social media at work [41], which can impact the ability of people to take a mental break at work.

A few field studies have examined the relationship of stress and interruptions. In a study that focused specifically on the role of e-mail interruptions, Kushlev and Dunn [30] found that limiting the amount of checking e-mail significantly reduced stress. Another field study in the workplace found that cutting off e-mail (and consequently reducing both internal and external interruptions) significantly reduced stress [36]. Cutting off smartphone notifications also significantly reduced inattention and symptoms of hyperactivity [31]. On the other hand, when e-mail notifications were turned off, another field study showed that some individuals increased their self-interruptions to check e-mail due to the lack of awareness of incoming e-mails [23]. It is theorized that people who multitask more and who are susceptible to interruptions may have lower ability to filter out irrelevant stimuli [11]. Other individual differences have been observed, such as the personality trait of higher neuroticism with higher task switching [35].

Field studies suggest that higher frequency of task switching is associated with lower perceived productivity [34, 38]. Several explanations have been proposed for this relationship, including the depletion of cognitive resources used in attending to interruptions, the redundancy of work when reorienting back to the task [34], and that a polychronic workstyle may be contrary to what most people prefer [5].

Observational studies reveal that people use strategies to manage interruptions. Whereas most people prefer monochronic work (finishing one task through to completion [5]), the demands of the workplace result in polychronic work (i.e., the consequent switching of attention to different tasks). Because of the expectation of working in an environment with interruptions, some people have been observed to develop strategies to adapt to the unpredictability of the working environment. Participants can externalize their memory of task information, for example in the form of artifacts such as sticky notes, the e-mail inbox (e-mails sent to oneself), or electronic planners, often updated throughout the day [16]. The challenge with conventional electronic planners is that they are generally not designed at a level of granularity to help people recover from interruptions from a partially completed task.

Technological solutions have also been implemented in the field to detect when people are interruptible, with the intent to minimize interruptions at inopportune times. Promising techniques tested in the field have shown that it is possible to predict when people are in cognitive states where they can be interrupted that can minimize interruptions, reduce stress, and thus minimize cognitive resources needed to reorient back to a task [14, 25, 51, 52].

Observational studies document the kinds of interruptions that people experience in their actual workplace. These studies are resource intensive to conduct and so often focus in on a small number of participants, giving a detailed and rich account of a particular work setting. We have learned from observational studies that workplace interruptions are extremely commonplace. Some of these interruptions reflect the fragmented nature of work: people work on different tasks and activities through the day, and this requires constant switching between them. People also seek out interactions with others—either by having conversations with colleagues or by communicating through social networking sites and e-mail. Consistent with the results from interruption experiments, observational studies also reveal that frequent interruptions result in feelings of reduced productivity. However, regular breaks from work are also necessary, and people return from breaks feeling energized and ready to resume their work.