14. Using Biometric Sensors to Measure Productivity

If we want to be productive, it would be great if we could track productivity in some way, such that it is possible to determine what factors help and hinder productivity. Biometric sensors may be helpful for such productivity tracking. But what does being productive mean? A simplistic notion of productivity is being able to pay attention without getting distracted. Indeed, to be productive in simple tasks such as filling out routine forms, one needs to carefully monitor one’s goals and ensure not to get distracted. On the other hand, for more complex tasks such as developing a new software architecture or implementing a complex function, one also needs creativity and outside-the-box thinking, which is incompatible with a singular focus. In other words, aspects of productivity such as creativity depend not on concentration but on its opposite: mind-wandering [1], which is a process of task-unrelated thinking. How would that work? Mind-wandering, when it involves thinking about other things while you are engaged in a task such as writing a computer program can help you to access new information that brings an alternative perspective on what you are doing. This means that when the contents of mind-wandering are monitored and are not too engrossing, it can in fact be very useful. Moreover, this also means that a singular focus does not always indicate productivity because, for example, being very concentrated on a single stupid task such as writing the same line of code over and over again is not very productive.

In summary, productivity requires sometimes singular focus and sometimes distraction. What is crucial is monitoring to ensure that attention is being paid to the most relevant goals and that the degree of attentional focus is in line with those goals. The attentional focus should be neither too narrow nor too wide and should be directed to the task that is most important at that moment.

Interestingly, most current attempts at developing biometric sensors focus on measuring attentional focus. Here I argue that another (albeit more technically challenging) target could be the goal-directedness of attention. A goal-directed attention is one that does not get pulled into patterns of thoughts that are difficult to disengage from, such as, for example, rumination and worry.

In this chapter, I will first discuss biometric sensors on the basis of eye tracking and electroencephalography (EEG) that simply track attention and then preview some new potential sensors that track the broader definition of productivity that depends on focusing on the most relevant goals and not being sidetracked by thoughts that pull one away.

Arguably the simplest method to measure attention is by following the eye gaze and the width of the pupil. In laboratory studies this is measured with fancy cameras that are following the eyes, but potentially similar functions could be provided by webcams that are present on almost every computer. In our lab we have demonstrated that webcam-based eye tracking is sensitive enough to predict upcoming choices from a set of stimuli presented on the screen.

So, what can you measure with eye tracking? In one experiment investigating distraction by external stimuli, we found that when we had a participant do a memory task on the screen but showed cat videos on a flanking screen, their eyes were drawn to the video [9]. The frequency with which the eyes were drawn to the cat video depended on the difficulty of the task, such that the more visual resources a task consumed (e.g., requiring poring over a visual image very precisely), the less likely a person was distracted by the cat videos. On the other hand, the more memory resources a task required (e.g., keeping in mind a series of numbers), the more likely the person’s eyes were drawn to the cat videos. In other words, video screens with moving images are a terrible idea on the work floor. In another study, we used eye tracking to examine whether a person was keeping a location on the computer screen in mind that they were trying to memorize [3]. We found that when they were distracted, as you would expect, people’s eyes were less fixated on the visual locations than when they were attentive. In short, when you are doing a task where your eyes have to be located at a specific spot (such as a coding window that occupies only part of the screen), then using eye gaze can be an effective measure of your attention.

However, most of the time, your work does not require your attention to be focused on a single spot. In that case, potentially we could still use eye-based biosensors but focus instead on the size of the pupil. Already for many decades, pupil size has been associated with a state of mental effort [4] and arousal [2]. For example, when we make the task more difficult, we tend to see an increase in pupil size. In addition, when we reward people for successfully performing a difficult task, their pupil size increases even more.

Many studies have associated mind-wandering with a decreased pupil size [3, 11], so another potential marker for being on the ball and being productive would be the size of your pupil. A larger pupil would be indicative of higher productivity. In fact, we have previously used pupil size as a marker for when it would be best to interrupt the user [5]. Interruptions are generally best when a person is experiencing low workload, i.e., when he or she is somewhere between subtasks, not when he or she is trying to remember something or manipulate complex information in his mind. The study showed that we were successful in finding low-workload moments and performance was better when we interrupted on low-workload moments. This suggests that pupil size can successfully be used even on a single-trial basis and is a good candidate for measuring mental effort as an index of productivity.

Another potential biomarker of productivity is EEG. EEG reflects the electrical activity emitted by the brain, as measured by electrodes on the scalp. EEG has frequently been used to track both mind-wandering and mental effort. A common finding is that when a person is mind-wandering, the brain activity evoked by a stimulus is reduced. This is thought to indicate a state in which the person is relatively disconnected from their environment with their attention more internally directed. While there has been long-standing research in the role of alpha waves—which are typically referred to as the brain’s “idling waves”—in mind-wandering, that research has not demonstrated clear mappings between these brain waves and mind-wandering.

The most advanced studies in this field have started to use machine learning classifiers to predict an individual’s attentional state. For example, a study by Mittner and colleagues [6] demonstrated that it was possible to predict with almost 80 percent accuracy whether a person was on-task or mind-wandering on the basis of a combination of behavioral and neural measures. These neural measures involved functional magnetic resonance imaging (fMRI). The problem with fMRI is that it is not a very suitable measure in an applied context because it requires an expensive and heavy MRI scanner in which the person has to lie down to be scanned. Moreover, MRI scanners produce a large amount of noise, making it not conducive for work. Nevertheless, recent work in our lab suggests that it is possible to achieve up to 70 percent accuracy in predicting mind-wandering using the more portable EEG. Moreover, in our study, this accuracy was achieved across two different behavioral tasks, suggesting that it can tap into a general mind-wandering measure, which is crucial for application in a work environment.

EEG has been used to measure not only mind-wandering but also mental effort. The most frequently used index of mental effort in EEG is the P3, an EEG potential that occurs roughly 300 to 800 ms after a stimulus has been shown to an individual [10]. This component is larger when a person exerts mental effort. This component is also smaller when a person is mind-wandering, suggesting that the P3 is potentially not a very unique index of mental effect. However, because this EEG component is time-locked to a discrete stimulus, it may be challenging to monitor such potentials in the office environment, unless you display periodic discrete stimuli to the individual with the purpose of measuring this P3 potential.

Taking these concerns into account, if EEG is potentially usable for monitoring distraction and productivity, then a problem to take into consideration is that despite that it is less unwieldy than MRI, an EEG system is typically still quite inconvenient and takes a lot of time to set up (usually somewhere between 15 and 45 minutes). A research-grade EEG system consists of a fabric cap in which anywhere between 32 and 256 electrodes are embedded, and for each of these electrodes, the connection with the scalp needs to be ascertained by means of an electrode gel and manual adjustments. On top of that, the cap needs to be connected to an amplifier that enhances the weak signals recorded on the scalp such that they are elevated above the noise. Only with these procedures a sufficiently clean signal can be collected. Clearly this would not be feasible for the workplace.

Luckily, recently there has been a boom in the development of low-cost EEG devices that have only between 1 and 8 sensors and that do not need extensive preparation (e.g., Emotiv and MUSE). If these electrodes were placed in the correct locations, they could potentially serve as productivity-monitoring devices. In fact, they are frequently marketed as devices that can record concentration. Despite these claims, however, I have found that when comparing a research-grade EEG system to these portable devices, that the portable EEG devices do not provide a reliable signal. Many place electrodes on the forehead, which are primarily expected to capture muscle activity instead of brain activity. Of course, muscle activity can be an index of how stressed a person is, since stress is associated with muscle tension, but it does not say much about a person’s mind-wandering and distraction. For example, it is possible to be quite tense while working on a software development project while being really relaxed and browsing social media. So, at this time EEG is really only a useful measure of productivity in a laboratory setting.

As mentioned, only measuring focus is not sufficient for productivity. In addition, a certain amount of mental flexibility and allocation of attention to relevant goals is crucial. This mental flexibility is difficult to monitor with biometric devices, but one related candidate signal is the one associated with “sticky mind-wandering”—a mind-wandering process that is very difficult to disengage from [12]. Sticky mind-wandering is a precursor of rumination (narrowly focused uncontrolled repetitive thinking that is mostly negatively balenced and self-referential [7]). For example, rumination may involve repeated thinking that “I am worthless, I am a failure,” supplemented by recall of experiences, such as a poor evaluation of a piece of work you delivered. This thinking repeatedly intrudes into a person’s consciousness, thereby making it difficult for them to concentrate, one of the major complaints that depressed people are suffering from. Sticky mind-wandering can take the form of recurrent worries, for example, about not being good enough, about their children, their future, and so on. These are the kinds of thoughts that are particularly harmful for productivity because they disrupt particular difficult thinking processes, which are crucial for software developers.

Recent work has started to map and experimentally manipulate these “sticky” forms of mind-wandering. We found that when people have a thought that they think is difficult to disengage from, then their task performance just prior to that moment tends to be worse and more variable in duration [12]. Other research where people were equipped with smart phones to measure their thoughts over the course of many days showed that sticky mind-wandering interfered more with ongoing activities and required more effort to inhibit. It was further suggested that a sticky form of mind-wandering is associated with reduced heart-rate variability compared to nonsticky mind-wandering [8]. In general, larger heart-rate variability is associated with increased well-being, and therefore reduced heart-rate variability is not desirable. This means that heart-rate variability is a potentially attractive target for biometric monitoring, especially because more and more low-cost heart-rate trackers are becoming available, such as those integrated in smart watches.

The studies discussed here together suggest that there are several ways in which it may be possible to measure productivity biometrically. Possibilities include pupil size, heart-rate variability, and EEG, which each has its own possibilities and limitations. Nevertheless, the majority of these measures were tested in a relatively simple and artificial laboratory context, in which only a limited set of events can happen. In contrast, in the real world, many more scenarios play out, and it is not clear how these biometric measures fare in those contexts. What is needed is a better understanding of the boundary conditions under which different biometric measures can work, and potentially a combination of different measures can give a suitably accurate index of distraction, thereby potentially differentiating between helpful mind-wandering and harmful mind-wandering.

Such an index could potentially be integrated into an interception system that makes the user aware of their distraction and then reminds them of their longer-term goals. Distraction usually arises when goals with short-term rewards or instant rewards such as social media are less active in our minds than longer-term goals. Even in the case of the stickier ruminative mind-wandering, a small reminder may be enough to allow a person to step out of this thought process and redirect attention to more productive long-term goals such as writing a paper or finishing a computer program.

In short, I have discussed what it means to be productive and how we can potentially measure this. Since most jobs require more than mechanical concentration on a single thing, measurement of productivity is nontrivial. Nevertheless, scientific studies on tracking attention provide a good starting point, and they demonstrate that eye movements, pupil size, heart rate variability, and EEG all provide some useful information about a person’s attentional state. On the other hand, none of these measures by themselves provides a fool-proof metric of productivity. Moreover, in many of them there are challenges to measuring it in a real-world context. For this reason, I think that the most productive use of biometric monitoring is not tracking productivity per se but rather helping the user to monitor himself or herself. The biometric sensors could be combined and in this way could help a user to become aware of potential lapses of productivity and remind them of their most important long-term goals.

The following are the key ideas from this chapter: