2.1 Workload
In the field of information systems, individual workload or the lack thereof is often operationalized simply as the general ease of use of an information system in a given task (e.g. ‘freedom from difficulty’ according to the technology acceptance model (TAM): Davis,
1989). While this conceptualization and construct has functioned as a workhorse in the bulk of the technology acceptance literature produced in the last decades (Lah et al.,
2020), it provides a limited and one-dimensional understanding of workload and the usability of information systems. Specifically, it does not take into consideration the barriers that would prevent system adoption (see Taylor & Todd,
2001), and it does not reveal how such perceptions are formed (see Mathieson,
1991). In this study, we are interested in the usability of a multimodal information system when used in a rich information processing task. Therefore, this paper ventures to investigate more holistic conceptualizations and constructs of workload. One instrument that is widely used in similar situations where attention and cognitive processes are limited by the context and complexity of the task is the NASA-TLX instrument, which measures the perceived workload of an individual in a given information processing task that can include mental, physical, frustration-inducing, temporal, performance-related and effort-related workload dimensions. Therefore, beyond investigating a novel context of using information systems, this study also builds bridges between fields in the frontier between adoption and ease of use research, in relation to new technologies.
Workload is largely considered as the effort or cost (e.g., physical, mental, emotional) an individual devotes to accomplish a task (Hart,
2006; Hart & Wickens,
1990). It can be influenced by internal aspects such as an individual’s motivation or their past experience and ability, as well as external aspects such as the type, novelty, difficulty, and number of tasks that an individual completes (Hart,
1982; Meshkati,
1988). Originally rooted in attempts to measure the effort of flight-related tasks (see e.g., Li et al.,
2020), the practice of assessing workload also became more and more relevant in contexts that transcend aviation. For example, due to the rapid technological advancements and the growing number of novel systems that aim at enhancing, among other factors, convenience, productivity and efficiency, it became increasingly relevant to scrutinize the workload demands of all sorts of information systems. Respectively, today workload is also used to evaluate the interface design of conventional computer systems and portable devices, including the technologies that support virtual and augmented vision (Hart,
2006). The goal often involves gaining a better understanding of how to design and improve systems so that the intended benefits are not compromised by excessive workload during their use. Generally speaking, ergonomists and designers of information technology are interested in creating technology in a way that reduces workload or at least keeps it within an acceptable range (e.g., Grier et al.,
2008), as workload management is vital for user acceptance (Dang et al.
2020), productivity, performance, and user health (Jung & Jung,
2001; MacDonald,
2003). An important consideration is that users can only deal with a finite capacity of workload. Kantowitz (
1987) describes the concept of spare capacity, which understands that as long as task demands are below an individual’s maximum workload capacity, then performance should not be impaired. However, with increasing task complexity or difficulty, the perceived workload intensifies and if the acceptable level is exceeded, performance will suffer. In the light of this rationale, it is not surprising that in the past decades, theories of task-technology-fit (Goodhue & Thompson
1995) and disciplines devoted to the usability of systems (e.g., Hoehle & Venkatesh,
2015; Lewis,
2014) have garnered great attention in the realm of human-computer-interaction.
The practical necessity to assess the workload involved in human-computer interaction brought forth a number of different evaluation approaches, including objective measures based on performance indicators and psychophysiological cues, as well as measures based on subjective experiences (e.g., Cain
2007; Tsang & Vidulich
2006). Whereas objective measures collect real-time performance data or measure physiological reactions (e.g., via electrodes), subjective measures rely on the self-assessment of the experienced workload by the subjects (Tsang & Vidulich,
2006). A general issue pertaining to the assessment of workload considers the circumstance that different tasks tend to be subject to different sources of workload (e.g., mental and physical), as well as the varying degrees to which each specific source is accountable for an individual’s perceived overall workload (i.e., weighted workload) (Hart & Staveland
1988). A weighting scheme aims at measuring workload more accurately, and requires users to evaluate the degree to which different dimensions of workload contribute to the overall workload of a specific task (Hart,
2006). One particular measurement that has been widely accepted and deemed functional to cover the multidimensional nature of workload and capable of accounting for the individual differences of humans with regard to their weighted perceptions of workload is the NASA Task Load Index (TLX) (Hart & Staveland,
1988). This index belongs to the class of subjective measurement instruments, and allows individuals to quantify their experienced workload via a weighted scheme and consists of six dimensions. These include (1) physical demand, (2) mental demand, (3) temporal demand, (4) performance, (5) effort, and (6) frustration. Table
1 specifies each dimension in more detail, based on the work by Hart (
2006). What should be mentioned here is that a single effort scale (combining physical effort and mental effort) cannot capture the information needed to address the specific source of demands (Hart & Staveland, 1998). Thus, instead of asking subjects to introspect about the amount of mental or physical effort exerted, the NASA TLX instrument requires them to assess the objective physical and mental demands that are placed on them (Hart & Staveland, 1998).
Table 1
Explanation of Each Dimension of Workload in NASA-TLX Based on Hart (
2006)
Mental demand | Perceived mental and perceptual activity required by an individual to accomplish a given task (e.g., thinking, deciding, calculating, remembering, looking, searching). |
Physical demand | Perceived physical activity required by an individual to accomplish a given task (e.g., pushing, pulling, turning, controlling, activating). |
Temporal demand | Perceived time pressure due to rate or pace of the given task. |
Effort | Perceived level of work (mental and physical) to realize performance level. |
Performance | Perceived success in accomplishing the goals that are tied to the performed tasks. |
Frustration | Perceived insecurity, discouragement, irritation, stress and annoyance versus perceived security, contentment, relaxation and complacency during task performance. |
2.2 Extended reality
Even though there seems to be a lack of consistency in the use of reality-related terms (e.g., Virtual Reality-VR, Augmented Reality-AR, Mixed Reality-MR, and Extended Reality-XR) in academic and professional fields (Flavián et al.,
2019), AR and VR have been considered as the two core reality-virtuality technologies. With the development of multi-sensory technologies and modalities and the deepened conceptualized understanding of AR and VR, there is a consensus that any sensory experience can be augmented in a digital way (Harley et al.,
2018) and also be virtualized (Boyd & Koles,
2019). In terms of AR, multimodal information such as smell, touch, taste and sound can be digitally overlaid on the current world (Azuma,
1997; Carmigniani et al.,
2011; Riar et al.,
2021) and AR users are not isolated from it (Rauschnabel,
2021). AR has been defined as the term for technologies for augmenting or altering the current reality (Riar et al.,
2021), while in VR, all of the sensory information and stimulus of the ‘real reality’ is rather blocked and inhibited (Manis & Choi,
2019; Yim et al.,
2017). Therefore, while VR has been considered as the digital technologies of choice for substituting the perceived reality (Xi & Hamari,
2021), AR and VR provide different kinds of experiences to users (Fromm et al.,
2021). Regarding AR, the “augmenting” information and content can bring users interactivity, vividness and novelty (McLean & Wilson,
2019; Yim et al.,
2017). Alternatively, VR has been believed to create immersiveness (Suh & Prophet,
2018), telepresence (Lee & Chung,
2008; Steuer,
1992), and the sense of “being there” (Heeter,
1992; IJsselsteijn & Riva,
2003). As a further extension, when combining AR and VR together, Augmented Virtuality (AV) can be constructed for a more hybrid experience. There are high expectations towards AR and VR on creating interesting, novel and playful experiences (Lin & Yeh,
2019; Raptis et al.,
2018), however, an increasing number of studies have shown that the use of AR and VR in activities and completing tasks requires various resources and costs.