Digital Transformation in Health Care: Augmented Reality for Hands-Free Service Innovation

Organizations today desire strategies that place them on the frontiers of service innovation (Kim et al. 2015). Transformation of the existing service systems to a smart service system is thus a promising endeavor. Smart service systems are configurations of people, technologies, and other resources that interact with other service systems to create mutual value (Maglio et al. 2009). Smart service systems use smart devices, such as smart glasses, as boundary objects to network resources and routinize interactions between the actors involved in a service system (Becker et al. 2013). Smart devices can observe their environment through sensors or actuators while being able to communicate over a network and, thus, they can be active actors in a service system rather than just passive objects (Beverungen et al. 2017b). Collaborative systems will enable human individuals to realize their full creative potential in delivering services to consumers (Bednar and Welch 2019). Artifacts that are intended for smart service systems should be designed and built with an eye towards user satisfaction and technology acceptance (Jafari Navimipour and Soltani 2016). This is especially true in the health care context, where patient trust is essential (O’Connor and O’Reilly 2018).

Several models and theories of technology acceptance have been adopted from the fields of sociology and psychology and were tested in various information systems related contexts over the last decades (Venkatesh et al. 2003). The most-cited models are the Technology Acceptance Model (Davis et al. 1989), the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh et al. 2003) and UTAUT2 (Venkatesh et al. 2012). Within our research, we decided to use UTAUT in order to evaluate our design artifact, because it was originally developed to explain employee technology acceptance and use (Venkatesh et al. 2012), whereas UTAUT2 focuses on consumer use (Venkatesh et al. 2012). UTAUT has previously been used in several instances to predict, define and enhance the use of information systems (Wills et al. 2008). The main factors in the UTAUT model are performance expectancy, effort expectancy and social influence that individuals experience when using an artifact. A vast body of research has shown that these constructs can predict an individual’s intention to use the artifact, which in turn predicts actual use.

Additionally we investigated the satisfaction of caregivers after using our artifacts. Satisfaction refers to the perceived discrepancy between prior expectation and perceived performance after consumption. When performance differs from expectation, dissatisfaction occurs (Oliver 1980). Past studies suggest, that perceptions of service quality and value affect satisfaction, and satisfaction, in turn, affects loyalty and post-behavior (Chen and Chen 2010). The service profit chain directly links employee satisfaction and loyalty to customer satisfaction and loyalty (Loveman 1998). It posits, that internal service quality influences employee satisfaction, which in turn influences their loyalty which then influences service quality (Loveman 1998). Increasing internal service quality and employee satisfaction is thus likely to improve overall service quality. Moreover, health care workers’ satisfaction with electronic patient records is considered to be a critical factor (Maillet et al. 2015). Finally, the adoption of new technology has been associated with increased job satisfaction (Bala and Venkatesh 2015).

Chronic wounds are a major problem in health care (Wüller et al. 2018). Every year 2–3 million German patients require wound treatment. Among those, approximately 900,000 patients suffer from chronic wounds. Chronic wounds are defined as wounds that require treatment for more than 8 weeks (Schubert and Köster 2015).

For the health care professionals in charge of treating the wounds, it is mandatory to provide an accurate documentation of the wound process (BVMed - Bundesverband Medizintechnologie e.V. 2015). This is not only required by German law but is also used as a basis for care quality assessment (BVMed - Bundesverband Medizintechnologie e.V. 2015). However, there is a lack of direction and there are no established standards on what tools to use for wound documentation (Ding et al. 2016). It has been noted that a more standardized approach to wound treatment documentation could help to shift from practice- to evidence-based wound treatment (Gillespie et al. 2015).

Wound documentation improves wound treatment outcomes, because relations between interventions and outcomes become evident (Ding et al. 2016). The wound documentation enables health care professionals to assess how the wound has changed over time. Thus, the wound documentation lays the foundation for professional therapy planning and ensures the best possible care for the patient.

The current practice of the wound documentation process is cumbersome to execute for the nursing staff and can lead to serious problems. These include the transmission of germs into open wounds. This may happen via the surfaces of digital cameras or similar devices, which are currently in use for image capture of wounds (Al-Masslawi et al. 2017). Nurses may touch such devices numerous times during the day without washing their hands, potentially spreading germs to others (Thomas et al. 2013).

Furthermore, the actual task of taking the photograph is challenging. Often, several nurses are required for such documentation. This makes it a time-consuming endeavor and adds to already high levels of workload. The documentation is typically written in the station room for hygienic reasons. The documentation is therefore written after the wound treatment is completed and is not done in a timely manner (Ding et al. 2016). Consequently, nurses have to remember specific details about the wound until they reach the station room. The resulting wound documentation is often described as inaccurate (Gillespie et al. 2015; Ding et al. 2016).

Several POC documentation systems, such as Physician-order entry systems or smartphone and tablet applications have been evaluated for use in the wound management context (Nuckols et al. 2014). However, POC systems whose functionality does not closely match the workflow are not valued by health care workers (Sockolow et al. 2014). A reoccurring problem with existing POC systems is that health care workers cannot use them when they do not have their hands free. This is especially problematic in health care where many procedures need to be performed aseptically (Hatscher et al. 2017). In this regard, smartphones, tablets and laptops are not well suited for documentation at the POC, because they should not be touched while the practitioner’s hands are sterile or soiled.

In contrast to such hand-held devices, smart glass applications are much better fit for aseptic use in health care because they can be used hands-free. Virtual information is displayed in the user’s field of view and technology interaction can be done through hands-free interaction modalities such as voice commands or eye blinking.

The goal of AR is to bring additional information as seamlessly as possible into the view of a user (Schwald and de Laval 2003). This is done by adding real-time interactive virtual three-dimensional (3D) elements into the user’s field of view (Azuma 1997). In previous research, usage of AR smart glasses has been evaluated in several service-related contexts (Klinker et al. 2018). In the field of Logistics, AR has been used to display assembly instructions and to support workers to pick the right parts from a shelf (Evans et al. 2017; Huck-Fries et al. 2017). Moreover, AR has been used for collaboration scenarios like minimal-invasive surgery (Chen et al. 2015).

Users of AR smart glasses need to be able to interact with the virtual objects surrounding them. As such devices are typically operated in mobile and 3D-environments, established interaction paradigms such as keyboard, mouse or the Windows Icons Menus Pointers (WIMP) are not a good fit for this technology (Jacob et al. 2008). Instead, physical 3D-interaction concepts like gestures, hand pointing, ray-casting with hand-held devices, eye blinking, gazing or audio-based interaction concepts like speech commands or natural language processing could be used (Bowman et al. 2008).

A selection technique for virtual objects has to provide means to indicate an object (object indication), a mechanism to confirm its selection (confirmation) and visual, haptic or audio feedback to guide the user during the selection task (feedback) (Argelaguet and Andujar 2013). Since selection consists of several subtasks, 3D user interfaces often leverage multimodal interaction techniques to achieve the synergizing effects of the division of labor. This can, for instance, be done by gazing at an object (indication) and speaking a voice command to select it (Klinker et al. 2017).

We implemented two software versions of our wound documentation application. Each of them corresponds to a treatment within our experiment. The two applications were identical, the only difference being the interaction method. One used voice commands while the other used eye blinking. For the eye blinking treatment, we employed a Wizard of Oz approach (Maulsby et al. 1993). During the experiment, the trial manager would watch the participant’s eyes and click on a Bluetooth device whenever the participant blinked twice with both eyes. We used the onboard voice command library of the HoloLens for the voice command version of the application. The available voice commands within an application screen were always displayed at the bottom of the participant’s field of view. As the third treatment, we asked the participants to evaluate the wound documentation that is currently in use in their facility.

Overall, a total of two manipulated scenarios and one baseline scenario were presented to subjects: (1) the wound documentation process currently in use at the health care facility; (2) The HoloLens application using voice commands and (3) the HoloLens application using eye blinking. We did not combine the use of voice and physical interaction.

We conducted the experiment with the staff of health care facilities in quiet rooms within their own facilities. We used a table, two chairs, and a laptop with an online questionnaire. The online questionnaire consisted of several parts. In the first part, participants were asked for demographic data (i.e., gender, age, working experience and daily smartphone usage). The succeeding parts were identical and filled out after each treatment.

Data was collected using a closed online questionnaire. It contained questions on: Performance Expectancy (4 items), Effort Expectancy (4 items), Patient Influence (2 items), Behavioral Intention (3 items), nurses’ satisfaction (4 items) and open comment sections after each treatment. Table 3 in the appendix lists all constructs and items as they were used in the study. Participants needed about 5 minutes on average to fill out one part of the questionnaire. Participants were asked to fill out one part of the questionnaire after each treatment.

The wording of the items used in prior studies was adapted to nursing practice. Since we were particularly interested in health care worker’s perception of how patients would react to smart glasses in the wound documentation process, we changed the subject of the social influence construct to “patient” instead of “people who are important to me”. Subsequently, one of the construct’s questions became redundant and we dropped it. We decided to refer to this construct as “patient influence” instead of “social influence” throughout this research paper. Moreover, we changed the word “friends” in one of the questions for the satisfaction construct to “colleagues”. Participants filled out a German version of the questionnaire. The question translations were derived from the appendices of published papers or upon request from German authors that used these constructs for evaluations (Nistor et al. 2014a, b; Rocznik et al. 2017).

At the beginning of the experiment, we briefed the participants that they would be asked to document wounds as they would in their daily work. We simulated different wounds by providing printed pictures of different wounds for each treatment. For the treatments involving the HoloLens, we asked the users to test the application until they felt confident enough to use it on their own. We would then let participants do the experiment version of that treatment once. After each treatment, the participants were asked to fill out a questionnaire. In order to prevent learning and boredom effects from influencing our results, we randomized the order of the treatments.

After having started one of the experimental treatments on the HoloLens, the user’s first task was to measure the length of the wound. This is done by gazing at the far edge of the wound using the white cursor (indication) and then performing the confirmation command (using voice/blinking). A little blue dot then appears at the point the user is gazing at. Then the user needs to gaze at the near edge of the wound and repeat the selection command. A second blue dot and a line appear, connecting the two dots. Furthermore, the distance between the two dots is displayed above the line as a decimal number measured in meters. Figure 3 shows a picture of the result. By performing the next confirmation command a picture of the wound is taken. Using a red arrow, the user’s attention is then guided to a virtual screen displaying the picture that was just taken. The user can either measure the wound again and take a new picture or proceed to the next steps.

The following screens are all checklists. On each screen, the user needs to check off all characteristics that the wound has. In total there are six checklist screens with four to six items each. Figure 4 shows a picture of one of the checklists. Once all checklists are filled in, the application returns to the main menu.

In total, we recruited 45 health care workers with wound management experience at four hospitals, three ambulant and two stationary health care providers. We approached senior level managers of the facilities and asked them for supporting our research by asking their staff to participate in an experiment about documenting wounds with a smart glass application. We then visited the facility and conducted the experiment with volunteers. The participants received no monetary compensation for their participation in the experiment.

The average age of the nurses was 40.48 years (standard deviation (SD) = 12.03) and they had 16.20 years of experience on average (SD = 11.63). The sample comprised 33 women (73.3%) and 12 men (26.6%). According to the German Federal Agency of Work, about 80% of nurses in the German health care sector are female (Bundesagentur für Arbeit 2018). Thus, we deem our experimental group to be a representative sample of nurses in the German health care sector.

Assuming a medium effect size (f = 0.25), with a power of 0.80 at alpha equals 0.05 significance level, the required sample size for each cell is 39 (Cohen 1992). Hence, 45 subjects for each experimental treatment is adequate for data analysis.

We used the standard procedure documented by Straub (Straub 1989) to validate the reflective constructs Performance Expectancy, Effort Expectancy, Patient Influence, Behavioral Intention, and Satisfaction. All factor loadings were significant, suggesting convergent validity. As suggested by Straub all constructs satisfy the threshold values for the average variance extracted (AVE > 0.50) and Cronbach’s alpha (α > 0.70) (Straub 1989). In order to evaluate construct reliability, we calculated composite reliability (CR) for all constructs. All constructs had a composite reliability significantly above the cut-off value of 0.70 and the constructs’ quality is therefore satisfactory. Table 4 in the appendix shows AVE, CR and Cronbach’s alpha for each construct used in this study.

During our ethnographic studies, wound experts mentioned several issues with the current wound documentation process that leave room for improvement. The most pressing issues stem from missing details in the documentation. For instance, wound details are often forgotten or wound measurements are often inaccurate. A second theme that emerged is related to the underlying process: In focus group meetings, wound managers described their present process to be inefficient and difficult to execute both time- and technology-wise. As an example, the issue of how to hold a ruler in place such that photos can be taken with a sense of scale has been raised. In addition, participants complained about the amount of material needed. Another major theme of issues relates to hygienic considerations: Wound managers expressed concerns about cameras not being disinfected, the need for frequent disinfection in the process, and the difficulty of adhering to hygienic standards. Taken together, while current processes seem to be based around a common core, they also seem to share deficiencies. The joint reports on unsatisfactory processes, e.g. regarding efficiency, and concerns regarding hygiene imply that health care practitioners find a lot to be desired in wound documentation.

To cope with deficiencies of current processes and to improve outcomes, health care workers employ various workarounds that may deviate from standard practices. Comments by participants have shown the following behavior: Some health care workers write down the wound size and other wound characteristics on paper or their hand directly after measuring in order to memorize them better (see Fig. 2). Others perform wound documentation with a second health care worker who writes down information on the paper ruler and takes the picture, while the other interacts with the patient and conducts the wound treatment. While such workarounds may increase outcome effectiveness, they also imply the risk of deviations in processes and thus consistency (Röder et al. 2014). Lack in consistency may be problematic since health care workers usually work in shifts and chances are, that future documentation will be carried out by different personnel, who then applies different standards. The resulting documentation may be hard to compare and thus contribute to the incomplete or unclear documentation.

Throughout the second phase of the project (Design and EVAL2), we identified several requirements and restrictions to the solution space. From the evaluations, it became apparent, that wound documentation systems need to be operated without having to touch them (e.g. by voice commands or body motions) and while the wound manager is at the patient’s bedside. We came to this conclusion by testing a tablet application with wound managers. The idea for testing the tablet application was that it might be sufficient for wound managers to document the wound directly after wound treatment while they are still in the patient’s room.

We evaluated the application in a focus group meeting. The wound managers in the focus group were not satisfied with the solution, because it did not improve the process sufficiently for them. Their concerns were, that they would not be able to look at the wound during documentation, because bandages would already be covering the wound. In this phase, we also tested a smart glass prototype on the Vuzix m100 that allowed for hands-free documentation using speech commands. We tested the prototype with wound managers in two focus group meetings and in six individual tests (EVAL 2). The findings of the focus group meetings are summarized in Table 1. The prototype was well received by the experts and viewed as a potential improvement to the wound documentation process. Wound mangers especially liked being able to document hands-free while performing the wound treatment. However, the experts wished for a feature that would help to measure the size of the wound. In the current process, paper rulers are being used for this. The experts stated that it is difficult to hold the paper rulers in place while taking a picture of the wound. Moreover, using a ruler holds the risk of transmitting germs into the wound and makes it difficult to stabilize patients.

Building on our design knowledge from the previous prototypes, we constructed our final prototype as a smart glass wound documentation application on the Microsoft HoloLens (see Fig. 3). We implemented a feature to measure the wound size using the Microsoft HoloLens’s depth sensing capabilities and tested it with a focus group of wound management experts from a hospital. The experts liked the measuring feature and being able to document hands-free during wound treatment.

In contrast to the Vuzix m100, the Microsoft HoloLens uses Augmented Reality. While the Vuzix m100 only displays information on a small screen, the HoloLens embeds virtual objects into the real world. One of the main challenges that emerged during the construction of the final prototype was interaction with the virtual objects. Nurses needed to be able to select and deselect checkboxes about wound characteristics and measure wound sizes using a 3D User interface (see Fig. 4). In line with existing research, we employed a multimodal interaction approach in our application (Bowman et al. 2008). We used gaze in order to indicate the object that should be selected and tested voice commands (audio based) and eye blinking (physical) for confirming the selection.

In the final project phase (EVAL3) we conducted an experimental evaluation of our artifact. The requirements throughout the project lead us to the conclusion, that a digital wound documentation system should be implemented on a mobile device that can be operated hands-free. This makes smart glasses a good fit for the task. From pre-tests of smart glass applications with health care professionals we deduced, that voice commands as well eye blinking (Aldaz et al. 2015) in combination with gaze were perceived as feasible technology interaction solutions. In order to assess what effect different technology interaction methods have on health care professionals, we conducted a repeated-measures experiment.

The main goal of this experiment was to test user performance, satisfaction, and technology acceptance of health care professionals for different types of 3D interaction with smart glasses in the context of wound documentation.

Data associated with technology acceptance and satisfaction outcomes were analyzed with a repeated-measure ANOVA test with three within-subject factors as independent variables: The tool wound managers are using in their current process (1), voice commands (2) and eye blinking (3) using the wound management application on the HoloLens. To test differences between the treatments contrast tests, based on the Wilcoxon signed-rank test were used (Wilcoxon 1946). All significant results (p < .05) of the contrast tests are reported in the “Comparison”-column of Table 1. In addition, Table 1 also reports means, variances and completion times. Due to problems with the logging system on the HoloLens only n = 33 datasets could be used for the evaluation of the completion times. In addition, the completion times of the current process were not measured during the experiment. The results of the experiment are reported in Table 2.

Gregor et al. specify a 2 × 2 framework of design science research contexts that can help to classify research endeavors into the quadrants of invention, improvement, exemption and routine design (Gregor and Hevner 2013). The axes of this framework are application domain maturity and solution maturity. The health care sector and specifically the process of wound management are a well-established problem context. Yet, AR smart glasses are an emerging technology that currently still inhibit at a low solution maturity. Thus, our research effort falls into the improvement quadrant. The goal of Design Science research in the improvement quadrant is to create better solutions in the form of more efficient and effective products, processes, services, technologies, or ideas (Gregor and Hevner 2013).

Health care providers all over the world are faced with the challenge to improve patient outcomes while containing costs. The digital transformation is recognized as a key component to tackle this challenge (Gopal et al. 2019). Due to hygienic requirements hands-free technology interaction is often required in the health care sector (Hatscher et al. 2017). Established smart devices like smartphones and tablets are not an optimal fit. Since it is possible to use AR smart glasses hands-free, we deem evaluation of AR smart glasses for process improvements in the health care sector to be a worthy research endeavor.

A necessary precondition for usage of AR smart glasses in the health care sector is technology acceptance of such devices by the health care workers. We used a design science approach to investigate design principles for AR smart glasses for usage in health care. Design Science research can make contributions on three Levels. These Levels range from specific instantiations in the form of products and processes at Level 1 to more abstract contributions at Level 2 in the form of nascent design theory (e.g., constructs, design principles, models, methods, technological rules), to well-developed design theories about the phenomena under study at Level 3 (Gregor and Hevner 2013). This research makes contributions on Levels 1 and 2. We have built several instantiations of wound management artifacts. From evaluations with these artifacts, we have derived knowledge that we consider to be contributions that specifically apply to the wound management process:

First of all, through an evaluation of a tablet application, we were able to confirm that hand-held devices are not viewed favorably by health care workers for the process of wound management. Secondly, it became apparent that health care workers prefer to fill in the wound documentation in the patient’s room while being able to look at the wound. Third, the evaluation of our two smart glass prototypes showed that health care workers require a measurement feature that allows them to measure wounds hands-free. Finally, our evaluation results suggest that health care workers would accept and be satisfied with using smart glasses for wound documentation.

Our study also allows us to put forward design principles that are not limited to the wound management context but pertain to the broader context of hands-free technology interaction. Previous studies in the fields of 3D-User Interfaces and AR have pointed out that real-world evaluations on usability and effectiveness of different technology interaction methods is necessary (Billinghurst et al. 2015; Datcu et al. 2015). We show that eye blinking in combination with gaze is a viable solution for hands-free technology interaction in the health care context. However, our study also shows that the preferred technology interaction methods are subject to individual preference. A possible solution to address this would be to offer alternative technology interaction modalities.

Practitioners will also profit from this research. Our research shows that AR smart glasses are a promising technology for supporting the wound management process. The description of our research and the proposed design principles can help smart glass manufacturers and software developers to design solutions tailored to the needs of health care workers. Since about 900.000 people in Germany require wound treatment on a regular basis this could be a promising business opportunity (Schubert and Köster 2015).

Moreover, the demographic change and increased life expectancy in our society will likely increase the demand for wound treatment and other health care services in the future. Therefore, more jobs are likely to be created in the health care sector. Patient care is an integral part of health care systems (Chandwani 2017). Yet, many health care facilities are already lacking personnel. One way of making job profiles in health care more attractive could be the transformation of existing service systems to smart services systems. Finally, positive innovation cases could incentivize health care providers to invest further into new innovative approaches in order to catch up to digitization levels of other service domains.