Addressing Emotions and Beliefs for Vulnerable Jobseekers with Virtual Reality

Virtual reality (VR) technologies have a number of affordances for education and workforce development. The characteristics of the technology that immerse the user in a digital setting and allow for interactivity with virtual environments, objects, and people make immersive media powerful for learning. This includes visualizing unseen phenomena, situating learning in virtual contexts that feel real, practicing procedural skills in low-stakes environments, and tapping into learners’ intrinsic motivation to engage them (Dalgarno & Lee, 2010). For these reasons, a myriad of VR applications have been developed for workforce learning in recent decades and, as the technology becomes more cost effective and pervasive, new industries are looking to it as an important tool to enhance their workforce development programs (Xie et al., 2021).

One promising use of VR in workforce training is to engage the affective domain of learning, including emotional and motivational dimensions that are intertwined with cognition but often lacking in traditional instruction (Chaffar & Frasson, 2012; Martin & Reigeluth, 2013). VR can give users a strong emotional response including joy, fear, relaxation, awe, and curiosity (Chirico et al., 2017; Markowitz & Bailenson, 2021; Urban, 2022). Studies using VR for learning applications find it enhances motivational dimensions of learning, including intrinsic motivation and self-efficacy, or their belief in their own capabilities (Mayer et al., 2022; Queiroz et al., 2022), and specifically can increase confidence in workforce training (Abich et al., 2021). This study contributes to this line of research by exploring the impact of VR on such emotional and motivational dimensions of workforce development for jobseekers who have been impacted by the criminal justice system, a particularly vulnerable group who struggle with confidence in job interviews due to stigma of their backgrounds (Park & Tietjen, 2021).

Despite an increased interest in VR, research on its effectiveness for learning in authentic workforce development contexts remains nascent. The Immersive Learning Knowledge Tree initiative outlines how research on immersive learning has suffered from a techno-centric approach focused on devices, rather than “the actual uses, practices, and strategies taking place with immersive learning” (Beck et al., 2021, p. p.4). Reviews have found that most research on learning with VR has been conducted in lab experiments comparing VR to less immersive devices, has focused on gains in content knowledge retention or procedural skills, and without grounding its design in learning theories (Abich et al., 2021; Hamilton et al., 2021; Radianti et al., 2020). These studies find mixed results in terms of the impact of VR on learning (Wu et al., 2020), but more consistently document impacts of the technology on learners’ interest, attitudes, and enjoyment of learning over other devices (Huang et al., 2021; Makransky et al., 2019; Parong & Mayer, 2018). Additionally, studies of using VR for workforce training show that it increases learners’ confidence (Abich et al., 2021).

Therefore, questions remain about the most fruitful areas in workforce development to leverage the affordances of VR, as well as why their impact varies across contexts. We explored these questions by studying some of VR’s most promising affordances to address affective dimensions of workforce development for vulnerable jobseekers, namely their emotions and beliefs, and by looking at VR as a pedagogical tool within a broader instructional system. To do so, we conducted a mixed-method case study to investigate the impact of a job interview VR simulation on the emotions, confidence, and self-efficacy beliefs of jobseekers who have been impacted by the criminal justice system. By observing the implementation of this application in an authentic workforce development context, we also addressed questions about how VR use and facilitation vary among participants. The findings contribute to growing interest in using VR in workforce development by providing evidence on how learners experience such programs and suggesting ways the experiences can be more effectively incorporated in training programs. The study also contributes to the need for evidence on applications and practices of VR in authentic learning environments (Beck et al., 2021; Southgate, 2020).

Based on this prior work, this study addresses gaps in the literature on using immersive technologies in workforce development, assessing a VR program that was implemented in an authentic workforce development program with vulnerable jobseekers and investigating its impact on emotions and beliefs related to the job search. Given the potential for VR to be a powerful tool for increasing learners’ positive emotions, beliefs in their ability to succeed, and confidence in job interviewing, we asked how a VR simulation tailored to support jobseekers who have been impacted by the criminal justice system supported their workforce development. Specifically, we asked:

To answer these questions, we conducted a case study of the implementation of Project OVERCOME, a VR application from Accenture that was piloted by Goodwill Industries International reentry programs in 2022. The case study employed concurrent mixed methods, in which both quantitative and qualitative data were collected at the same time (Creswell & Plano Clark, 2018). As a case study, the aim of this research is to explore the experiences of this small sample to portray a rich picture of their experiences with and the impact of the VR program, without prioritizing causal inference and generalizability (Gerring, 2006). Our use of quantitative surveys combined with qualitative interviews was pragmatic in terms of answering research questions about whether participants experienced changes in their emotions, beliefs, and attitudes in a systematic way, as well as how the experience impacted them and to allow themes to surface that had not been previously hypothesized (Creswell & Plano Clark, 2018; Teddlie & Tashakkori, 2010). We were especially interested in triangulating findings between different data sources and analytical methods to assess their validity and raising questions for future research based on any disagreements.

Here we present results of our analysis pertaining to each of our four research questions.

Analysis of the survey data finds that participants’ negative affect scores, which estimate their emotions related to the job search, decreased from before to after using the VR application. Figure 3 illustrates participants’ pre-post scores. Participants’ mean negative affect score was 2.1 before and 1.7 after using Project OVERCOME (see Table 4), meaning they were less likely to report feeling afraid, upset, scared, distressed, and nervous about their job search. On average, participants’ negative affect scores fell by 0.6 points, and the difference from mean pre- to post- scores was statistically significant (t(19)=2.88,p=.01). While on average this appears to be a small change, for many participants the impact moved their negative affect from feeling these emotions “moderately” or “a little” to “very slightly or not at all.”

Results from the qualitative analysis comparing participants’ pre- to post- interviews supports this finding as well, in which we identified six of the eight participants indicated a reduction in nerves or anxiety, often described alongside an increase in comfort. Table 5 illustrates how participants described their emotions before using the VR and after, including how they felt “normal,” “better,” and “confident.” Furthermore, the NLP emotion, sentiment, and tone analysis also indicated a decrease in negative emotions for these same participants based on their pre- and post-interviews, indicating their language decreased in its level of fear, anger, and sadness, depicted in Table 6.

On the other hand, we did not find consistent evidence of increases in participants’ positive affect and emotions about their job search. Figure 3 and Table 4 illustrate how participants’ positive affect scores on the surveys remained stable from pre- to post-VR. However, the analysis of interviews found that participants did describe increased positive emotions like feeling more comfortable (see Table 5). NLP analysis triangulates the increase in positive emotions, also finding an increase in joy based on participants’ interview responses, depicted in Table 6.

Table 6 illustrates how the different sources of data triangulate our findings, as participants who reported decreases in their negative affect on the surveys also exhibited decreased anxiety in qualitative interviews, and NLP analysis also identified decreases in negative sentiment and tone such as fear, anger, or sadness. These participants’ results are highlighted in green. We also find the results triangulate for the one participant who reported feeling less confident and whose negative affect increased, highlighted in red.

We also did not find consistent evidence of change in participants’ self-efficacy beliefs and confidence from before to after using the VR simulation. Figure 3 and Table 4 illustrate how participants reported their self-efficacy beliefs as generally stable from the pre- to post-surveys. Yet when asked on the post-survey what impact the VR experience had on their confidence, participants reported mixed results, with 10 (44%) saying they felt more confident, 12 (52%) reporting they felt equally confident, and one reporting they felt less confident (4%). In the interview analysis, illustrated in Table 5, we find that five of eight participants responses suggested improvements in confidence and comfort, alongside the stronger finding of decrease in negative affective emotions such as anxiety and nervousness. The quotes in Table 5 help illustrate how participants described increases in their confidence but did not necessarily describe increased belief in their capabilities as self-efficacy would be defined. This raises questions about confidence as an emotion versus self-efficacy as a belief.

Analysis of the interviews confirmed that participants had a high degree of negative emotions related to their job search and interviews, with many saying they feel nervous and anxious. Table 7 depicts interview quotes that illustrate the emotional toll of the job search, with Angela expressing fear and nervousness, Heather indicating hopelessness, and Frank and Georgia describing feeling nervous about job interviews specifically. These interviews referred to other emotions on the negative affect scale, illustrating how participants also felt afraid, upset, and at times distressed about the job search, including frequently wondering why they are not able to get job offers or promotion, such as Angela wondering “what am I doing wrong?”

In post-interviews, participants described how the VR simulation felt relaxing, put them at ease, and generally made them feel positive emotions (see Table 8). The way the simulation did this was by taking pressure off having to talk to a real person, as Emma described “knowing the person wasn’t actually directly in front of me.” The way participants described feeling in the VR simulation also indicated how it supported emotions on the positive affect scale, such as feeling alert, enthusiastic, or inspired. For example, Georgia’s post-interview quote in Table 5 illustrates how participants who exhibited increased confidence also felt more inspired and enthusiastic, as she described wanting to walk out of the training and apply for every job. Their experiences of the VR simulation itself also indicate a feeling of being alert, including how Brian describes feeling like the interviewer’s presence made him “want to sit up.”

When participants described their experiences with the VR application, they discussed how the responsiveness of the interviewer made it feel like an authentic task, and for most of them they felt the scenario and task felt authentic to them. For example, they described how his facial expressions and body language made it feel as though it was a real interview. They also described how answering the questions in the role of Nadia felt authentic because it related to their own situations and they could empathize with her (Table 9).

We observed variation in the facilitation and use of the VR program, summarized in Table 10. For example, only one of the facilitators encouraged participants to use the journeys portion of the VR application and reviewed the feedback with participants after they completed the interview simulation, resulting in two participants receiving this facilitation (Denise and Frank). This facilitator also asked participants to reflect on what they were seeing and hearing in the VR program before they started the simulation, asking what they thought about the people’s stories in the Journeys and what advice from the career counselor resonated with them. However, we observed the program did not accommodate this discussion as the voice activation of the VR application made it difficult for the facilitator to talk to the participant while the application was running. Other facilitators showed participants how to use the VR headset and start the application without any other engagement but did not instruct them to use the journeys and did not give them feedback during or after. Additionally, participants varied in whether they reached the elevator speech portion of the simulation, with five of the eight participants practicing the elevator speech. This shows how the branched narrative also contributes to variation in how participants use and experience the application.

The participants in this study are too small a sample to connect the characteristics of the facilitation and use of the program to their outcomes, as no clear pattern exists in the differences between use of journeys, the elevator speech, or feedback and changes in their beliefs, emotions, and confidence. However, the variation in facilitation and use raises questions about aligning the goals of a VR experience with what participants will engage in. During this implementation of Project OVERCOME, there were time constraints as facilitators attempted to get as many people as possible to use the program. If using the journeys and practicing the elevator speech are necessary to achieve the desired outcomes, however, these findings suggest participants need more time and potentially to use the application multiple times.

We find compelling evidence that a VR job interview simulation tailored to the experiences of people impacted by the criminal justice system can alleviate some of the emotional toll the job search takes on this vulnerable population. This impact was apparent across our data sources, from surveys, qualitative interview analysis, and NLP interview analysis. Because participants described negative emotions like anxiety and nervousness in interviews, this seems to be an especially promising use of immersive technologies in workforce development. This finding confirms prior research on the emotional impacts of VR experiences (Markowitz & Bailenson, 2021), and helps address calls for more research focused on the affective impacts of VR on learning and workforce training rather than a narrow focus on knowledge and procedural skills outcomes (Hamilton et al., 2021). It also addresses the need within immersive learning research go beyond a techno-centric view of the impact of VR on learning and instead provides a study of the technology within context that attends to a specific population of learners (Beck et al., 2021).

Additionally, the mixed method approach of this study goes beyond a “black box” approach of looking at the changes in participants’ outcomes from before and after using VR to also look at how its design and implementation affected their use and outcomes. We find that participants described the simulation feeling authentic both in the environment and the task. Interview responses suggest the branched narrative structure allowed for a realistic feeling as the interviewer was an actor filmed to respond to each interview answer in an authentic way. Participants also indicated the scenario of role playing someone who has been incarcerated spoke to their own experience and was relatable. Together, this suggests a personalized VR experience that is tailored to the needs of a specific population may be especially effective at making them feel relaxed, comfortable, and less anxious about job interviewing. Our findings that such a branched narrative structure can feel authentic to participants contrasts many in VR who call for experiences to be maximally interactive and freeform in order to make full use of the technology’s immersive qualities (Lanier, 2017), and rather supports work showing how narrative and symbolism drive immersion as well (Dede, 2009; Harrell, 2013; Slater, 2009).

On the other hand, we did not observe an increase in their self-efficacy, or more deeply held beliefs about their capability to perform well in an interview. These findings contradict other studies which have shown even brief VR experiences to have an impact on learners’ self-efficacy beliefs, motivation, and even identity (Johnson-Glenberg et al., 2023; Mayer et al., 2022; Queiroz et al., 2022). One explanation for the mixed findings is that the results are sensitive to whether the measures are capturing something more immediate like the emotional responses we identified, or more long-term and deeply held beliefs outside the VR experience itself. Another is that the population matters, as does the task they engage in using VR. Other studies have been conducted with younger learners or with less high-stakes activities than preparing for a job interview.

We also find that how the program is facilitated is likely to affect how participants use it, including whether they spend time reviewing feedback on their performance. This highlights the need to consider VR as a pedagogical tool best implemented within a broader program. While some use cases of VR see it as a replacement for skills training, our findings suggest it be used to augment existing training. In the case of interview simulations for vulnerable jobseekers, this may be an intervention used to help alleviate participants’ anxieties, akin to VR programs used in exposure therapy to treat phobias (Parsons & Rizzo, 2008). However, our observation of the use of this program suggests that participants also need instruction on interview techniques and skills practice with an instructor.

Our use of multiple methods to triangulate our findings also provides a promising way to measure participants’ experiences and outcomes in larger samples by collecting both survey data and open-ended data that can be analyzed with NLP. Our findings suggest these methods are complementary, and that a simple survey measure asking participants if they feel more confident can be a meaningful measure that aligns with interview data. We therefore help contribute methods for researchers to measure the impact of VR on affective dimensions of workforce learning, in particular an application of the PANAS, measures of jobseekers’ beliefs such as self-efficacy, and the usefulness of text analysis for analyzing qualitative data. In this way, our study also contributes to the Immersive Learning Knowledge Tree initiative, which calls for greater sharing of fruitful research methodologies and instruments for the study of VR in education and training (Beck et al., 2021).

These findings point to a number of implications for research on and implementation of VR in workforce development contexts. For one, future research should continue to use multiple methods to develop and validate measures of participants’ emotions and beliefs in response to VR programs, including teasing apart the distinctions between emotions, beliefs, and confidence. Further, VR remains a technology suited to brief interventions, and it may not be as impactful as longer-term experiences. Our study raises questions about the appropriate “dosage” of job interview simulations, as well as the likelihood of transfer to an interview context. Future work should investigate whether workforce development VR programs are effective at shifting participants’ skills and beliefs if used multiple times, especially if participants receive feedback and have time to implement it. This research would also help to address questions as to whether affective outcomes are impacted by the novelty of the technology similarly or differently from learning outcomes and sense of presence in VR (Huang et al., 2021). Future research should also investigate whether the emotional impacts of VR transfer to real interview environments, or if the relaxing and comfortable feelings only apply in VR because participants are aware it is not real. This would extend research on transfer in VR beyond narrow learning outcomes like content retention and procedural skills performance to the affective domain of learning (Hamilton et al., 2021). Drawing on VR exposure therapy treatments to help participants face their fears over time may be particularly fruitful (Parsons & Rizzo, 2008).

The findings also have implications for practice in suggesting ways VR can be most effectively implemented. An experiential learning framework may be particularly beneficial at helping participants prepare for the simulated interview (Petruzziello et al., 2021; Tay et al., 2006). For example, participants could receive instruction and coaching, plan what they will aim to achieve within the simulation, engage in the VR experience, and then receive and review feedback on their performance. They could then repeat this cycle by processing the feedback to plan new goals, engage in the simulation, and receive feedback on their progress. The current implementation observed did not allow for the structure of planning and reflecting on feedback needed to engage in this cycle. The implementation of such immersive VR simulations could draw on less-immersive simulations with similar populations to do so, such as Smith et al. (2022) which allowed incarcerated individuals to use the simulation multiple times at increasing levels of difficulty.

This exploratory study does have limitations. The case study was not designed to isolate causal mechanisms, and therefore the effects observed may not be generalizable to other contexts, populations, and VR applications. We may not have observed statistically significant changes in self-efficacy and positive affect due to the small sample, or the measures may have been too blunt to capture changes, as participants’ baseline levels were relatively high to begin with. Further research should assess the impact of job interview simulations on participants’ emotions and beliefs on larger samples and in other contexts to replicate these findings, as well as measure learning outcomes to model the impact of changes in beliefs and emotions on interviewing skills. Additionally, this study is not able to validate longer-term changes in participant emotions or beliefs, nor can it account for any effect the novelty of the technology may have had on participant outcomes, as we were unable to track participant outcomes over time and the VR program was only offered to participants for a single session. Future research should investigate the usage of the VR and participants’ outcomes over time to validate our findings of the impact of VR on jobseekers’ emotions.

This study employed both quantitative and qualitative methods to investigate to what extent and how a VR job interview simulation impacts vulnerable jobseekers’ emotions and beliefs. By conducting a case study of one site utilizing VR in an authentic workforce development program, it contributes to a need for more research in learning environments and focused on the actual use and practical application of VR in context. The findings highlight the promise of VR to help jobseekers reduce their negative emotions surrounding the job search and increase their feelings of confidence. On the other hand, we did not find consistent evidence that the simulation increased participants’ self-efficacy beliefs, raising questions about the difference between confidence as an emotion and self-efficacy as a belief. This suggests the potential for VR to engage affective dimensions of learning depends on how such dimensions are defined and measured, the way the application is implemented including giving participants feedback and incorporating sound instructional design, and the length of an intervention in terms of its dosage. We conclude that future VR for workforce development designs should consider how they can impact the affective dimensions of learning such as easing negative emotions, and that such programs should consider incorporating VR into instructional models such as experiential learning. Our findings also raise questions for future research such as how to best measure changes in participants’ affect and beliefs using surveys, interviews, and NLP, testing the association between VR simulations and participants’ emotions at a larger scale, and assessing the impact of VR as its novelty wanes.