Basic NeuroscienceBuilding virtual reality fMRI paradigms: A framework for presenting immersive virtual environments
Highlights
► We develop informatical concepts, which allow the creation of own VR-fMRI paradigms. ► We develop neuroinformatical techniques which provide real-time VR-fMRI studies. ► We embed an easy-to-handle integration concept for virtual environment files. ► We validate the application in a real-time VR-fMRI study with spatial memory topic. ► Subjects indicate higher interaction and more attention than in common fMRI studies.
Introduction
In the last years, several functional magnetic resonance imaging (fMRI) studies used virtual reality (VR) environments when presenting their stimuli. Virtual reality refers to an artificial computer-generated environment, with which users act and interact as if in a known real environment. Furthermore, users can experience things that would otherwise be very difficult or even impossible in a magnetic resonance scanner or with an electroencephalograph.
The big advantage of virtual environments lies in the presentation of realistic stimuli. Instead of passively watching a simple movie stimulus, subjects can interact actively with the paradigm, for example, navigating and exploring an artificial environment. Referred to this, several virtual environment studies have been used to investigate the role of the hippocampus and the parahippocampal area in topographical, spatial and episodic memory processes (Aguirre, 1998, Maguire et al., 2006, Mellet et al., 2010). In addition, the location of so-called place cells in the human brain was researched with virtual environments (Aguirre et al., 1996, Doeller et al., 2010).
Another example of how virtual environments are used in neurosciences are neurofeedback studies and human–brain-interfaces (HBI). By using operant conditioning, subjects learn to regulate their own brain activation with the aid of a neurofeedback signal. In such cases, fMRI image data need to be analyzed in real-time, which means as fast as they are acquired, i.e., within a single repetition time (TR) (Gembris et al., 2000, Weiskopf et al., 2007, Scharnowksi et al., 2009). Specific examples include a classification algorithm to control the movement of an avatar in a two-dimensional maze (Yoo et al., 2004) or reducing the pain intensity in the right anterior cingulate cortex with neurofeedback training and a virtual flame (DeCharms, 2005).
Similarly, several psychological studies used virtual environments in their therapies: burn pain therapy (Hoffman et al., 2000), reduction of claustrophobic symptoms (Garcia-Palacios et al., 2007), and therapy of posttraumatic stress disorder based on the virtual reality exposure therapy (Rothbaum et al., 1999). The basic aspect of these psychological therapies is the immersive experience, which helps to evoke relevant neuronal activity and provides the essential feeling of being inside of a real world.
Alongside these benefits, the main difficulty of using virtual reality in neuroscientific experiments are the time-consuming implementation, the combination of neuroscientific and computational methods, and the integration into an existing acquisition framework. In spite of the increasing number of neuroscientific studies using virtual reality, few virtual environment stimulus applications can handle such experiments in their entirety. Since the technical complexity and the required knowledge in computer sciences demand high personnel and financial resources, several scientific groups work with virtual environment stimuli using static environments from computer games. These game frameworks contain only one virtual environment, whose technical restrictions dictate the design of the neuroscientific experiment. Our aim was to implement an adaptive, flexible, and extendable virtual environment stimulus application that includes several scenes for fMRI paradigms. The user should be able to import a VR scene with just a few mouse clicks and present this scene to a subject. Furthermore, the application should provide a highly realistic stimulus output which fulfills the immersive requirements necessary for using virtual reality in psychological therapies (Regenbrecht et al., 1998). To support real-time fMRI experiments we integrated functionalities for exchanging data values with external applications (Mathiak and Posse, 2001).
Section snippets
Experimental infrastructure
For the presentation of virtual environments we used a multimodal stimulus environment concept that comprises three basic systems. See Fig. 1 for a detailed overview of this concept.
The first basic system is the presentation system which is represented by the virtual environment stimulus application. This system provides visual and auditory stimuli and presents them to the subject.
The second is the data acquisition system represented in our case by the magnetic resonance imaging scanner which
Application performance tests
The highest layer in the hierarchy, the user interface layer, was tested in several capacity tests and stress tests. All paradigms settings either from generated test data or from the VR-fMRI experiment were loaded and processed successfully. The data obtained were used to analyze the paradigm creation process and the consistency check algorithm embedded in this software layer. In addition, the observation of corresponding process calls and the evaluation of memory usage showed no data loss or
Discussion
The virtual environment stimulus application was successfully implemented and evaluated during 12 real-time fMRI experiments with four subjects and several performance tests. Neuroscientists were able to design and to load several fMRI experiments either in block design or event-related design and could apply experimental settings to the provided virtual environment scenes.
The implemented plugin framework provides high expandability and a custom access to internal functionalities. Plugins have
Acknowledgement
This work was supported by the Ministry of Education, Saxony-Anhalt, Germany. Grant-number: 5162/AD/0308T.
References (27)
- et al.
A new concept of a unified parameter management, experiment control, and data analysis in fMRI: application to real-time fMRI at 3 T and 7 T
J Neurosci Methods
(2008) - et al.
Real-time functional magnetic resonance imaging: methods and applications
Magn Reson Imaging
(2007) Neural components of topographical representation
Proc Natl Acad Sci USA
(1998)- et al.
The parahippocampus subserves topographical learning in man
Cereb Cortex
(1996) - et al.
Event-related functional magnetic resonance imaging (efMRI) of depth-by-disparity perception: additional evidence for right-hemispheric lateralization
Exp Brain Res
(2009) - et al.
A virtual reality system for neurobehavioral and functional MRI studies
Cyberpsychol Behav
(2003) - et al.
Evaluation of spatial processing in virtual reality using functional magnetic resonance imaging (fMRI)
Cyberpsychol Behav Soc Netw
(2010) - Brain Innovation. Turbo BrainVoyager 2012. URL...
Control over brain activation and pain learned by using real-time functional MRI
Proc Natl Acad Sci USA
(2005)- et al.
Evidence for grid cells in a human memory network
Nature
(2010)
Computer architecture performance evaluation methods
Synth Lect Comput Archit
Use of virtual reality distraction to reduce claustrophobia symptoms during a mock magnetic resonance imaging brain scan: a case report
Cyberpsychol Behav
Functional magnetic resonance imaging in real time (FIRE): sliding-window correlation analysis and reference-vector optimization
Magn Reson Med
Cited by (41)
Evoking stress reactivity in virtual reality: A systematic review and meta-analysis
2022, Neuroscience and Biobehavioral ReviewsCitation Excerpt :Meta-analyses on current literature in VR stress research provides an avenue to expand knowledge of stressors beyond the VR-TSST. This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines (Moher et al., 2009). Relevant literature was searched for, identified, and screened.
In search of a naturalistic neuroimaging approach: Exploration of general feasibility through the case of VR-fMRI and application in the domain of episodic memory
2022, Neuroscience and Biobehavioral ReviewsCitation Excerpt :Finally, many options make it possible to adapt immersion and optimise the sense of presence in the VE in fMRI. Granted, to adapt immersion, a good real-time performance is required (Mueller et al., 2012) and to enhance realism, a qualitative game such as Unity 3D is an asset. The view of the VE has to be adjusted in real-time as the subject evolves through it by calculating the interactions and/or navigation between the subject and the VE, and the computer used for the VE should not lose any performance or data over time.
Movies and narratives as naturalistic stimuli in neuroimaging
2021, NeuroImageNeural representations of social valence bias economic interpersonal choices
2020, NeuropsychologiaStudying the visual brain in its natural rhythm
2020, NeuroImage
- 1
Institut Für Biometrie und Medizinische Informatik, Medizinische Fakultät, Universität Magdeburg, Leipziger Str. 44, 39120 Magdeburg, Germany. Tel.: +49 391 67 13537; fax: +49 391 67 13536.