Extended Reality

Virtual reality (VR) technology has emerged as a promising tool for rehabilitation, offering innovative approaches to address conditions such as Carpal Tunnel Syndrome (CTS). This study presents the development and validation of a Virtual Rehabilitation System tailored specifically for CTS rehabilitation. The system integrates VR environments created using Unity graphics engine with haptic equipment to capture real-time hand movements and virtual reality headsets for user interaction. A diverse sample of individuals diagnosed with CTS participated in the validation process, undergoing a series of rehabilitation sessions using the developed system. Results indicate significant improvements in finger mobility and overall usability perception, with a System Usability Scale (SUS) score of 83/100. The proposed system demonstrates favorable outcomes in terms of usability and effectiveness, supporting its potential integration into clinical settings for CTS rehabilitation. Future research may explore advanced VR techniques and AI integration to further enhance rehabilitation outcomes and extend the application of VR in physiotherapy beyond CTS.

In recent years, research has increasingly explored the efficacy of art therapy for conditions such as stroke, uncovering benefits like the “Michelangelo effect”, which enhances motor performance in individuals exposed to aesthetically pleasing artwork during specific activities. While standardized stimuli are often used, their clinical effectiveness is questionable. Providing a positive experience with subjectively appreciated visual stimuli is crucial, as it enhances patient engagement and therapeutic outcomes. For this reason, our study aims to understand which paintings, starting from an existing dataset, are most appreciated and which characteristics make them more interesting for observers. We enrolled 59 healthy subjects who evaluated 60 abstract paintings based on pleasantness and element salience. Results highlighted preferences for warm colors, prominent elements, and a sense of movement influencing aesthetic appreciation. Possible hypotheses of these results are linked to the importance of spatial attention, depth perception and visuo-motor integration when viewing paintings. These findings informed the selection of stimulus sets for future motor neurorehabilitation protocols.

Markerless systems and inertial sensors, such as inertial measurement units, offer significant advantages over traditional marker-based methods by eliminating the need for laboratory conditions or markers placement. Virtual reality is an innovative technology used in rehabilitation, recently investigated for obtaining kinematic motion data. This research aims to validate a markerless system using a MediaPipe-based algorithm to reconstruct joint kinematics during exercises with the Nirvana virtual reality rehabilitation system. In the first phase, the markerless system was evaluated against the Xsens inertial measurement unit system, analyzing knee, hip, elbow and shoulder movements of 7 healthy participants at the Politecnico di Milano. The MediaPipe algorithm showed high performance in tracking lower limb movements but was less accurate for upper limbs, as indicated by RMSE and %RMSE values. However, it accurately replicated the time course of all angles based on ICC and Spearman’s coefficient, suggesting virtual reality’s feasibility in rehabilitation and movement analysis. In the second phase, a pilot study was carried out at the Fondazione TOG on 2 participants with left hemiparesis to assess the efficacy of the system in a pathological context. The results provide a basis for evaluating the potential clinical applicability of the system.

Emerging technologies like virtual reality (VR) and telerehabilitation (TR) may provide numerous advantages for treatment of patients with neuromuscular and musculoskeletal disorders. Spinal muscular atrophy (SMA), a progressive neuromuscular disorder presents challenges in motor function rehabilitation. However, to date no studies have evaluated the use and usability of new technologies in the treatment of patients with SMA. This case study examines the effectiveness of a particular VR-based telerehabilitation system, Khymeia – Virtual Reality Rehabilitation System, in improving upper limb motor recovery in a 13-year-old patient with Type 2 SMA who is receiving gene-based therapy (GBT) with Risdiplam. Motor function was assessed using the Revised Upper Limb Module (RULM), and usability of K-VRRS was evaluated with the System Usability Scale (SUS). The results indicated stable motor function during the baseline phase, with significant improvement observed at the post-treatment assessment. The SUS score reflected high usability of the K-VRRS. This suggests that combining GBT with VR-based telerehabilitation could enhance upper limb motor function in SMA patients.

Further research is needed to confirm these findings and investigate the potential of K-VRRS in enhancing patient motivation and treatment adherence in SMA rehabilitation.

Many applications involving virtual humans in the area of hearing disabilities focus on sign language, disregarding other rehabilitation methods. This article presents Labio, a highly immersive 3D virtual environment designed to support hearing rehabilitation methods based on visual strategies such as lip reading. The Labio application offers a range of exercises primarily for the learning phase. These exercises are intended to be used in conjunction with a speech therapy program, where the therapist will guide the subject in performing the exercises.

This research focuses on the integration of augmented reality (AR) technology into Supervisory Control and Data Acquisition (SCADA) systems for industrial applications. A SCADA system in an AR environment was developed, enabling users to monitor and control industrial processes through an immersive and interactive interface. The system’s architecture and implementation methodology are described, including communication protocols, data exchange, and 3D object manipulation. A mobile application was also implemented to enhance connectivity and accessibility. The usability and functionality of the AR SCADA system were validated through user surveys and statistical analysis. The results demonstrated that the application meets established usability parameters, providing an intuitive interface and satisfactory performance in controlling and monitoring industrial variables. The successful integration of AR into SCADA systems presents opportunities to enhance operational efficiency and safety in industrial environments. This study highlights the potential of AR technology in modernizing industrial control systems and paves the way for further advancements in this field.

Traditionally, robot control is done through fixed workstations or wired communication, which affects system performance and flexibility. This limitation is overcome by augmented reality by offering intuitive and flexible user interfaces that allow controlling and monitoring robots remotely and in real-time, freeing users from physical constraints and facilitating dynamic interaction with robots in various environments and situations. This paper presents the development of a mobile AR system, using Unity and Vuforia to control the UFactory Lite 6 Robotic Arm. Three modules were built i) Overview, ii) Test; and, iii) AR. Two processes were simulated: pick and place and welding, using an API for communication between the cell phone and the robot. HTTP and TCP protocols were used for the connection between the different components. Latency analysis using Wireshark revealed execution times of approximately 6.54 ms and 2.24 ms for the welding and pick and place processes, respectively. Finally, a usability test (SUS) was performed with a group of five people, obtaining a score above 70, indicating that the system is adequate but requires improvement.

This paper proposes the development of a virtual system as a tool for teaching and learning the ISA 5.1 standard, which is used for symbology and instrument identification in piping and instrumentation diagrams (P&IDs). A teaching methodology based on level control for water inlet into a steam boiler, characterized in the Unity graphics engine, is proposed. The immersive virtual environment allows students to be trained in subjects related to process control and industrial instrumentation. The teaching methodology involves interaction between the virtual environment and the P&ID, guiding students to recognize the symbology used in the standard and the location of instruments within the virtual environment, and then to recognize the real instruments in the didactic station. The results obtained show the utility of the virtual system as a tool in teaching the ISA 5.1 standard and the improvement in student performance.

In this article, a 3D virtual system of a two-phase natural gas separator was developed in Unity 3D software modeled from the piping and instrumentation diagram (P&ID), the created animations and instrumentation (manometers, Smart transmitters, smart control valves, human machine interface) are accord to the current technology that are preset in the oil industries, A mathematical model of the gas pressure process was scripted through a TCP-IP communication interacts with a PID controller developed in LabVIEW software. The proposed methodology for the research allows operators to be trained in automatic control of the two-phase separator, through the visualization of indicators and signals of the process’ variables presented in the industrial transmitters, change of valve positions (close and open), visualization and automatic control of the pressure’s variable for trend and history screens are available in the Human Machine Interface (HMI).

The construction industry, much like other labor-intensive fields, faces a challenge with its continually diminishing pool of experienced workers. Currently, there is a steady decline in skilled workforce availability, creating a shortage just when their expertise is most needed. Typically, mastering the necessary skills in this industry requires years of experience, which is not feasible given the immediate and crucial demand for workers. Additionally, the industry grapples with a reputation for subpar quality. To address these issues, this research proposed a new approach to training a novice worker, by focusing on enhancing their ability to properly locate and place construction materials. The study utilized augmented reality technology to guide students in placing plumbing pipes accurately within a simulated construction environment. Students found the experience highly educational. It provided them with a sense of practical work experience crucial for their learning process. This study introduces a valuable tool for student education and suggests potential for exploring more advanced technologies to improve accuracy – with the secondary intent of providing a tool for practitioners to use while conducting construction industry inspections. It underscores the importance of innovative approaches in preparing the next generation of construction workers.

The paper presents the development of a learning tool to enhance the teaching of industrial process control. The implementation of the Hardware-in-the-Loop technique in the teaching-learning process allows students to interact with a virtualized industrial process, understanding the real behavior of actuators and sensors. To comprehend, modify, and control the process, this work demonstrates the design and implementation of a water bottling plant. The developed tool enables real-time communication between the simulation part and the physical part. The simulation is based on the development of a virtual environment in Unity, and the process control is managed through the Ladder language in TIA Portal v16. The physical part of the process control is developed using a didactic module consisting of a PLC 1200 S7 (1214 AC/DC/RLY version 4.2). The learning tool was evaluated through an acceptance survey conducted with a group of 20 students, yielding a high level of acceptance. It was determined that control in the virtual process is intuitive and easy to understand, making the system easy to manage.

The Virtual Electrician Training (VET) system, designed to provide effective and innovative technical training in a virtual reality (VR) environment, allows trainees to practice operational tasks safely and realistically on an electrical grid. By integrating advanced VR technology with flexible tools for managing training content, the VET system offers immersive and interactive simulations that enhance skill acquisition and operational proficiency. Trainees engage with detailed 3D models and scenarios of electrical infrastructure, including transformers and switchgear, enabling them to perform complex procedures such as circuit switching and live-line work in a controlled and risk-free setting. The system employs a semantic database managed by Apache Fuseki, which seamlessly links 3D models, scenes, and training scenarios, facilitating efficient content creation, modification, and updates. Key components include a central server, stationary and mobile training stations equipped with VR headsets and controllers, and auxiliary equipment like motion sensors and communication devices. The VET system’s architecture supports easy scalability and adaptability, ensuring that training programs remain current with industry standards and evolving operational requirements. This comprehensive training tool significantly improves safety, reduces training costs, and enhances the overall quality and efficiency of technical training in the dynamic energy sector.

This paper introduces Model Checker, an open-source plugin for Blender designed to analyse distance discrepancies between 3D models and visualize the results using heatmaps. It is designed for analysing discrepancies during retopology and restoration processes, seamlessly integrated into the Blender environment. The plugin extracts minimum, maximum, mean, and standard deviation distances between models, visualized with heatmaps based on adaptable thresholds. It also provides change tracking in the heatmaps, facilitating comparison between different project versions. Heatmaps are exportable as textures and compatible with immersive Virtual Reality (iVR) visualization within Blender, enabling convenient analysis and visualization within the workspace. Comparative analysis with CloudCompare reveals an error margin of maximum 1mm for Model Checker, making it suitable for tasks requiring similar precision. The results indicate its suitability as a viable alternative in projects requiring millimetric precision restoration, particularly notable for its integration into modelling software, change control, and iVR visualization capabilities. Future development efforts should focus on optimizing the process to enhance both accuracy and analysis time.

The experience of a musical performance in a concert hall transcends the acoustic dimension, engaging all senses in an integrative manner to yield a holistic perceptual outcome. Despite this integrative nature, studies focusing on the auditory aspects of performances, employing spatial audio technologies, virtual acoustics, and immersive audiovisual renderings, contribute to a deeper understanding of performance space acoustics as a form of intangible cultural heritage. XR experiences could be considered not merely a product of research but also a process that could be analyzed through tests and surveys. Testing the model can be useful not only to evaluate its Quality of Experience and the realism of the rendering, but also for accessing data about the object of study, impossible to obtain otherwise. This research examines the application of tests and surveys within the XR project Listen To The Theatre, elucidating its utility and implications.

This paper details and discusses the methodology employed for developing a virtual auralisation of Napoleon Hall, situated beneath the Louvre pyramid, as part of an acoustic consulting study. The aim of the project was to improve the hall’s acoustics and mitigate the disruptive impact of visitor-generated noise during operational hours. The virtual acoustic auralisation was created based on a 3D geometrical acoustic model calibrated using onsite measurements. A 3D visual model of the hall was created in parallel to facilitate the generation of immersive \({360}^{\circ }\) viewpoints corresponding to each rendered scenario. Combining the auralisations and visualisations, a virtual reality application was developed, enabling project stakeholders to navigate through different positions within the hall and explore the various acoustic treatment conditions.

The basilica of SS. Medici represents an extremely important urban hub for the Alberobello community, but today's monumental building appears incomplete: although the original project made by the architect Antonio Curri included a 23-m-high dome, this structure is absent. Luckily, contemporary Extended Reality technologies enable architects and other experts to give the complete sacred space back to the citizens in a virtual way in order to both encourage the development of a more informed debate between community and authorities on the topic of interventions on historical and architectural heritage and promote an architectural intervention.

To achieve this goal a Virtual Reality immersive application has been realized and shown to citizens during a public event. A digital twin of the church has been obtained through photogrammetry. Subsequently, a 3D model of the base and elevation was created on the basis of images and plans and, finally, a three-dimensional file of the dome provided by Professor Fallacara was optimized and put in the project. Once the virtual project was done – joining the 3D dome, roof and walls –, Unreal Engine 4 was used to program a Virtual Reality application.

The evolution of digitisation technologies and the growing transition to the digital age have reshaped the cultural heritage landscape, expanding possibilities beyond digital documentation or additionally immersive interactive experiences. This transition has also fostered innovative approaches for the (re)use of cultural heritage digital data. This paper proposes an integrated approach of digital and virtual technologies, by effectively integrating and (re)using different available digital data with the final aim of reconstructing a selected portion of the Tiwanaku UNESCO complex in a Virtual Environment, with work still in progress. The research methodology involved gathering 2D and 3D digital products for the 3D modeling of two areas within Tiwanaku, namely the Kalasasaya and the Semi-Subterranean temples, and the 3D reconstruction of their monuments. Finally, all the reconstructed elements were integrated into a Virtual Environment where the virtual experience goes beyond mere site navigation. It provides an immersive and interactive heritage learning experience through object interaction, historical context storytelling, and also offers a unique opportunity to imagine how the an-cient city of Tiwanaku looked like by introducing some 3D-reconstructed monuments to the modeled site in their original configuration and position.

The quick development of generative artificial intelligence (GenAI) techniques is a promising step toward automated processes in the field of cultural heritage (CH). The recent rise of powerful Large Language Models (LLMs) like ChatGPT has made them a commonly utilized tool for a wide range of tasks across various fields. In this paper, we introduce LLMs as a guide in the three-dimensional (3D) panoramic virtual tour of the Civic Art Gallery of Ascoli to enable visitors to express their interest and show them the requested content. The input to our algorithm is a user request in natural language. The processing tasks are performed with the OpenAI’s Generative Pre-trained Transformer (GPT) 4o model. Requests are handled through the OpenAI’s API. We demonstrate all the functionalities within a developed local web-based application. This novel approach is capable of solving the problem of generic guided tours in the museum and offers a solution for the more automatized and personalized ones.

Museums play a crucial role in preserving society's history and heritage. Recent advancements in Virtual Reality (VR) offer exciting possibilities for creating immersive, interactive experiences. This study explored the acceptance of a VR multisensory system integrated into a public weapons exhibition. Using a haptic bow in a virtual medieval environment, 60 tourists participated in the study, evaluating immersion, engagement, intention to use, and social interaction through an adapted questionnaire. Statistical analysis revealed significant correlations between factors, confirming the proposed research model. Interestingly, engagement exerted a stronger influence on user intention than immersion, suggesting that engaging VR experiences are more likely to motivate museum visits. While immersion had a statistically significant impact, it did not fully explain the variation in user intent to use the system. This research highlights the potential of VR for enhancing museum experiences and provides valuable insights for future VR implementation strategies.

In today's landscape, numerous factors threaten diverse cultural heritage assets, whether they reside within museums and galleries or as historical landmarks. Current geopolitical challenges such as the COVID-19 pandemic, military conflicts, and food crises contribute to financial instability, reducing efforts to conserve and promote cultural heritage. However, with the emergence of blockchain technology, there is a potential avenue to address this situation. The system proposed in this research targets three key groups: the initial stakeholders, including museums, galleries, local and national authorities, and companies; investors, such as the general public, companies, and NGOs; and visitors. This system facilitates financing for cultural objectives through smart contracts, allowing investors to support curated lists of cultural assets provided by the initial stakeholders. These assets are digitized into unique digital forms and offered to investors during the initial “acquisition” phase. Regular visitors can then use an application to rate these assets during the exploitation phase. Based on these ratings, investors can receive an income proportional to the score given by visitors within the bounds of a portion of the entrance ticket proceeds. Through a case study, we have observed promising results: initial capital can be generated to preserve cultural heritage while investors become advocates for these assets, actively raising awareness. Visitors are incentivized to support what they enjoy, knowing that part of the funds directly benefits the owners, backers, or investors of those specific assets. Moreover, digital assets (e.g. digital twins, XR heritage) can be included within the proposed framework with no additional effort. Future studies will explore methods to reduce system costs, potentially leveraging technologies like Optimistic/ZK Rollups and integrating augmented/virtual reality to enhance this system further.

This contribution stems from research on the enhancement of museum houses through the use of augmented reality, specifically expanding the modes of viewing a space. This study involves an interdisciplinary relationship between architecture and digital humanities by analyzing two case studies. The first is Kundmanngasse Haus (Ludwig Wittgenstein, Vienna, 1925–1929), a partially musealized house without the use of new technologies, a case study propaedeutic to get to know architecture whose design process, experience of use, and interactions are mediated by computers and devices. This case study is also exemplary as an intersection of philosophy and architectural theory. The second considered case study is Casa Batlló (Antoni Gaudí, Barcelona, 1904–06), a pioneering museum in the application of augmented reality to a museum house. Despite considering different architectural styles and theoretical premises, the case studies have more than something in common, especially in identifying elements that enrich the experience of cultural heritage through the use of new technologies, particularly with augmented reality.

This paper discusses the design of a system for Extended Realities (XR) in live arts performances. XR, a broad concept and medium, enhances the capabilities of Virtual and Augmented Reality (VR/AR) by incorporating spatial audio and multiple projections for audiences. In live performance scenarios, XR has the potential to define a novel, specific type of audience engagement, which is largely unexplored. This paper specifically examines the modular structure of the system, prioritizing a high-level description over detailed implementation specifics. However, details are included as needed for clarity and to provide context within the XR medium. According to the design, a VR/AR performer wearing a headset can interact with virtual objects, controlling the audio and visuals generated during the performance. The system is divided into modules dedicated to specific functions: audio generation, video generation, event scheduling, parameter mapping, and inter-machine connections (typically, the system requires several machines to run in parallel due to performance and input/output necessities). These modules are overseen by a main module, the VR/AR performer module.

The system is designed to explore and leverage specific affordances of XR as a medium. It envisions XR performance as a stratified experience of multiple realities, such as the simultaneous existence of VR/AR and screen-based content. Moreover, it ensures that these different types of content are connected and interdependent, a feature crucial for delivering immersive experiences to audiences who cannot directly experience VR/AR content. To achieve this, a dynamic ecosystem of devices is supported to accommodate various perspectives and dimensions of reality. The system has been entirely designed to cater to these needs and facilitate rapid content creation and prototyping.

Digital transformation is reshaping business strategies and innovation, by creating new opportunities in different industries. Digital technologies, like eXtended Reality (XR) and Internet of Things (IoT), are rapidly advancing in the field of virtual heritage and tourism industry for improving visitors’ experiences, facilitating user interaction, and promoting cultural knowledge. Despite several digital solutions are proposed in the Cultural Heritage (CH) literature, there is still the need to address some limitations, including the need of differentiating commercial offers in the market. Before designing a digital solution, such as a cloud-based platform for tourists, it is fundamental to identify the key needs and expectations of companies operating in the CH and tourism industry. Through a survey-based questionnaire, this paper aims to investigate the most relevant features, services, and functionalities that CH companies expect to integrate within such solutions. Findings reveal the need to implement cloud-based platforms with XR technologies, IoT, sensors and smart devices to gather data and provide with an immersive, recommended, and customized experience to visitors.

In the intersection of archaeological heritage, performing arts, and technology, the E.T.E.R.I.A. project exemplifies a new experimentation of methods and tools: the aim is to integrate extended reality technologies into social theatre and archaeological heritage across Europe. Spearheaded by the Italian Centre of the International Theatre Institute (ITI Italy) and supported by the Creative Europe Program, it fosters innovative models for audience engagement and conflict transformation through immersive storytelling. The project explores themes of conflict and cultural heritage through a series of co-working initiatives and artistic residencies in historical sites. Central to the project approach is the integration of cutting-edge XR technologies, enhancing traditional theatre with interactive experiences that deepen the connection with audience and the narrative impact. This contribution focuses on the AVR Lab’s contribution that drives the creation of immersive installations for the project’s performance, by means of digitization of archaeological heritage and Spatial Augmented Reality techniques. By leveraging digital innovation, E.T.E.R.I.A. aims to transcend boundaries, stimulate dialogue, and enrich community interactions through cultural heritage and transformative artistic encounters.

The convergence of technology and cultural heritage has inaugurated a new era of exploration and engagement within museums and historical sites worldwide. Extended Reality (XR) is at the forefront of this transformation, revolutionizing traditional museum experiences by offering interactive, multi-dimensional encounters that surpass the limitations of static exhibits. In this scenario the MAB Institution of Gallipoli (Museo, Archivio e Biblioteca) with the project titled “Gallipoli, qui dove finisce il mare ed inizia la terra”, supported by the Ministry of Culture’s “Fondo Cultura,” epitomizes the commitment to enhancing cultural heritage through new digital media. By integrating virtual elements and employing user-centered design principles, the project aims to create dynamic, immersive experiences that deal with diverse audiences locally and globally. Through strategic collaborations and technological advancements, this project strives to extend the tourism season, deepen visitor engagement, and reinforce Gallipoli’s identity as a cultural destination steeped in historical richness. This paper provides a first study of the design and development of digital experiences within Gallipoli, by a description of the XR applications developed and the methodology adopted.

In recent years, there has been increasing talk about the Metaverse, eXtended Reality, and the educational opportunities that can result from the crossover of these technologies. Recent studies on the use of Virtual, Augmented, and Mixed Reality in educational processes aim to understand if these technologies, connected to the larger world of the Metaverse, can provide alternative methods for more inclusive and engaging teaching. This case study focuses on developing a Virtual Reality application for the humanities by making it accessible through the Spatial Metaverse platform. Specifically, it brings to life the literary world of Italo Calvino’s work, “Il Barone Rampante” (“The Baron in The Trees”). The goal is to explore how these immersive technologies can enhance the educational experience, making it more interactive and accessible. By doing so, this work introduces an innovative approach to teaching literature, expanding opportunities to engage young users and improving the learning process through innovative technologies.

This paper introduces a novel approach for optimizing image capture using View-Planning (VP) to enhance Gaussian splatting for 3D reconstruction of archaeological sites, specifically focusing on Castellaraccio di Monteverdi. Traditional photogrammetry often produces over-smoothed models with artifacts. Our proposed approach leverages VP planning to select optimal viewpoints, ensuring comprehensive image coverage. We integrate this with Gaussian splatting which outputs highly-realistic 3D reconstructions. Initial evaluations on sample datasets demonstrate the potential of VP-enhanced Gaussian splatting to surpass traditional methods in terms of quality. The paper details our approach, discusses the challenges of 3D reconstruction without VP, and outlines our ongoing and future work, including upcoming tests at Castellaraccio di Monteverdi. Our findings aim to contribute to the field of archaeological documentation and analysis, supporting better preservation and understanding of historical sites.