Nina Rosa
ABSTRACT
Augmented reality research is finally moving towards multimodal experiences: more and more applications do not only include visuals, but also audio and even haptics. The purpose of multimodality in these applications can be to increase realism or to increase the amount or quality of communicated information. One particularly interesting and increasingly important application area is AR gaming, where the player can experience the virtual game integrated into the real environment and interact with it in a multimodal fashion. Currently, many games are set up such that the interaction is local (direct), however there are many cases in which remote (indirect) interaction will be useful or even necessary. In the latter case, the actions can be expressed through a virtual avatar, while the player's real body is also still perceivably present. The player then controls the motions and actions of the avatar, and receives multimodal feedback associated to the events occurring in the game. Can it be that the player starts to perceive the avatar as a (part of) him- or herself? Or does something even more intense take place? What are the benefits of this experience? The core of this research is to understand how multimodal perceptual configuration plays a role in the relation between a player and their in-game avatar.
- P. Bayliss. Beings in the game-world: characters, avatars, and players. In Proceedings of the 4th Australasian Conference on Interactive Entertainment, number 4. RMIT University, 2007. Google Scholar
Digital Library
- B. Forster, C. Cavina-Pratesi, S. M. Aglioti, and G. Berlucchi. Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time. Experimental Brain Research, 143(4):480–487, 2002.Google ScholarCross Ref
- B. Goldiez and J. Dawson. Is presence present in augmented reality systems. In Proceedings of Presence 2004. VII. International Workshop on Presence, pages 294–297, 2004.Google Scholar
- W. Hürst, N. Rosa, and J.-P. van Bommel. Vibrotactile experiences for augmented reality. In Proceedings of the 2016 ACM Multimedia Conference. ACM, 2016. (Accepted). Google ScholarDigital Library
- W. IJsselsteijn, Y. De Kort, and A. Haans. Is this my hand i see before me? the rubber hand illusion in reality, virtual reality, and mixed reality. Presence, 15(4):455–464, 2006. Google ScholarDigital Library
- P. Martin. A phenomenological account of the playing body in avatar-based action game. In Philosophy of Computer Games Conference, pages 29–31, 2012.Google Scholar
- N. Rosa, W. Hürst, W. Vos, and P. Werkhoven. The influence of visual cues on passive tactile sensations in a multimodal immersive virtual environment. In Proceedings of the 2015 ACM International Conference on Multimodal Interaction, pages 327–334. ACM, 2015. Google ScholarDigital Library
- N. Rosa, W. Hürst, P. Werkhoven, and R. Veltkamp. Visuotactile integration for depth perception in augmented reality. In Proceedings of the 2016 ACM International Conference on Multimodal Interaction. ACM, 2016. (Accepted). Google ScholarDigital Library
- B. Thomas. A survey of visual, mixed, and augmented reality gaming. Computers in Entertainment (CIE), 10(1):3, 2012. Google ScholarDigital Library
- J. van Erp, F. Kooi, A. Bronkhorst, D. van Leeuwen, M. van Esch, and S. van Wijngaarden. Multimodal interfaces: a framework based on modality appropriateness. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 50, pages 1542–1546. SAGE Publications, 2006.Google Scholar
- Y. Visell. Tactile sensory substitution: Models for enaction in hci. Interacting with Computers, 21(1-2):38–53, 2009. Google ScholarDigital Library
- F. ˇ Skola, S. Fialek, and F. Liarokapis. Augmenting the rubber hand illusion. ERCIM News, 2015. Special Theme: Augmented Reality.Google Scholar
- U. Wilhelmsson. What is a game ego: or how the embodied mind plays a role in computer game environments. Affective and emotional aspects of human-computer interaction: Game-based and innovative learning approaches, pages 45–58, 2006.Google Scholar
Index Terms
- Player/Avatar body relations in multimodal augmented reality games
Recommendations
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented RealityAugmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
(Re-)examination of multimodal augmented reality
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented RealityThe majority of augmented reality (AR) research has been concerned with visual perception, however the move towards multimodality is imminent. At the same time, there is no clear vision of what multimodal AR is. The purpose of this position paper is to ...
Augmented invaders: a mixed reality multiplayer outdoor game
VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and TechnologyMany virtual and mixed reality games focus on single player experiences. In this paper, we describe the concept and prototype implementation of a mixed reality multiplayer game that can be played with a smartphone and an HMD in outdoor environments. ...
Comments