Skip to main content
Top

2016 | OriginalPaper | Chapter

Strong Concepts for Designing Non-verbal Interactions in Mixed Reality Narratives

Author : Joshua A. Fisher

Published in: Interactive Storytelling

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

As next-generation augmented reality (AR) devices, referred to by companies like Microsoft as mixed reality (MR) headsets, enter the market non-verbal interaction paradigms for storytelling should be designed to take advantage of the new technical affordances. By ascertaining strong concepts, a model used in the HCI community to describe intermediate knowledge that sits between theory and practice for generating new interaction designs, a foundation for future work in MR, based on the existing technical affordances and interaction behaviors of AR and virtual reality (VR), can be developed. Strong concepts free interaction designers to explore potential interaction gestalts based on current behaviors and technologies. The framework is useful for speculating on effective non-verbal interactions in MR narratives as the platform is still being established. Interaction designers can use these foundational strong concepts as a starting point to develop a toolset for non-verbal interactions in future MR interactive digital stories.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Milgram, P., Haruo, T., Akira, U., Fumio, K.: Augmented reality: a class of displays on the reality-virtuality continuum. In: Telemanipulator and Telepresence Technologies (1995) Milgram, P., Haruo, T., Akira, U., Fumio, K.: Augmented reality: a class of displays on the reality-virtuality continuum. In: Telemanipulator and Telepresence Technologies (1995)
4.
go back to reference Murray, J.H.: The Aesthetics of the Medium. Hamlet on the Holodeck: The Future of Narrative in Cyberspace, pp. 97–154. MIT, Cambridge (1998) Murray, J.H.: The Aesthetics of the Medium. Hamlet on the Holodeck: The Future of Narrative in Cyberspace, pp. 97–154. MIT, Cambridge (1998)
5.
go back to reference Mateas, M., Stern, A.: Interaction and narrative. In: The Game Design Reader: A Rules of Play Anthology, pp. 642–669 (2006) Mateas, M., Stern, A.: Interaction and narrative. In: The Game Design Reader: A Rules of Play Anthology, pp. 642–669 (2006)
6.
go back to reference Akimoto, T., Ogata, T.: Macro Structure and Basic Methods in the Integrate Narrative Generation System By Introducing Narratological Knowledge, pp. 1–25 (1999) Akimoto, T., Ogata, T.: Macro Structure and Basic Methods in the Integrate Narrative Generation System By Introducing Narratological Knowledge, pp. 1–25 (1999)
7.
go back to reference Lawson, B.: How Designers Think. Butterworth Architecture, London (1990) Lawson, B.: How Designers Think. Butterworth Architecture, London (1990)
8.
go back to reference Bolter, J.D., Grusin, R.: Remediation: Understanding New Media. MIT, Cambridge (1999) Bolter, J.D., Grusin, R.: Remediation: Understanding New Media. MIT, Cambridge (1999)
10.
go back to reference Feng, Z, Been-Lirn Duh, H., Billinghurst, M.: Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR 2008), pp. 193–202. IEEE Computer Society (2008) doi:10.1109/ISMAR.2008.4637362 Feng, Z, Been-Lirn Duh, H., Billinghurst, M.: Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR 2008), pp. 193–202. IEEE Computer Society (2008) doi:10.​1109/​ISMAR.​2008.​4637362
11.
go back to reference Yi-fu, T.: Space and Place: The Perspective of Experience. University of Minnesota, Minneapolis (1977). Print Yi-fu, T.: Space and Place: The Perspective of Experience. University of Minnesota, Minneapolis (1977). Print
13.
go back to reference Dostal, J., Hinrichs, U., Kristensson, P.O., Quigley, A.: SpiderEyes: designing attention-and proximity-aware collaborative interfaces for wall-sized displays. In: Proceedings of the 19th international conference on Intelligent User Interfaces. (2014) Dostal, J., Hinrichs, U., Kristensson, P.O., Quigley, A.: SpiderEyes: designing attention-and proximity-aware collaborative interfaces for wall-sized displays. In: Proceedings of the 19th international conference on Intelligent User Interfaces. (2014)
14.
go back to reference Whelan, T., Kaess, M., Fallon, M.: Kintinuous: Spatially extended kinectfusion. In: RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras, p. 7 (2012) Whelan, T., Kaess, M., Fallon, M.: Kintinuous: Spatially extended kinectfusion. In: RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras, p. 7 (2012)
16.
go back to reference Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR (2001) Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR (2001)
17.
go back to reference Heidegger, M.: The question concerning technology. In: Technology and Values: Essential Readings, pp. 99–113 (1954) Heidegger, M.: The question concerning technology. In: Technology and Values: Essential Readings, pp. 99–113 (1954)
18.
go back to reference O’Brien, M.: Commentary on Heidegger’s “The Question Concerning Technology”. In: Thinking Together: Proceedings of the IWM Junior Fellows’ Conference, vol. 16, XVI, 2 (2004) O’Brien, M.: Commentary on Heidegger’s “The Question Concerning Technology”. In: Thinking Together: Proceedings of the IWM Junior Fellows’ Conference, vol. 16, XVI, 2 (2004)
19.
go back to reference Murray, J.: Did it make you cry? Creating dramatic agency in immersive environments. In: Subsol, G. (ed.) ICVS-VirtStory 2005. LNCS, vol. 3805, pp. 83–94. Springer, Heidelberg (2005) Murray, J.: Did it make you cry? Creating dramatic agency in immersive environments. In: Subsol, G. (ed.) ICVS-VirtStory 2005. LNCS, vol. 3805, pp. 83–94. Springer, Heidelberg (2005)
22.
go back to reference Collins English Dictionary. HarperCollins (2011) Collins English Dictionary. HarperCollins (2011)
23.
go back to reference Hugues, O., Fuchs, P., Nannipieri, O.: New augmented reality taxonomy: technologies and features of augmented environment. In: Handbook of Augmented Reality (2011). doi:10.1007/978-1-4614-0064-6 Hugues, O., Fuchs, P., Nannipieri, O.: New augmented reality taxonomy: technologies and features of augmented environment. In: Handbook of Augmented Reality (2011). doi:10.​1007/​978-1-4614-0064-6
25.
go back to reference Manovich, L.: The poetics of augmented space. Visual Commun. 5(2), 219–240 (2006)CrossRef Manovich, L.: The poetics of augmented space. Visual Commun. 5(2), 219–240 (2006)CrossRef
28.
go back to reference Tuters, M.: The locative commons: situating location-based media in urban public space. In: Electronic Proceedings of the 2004 Futuresonic Conference (2004) Tuters, M.: The locative commons: situating location-based media in urban public space. In: Electronic Proceedings of the 2004 Futuresonic Conference (2004)
29.
go back to reference Deleuze, G.: Pure Immanence, p. 27. Zone, New York (2001) Deleuze, G.: Pure Immanence, p. 27. Zone, New York (2001)
30.
go back to reference Dow, S., Mehta, M., Lausier, A., MacIntyre, B., Mateas, M.: Initial lessons from AR Façade, an interactive augmented reality drama. In: Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Article No.: 28 (2006). doi:10.1145/1178823.1178858 Dow, S., Mehta, M., Lausier, A., MacIntyre, B., Mateas, M.: Initial lessons from AR Façade, an interactive augmented reality drama. In: Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Article No.: 28 (2006). doi:10.​1145/​1178823.​1178858
31.
go back to reference Thomas, B., Close, B., Donoghue, J., Squires, J., De Bondi, P., Piekarski, W.: First person indoor/outdoor augmented reality application: ARQuake. Pers. Ubiquit. Comput. 6(1), 75–86 (2002). doi:10.1007/s007790200007 CrossRef Thomas, B., Close, B., Donoghue, J., Squires, J., De Bondi, P., Piekarski, W.: First person indoor/outdoor augmented reality application: ARQuake. Pers. Ubiquit. Comput. 6(1), 75–86 (2002). doi:10.​1007/​s007790200007 CrossRef
32.
go back to reference Rekimoto, J., Nagao, K.: The world through the computer: computer augmented interaction with real world environments. In: Proceedings of the 8th Annual ACM Symposium User Interface and Software Technology UIST, pp. 29–36. ACM Press (1995). doi:10.1145/215585.215639 Rekimoto, J., Nagao, K.: The world through the computer: computer augmented interaction with real world environments. In: Proceedings of the 8th Annual ACM Symposium User Interface and Software Technology UIST, pp. 29–36. ACM Press (1995). doi:10.​1145/​215585.​215639
33.
go back to reference Lee, T., Hollerer, T.: Handy AR: Markerless inspection of augmented reality objects using fingertip tracking. In: International Symposium on Wearable Computers, October 2007 (2007) Lee, T., Hollerer, T.: Handy AR: Markerless inspection of augmented reality objects using fingertip tracking. In: International Symposium on Wearable Computers, October 2007 (2007)
34.
go back to reference Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: Proceedings of the Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), Nara, Japan, November 2007 (2007) Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: Proceedings of the Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), Nara, Japan, November 2007 (2007)
35.
go back to reference Lepetit, V., Fua, P.: Monocular model-based 3d tracking of rigid objects. Found. Trends Comput. Graph. Vis. 1(1), 1–89 (2006)CrossRef Lepetit, V., Fua, P.: Monocular model-based 3d tracking of rigid objects. Found. Trends Comput. Graph. Vis. 1(1), 1–89 (2006)CrossRef
36.
go back to reference Cavazza, M., Charles, F., Mead, S.: Interacting with virtual characters in interactive storytelling. Agents and Multiagent Systems: Part 1, pp. 241–318 (2002). doi:10.1145/544741.544819 Cavazza, M., Charles, F., Mead, S.: Interacting with virtual characters in interactive storytelling. Agents and Multiagent Systems: Part 1, pp. 241–318 (2002). doi:10.​1145/​544741.​544819
37.
39.
go back to reference Lee, T., Höllerer, T.: Hybrid feature tracking and user interaction for markerless augmented reality. In: Proceedings - IEEE Virtual Reality, pp. 145–152 (2008). doi:10.1109/VR.2008.4480766 Lee, T., Höllerer, T.: Hybrid feature tracking and user interaction for markerless augmented reality. In: Proceedings - IEEE Virtual Reality, pp. 145–152 (2008). doi:10.​1109/​VR.​2008.​4480766
41.
go back to reference Lee, M., Green, R., Billinghurst, M.: 3D natural hand interaction for AR applications. In: 2008 23rd International Conference Image and Vision Computing New Zealand, IVCNZ, pp. 2–7 (2008). doi:10.1109/IVCNZ.2008.4762125 Lee, M., Green, R., Billinghurst, M.: 3D natural hand interaction for AR applications. In: 2008 23rd International Conference Image and Vision Computing New Zealand, IVCNZ, pp. 2–7 (2008). doi:10.​1109/​IVCNZ.​2008.​4762125
43.
go back to reference Reifinger, S., Wallhoff, F., Ablassmeier, M., Poitschke, T., Rigoll, G.: Static and Dynamic Hand-Gesture Recognition for Augmented Reality Applications. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 728–737. Springer, Heidelberg (2007). doi:10.1007/978-3-540-73110-8 CrossRef Reifinger, S., Wallhoff, F., Ablassmeier, M., Poitschke, T., Rigoll, G.: Static and Dynamic Hand-Gesture Recognition for Augmented Reality Applications. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 728–737. Springer, Heidelberg (2007). doi:10.​1007/​978-3-540-73110-8 CrossRef
44.
go back to reference Bolter, J., Gromala, D.: Transparency and reflectivity: digital art and the aesthetics of interface design. In: Aesthetic Computing, 7 (2004) Bolter, J., Gromala, D.: Transparency and reflectivity: digital art and the aesthetics of interface design. In: Aesthetic Computing, 7 (2004)
45.
go back to reference Buchmann, V., Violich, S., Billinghurst, M., Cockburn, A.: FingARtips – Gesture based direct manipulation in augmented reality. In: Proceedings GRAPHITE 2004 - 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, 1(212), pp. 212–221 (2004). doi:10.1145/988834.988871 Buchmann, V., Violich, S., Billinghurst, M., Cockburn, A.: FingARtips – Gesture based direct manipulation in augmented reality. In: Proceedings GRAPHITE 2004 - 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, 1(212), pp. 212–221 (2004). doi:10.​1145/​988834.​988871
Metadata
Title
Strong Concepts for Designing Non-verbal Interactions in Mixed Reality Narratives
Author
Joshua A. Fisher
Copyright Year
2016
Publisher
Springer International Publishing
DOI
https://doi.org/10.1007/978-3-319-48279-8_26

Premium Partner