Top

Published in:

2022 | OriginalPaper | Chapter

Multimodal Interaction with Mid-Air Haptics

Author : Jin Ryong Kim

Published in: Ultrasound Mid-Air Haptics for Touchless Interfaces

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Multimodal human–computer interfaces utilize sensory information from different modalities to create richer and more immersive user experiences. The use of ultrasound haptics in a multimodal interface is compelling: It requires no physical contact, has high spatial and temporal resolution, and can even be physically co-located with other sensory information (e.g., visual content in mid-air). In this chapter, we explore examples of multimodal interface that use ultrasound haptics, and reflect on the technical and design challenges of creating a high-quality multimodal ultrasound haptic interface.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Ultrasound Mid-Air Tactile Feedback for Immersive Virtual Reality Interaction

next chapter Touchless Tactile Interaction with Unconventional Permeable Displays

Bhardwaj A, Chae J, Noeske R, Kim JR (2021) Tangibledata: interactive data visualization with mid-air haptics. In: Proceedings of the 27th ACM symposium on virtual reality software and technology, ACM. https://doi.org/10.1145/3489849.3489890

Corenthy L, Giordano M, Hayden R, Griffiths D, Jeffrey C, Limerick H, Georgiou O, Carter T, Müller J, Subramanian S (2018) Touchless tactile displays for digital signage: mid-air haptics meets large screens. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems, ACM, pp 1–4. https://doi.org/10.1145/3170427.3186533

Darian-Smith I, Johnson KO (1977) Thermal sensibility and thermoreceptors. J Invest Dermatol 69(1):146–153CrossRef

Dzidek B, Frier W, Harwood A, Hayden R (2018) Design and evaluation of mid-air haptic interactions in an augmented reality environment. In: International conference on human haptic sensing and touch enabled computer applications, Springer, pp 489–499. https://doi.org/10.1007/978-3-319-93399-3_42

Freeman E (2021) Enhancing ultrasound haptics with parametric audio effects. In: Proceedings of 23rd ACM international conference on multimodal interaction - ICMI ’21, ACM. https://doi.org/10.1145/3462244.3479951

Freeman E, Brewster S, Lantz V (2014) Tactile feedback for above-device gesture interfaces: adding touch to touchless interactions. In: Proceedings of the 16th international conference on multimodal interaction - ICMI ’14, ACM, pp 419–426. https://doi.org/10.1145/2663204.2663280

Freeman E, Vo DB, Brewster S (2019) HaptiGlow: helping users position their hands for better mid-air gestures and ultrasound haptic feedback. In: Proceedings of IEEE world haptics conference 2019, the 8th joint eurohaptics conference and the IEEE haptics symposium, IEEE, p TP2A.09. https://doi.org/10.1145/3025453.3025518

Freeman E, Wilson G, Vo DB, Ng A, Politis I, Brewster S (2017) The handbook of multimodal-multisensor interfaces, association for computing machinery and morgan and claypool, chap multimodal feedback in HCI: haptics, non-speech Audio, and Their Applications, pp 277–317. https://doi.org/10.1145/3015783.3015792

Georgiou O, Jeffrey C, Chen Z, Tong BX, Chan SH, Yang B, Harwood A, Carter T (2018) Touchless haptic feedback for vr rhythm games. In: Proceedings of the 2018 IEEE conference on virtual reality and 3D user interfaces (VR), IEEE, pp 553–554. https://doi.org/10.1109/VR.2018.8446619

Georgiou O, Limerick H, Corenthy L, Perry M, Maksymenko M, Frish S, Müller J, Bachynskyi M, Kim JR (2019) Mid-air haptic interfaces for interactive digital signage and kiosks. In: Extended abstracts of the 2019 CHI conference on human factors in computing systems, ACM, pp 1–9. https://doi.org/10.1145/3290607.3299030

Gil H, Son H, Kim JR, Oakley I (2018) Whiskers: exploring the use of ultrasonic haptic cues on the face. In: Proceedings of the 2018 CHI conference on human factors in computing systems, pp 1–13

Han PH, Chen YS, Lee KC, Wang HC, Hsieh CE, Hsiao JC, Chou CH, Hung YP (2018) Haptic around: multiple tactile sensations for immersive environment and interaction in virtual reality. In: Proceedings of the 24th ACM symposium on virtual reality software and technology - VRST ’18, pp 1–10

Hashizume S, Koike A, Hoshi T, Ochiai Y (2017) Sonovortex: Rendering multi-resolution aerial haptics by aerodynamic vortex and focused ultrasound. In: Proceedings of ACM SIGGRAPH 2017 posters, ACM, pp 1–2. https://doi.org/10.1145/3102163.3102178

Hülsmann F, Fröhlich J, Mattar N, Wachsmuth I (2014) Wind and warmth in virtual reality: implementation and evaluation. In: Proceedings of the 2014 virtual reality international conference, pp 1–8

Hwang I, Son H, Kim JR (2017) Airpiano: enhancing music playing experience in virtual reality with mid-air haptic feedback. In: Proceedings of the 2017 IEEE world haptics conference (- IEEE WHC ’17, IEEE), pp 213–218

Iwai D, Aoki M, Sato K (2018) Non-contact thermo-visual augmentation by IR-RGB projection. IEEE Trans Visualization Comput Graph 25(4):1707–1716CrossRef

Jones LA, Ho HN (2008) Warm or cool, large or small? the challenge of thermal displays. IEEE Trans Hapt 1(1):53–70CrossRef

Kim JR, Chan S, Huang X, Ng K, Fu LP, Zhao C (2019) Demonstration of refinity: an interactive holographic signage for new retail shopping experience. In: Extended abstracts of the 2019 CHI conference on human factors in computing systems, pp 1–4

Limerick H, Hayden R, Beattie D, Georgiou O, Müller J (2019) User engagement for mid-air haptic interactions with digital signage. In: Proceedings of the 8th ACM international symposium on pervasive displays, pp 1–7

Martinez J, Griffiths D, Biscione V, Georgiou O, Carter T (2018) Touchless haptic feedback for supernatural vr experiences. In: 2018 IEEE conference on virtual reality and 3D user interfaces (VR), IEEE, pp 629–630

Matsubayashi A, Makino Y, Shinoda H (2019) Direct finger manipulation of 3d object image with ultrasound haptic feedback. In: Proceedings of the 2019 CHI conference on human factors in computing systems, ACM, p paper 87. https://doi.org/10.1145/3290605.3300317

Monnai Y, Hasegawa K, Fujiwara M, Yoshino K, Inoue S, Shinoda H (2014) Haptomime: mid-air haptic interaction with a floating virtual screen. In: Proceedings of the 27th annual ACM symposium on User interface software and technology, pp 663–667

Ochiai Y, Kumagai K, Hoshi T, Hasegawa S, Hayasaki Y (2016) Cross-field aerial haptics: rendering haptic feedback in air with light and acoustic fields. In: Proceedings of the 2016 CHI conference on human factors in computing systems, pp 3238–3247

Ozkul C, Geerts D, Rutten I (2020) Combining auditory and mid-air haptic feedback for a light switch button. In: Proceedings of the 2020 international conference on multimodal interaction, ACM, pp 60–69. https://doi.org/10.1145/3382507.3418823

Puertolas Balint L, Althoefer K et al (2018) Medical virtual reality palpation training using ultrasound based haptics and image processing. In: Proceedings of the conference on computer/robot assisted surgery - CRAS ’18

Rakkolainen I, Freeman E, Sand A, Raisamo R, Brewster S (2020) A survey of mid-air ultrasound haptics and its applications. IEEE Trans Hapt 14:2–19. https://doi.org/10.1109/TOH.2020.3018754CrossRef

Romanus T, Frish S, Maksymenko M, Frier W, Corenthy L, Georgiou O (2019) Mid-air haptic bio-holograms in mixed reality. In: 2019 IEEE International symposium on mixed and augmented reality adjunct (ISMAR-Adjunct), IEEE, pp 348–352

Shaw E, Roper T, Nilsson T, Lawson G, Cobb SV, Miller D (2019) The heat is on: Exploring user behaviour in a multisensory virtual environment for fire evacuation. In: Proceedings of the 2019 CHI conference on human factors in computing systems - CHI ’19, pp 1–13

Singhal Y, Wang H, Gil H, Kim JR (2021) Mid-air thermo-tactile feedback using ultrasound haptic display. In: Proceedings of the 27th ACM symposium on virtual reality software and technology, ACM, p Article 28, https://doi.org/10.1145/3489849.3489889

van Beers RJ, Wolpert DM, Haggard P (2002) When feeling is more important than seeing in sensorimotor adaptation. Current biology 12(10):834–837. https://doi.org/10.1016/S0960-9822(02)00836-9CrossRef

Van den Bogaert L, Geerts D, Rutten I (2019) Grasping the future: identifying potential applications for mid-air haptics in the home. In: Extended Abstracts of the 2019 CHI conference on human factors in computing Systems, ACM, p LBW1415. https://doi.org/10.1145/3290607.3312911

Title: Multimodal Interaction with Mid-Air Haptics
Author: Jin Ryong Kim
Publisher: Springer International Publishing
Book: Ultrasound Mid-Air Haptics for Touchless Interfaces
Print ISBN: 978-3-031-04042-9

Electronic ISBN: 978-3-031-04043-6

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-031-04043-6_7