Top

Published in:

2022 | OriginalPaper | Chapter

Ultrasound Haptic Feedback for Touchless User Interfaces: Design Patterns

Author : Euan Freeman

Published in: Ultrasound Mid-Air Haptics for Touchless Interfaces

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Touchless user interfaces enable people to interact with digital services and information without physically touching an input device. There are numerous benefits to touchless interaction (including convenience, hygiene and the potential for more expressive input), and sensing technologies have advanced significantly in recent years. As a result, touchless user interfaces have been adopted on a wider scale across a variety of application areas, e.g. automotive, digital signage and gaming. However, usability remains a key concern; touchless gesture input poses several interaction challenges, many related to uncertainty and the inherent loss of tactile cues. Ultrasound haptic feedback has shown promise in helping users overcome such interaction challenges, restoring the missing sense of touch and closing the feedback loop for effective haptic interaction. This chapter explores how ultrasound haptic feedback has been used in touchless user interface design and presents design patterns used by industry and academia alike.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

previous chapter User Experience and Mid-Air Haptics: Applications, Methods, and Challenges

next chapter Opportunities for Multisensory Mid-Air Interactions Featuring Ultrasound Haptic Feedback

Ablart D, Velasco C, Obrist M (2017) Integrating mid-air haptics into movie experiences. In: Proceedings of the 2017 ACM international conference on interactive experiences for TV and online video—TVX ’17, ACM Press, pp 77–84, https://doi.org/10.1145/3077548.3077551, https://dl.acm.org/citation.cfm?id=3077548.3077551

Bellotti V, Back M, Edwards WK, Grinter RE, Henderson A, Lopes C (2002) Making sense of sensing systems: five questions for designers and researchers. In: Proceedings of the SIGCHI conference on human factors in computing systems—CHI ’02, ACM Press, pp 415–422. https://doi.org/10.1145/503376.503450, http://dl.acm.org/citation.cfm?id=503450

Carter T, Seah SA, Long B, Drinkwater B, Subramanian S (2013) UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In: Proceedings of the 26th symposium on user interface software and technology—UIST ’13. ACM Press, pp 505–514, https://doi.org/10.1145/2501988.2502018, http://dl.acm.org/citation.cfm?id=2502018

Corenthy L, Giordano M, Hayden R, Griffiths D, Jeffrey C, Limerick H, Georgiou O, Carter T, Müller J, Subramanian S (2018) Touchless tactile displays for digital signage. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems—CHI ’18. ACM Press, p Demo 103. https://doi.org/10.1145/3170427.3186533, http://dl.acm.org/citation.cfm?doid=3170427.3186533

Cornelio-Martinez PI, Pirro SD, Vi CT, Subramanian S (2017) Agency in mid-air Interfaces. In: Proceedings of the 2017 CHI conference on human factors in computing systems—CHI ’17, ACM Press, pp 2426–2439. https://doi.org/10.1145/3025453.3025457, http://dl.acm.org/citation.cfm?id=3025457

Cronin S, Doherty G (2019) Touchless computer interfaces in hospitals: a review. Health Informatics J 25(4):1325–1342. https://doi.org/10.1177/1460458217748342, https://journals.sagepub.com/doi/full/10.1177/1460458217748342

Dzidek B, Frier W, Harwood A, Hayden R (2018) Design and evaluation of mid-air haptic interactions in an augmented reality environment. In: Proceedings of EuroHaptics 2018 in LNCS 10894—EuroHaptics ’18, Springer International Publishing, pp 489–499. https://doi.org/10.1007/978-3-319-93399-3_42, https://link.springer.com/chapter/10.1007/978-3-319-93399-3_42

Freeman E (2021) Enhancing ultrasound haptics with parametric audio effects. In: Proceedings of the 2021 international conference on multimodal interaction. Association for Computing Machinery, New York, NY, USA, ICMI ’21, pp 692–696. https://doi.org/10.1145/3462244.3479951, https://doi.org/10.1145/3462244.3479951

Freeman E, Wilson G (2021) Perception of ultrasound haptic focal point motion. In: Proceedings of the 2021 international conference on multimodal interaction. Association for Computing Machinery, New York, NY, USA, ICMI ’21, pp 697–701. https://doi.org/10.1145/3462244.3479950, https://doi.org/10.1145/3462244.3479950

Freeman E, Brewster S, Lantz V (2014) Tactile feedback for above-device gesture interfaces: adding touch to touchless interactions. In: Proceedings of the 16th international conference on multimodal interaction—ICMI ’14. ACM Press, pp 419–426. https://doi.org/10.1145/2663204.2663280, http://dl.acm.org/citation.cfm?id=2663280 http://dl.acm.org/citation.cfm?doid=2663204.2663280

Freeman E, Brewster S, Lantz V (2015) Towards In-Air Gesture Control of Household Appliances with Limited Displays. In: Proceedings of INTERACT ’15 in LNCS 9299, IFIP, pp 611–615, https://doi.org/10.1007/978-3-319-22723-8_73, http://link.springer.com/10.1007/978-3-319-22723-8_73

Freeman E, Brewster S, Lantz V (2016) Do that, there: an interaction technique for addressing in-air gesture systems. In: Proceedings of the 2016 CHI conference on human factors in computing systems—CHI ’16. ACM Press, pp 2319–2331, https://doi.org/10.1145/2858036.2858308, http://dl.acm.org/citation.cfm?doid=2858036.2858308

Freeman E, Vo DB, Brewster S (2019) HaptiGlow: helping users position their hands for better mid-air gestures and ultrasound haptic feedback. In: Proceedings of the IEEE world haptics conference 2019, the 8th joint Eurohaptics conference and the IEEE haptics symposium, IEEE, pp 289–294, https://doi.org/10.1109/WHC.2019.8816092, https://ieeexplore.ieee.org/document/8816092

Frier W, Ablart D, Chilles J, Long B, Giordano M, Obrist M, Subramanian S (2018) Using spatiotemporal modulation to draw tactile patterns in mid-air. In: Proceedings of EuroHaptics 2018. Springer, Berlin

Frier W, Pittera D, Ablart D, Obrist M, Subramanian S (2019) Sampling strategy for ultrasonic mid-air haptics. In: Proceedings of the 2019 CHI conference on human factors in computing systems proceedings—CHI ’19. ACM Press, p Paper 121, https://doi.org/10.1145/3290605.3300351, https://dl.acm.org/citation.cfm?id=3290605.3300351

Georgiou O, Griffiths D (2017) Haptic in-vehicle gesture controls. In: Adjunct proceedings of the 9th international ACM conference on automotive user interfaces and interactive vehicular applications—AutomotiveUI ’17, ACM Press, pp 233–238. https://doi.org/10.1145/3131726.3132045

Georgiou O, Jeffrey C, Chen Z, Tong BX, Chan SH, Yang B, Harwood A, Carter T (2018) Touchless haptic feedback for VR rhythm games. In: Proceedings of 25th IEEE conference on virtual reality and 3D user interfaces—IEEE VR ’18. IEEE, pp 2–3, https://doi.org/10.1109/VR.2018.8446619, https://ieeexplore.ieee.org/document/8446619

Hajas D, Pittera D, Nasce A, Georgiou O, Obrist M (2020) Mid-air haptic rendering of 2D geometric shapes with a dynamic tactile pointer. IEEE Trans Haptics 13(1):1–12. https://doi.org/10.1109/TOH.2020.2966445CrossRef

Hirayama R, Martinez Plasencia D, Masuda N, Subramanian S (2019) A volumetric display for visual, tactile and audio presentation using acoustic trapping. Nature 575

Hoshi T (2011) Development of aerial-input and aerial-tactile-feedback system. In: Proceedings of the 2011 IEEE world haptics conference, IEEE, pp 569–573. https://doi.org/10.1109/WHC.2011.5945548, http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5945548

Hoshi T, Takahashi M, Iwamoto T, Shinoda H (2010) Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Trans Haptics 3(3):155–165CrossRef

Hwang I, Son H, Kim JR (2017) AirPiano: enhancing music playing experience in virtual reality with mid-air haptic feedback. In: 2017 IEEE world haptics conference—WHC ’17. IEEE, pp 213–218. https://doi.org/10.1109/WHC.2017.7989903, http://ieeexplore.ieee.org/document/7989903/

Iwamoto T, Tatezono M, Shinoda H (2008) Non-contact method for producing tactile sensation using airborne ultrasound. In: Proceedings of EuroHaptics 2008. Springer, Berlin, pp 504–513. https://doi.org/10.1007/978-3-540-69057-3_64, http://www.springerlink.com/index/X41J595757401387.pdf

Korres G, Eid M (2016) Haptogram: ultrasonic point-cloud tactile stimulation. IEEE Access 4:7758–7769CrossRef

Limerick H (2020) Call to interact: communicating interactivity and affordances for contactless gesture controlled public displays. In: Proceedings of the 9th ACM international symposium on pervasive displays—PerDis ’20, pp 63–70. https://doi.org/10.1145/3393712.3395338

Limerick H, Hayden R, Beattie D, Georgiou O, Müller J (2019) User engagement for mid-air haptic interactions with digital signage. In: Proceedings of the 8th ACM international symposium on pervasive displays—PerDis ’19. ACM Press, p to appear

Long B, Seah SA, Carter T, Subramanian S (2014) Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Trans Graphics 33(6):Article 181. https://doi.org/10.1145/2661229.2661257, http://dl.acm.org/citation.cfm?id=2661257

Martinez J, Griffiths D, Biscione V, Georgiou O, Carter T (2018) Touchless haptic feedback for supernatural VR experiences. In: Proceedings of the 25th IEEE conference on virtual reality and 3D user interfaces, pp 629–630. https://doi.org/10.1109/VR.2018.8446522

Monnai Y, Hasegawa K, Fujiwara M, Yoshino K, Inoue S, Shinoda H (2014) HaptoMime: mid-air haptic interaction with a floating virtual screen. In: Proceedings of the 27th symposium on user interface software and technology—UIST ’14. ACM Press, pp 663–667. https://doi.org/10.1145/2642918.2647407, http://dl.acm.org/citation.cfm?id=2642918.2647407

O’Hara K, Gonzalez G, Sellen A, Penney G, Varnavas A, Mentis H, Criminisi A, Corish R, Rouncefield M, Dastur N, Carrell T (2014) Touchless interaction in surgery. Commun ACM 57(1):70–77CrossRef

Rakkolainen I, Freeman E, Sand A, Raisamo R, Brewster S (2020) A survey of mid-air ultrasound haptics and its applications. IEEE Trans Haptics 14(1):2–19CrossRef

Rutten I, Frier W, Van de Bogaert L, Geerts D (2019) Invisible touch: how identifiable are mid-air haptic shapes? In: Proceedings of extended abstracts on human factors in computing systems on—CHI EA ’19 p LBW0283

Sand A, Rakkolainen I, Isokoski P, Kangas J, Raisamo R, Palovuori K (2015) Head-mounted display with mid-air tactile feedback. In: Proceedings of the 21st ACM symposium on virtual reality software and technology—VRST ’15. ACM Press, pp 51–58. https://doi.org/10.1145/2821592.2821593, http://dl.acm.org/citation.cfm?doid=2821592.2821593

Shakeri G, Williamson JH, Brewster S (2018) May the force be with you: ultrasound haptic feedback for mid-air gesture interaction in cars. In: Proceedings of automotive UI 2018—AutoUI ’18. ACM Press

Sridhar S, Feit AM, Theobalt C, Oulasvirta A (2015) Investigating the dexterity of multi-finger input for mid-air text entry. In: Proceedings of the SIGCHI conference on human factors in computing systems—CHI ’15. ACM Press, pp 3643–3652. https://doi.org/10.1145/2702123.2702136, https://dl.acm.org/citation.cfm?id=2702136

Takahashi R, Hasegawa K, Shinoda H (2018) Lateral modulation of midair ultrasound focus for intensified vibrotactile stimuli. In: Proceedings of EuroHaptics 2018 in LNCS 10894—EuroHaptics ’18. Springer International Publishing, pp 276–288. https://doi.org/10.1007/978-3-319-93399-3_25, http://link.springer.com/10.1007/978-3-319-93445-7

Takahashi R, Hasegawa K, Shinoda H (2019) Tactile stimulation by repetitive lateral movement of midair ultrasound focus. IEEE Trans Haptics

Ultrahaptics (2017) Sensation Editor: The first 10 sensation templates. https://developer.ultrahaptics.com/knowledgebase/the-first-10-sensation-templates/

Ultraleap (2019) UCA tutorial 8: the forcefield sensation. https://developer.ultrahaptics.com/knowledgebase/unity-tutorial-eight/

Vo DB, Brewster S (2015) Touching the invisible: localizing ultrasonic haptic cues. In: Proceedings of world haptics conference 2015—WHC ’15, IEEE, pp 368–373. https://doi.org/10.1109/WHC.2015.7177740, http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7177740

Waugh K, Robertson J (2021) Don’t touch me! a comparison of usability on touch and non-touch inputs. In: Proceedings of the IFIP conference on human-computer interaction—Interact ’21. Springer, Berlin, pp 400–404. https://doi.org/10.1007/978-3-030-85607-6_46

Wilson G, Carter T, Subramanian S, Brewster S (2014) Perception of ultrasonic haptic feedback on the hand: localisation and apparent motion. In: Proceedings of the SIGCHI conference on human factors in computing systems—CHI ’14. ACM Press, pp 1133–1142. https://doi.org/10.1145/2556288.2557033, http://dl.acm.org/citation.cfm?id=2557033

Title: Ultrasound Haptic Feedback for Touchless User Interfaces: Design Patterns
Author: Euan Freeman
Publisher: Springer International Publishing
Book: Ultrasound Mid-Air Haptics for Touchless Interfaces
Print ISBN: 978-3-031-04042-9

Electronic ISBN: 978-3-031-04043-6

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-031-04043-6_3