Ruolin Wang
ABSTRACT
Interacting with a smartphone using touch input and speech output is challenging for visually impaired people in mobile and public scenarios, where only one hand may be available for input (e.g., while holding a cane) and using the loudspeaker for speech output is constrained by environmental noise, privacy, and social concerns. To address these issues, we propose EarTouch, a one-handed interaction technique that allows the users to interact with a smartphone using the ear to perform gestures on the touchscreen. Users hold the phone to their ears and listen to speech output from the ear speaker privately. We report how the technique was designed, implemented, and evaluated through a series of studies. Results show that EarTouch is easy, efficient, fun and socially acceptable to use.
Supplemental Material
Available for Download
srt
paper024vc.srt (2.6 KB)
Video figure captions
zip
paper024pvc.zip (602 B)
Preview video captions
- Ali Abdolrahmani, Ravi Kuber, and Amy Hurst. 2016. An Empirical Investigation of the Situationally-induced Impairments Experienced by Blind Mobile Device Users. In Proceedings of the 13th Web for All Conference (W4A '16). ACM, New York, NY, USA, Article 21, 8 pages. Google Scholar
Digital Library
- Tousif Ahmed, Roberto Hoyle, Kay Connelly, David Crandall, and Apu Kapadia. 2015. Privacy Concerns and Behaviors of People with Visual Impairments. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3523--3532. Google ScholarDigital Library
- Maraim Alnfiai and Srinivas Sampalli. 2016. An Evaluation of SingleTapBraille Keyboard: A Text Entry Method that Utilizes Braille Patterns on Touchscreen Devices. In International ACM Sigaccess Conference on Computers and Accessibility. 161--169. Google ScholarDigital Library
- Shiri Azenkot, Sanjana Prasain, Alan Borning, Emily Fortuna, Richard E. Ladner, and Jacob O. Wobbrock. 2011. Enhancing Independence and Safety for Blind and Deaf-blind Public Transit Riders. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 3247--3256. Google ScholarDigital Library
- Shiri Azenkot, Kyle Rector, Richard Ladner, and Jacob Wobbrock. 2012. PassChords: Secure Multi-touch Authentication for Blind People. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '12). ACM, New York, NY, USA, 159--166. Google ScholarDigital Library
- Nikola Banovic, Koji Yatani, and Khai N. Truong. 2013. EscapeKeyboard: A Sight-Free One-Handed Text Entry Method for Mobile Touch-screen Devices. Int. J. Mob. Hum. Comput. Interact. 5, 3 (July 2013), 42--61. Google ScholarDigital Library
- Sebastian Boring, David Ledo, Xiang 'Anthony' Chen, Nicolai Marquardt, Anthony Tang, and Saul Greenberg. 2012. The Fat Thumb: Using the Thumb's Contact Size for Single-handed Mobile Interaction. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services Companion (MobileHCI '12). ACM, New York, NY, USA, 207--208. Google ScholarDigital Library
- Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, and Amaury Trujillo. 2017. Analyzing Visually Impaired People's Touch Gestures on Smartphones. Multimedia Tools Appl. 76, 4 (Feb. 2017), 5141--5169. Google ScholarDigital Library
- Sabine Croxford and Cathy Rundle. 2013. Blind people and apps on mobile phones and tablets Challenges and possibilities.Google Scholar
- Rafael Jeferson Damaceno, Juliana Cristina Braga, and Jesús Pascual Mena-Chalco. 2018. Mobile Device Accessibility for the Visually Impaired: Problems Mapping and Recommendations. Univers. Access Inf. Soc. 17, 2 (June 2018), 421--435. Google ScholarDigital Library
- Krzysztof Dobosz and MichaÅ Szucik. 2018. OneHandBraille: An Alternative Virtual Keyboard for Blind People.Google Scholar
- M. Fakrudeen, M. Ali, S. Yousef, and A. H. Hussein. 2013. Analysing the Mental Model of Blind Users in Mobile Touch Screen Devices for Usability. Lecture Notes in Engineering & Computer Science 2205, 1 (2013).Google Scholar
- Yoshitomo Fukatsu, Buntarou Shizuki, and Jiro Tanaka. 2013. No-look Flick: Single-handed and Eyes-free Japanese Text Input System on Touch Screens of Mobile Devices. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '13). ACM, New York, NY, USA, 161--170. Google ScholarDigital Library
- Audrey Girouard, Jessica Lo, Md Riyadh, Farshad Daliri, Alexander Keith Eady, and Jerome Pasquero. 2015. One-Handed Bend Interactions with Deformable Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1509--1518. Google ScholarDigital Library
- William Grussenmeyer and Eelke Folmer. 2017. Accessible Touchscreen Technology for People with Visual Impairments: A Survey. ACM Trans. Access. Comput. 9, 2, Article 6 (Jan. 2017), 31 pages. Google ScholarDigital Library
- Kotaro Hara, Christine Chan, and Jon E. Froehlich. 2016. The Design of Assistive Location-based Technologies for People with Ambulatory Disabilities: A Formative Study. In CHI Conference on Human Factors in Computing Systems. 1757--1768. Google ScholarDigital Library
- Chris Harrison, Julia Schwarz, and Scott E. Hudson. 2011. TapSense: Enhancing Finger Interaction on Touch Surfaces. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 627--636. Google ScholarDigital Library
- Khalad Hasan, Junhyeok Kim, David Ahlström, and Pourang Irani. 2016. Thumbs-Up: 3D Spatial Thumb-Reachable Space for One-Handed Thumb Interaction on Smartphones. In Proceedings of the 2016 Symposium on Spatial User Interaction (SUI '16). ACM, New York, NY, USA, 103--106. Google ScholarDigital Library
- João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2015. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3 (2015), 583--596.Google ScholarDigital Library
- Ken Hinckley, Seongkook Heo, Michel Pahud, Christian Holz, Hrvoje Benko, Abigail Sellen, Richard Banks, Kenton O'Hara, Gavin Smyth, and William Buxton. 2016. Pre-touch sensing for mobile interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2869--2881. Google ScholarDigital Library
- Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3011--3014. Google ScholarDigital Library
- Shaun K. Kane, Jeffrey P. Bigham, and Jacob O. Wobbrock. 2008. Slide Rule: Making Mobile Touch Screens Accessible to Blind People Using Multi-touch Interaction Techniques. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '08). ACM, New York, NY, USA, 73--80. Google ScholarDigital Library
- Shaun K. Kane, Chandrika Jayant, Jacob O. Wobbrock, and Richard E. Ladner. 2009. Freedom to Roam: A Study of Mobile Device Adoption and Accessibility for People with Visual and Motor Disabilities. In Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '09). ACM, New York, NY, USA, 115--122. Google ScholarDigital Library
- Shaun K. Kane, Jacob O. Wobbrock, and Richard E. Ladner. 2011. Usable Gestures for Blind People: Understanding Preference and Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 413--422. Google ScholarDigital Library
- Amy K. Karlson and Benjamin B. Bederson. 2007. ThumbSpace: Generalized One-handed Input for Touchscreen-based Mobile Devices. In Proceedings of the 11th IFIP TC 13 International Conference on Humancomputer Interaction (INTERACT'07). Springer-Verlag, Berlin, Heidelberg, 324--338. http://dl.acm.org/citation.cfm?id=1776994.1777034 Google ScholarDigital Library
- Takashi Kikuchi, Yuta Sugiura, Katsutoshi Masai, Maki Sugimoto, and Bruce H. Thomas. 2017. EarTouch: Turning the Ear into an Input Surface. In Proceedings of the 19th International Conference on HumanComputer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 27, 6 pages. Google ScholarDigital Library
- Jee-Eun Kim, Masahiro Bessho, Shinsuke Kobayashi, Noboru Koshizuka, and Ken Sakamura. 2016. Navigating Visually Impaired Travelers in a Large Train Station Using Smartphone and Bluetooth Low Energy. In Proceedings of the 31st Annual ACM Symposium on Applied Computing (SAC '16). ACM, New York, NY, USA, 604--611. Google ScholarDigital Library
- Andreas Kunz, Klaus Miesenberger, Max MÃhlhÃuser, Ali Alavi, Stephan PÃlzer, Daniel PÃll, Peter Heumader, and Dirk SchnelleWalka. 2014. Accessibility of Brainstorming Sessions for Blind People. In International Conference on Computers Helping People with Special Needs. 237--244.Google Scholar
- Jianwei Lai and Dongsong Zhang. 2014. ExtendedThumb: A Motionbased Virtual Thumb for Improving One-handed Target Acquisition on Touch-screen Mobile Devices. In Proceedings of the Extended Abstracts of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI EA '14). ACM, New York, NY, USA, 1825--1830. Google ScholarDigital Library
- Barbara Leporini, Maria Claudia Buzzi, and Marina Buzzi. 2012. Interacting with Mobile Devices via VoiceOver: Usability and Accessibility Issues. In Proceedings of the 24th Australian Computer-Human Interaction Conference (OzCHI '12). ACM, New York, NY, USA, 339--348. Google ScholarDigital Library
- Frank Chun Yat Li, David Dearman, and Khai N. Truong. 2010. Leveraging Proprioception to Make Mobile Phones More Accessible to Users with Visual Impairments. In Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '10). ACM, New York, NY, USA, 187--194. Google ScholarDigital Library
- Roman Lissermann, Jochen Huber, Aristotelis Hadjakos, and Max Mühlhäuser. 2013. EarPut: Augmenting Behind-the-ear Devices for Ear-based Interaction. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, New York, NY, USA, 1323-- 1328. Google ScholarDigital Library
- Roman Lissermann, Jochen Huber, Aristotelis Hadjakos, Suranga Nanayakkara, and Max Mühlhäuser. 2014. EarPut: Augmenting Earworn Devices for Ear-based Interaction. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design (OzCHI '14). ACM, New York, NY, USA, 300--307. Google ScholarDigital Library
- Martez E. Mott, Radu-Daniel Vatavu, Shaun K. Kane, and Jacob O. Wobbrock. 2016. Smart Touch: Improving Touch Accuracy for People with Motor Impairments with Template Matching. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1934--1946. Google ScholarDigital Library
- Maia Naftali and Leah Findlater. 2014. Accessibility in Context: Understanding the Truly Mobile Experience of Smartphone Users with Motor Impairments. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS '14). ACM, New York, NY, USA, 209--216. Google ScholarDigital Library
- Uran Oh and Leah Findlater. 2014. Design of and Subjective Response to On-body Input for People with Visual Impairments. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS '14). ACM, New York, NY, USA, 115--122. Google ScholarDigital Library
- Uran Oh, Lee Stearns, Alisha Pradhan, Jon E. Froehlich, and Leah Findlater. 2017. Investigating Microinteractions for People with Visual Impairments and the Potential Role of On-Body Interaction. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, New York, NY, USA, 22--31. Google ScholarDigital Library
- Kyudong Park, Taedong Goh, and Hyo-Jeong So. 2014. Toward Accessible Mobile Application Design: Developing Mobile Application Accessibility Guidelines for People with Visual Impairment. In Proceedings of HCI Korea (HCIK '15). Hanbit Media, Inc., South Korea, 31--38. http://dl.acm.org/citation.cfm?id=2729485.2729491 Google ScholarDigital Library
- Huimin Qian, Ravi Kuber, and Andrew Sears. 2013. Developing Tactile Icons to Support Mobile Users with Situationally-induced Impairments and Disabilities. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13). ACM, New York, NY, USA, Article 47, 2 pages. Google ScholarDigital Library
- André Rodrigues, Kyle Montague, Hugo Nicolau, and Tiago Guerreiro. 2015. Getting Smartphones to Talkback: Understanding the Smartphone Adoption Process of Blind Users. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS '15). ACM, New York, NY, USA, 23--32. Google ScholarDigital Library
- Andrew Sears, Min Lin, Julie Jacko, and Yan Xiao. 2003. When Computers Fade Pervasive Computing and Situationally-Induced Impairments and Disabilities.Google Scholar
- Karsten Seipp and Kate Devlin. 2014. Backpat: Improving One-handed Touchscreen Operation by Patting the Back of the Device. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, New York, NY, USA, 555--558. Google ScholarDigital Library
- Karsten Seipp and Kate Devlin. 2014. BackPat: One-handed Off-screen Patting Gestures. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI '14). ACM, New York, NY, USA, 77--80. Google ScholarDigital Library
- K. Seipp and K. Devlin. 2014. The one hand wonder:: A framework for enhancing one-handed website operation on touchscreen smartphones. In International Conference on Web Information Systems and Technologies. 5--13.Google Scholar
- Kristen Shinohara and Jacob O. Wobbrock. 2011. In the Shadow of Misperception: Assistive Technology Use and Social Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 705--714. Google ScholarDigital Library
- Ke Sun, Yuntao Wang, Chun Yu, Yukang Yan, Hongyi Wen, and Yuanchun Shi. 2017. Float: One-Handed and Touch-Free Target Selection on Smartwatches. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 692--704. Google ScholarDigital Library
- Jacob O. Wobbrock, Shaun K. Kane, Krzysztof Z. Gajos, Susumu Harada, and Jon Froehlich. 2011. Ability-Based Design: Concept, Principles and Examples. ACM Trans. Access. Comput. 3, 3, Article 9 (April 2011), 27 pages. Google ScholarDigital Library
- Pui Chung Wong, Hongbo Fu, and Kening Zhu. 2016. Back-Mirror: Back-of-device One-handed Interaction on Smartphones. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications (SA '16). ACM, New York, NY, USA, Article 10, 5 pages. Google ScholarDigital Library
- Koki Yamashita, Takashi Kikuchi, Katsutoshi Masai, Maki Sugimoto, Bruce H. Thomas, and Yuta Sugiura. 2017. CheekInput: Turning Your Cheek into an Input Surface by Embedded Optical Sensors on a Headmounted Display. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST '17). ACM, New York, NY, USA, Article 19, 8 pages. Google ScholarDigital Library
- Hanlu Ye, Meethu Malu, Uran Oh, and Leah Findlater. 2014. Current and Future Mobile and Wearable Device Use by People with Visual Impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3123--3132. Google ScholarDigital Library
- Hui-Shyong Yeo, Xiao-Shen Phang, Steven J. Castellucci, Per Ola Kristensson, and Aaron Quigley. 2017. Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4194--4202. Google ScholarDigital Library
- Neng-Hao Yu, Da-Yuan Huang, Jia-Jyun Hsu, and Yi-Ping Hung. 2013. Rapid Selection of Hard-to-access Targets by Thumb on Mobile Touchscreens. In Proceedings of the 15th International Conference on Humancomputer Interaction with Mobile Devices and Services (MobileHCI '13). ACM, New York, NY, USA, 400--403. Google ScholarDigital Library
- Adam Zarek, Daniel Wigdor, and Karan Singh. 2012. SNOUT: Onehanded Use of Capacitive Touch Devices. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '12). ACM, New York, NY, USA, 140--147. Google ScholarDigital Library
Index Terms
- EarTouch: Facilitating Smartphone Use for Visually Impaired People in Mobile and Public Scenarios
Recommendations
Smartphone Usage by Expert Blind Users
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsPeople with vision impairments access smartphones with the help of screen reader apps such as TalkBack for Android and VoiceOver for iPhone. Prior research has mostly focused on understanding touchscreen phone adoption and typing performance of novice ...
LucentMaps: 3D Printed Audiovisual Tactile Maps for Blind and Visually Impaired People
ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and AccessibilityTactile maps support blind and visually impaired people in orientation and to familiarize with unfamiliar environments. Interactive approaches complement these maps with auditory feedback. However, commonly these approaches focus on blind people. We ...
Impact of mobile tangible slider design and its reachability on pointing performance
IHM '16: Actes de la 28ième conference francophone sur l'Interaction Homme-MachineTouch interaction is almost omnipresent on mobile devices and technological advances have made screens bigger. The users can then interact with more content displayed on screen but it is more difficult for the users to reach all the parts of the screen ...
Comments