ABSTRACT
We present two contributions toward improving the accessibility of touch screens for people with motor impairments. First, we provide an exploration of the touch behaviors of 10 people with motor impairments, e.g., we describe how touching with the back or sides of the hand, with multiple fingers, or with knuckles creates varied multi-point touches. Second, we introduce Smart Touch, a novel template-matching technique for touch input that maps any number of arbitrary contact-areas to a user's intended (x,y) target location. The result is that users with motor impairments can touch however their abilities allow, and Smart Touch will resolve their intended touch point. Smart Touch therefore allows users to touch targets in whichever ways are most comfortable and natural for them. In an experimental evaluation, we found that Smart Touch predicted (x,y) coordinates of the users' intended target locations over three times closer to the intended target than the native Land-on and Lift-off techniques reported by the built-in touch sensors found in the Microsoft PixelSense interactive tabletop. This result is an important step toward improving touch accuracy for people with motor impairments and others for whom touch screen operation was previously impossible.
Supplemental Material
Available for Download
pn1750-file4.zip
- Pär-Anders Albinsson and Shumin Zhai. 2003. High precision touch screen interaction. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '03), ACM Press, 105--112. http://doi.org/10.1145/642611.642631 Google ScholarDigital Library
- Lisa Anthony, YooJin Kim, and Leah Findlater. 2013. Analyzing user-generated Youtube videos to understand touchscreen use by people with motor impairments. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '13), ACM Press, 1223--1232. http://doi.org/10.1145/2470654.2466158 Google ScholarDigital Library
- Ravin Balakrishnan. 2004. "Beating" Fitts' law: Virtual enhancements for pointing facilitation. International Journal of Human-Computer Studies 61, 6: 857--874. http://doi.org/10.1016/j.ijhcs.2004.09.002 Google ScholarDigital Library
- Hrvoje Benko, Andrew D. Wilson, and Patrick Baudisch. 2006. Precise selection techniques for multitouch screens. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '06), ACM, 1263--1272. http://doi.org/10.1145/1124772.1124963 Google ScholarDigital Library
- Pradipta Biswas and Patrick Langdon. 2012. Developing multimodal adaptation algorithm for mobility impaired users by evaluating their hand strength. International Journal of Human-Computer Interaction 28, 9: 576--596. http://doi.org/10.1080/10447318.2011.636294Google ScholarCross Ref
- Xiaojun Bi, Yang Li, and Shumin Zhai. 2013. FFitts law: Modeling finger touch with Fitts' law. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '13), ACM, 1363--1372. http://doi.org/10.1145/2470654.2466180 Google ScholarDigital Library
- Xiaojun Bi and Shumin Zhai. 2013. Bayesian touch: A statistical criterion of target selection with finger touch. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '13), ACM Press, 51--60. http://doi.org/10.1145/2501988.2502058 Google ScholarDigital Library
- Karen B. Chen, Anne B. Savage, Amrish O. Chourasia, Douglas A. Wiegmann, and Mary E. Sesto. 2013. Touch screen performance by individuals with and without motor control disabilities. Applied Ergonomics 44, 2: 297--302. http://doi.org/10.1016/j.apergo.2012.08.004Google ScholarCross Ref
- Morgan Dixon, James Fogarty, and Jacob Wobbrock. 2012. A general-purpose target-aware pointing enhancement using pixel-level analysis of graphical interfaces. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '12), ACM Press, 3167--3176. http://doi.org/10.1145/2207676.2208734 Google ScholarDigital Library
- Sacha N. Duff, Curt B. Irwin, Jennifer L. Skye, Mary E. Sesto, and Douglas A. Wiegmann. 2010. The effect of disability and approach on touch screen performance during a number entry task. Proceedings of the Human Factors and Ergonomics Society, 566--570.Google Scholar
- Abigail Evans and Jacob Wobbrock. 2012. Taming wild behavior: The input observer for obtaining text entry and mouse pointing measures from everyday computer use. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '12), ACM, 1947--1956. http://doi.org/10.1145/2207676.2208338 Google ScholarDigital Library
- Leah Findlater, Alex Jansen, Kristen Shinohara, Morgan Dixon, Peter Kamb, Joshua Rakita, and Jacob O. Wobbrock. 2010. Enhanced area cursors: Reducing fine pointing demands for people with motor impairments. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '10), ACM Press, 153--162. http://doi.org/10.1145/1866029.1866055 Google ScholarDigital Library
- Jon Froehlich, Jacob O. Wobbrock, and Shaun K. Kane. 2007. Barrier pointing: Using physical edges to assist target acquisition on mobile device touch screens. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '07), ACM Press, 19--26. http://doi.org/10.1145/1296843.1296849 Google ScholarDigital Library
- Krzysztof Z. Gajos, Jacob O. Wobbrock, and Daniel S. Weld. 2007. Automatically generating user interfaces adapted to users' motor and vision capabilities. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '07), ACM Press, 231--240. http://doi.org/10.1145/1294211.1294253 Google ScholarDigital Library
- Krzysztof Z. Gajos, Jacob O. Wobbrock, and Daniel S. Weld. 2008. Improving the performance of motorimpaired users with automatically-generated, abilitybased interfaces. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '08), ACM, 1257--1266. http://doi.org/10.1145/1357054.1357250 Google ScholarDigital Library
- Tiago João Vieira Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2010. Assessing mobile touch interfaces for tetraplegics. Proceedings of the ACM Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '10), ACM Press, 31--34. http://doi.org/10.1145/1851600.1851608 Google ScholarDigital Library
- Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2010. Towards accessible touch interfaces. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '10), ACM, 19--26. http://doi.org/10.1145/1878803.1878809 Google ScholarDigital Library
- Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 2: 65--70.Google Scholar
- Christian Holz and Patrick Baudisch. 2010. The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '10), ACM, 581--590. http://doi.org/10.1145/1753326.1753413 Google ScholarDigital Library
- Christian Holz and Patrick Baudisch. 2011. Understanding touch. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '11), ACM, 2501--2510. http://doi.org/10.1145/1978942.1979308 Google ScholarDigital Library
- Curt B. Irwin and Mary E. Sesto. 2012. Performance and touch characteristics of disabled and non-disabled participants during a reciprocal tapping task using touch screen technology. Applied Ergonomics 43, 6: 1038--1043. http://doi.org/10.1016/j.apergo.2012.03.003Google ScholarCross Ref
- Shaun K. Kane, Chandrika Jayant, Jacob O. Wobbrock, and Richard E. Ladner. 2009. Freedom to roam: A study of mobile device adoption and accessibility for people with visual and motor disabilities. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '09), ACM, 115--122. http://doi.org/10.1145/1639642.1639663 Google ScholarDigital Library
- R. C. Littell, P. R. Henry, and C. B. Ammerman. 1998. Statistical analysis of repeated measures data using SAS procedures. Journal of Animal Science 76, 4: 1216--1231.Google ScholarCross Ref
- Tom M. Mitchell. 1997. Machine Learning. McGrawHill Education, NY. Google ScholarDigital Library
- Kyle Montague, Vicki L. Hanson, and Andy Cobley. 2012. Designing for individuals: Usable touch-screen interaction through shared user models. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '12), ACM Press, 151--158. http://doi.org/10.1145/2384916.2384943 Google ScholarDigital Library
- Kyle Montague, Hugo Nicolau, and Vicki L. Hanson. 2014. Motor-impaired touchscreen interactions in the wild. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '14), ACM, 123--130. http://doi.org/10.1145/2661334.2661362 Google ScholarDigital Library
- Maia Naftali and Leah Findlater. 2014. Accessibility in context: Understanding the truly mobile experience of smartphone users with motor impairments. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '14), ACM Press, 209--216. http://doi.org/10.1145/2661334.2661372 Google ScholarDigital Library
- Pekka Parhi, Amy K. Karlson, and Benjamin B. Bederson. 2006. Target size study for one-handed thumb use on small touchscreen devices. Proceedings of the ACM Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '06), ACM Press, 203--210. http://doi.org/10.1145/1152215.1152260 Google ScholarDigital Library
- R. L. Potter, L. J. Weldon, and B. Shneiderman. 1988. Improving the accuracy of touch screens: An experimental evaluation of three strategies. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '88), ACM, 27--32. http://doi.org/10.1145/57167.57171 Google ScholarDigital Library
- Julia Schwarz, Scott Hudson, Jennifer Mankoff, and Andrew D. Wilson. 2010. A framework for robust and flexible handling of inputs with uncertainty. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '10), ACM Press, 47--56. http://doi.org/10.1145/1866029.1866039 Google ScholarDigital Library
- Julia Schwarz, Jennifer Mankoff, and Scott Hudson. 2011. Monte Carlo methods for managing interactive state, action and feedback under uncertainty. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '11), ACM, 235--244. http://doi.org/10.1145/2047196.2047227 Google ScholarDigital Library
- Julia Schwarz, Jennifer Mankoff, and Scott E. Hudson. 2015. An architecture for generating interactive feedback in probabilistic user interfaces. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '15), ACM Press, 2545--2554. http://doi.org/10.1145/2702123.2702228 Google ScholarDigital Library
- Andrew Sears, Min Lin, Julie Jacko, and Yan Xiao. 2003. When computers fade -- pervasive computing and situationally-induced impairments and disabilities. Proceedings of the 10th International Conference on Human-Computer Interaction (HCI Int'l' 03), 1298--1302.Google Scholar
- Andrew Sears and Ben Shneiderman. 1991. High precision touchscreens: Design strategies and comparisons with a mouse. International Journal of Man-Machine Studies 34, 4: 593--613. http://doi.org/10.1016/0020--7373(91)90037-8 Google ScholarDigital Library
- Shari Trewin, Cal Swart, and Donna Pettick. 2013. Physical accessibility of touchscreen smartphones. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13), ACM, 19:1--19:8. http://doi.org/10.1145/2513383.2513446 Google ScholarDigital Library
- Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock. 2012. Gestures as point clouds: A $P recognizer for user interface prototypes. Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '12), ACM, 273--280. http://doi.org/10.1145/2388676.2388732 Google ScholarDigital Library
- Daniel Vogel and Patrick Baudisch. 2007. Shift: A technique for operating pen-based interfaces using touch. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '07), ACM, 657--666. http://doi.org/10.1145/1240624.1240727 Google ScholarDigital Library
- Chat Wacharamanotham, Jan Hurtmanns, Alexander Mertens, Martin Kronenbuerger, Christopher Schlick, and Jan Borchers. 2011. Evaluating swabbing: A touchscreen input method for elderly users with tremor. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '11), ACM Press, 623--626. http://doi.org/10.1145/1978942.1979031 Google ScholarDigital Library
- Daryl Weir. 2012. Machine learning models for uncertain interaction. Adjunct Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12), ACM, 31--34. http://doi.org/10.1145/2380296.2380313 Google ScholarDigital Library
- Daryl Weir, Simon Rogers, Roderick Murray-Smith, and Markus Löchtefeld. 2012. A user-specific machine learning approach for improving touch accuracy on mobile devices. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '12), ACM Press, 465--476. http://doi.org/10.1145/2380116.2380175 Google ScholarDigital Library
- Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell, and Chia Shen. 2007. Lucid Touch: A seethrough mobile device. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '07), ACM Press, 269--278. http://doi.org/10.1145/1294211.1294259 Google ScholarDigital Library
- Jacob O. Wobbrock. 2014. Improving pointing in graphical user interfaces for people with motor impairments through ability-based design. In Assitive Technologies and Computer Access for Motor Disabilities. IGI Global, Hershey, PA, 206--253.Google Scholar
- Jacob O. Wobbrock, James Fogarty, Shih-Yen (Sean) Liu, Shunichi Kimuro, and Susumu Harada. 2009. The Angle Mouse: Target-agnostic dynamic gain adjustment based on angular deviation. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '09), ACM, 1401--1410. http://doi.org/10.1145/1518701.1518912 Google ScholarDigital Library
- Jacob O. Wobbrock, Shaun K. Kane, Krzysztof Z. Gajos, Susumu Harada, and Jon Froehlich. 2011. Ability-based design: Concept, principles and examples. ACM Transactions on Accessible Computing 3, 3: 9:1--9:27. http://doi.org/10.1145/1952383.1952384 Google ScholarDigital Library
- Jacob O. Wobbrock, Brad A. Myers, and John A. Kembel. 2003. EdgeWrite: A stylus-based text entry method designed for high accuracy and stability of motion. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '03), ACM Press, 61--70. http://doi.org/10.1145/964696.964703 Google ScholarDigital Library
- Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '07), ACM, 159--168. http://doi.org/10.1145/1294211.1294238 Google ScholarDigital Library
Index Terms
- Smart Touch: Improving Touch Accuracy for People with Motor Impairments with Template Matching
Recommendations
Cluster Touch: Improving Touch Accuracy on Smartphones for People with Motor and Situational Impairments
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsWe present Cluster Touch, a combined user-independent and user-specific touch offset model that improves the accuracy of touch input on smartphones for people with motor impairments, and for people experiencing situational impairments while walking. ...
Accessible Touch Input for People with Motor Impairments
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing SystemsTouch input has emerged as a dominant form of interaction for billions of interactive computing devices like smartphones, tablets, and interactive surfaces. Although touch input is widely used, it remains largely inaccessible for many people with motor ...
MagicPAPER: An Integrated Shadow-Art Hardware Device Enabling Touch Interaction on Paper
DIS '18 Companion: Proceedings of the 2018 ACM Conference Companion Publication on Designing Interactive SystemsTouch interaction is usually implemented on a computer screen or a projected surface. However, tactile quality of paper is attractive and intuitive for people to manipulate in traditional paintings and storytelling. In order to enable touch interaction ...
Comments