Jacqui Fashimpaur
ABSTRACT
Text entry is an integral component to many use cases in virtual and augmented reality. We present PinchType: A new method of virtual text entry that combines users' existing knowledge of the QWERTY keyboard layout with simple thumb and finger interactions. Users pinch with the thumb and fingertip to select from the same group of letters the finger would press on a QWERTY keyboard; a language model disambiguates. In a preliminary study with 14 participants, we investigated PinchType's speed and accuracy on initial use, as well as its physical comfort relative to a mid-air keyboard. After entering 40 phrases, most people reported that PinchType was more comfortable than the mid-air keyboard. Most participants reached a mean speed of 12.54 WPM, or 20.07 WPM without the time spent correcting errors. This compares favorably to other thumb-to-finger virtual text entry methods.
Supplemental Material
Available for Download
Preview video captions
- 2019. T9 Text Input: The Global Standard for Mobile Text Input. (2019). https://www.nuance.com/mobile/mobile-solutions/text-input-solutions/t9.htmlGoogle Scholar
- D. A. Bowman, C. A. Wingrave, J. M. Campbell, V. Q. Ly, and C. J. Rhoton. 2002. Novel Uses of Pinch GlovesTM for Virtual Environment Interaction Techniques. Virtual Reality 6, 3 (Jan 2002), 122--129. DOI:http://dx.doi.org/10.1007/s100550200013Google ScholarCross Ref
- John Dudley, Hrvoje Benko, Daniel Wigdor, and Per Ola Kristensson. 2019. Performance envelopes of virtual keyboard text input strategies in virtual reality. Proceedings of the 17th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2019) (2019). DOI: http://dx.doi.org/10.17863/CAM.41874Google ScholarCross Ref
- Nathan Green, Jan Kruger, Chirag Faldu, and Robert St. Amant. 2004. A reduced QWERTY keyboard for mobile text entry. Extended abstracts of the 2004 conference on Human factors and computing systems - CHI 04 (2004). DOI: http://dx.doi.org/10.1145/985921.986082Google ScholarDigital Library
- Shangchen Han, Beibei Liu, Robert Wang, Yuting Ye, Christopher D. Twigg, and Kenrick Kin. 2018. Online optical marker-based hand tracking with deep labels. ACM Transactions on Graphics 37, 4 (2018), 1--10. DOI:http://dx.doi.org/10.1145/3197517.3201399Google ScholarDigital Library
- Haiyan Jiang, Dongdong Weng, Zhenliang Zhang, and Feng Chen. 2019. HiFinger: One-Handed Text Entry Technique for Virtual Environments Based on Touches between Fingers. Sensors 19, 14 (Nov 2019), 3063. DOI:http://dx.doi.org/10.3390/s19143063Google ScholarCross Ref
- Falko Kuester, Michelle Chen, Mark E. Phair, and Carsten Mehring. 2005. Towards keyboard independent touch typing in VR. Proceedings of the ACM symposium on Virtual reality software and technology - VRST 05 (2005), 86--95. DOI: http://dx.doi.org/10.1145/1101616.1101635Google ScholarDigital Library
- I. Scott Mackenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. CHI 03 extended abstracts on Human factors in computer systems - CHI 03 (2003). DOI: http://dx.doi.org/10.1145/765968.765971Google ScholarCross Ref
- Anders Markussen, Mikkel R. Jakobsen, and Kasper Hornbæk. 2013. Selection-Based Mid-Air Text Entry on Large Displays. Human-Computer Interaction INTERACT 2013 Lecture Notes in Computer Science (2013), 401--418. DOI: http://dx.doi.org/10.1007/978--3--642--40483--2_28Google ScholarCross Ref
- Anders Markussen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2014. Vulture: A Mid-Air Word-Gesture Keyboard. Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI 14 (2014), 1073--1082. DOI: http://dx.doi.org/10.1145/2556288.2556964Google ScholarDigital Library
- Anna Peshock, Lucy E. Dunne, and Julia Duvall. 2014. Argot: A wearable one-handed keyboard glove. Proceedings of the 2014 ACM International Symposium on Wearable Computers Adjunct Program - ISWC 14 Adjunct (2014). DOI: http://dx.doi.org/10.1145/2641248.2641266Google ScholarDigital Library
- Eric Whitmire, Mohit Jain, Divye Jain, Greg Nelson, Ravi Karkar, Shwetak Patel, and Mayank Goel. 2017. DigiTouch: Reconfigurable Thumb-to-Finger Input and Text Entry on Head-mounted Displays. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (Sept 2017). DOI: http://dx.doi.org/10.1145/3130978Google ScholarDigital Library
- Pui Chung Wong, Kening Zhu, and Hongbo Fu. 2018. FingerT9: Leveraging Thumb-to-finger Interaction for Same-side-hand Text Entry on Smartwatches. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI 18 (2018). DOI: http://dx.doi.org/10.1145/3173574.3173752Google ScholarDigital Library
- Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data. Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology UIST 15 (2015), 539--548. DOI: http://dx.doi.org/10.1145/2807442.2807504Google ScholarDigital Library
Index Terms
- PinchType: Text Entry for Virtual and Augmented Reality Using Comfortable Thumb to Fingertip Pinches
Recommendations
HawKEY: Efficient and Versatile Text Entry for Virtual Reality
VRST '19: Proceedings of the 25th ACM Symposium on Virtual Reality Software and TechnologyText entry is still a challenging task in modern Virtual Reality (VR) systems. The lack of efficient text entry methods limits the applications that can be used productively in VR. Previous work has addressed this issue through virtual keyboards or ...
iText: Hands-free Text Entry on an Imaginary Keyboard for Augmented Reality Systems
UIST '21: The 34th Annual ACM Symposium on User Interface Software and TechnologyText entry is an important and frequent task in interactive devices including augmented reality head-mounted displays (AR HMDs). In current AR HMDs, there are still two main open challenges to overcome for efficient and usable text entry: arm fatigue ...
Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsEntering text is one of the most common tasks when interacting with computing systems. Virtual Reality (VR) presents a challenge as neither the user's hands nor the physical input devices are directly visible. Hence, conventional desktop peripherals are ...
Comments