Haimo Zhang
ABSTRACT
Stroke gestures are intuitive and efficient but often require gesture-capable input hardware such as a touchscreen. In this paper, we present GestKeyboard, a novel technique for gesturing over an ordinary, unmodified physical keyboard that remains the major input modality for existing desktop and laptop computers. We discuss an exploratory study for understanding the design space of gesturing on a physical keyboard and our algorithms for detecting gestures in a modeless way, without interfering with the keyboard's major functionality such as text entry and shortcuts activation. We explored various features for detecting gestures from a keyboard event stream. Our experiment based on the data collected from 10 participants indicated it is feasible to reliably detect gestures from normal keyboard use, 95% detection accuracy within a maximum latency of 200ms.
Supplemental Material
- Bailly, G., Pietrzak, T., Deber, J., and Wigdor, D. Métamorphe: augmenting hotkey usage with actuated keys. In Proc. CHI 2013, ACM Press (2013), 563--572. Google Scholar
Digital Library
- Block, F., Gellersen, H., and Villar, N. Touch-Display Keyboards: Transforming Keyboards into Interactive Surfaces. In Proc. CHI 2010, ACM Press (2010), 1145--1154. Google ScholarDigital Library
- Dietz, P., Eidelson, B., Westhues, J., and Bathiche, S. A Practical Pressure Sensitive Computer Keyboard. In Proc. UIST 2009, ACM Press (2009), 55--58. Google ScholarDigital Library
- Dribin, D. Keyboard Matrix Help. http://www.dribin.org/dave/keyboard/one_html/.Google Scholar
- Jannotti, J. Iconic Text Entry Using a Numeric Keypad. Unpublished, 2002. http://pdos.csail.mit.edu/~jj/jannotti.com/papers/iconicuist02.pdf.Google Scholar
- Kane, S., Avrahami, D., Wobbrock, J., Harrison, B., Rea, A., Philipose, M., and LaMarca, A. Bonfire: A Nomadic System for Hybrid Laptop-Tabletop Interaction. In Proc. UIST 2009, ACM Press (2009), 129--138. Google ScholarDigital Library
- Keyboard Ghosting Explained. http://www.microsoft.com/appliedsciences/antighostingexplained.mspx. Retrieved September 15, 2013.Google Scholar
- Leap Motion. https://www.leapmotion.com/. Retrieved September 15, 2013.Google Scholar
- Li, Y., Protractor: A Fast and Accurate Gesture Recognizer. In Proc. CHI 2010, ACM Press (2010), 2169--2172. Google ScholarDigital Library
- Wigdor, D., Benko, H., Pella, J., Lombardo, J., and Williams, S. Rock & Rails: Extending Multi-Touch Interaction with Shape Gestures to Enable Precise Spatial Manipulations. In Proc. CHI 2011, ACM Press (2011), 1581--1590. Google ScholarDigital Library
- Wilson, A. Robust Computer Vision-Based Detection of Pinching for One and Two-Handed Gesture Input, In Proc. UIST 2006, ACM Press (2006), 255--258. Google ScholarDigital Library
- Wobbrock, J., Myers, B., and Rothrock, B. Few-Key Text Entry Revisited: Mnemonic Gestures on Four Keys. In Proc. CHI 2006, ACM Press (2006), 489--492. Google ScholarDigital Library
- Wobbrock, J., Wilson, A., and Li., Y. Gestures without libraries, toolkits or Training: a $1.00 Recognizer for User Interface Prototypes. In Proc. UIST 2007, ACM Press (2007), 159--168. Google ScholarDigital Library
- Zhai, S., Kristensson, P. O., Gong, P., Greiner, M., Peng, S. A., Liu, L. M., and Dunnigan, A. ShapeWriter on the iPhone - From the Laboratory to the Real World. In Proc. CHI EA 2009, ACM Press (2009), 2667--2670. Google ScholarDigital Library
Index Terms
- GestKeyboard: enabling gesture-based interaction on ordinary physical keyboard
Recommendations
Efficient Typing on a Visually Occluded Physical Keyboard
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing SystemsThe rise of affordable head-mounted displays (HMDs) has raised questions about how to best design user interfaces for this technology. This paper focuses on the use of HMDs for home and office applications that require substantial text input. A physical ...
GestKeyboard: enabling gesture-based interaction on ordinary physical keyboard
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing SystemsStroke gestures are intuitive and efficient but often require gesture-capable input hardware such as a touchscreen. In this paper, we present GestKeyboard, a novel technique for gesturing over an ordinary, unmodified physical keyboard - that remains the ...
DownChord and UpChord: A New Style of Keyboard Shortcuts based on Simultaneous Key-down and Key-up Events
Chinese CHI '16: Proceedings of the Fourth International Symposium of Chinese CHITypical typing, as well as many other keyboard operations including using modifier-based shortcuts, is dependent upon the order of key press and avoids simultaneous key pressing. This paper introduces DownChord and UpChord, novel keyboard shortcut ...
Comments