Fraser Anderson
ABSTRACT
Gesture-based interfaces are becoming more prevalent and complex, requiring non-trivial learning of gesture sets. Many methods for learning gestures have been proposed, but they are often evaluated with short-term recall tests that measure user performance, rather than learning. We evaluated four types of gesture guides using a retention and transfer paradigm common in motor learning experiments and found results different from those typically reported with recall tests. The results indicate that many guide systems with higher levels of guidance exhibit high performance benefits while the guide is being used, but are ultimately detrimental to user learning. We propose an adaptive guide that does not suffer from these drawbacks, and that enables a smooth transition from novice to expert. The results contrasting learning and performance can be explained by the guidance hypothesis. They have important implications for the design and evaluation of future gesture learning systems.
- Albaret, J. M. and Thon, B. Differential effects of task complexity on contextual interference in a drawing task. Acta Psychologia, 100(1-2), 1998, pp. 9--24.Google Scholar
Cross Ref
- Appert, C. and Bau, O. Scale Detection for a priori Gesture Recognition. In Proc. ACM CHI 2010. pp. 879--882. Google ScholarDigital Library
- Appert C. and Zhai, S. Using strokes as command shortcuts: cognitive benefits and toolkit support. In Proc. ACM CHI 2009, pp. 2289--2298. Google ScholarDigital Library
- Bau O. and Mackay W. OctoPocus: a dynamic guide for learning gesture-based command sets. In Proc. ACM UIST 2009, pp. 37--46. Google ScholarDigital Library
- Bennett, M. McCarthy, K., O'Modhrain, S. and Smyth, B. Simpleflow: enhancing gestural interaction with gesture prediction, abbreviation and autocompletion. In Proc. INTERACT 2011, pp. 591--608. Google ScholarDigital Library
- Bragdon, A., Uguray, A., Wigdor, D., Anagnostopoulos, S., Zeleznik, R., and Feman, R. Gesture play: motivating online gesture learning with fun, positive reinforcement and physical metaphors. In Proc. ACM ITS 2010, pp. 39--48. Google ScholarDigital Library
- Bragdon, A., Zeleznik, R., Williamson, B., Miller, T., and LaViola J. J. GestureBar: improving the approachability of gesture-based interfaces. In Proc. ACM UIST 2008, pp. 2269--2278. Google ScholarDigital Library
- Brydges, R., Carnahan, H., Backstein D. and Dubrowski, A. Application of Motor Learning Principles to Complex Surgical Tasks: Searching for the Optimal Practice Schedule. Journal of Motor Behavior, 39(1), 2007, pp. 40--48.Google ScholarCross Ref
- Cao, X. and Zhai, S. Modeling human performance of pen stroke gestures. In Proc. ACM CHI 2007, pp. 1495--1504. Google ScholarDigital Library
- Chapman, C. S., Gallivan, J. P., Wood, D. W., Milne, J. L., Culham, J. C. and Goodale, M. A. Reaching for the unknown: Multiple target encoding and real-time decision-making in a rapid reach task. Cognition, 2010, 116(2), pp. 168--176.Google ScholarCross Ref
- Cockburn, A., Kristensson, P., Alexander, J. and Zhai, S. Hard lessons: effort-inducing interfaces benefit spatial learning. In Proc. ACM CHI 2007, pp. 1571--1580. Google ScholarDigital Library
- Fitts, P. M. and Posner, M. I. Human Performance. Brooks and Cole, Oxford, England, 1967.Google Scholar
- Freeman, D., Benko, H., Morris, M. R. and Wigdor, D. ShadowGuides: Visualizations for in-situ learning of multi-touch and whole-hand gestures. In Proc. of ACM ITS 2009, pp. 165--172. Google ScholarDigital Library
- Grossman, T., Dragicevic, P. and Balakrishnan, R. Strategies for accelerating online learning of hotkeys. In Proc. ACM CHI 2007, 1591--1600. Google ScholarDigital Library
- Helsen, W. F., Hodges, N. J., Van Winckel, J. and Starkes, J. L. The roles of talent, physical precocity and practive in the development of soccer expertise. Journal of Sports Sciences, 18(9), 2000, pp. 727--736.Google ScholarCross Ref
- Kristensson, P. O. and Denby, L. C. Continuous recognition and visualization of pen strokes and touch-screen gestures. In Proc. ACM Eurographics SBIM 2011, pp. 95--102. Google ScholarDigital Library
- Kristensson, P. O. and Zhai, S. SHARK: a large vocabulary shorthand writing system for pen-based computers. In Proc. ACM UIST 2004, pp. 43--52. Google ScholarDigital Library
- Kurtenbach, G. P., Sellen, A. J., and Buxton, W. A. S. An empirical evaluation of some articulatory and cognitive aspects of marking menus. Journal of Human Computer Interaction, 8(1), 1993, pp. 1--23. Google ScholarDigital Library
- Kurtenbach, G., Moran, T. P., and Buxton, W. Contextual animation of gestural commands. Computer Graphics Forum, 13(5), 1994, pp. 305--314.Google ScholarCross Ref
- Lai, Q. and Shea, C. H. The role of reduced frequency of knowledge of results during constant practice. Research Quarterly for Exercise and Sport, 70, 1999, pp. 33--40.Google ScholarCross Ref
- Lepinski, J., Grossman, T., and Fitzmaurice, G. The design and evaluation of multitouch marking menus. In Proc. CHI 2010, pp. 2233--2242. Google ScholarDigital Library
- Oldfield, R. C. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9(1), 1971, pp. 97--113.Google ScholarCross Ref
- Ouyang, T. and Li, Y. Bootstrapping Personal Gesture Shortcuts with the Wisdom of the Crowd and Handwriting Recognition. In Proc. CHI 2012, pp. 2895--2904. Google ScholarDigital Library
- Park, J., Shea, C. H. and Wright, D. L. Reduced frequency concurrent and terminal feedback: A test of the guidance hypothesis. Journal of Motor Behavior, 32, 2000, pp. 278--296.Google ScholarCross Ref
- Panzer S., Krueger, M., Muehlbauer, T. and Shea, C. H. Asymmetric effector transfer of complex movement sequences. Human Movement Science, 2010 29(1), pp. 62--72.Google ScholarCross Ref
- Rappin, N., Guzdial, M. Realff, M., and Ludovice, P. Balancing usability and learning in an interface. In Proc. ACM CHI 1997, pp. 470--486. Google ScholarDigital Library
- Rubine, D. Specifying gestures by example. In SIGGRAPH 1991, pp. 239--337. Google ScholarDigital Library
- Sainburg, R. L. and Wang, J. Interlimb transfer of visuomotor rotations: independence of direction and final position information. Experimental Brain Research, 2002, 145(4), pp. 437--447.Google ScholarCross Ref
- Savion-Lemieux, T. and Penhune, V. The effects of practice and delay on motor skill learning and retention. Experimental Brain Research, 161(4), 2005, pp. 423--431.Google ScholarCross Ref
- Schmidt, R. A. Frequent augmented feedback can degrade learning: Evidence and interpretations. In J. Requin and G. E. Stelmach (Eds.), Tutorials in motor neuroscience, 1991, pp. 59--75.Google Scholar
- Schmidt, R. Motor Learning Concepts and Research Methods. In Motor Control and Learning: A Behavioral Emphasis, 2011, pp. 325--346.Google Scholar
- Schmidt, R. A. and Bjork, R. A. New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 1992, pp. 207--217.Google ScholarCross Ref
- Schmidt, R. A. and Wulf, G. Continuous concurrent feedback degrades skill learning: Implications for training and simulation. Human Factors, 39, 1997, pp. 509--525.Google ScholarCross Ref
- Schmidt, R. A., Young, D. E., Swinnen, S. and Shapiro, D. E. Summary knowledge of results for skill acquisition: Support for the guidance hypothesis. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 1989, pp. 352--359.Google ScholarCross Ref
- Shea, J. B. and Morgan, R. Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5(2), 1979, pp. 179--187.Google ScholarCross Ref
- Sigrist, R. Rauter, G., Riener, R. and Wolf, P. Augmented visual, auditory, haptic and multimodal feedback in motor learning: A review. Psychonomic Bulletin and Review, 2012, pp. 1--33.Google Scholar
- van Nimwegen, C. and van Oostendorp, H. Guidance in the interface and transfer of task performance. In Proc. ACM ECCE, 2007, pp. 225--232. Google ScholarDigital Library
- Winstein, C. J. and Schmidt, R. A. Reduced frequency of knowledge of results enhances motor skill learning. Journal of Experimental Psychology: Learning, Memory and Cognition, 16, 1990, pp. 677--691.Google ScholarCross Ref
- Wobbrock, J. O., Morris, M. R. and Wilson, A. D. User-defined gestures for surface computing. In Proc. ACM UIST 2003, pp. 193--202.Google Scholar
- Wobbrock, J., Wilson, A. and Li, Y. Gestures without libraries, toolkits, or training: a $1 recognizer for user interface prototypes. In Proc. ACM UIST 2007, pp. 159--168. Google ScholarDigital Library
- Zajicek, M. Aspects of HCI research for older people. Universal Access in the Information Society. 5(3), 2006, pp. 279--286. Google ScholarDigital Library
- Zhai, S. and Kristensson, P. O. Modeling Human Performance of Pen Stroke Gestures. In Proc. ACM CHI 2003, pp. 97--104.Google Scholar
- Zhao, S. and Balakrishnan, R. Simple vs. computing mark hierarchical marking menus. In Proc. ACM UIST 2004, pp. 33--42. Google ScholarDigital Library
Index Terms
- Learning and performance with gesture guides
Recommendations
Haptic Learning of Semaphoric Finger Gestures
UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and TechnologyHaptic learning of gesture shortcuts has never been explored. In this paper, we investigate haptic learning of a freehand semaphoric finger tap gesture shortcut set using haptic rings. We conduct a two-day study of 30 participants where we couple haptic ...
Gesture play: motivating online gesture learning with fun, positive reinforcement and physical metaphors
ITS '10: ACM International Conference on Interactive Tabletops and SurfacesLearning a set of gestures requires a non-trivial investment of time from novice users. We propose a novel approach based on positive reinforcement for motivating the online learning of multi-touch gestures: introducing simple, game-like elements to ...
Comments