Katherine Fennedy
Angad Srivastava
Simon T. Perrault
Skip Abstract Section
Abstract
We advocate for the usage of hotkeys on touch-based devices by capitalising on soft keyboards through four studies. First, we evaluated visual designs and recommended icons with command names for novices while letters with command names for experts. Second, we investigated the discoverability by asking crowdworkers to use our prototype, with some tasks only doable upon successfully discovering the technique. Discovery rates were high regardless of conditions that vary the familiarity and saliency of modifier keys. However, familiarity with desktop hotkeys boosted discoverability. Our third study focused on how prior knowledge of hotkeys could be leveraged and resulted in a 5% selection time improvement and identified the role of spatial memory in retention. Finally, we compared our soft keyboard layout with a grid layout similar to FastTap. The latter offered a 12–16% gain on selection speed, but at a high cost in terms of screen estate and low spatial stability.
- [1] . 2020. Where is that feature? Designing for cross-device software learnability. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. Association for Computing Machinery, New York, NY, 1103–1115.
DOI: https://doi.org/10.1145/3357236.3395506 Google Scholar
Digital Library
- [2] . 1983. The Architecture of Cognition. Harvard University Press. Google ScholarDigital Library
- [3] . 1974. Interference in memory for multiple contexts. Memory & Cognition 2, 3 (1974), 509–514.Google ScholarCross Ref
- [4] . 2012. Dwell-and-spring: Undo for direct manipulation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1957–1966.
DOI: https://doi.org/10.1145/2207676.2208339 Google ScholarDigital Library - [5] . 2009. Using Strokes as command shortcuts: Cognitive benefits and toolkit support. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 2289–2298.
DOI: https://doi.org/10.1145/1518701.1519052 Google ScholarDigital Library - [6] . 2020. Retrieved on 01 Dec, 2020 from https://developer.apple.com/design/human-interface-guidelines/ios/user-interaction/undo-and-redo/.Google Scholar
- [7] . 2012. Paired-Associate Learning. Springer US, Boston, MA, 2551–2552.
DOI: https://doi.org/10.1007/978-1-4419-1428-6_1038Google Scholar - [8] . 2016. Surveying expert-level gesture use and adoption on multi-touch tablets. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. Association for Computing Machinery, New York, NY, 577–581.
DOI: https://doi.org/10.1145/2901790.2901895 Google ScholarDigital Library - [9] . 2007. Wave menus: Improving the novice mode of hierarchical marking menus. In Proceedings of the 11th IFIP TC 13 International Conference on Human-Computer Interaction. Springer-Verlag, Berlin, 475–488. Google ScholarDigital Library
- [10] . 2016. Visual menu techniques. ACM Computing Surveys 49, 4 (
Dec. 2016), Article 60, 41 pages.DOI: https://doi.org/10.1145/3002171 Google ScholarDigital Library - [11] . 2008. OctoPocus: A dynamic guide for learning gesture-based command sets. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, 37–46.
DOI: https://doi.org/10.1145/1449715.1449724 Google ScholarDigital Library - [12] . 2000. The strategic use of complex computer systems. Human-Computer Interaction 15, 2 (
Sept. 2000), 107–137.DOI: https://doi.org/10.1207/S15327051HCI1523_3 Google ScholarDigital Library - [13] . 2010. Touch-display keyboards: Transforming keyboards into interactive surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1145–1154.
DOI: https://doi.org/10.1145/1753326.1753498 Google ScholarDigital Library - [14] . 1980. The keystroke-level model for user performance time with interactive systems. Communications of the ACM 23, 7 (
July 1980), 396–410.DOI: https://doi.org/10.1145/358886.358895 Google ScholarDigital Library - [15] . 1987. Paradox of the Active User. MIT Press, Cambridge, MA, 80–111. Google ScholarDigital Library
- [16] . 2014. Supporting novice to expert transitions in user interfaces. ACM Computing Surveys 47, 2 (
Nov. 2014), Article 31, 36 pages.DOI: https://doi.org/10.1145/2659796 Google ScholarDigital Library - [17] . 2007. Hard lessons: Effort-inducing interfaces benefit spatial learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1571–1580.
DOI: https://doi.org/10.1145/1240624.1240863 Google ScholarDigital Library - [18] . 2003. Inter-Usability of Multi-Device Systems – A Conceptual Framework. John Wiley & Sons, Ltd, Chapter 17, 373–385.
DOI: https://doi.org/10.1002/0470091703.ch17 arXiv: https://onlinelibrary.wiley.com/doi/pdf/10.1002/0470091703.ch17.Google Scholar - [19] . 1997. Human-Computer Interaction. Prentice-Hall, Inc. Google ScholarDigital Library
- [20] . 2019. Multitouch keyboard revisited: Enhancing moded interaction through redesigning structure and switching techniques. In Proceedings of the 5th International ACM In-Cooperation HCI and UX Conference. Association for Computing Machinery, New York, NY, 36–45.
DOI: https://doi.org/10.1145/3328243.3328249 Google ScholarDigital Library - [21] . 2020. Investigating performance and usage of input methods for soft keyboard hotkeys. In Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. Association for Computing Machinery, New York, NY, Article 29, 12 pages.
DOI: https://doi.org/10.1145/3379503.3403552 Google ScholarDigital Library - [22] . 2020. The Collaborative Interface Design Tool. Retrieved on 01 Dec, 2020 from https://www.figma.com/.Google Scholar
- [23] . 2009. Wavelet menus: A stacking metaphor for adapting marking menus to mobile devices. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, NY, Article 49, 4 pages.
DOI: https://doi.org/10.1145/1613858.1613919 Google ScholarDigital Library - [24] . 2017. IconHK: Using toolbar button icons to communicate keyboard shortcuts. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 4715–4726.
DOI: https://doi.org/10.1145/3025453.3025595 Google ScholarDigital Library - [25] . 2018. Storyboard-based empirical modeling of touch interface performance. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1–12.
DOI: https://doi.org/10.1145/3173574.3174019 Google ScholarDigital Library - [26] . 2019. Reducing error aversion to support novice-to-expert transitions with fastTap. In Proceedings of the 31st Conference on l’Interaction Homme-Machine. Association for Computing Machinery, New York, NY, Article 1, 10 pages.
DOI: https://doi.org/10.1145/3366550.3372247 Google ScholarDigital Library - [27] . 2018. Improving Discoverability and Expert Performance in Force-Sensitive Text Selection for Touch Devices with Mode Gauges. Association for Computing Machinery, New York, NY, 1–12.
DOI: https://doi.org/10.1145/3173574.3174051 Google ScholarDigital Library - [28] . 2004. A reduced QWERTY keyboard for mobile text entry. In Proceedings of the CHI’04 Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1429–1432.
DOI: https://doi.org/10.1145/985921.986082 Google ScholarDigital Library - [29] . 2007. Strategies for accelerating on-line learning of hotkeys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1591–1600.
DOI: https://doi.org/10.1145/1240624.1240865 Google ScholarDigital Library - [30] . 2015. Testing the rehearsal hypothesis with two FastTap interfaces. In Proceedings of the 41st Graphics Interface Conference. 223–231. Google ScholarDigital Library
- [31] . 2014. Faster command selection on tablets with FastTap. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 2617–2626.
DOI: https://doi.org/10.1145/2556288.2557136 Google ScholarDigital Library - [32] . 2020. Investigating the necessity of delay in marking menu invocation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1–13.
DOI: https://doi.org/10.1145/3313831.3376296 Google ScholarDigital Library - [33] . 2020. Mobile-App Onboarding: An Analysis of Components and Techniques. Retrieved on 01 Dec, 2020 from https://www.nngroup.com/articles/mobile-app-onboarding/.Google Scholar
- [34] . 2020. Mobile Tutorials: Wasted Effort or Efficiency Boosts? Retrieved on 01 Dec, 2020 from https://www.nngroup.com/articles/mobile-tutorials/.Google Scholar
- [35] . 1973. Attention and Effort. Vol. 1063. Citeseer.Google Scholar
- [36] . 2018. Swype Keyboard Has Been Discontinued. Retrieved on 01 Dec, 2020 from https://www.theverge.com/2018/2/21/17030216/nuance-discontinues-swype-keyboard.Google Scholar
- [37] . 2011. Two-handed marking menus for multitouch devices. ACM Transactions on Computer-Human Interaction 18, 3 (
Aug. 2011), Article 16, 23 pages.DOI: https://doi.org/10.1145/1993060.1993066 Google ScholarDigital Library - [38] . 2008. Training towards mastery: Overcoming the active user paradox. In Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges. Association for Computing Machinery, New York, NY, 239–248.
DOI: https://doi.org/10.1145/1463160.1463186 Google ScholarDigital Library - [39] . 1994. User learning and performance with marking menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 258–264.
DOI: https://doi.org/10.1145/191666.191759 Google ScholarDigital Library - [40] . 2017. Investigating the post-training persistence of expert interaction techniques. ACM Transactions on Computer-Human Interaction 24, 4 (
Aug. 2017), Article 29, 46 pages.DOI: https://doi.org/10.1145/3119928 Google ScholarDigital Library - [41] . 2016. Faster command selection on touchscreen watches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 4663–4674.
DOI: https://doi.org/10.1145/2858036.2858166 Google ScholarDigital Library - [42] . 2005. Hidden costs of graphical user interfaces: Failure to make the transition from menus and icon toolbars to keyboard shortcuts. International Journal of Human -Computer Interaction 18, 2 (
May 2005), 133–144.DOI: https://doi.org/10.1207/s15327590ijhc1802_1Google ScholarCross Ref - [43] . 2020. Shortcut gestures for mobile text editing on fully touch sensitive smartphones. ACM Transactions on Computer-Human Interaction 27, 5 (
Aug. 2020), Article 33, 38 pages.DOI: https://doi.org/10.1145/3396233Google ScholarDigital Library - [44] . 2010. The design and evaluation of multitouch marking menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 2233–2242.
DOI: https://doi.org/10.1145/1753326.1753663 Google ScholarDigital Library - [45] . 2020. KeyMap: Improving keyboard shortcut vocabulary using norman’s mapping. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1–10.
DOI: https://doi.org/10.1145/3313831.3376483 Google ScholarDigital Library - [46] . 2020. Longer delays in rehearsal-based interfaces increase expert use. ACM Transactions on Computer-Human Interaction 27, 6 (
Nov. 2020), Article 45, 41 pages.DOI: https://doi.org/10.1145/3418196 Google ScholarDigital Library - [47] . 2013. Promoting hotkey use through rehearsal with ExposeHK. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 573–582.
DOI: https://doi.org/10.1145/2470654.2470735 Google ScholarDigital Library - [48] . 2013. Skillometers: Reflective widgets that motivate and help users to improve performance. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, 321–330.
DOI: https://doi.org/10.1145/2501988.2501996 Google ScholarDigital Library - [49] . 2018. How to communicate new input techniques. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction. Association for Computing Machinery, New York, NY, 460–472.
DOI: https://doi.org/10.1145/3240167.3240176 Google ScholarDigital Library - [50] . 2020. Design that works. https://maze.design/.Google Scholar
- [51] . 2011. Categorization costs for hierarchical keyboard commands. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 2765–2768.
DOI: https://doi.org/10.1145/1978942.1979351 Google ScholarDigital Library - [52] . 1993. Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA. Google ScholarCross Ref
- [53] . 1983. Design principles for human-computer interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1–10.
DOI: https://doi.org/10.1145/800045.801571 Google ScholarDigital Library - [54] . 1988. The Psychology of Everyday Things.Basic books.Google Scholar
- [55] . 2010. Natural user interfaces are not natural. Interactions 17, 3 (
May 2010), 6–10.DOI: https://doi.org/10.1145/1744161.1744163 Google ScholarDigital Library - [56] . 2010. Gestural interfaces: A step backward in usability. Interactions 17, 5 (
Sept. 2010), 46–49.DOI: https://doi.org/10.1145/1836216.1836228 Google ScholarDigital Library - [57] . 2017. What Are the Most Common Swype Gestures? Retrieved on 01 Dec, 2020 from https://nuancemobility.zendesk.com/hc/en-us/articles/212253703-What-are-the-most-common-Swype-gestures-.Google Scholar
- [58] . 2004. Toolglasses, marking menus, and hotkeys: A comparison of one and two-handed command selection techniques. In Proceedings of Graphics Interface 2004. Canadian Human-Computer Communications Society, School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada, 17–24. Retrieved on 01 Dec, 2020 from http://dl.acm.org/citation.cfm?id=1006058.1006061. Google ScholarDigital Library
- [59] . 2013. Typing on Windows 8 / Windows RT Onscreen Touch Keyboard + Features. Youtube. Retrieved from https://youtu.be/Dep_O1508Ug.Google Scholar
- [60] . 1969. On the use of transition diagrams in the design of a user interface for an interactive computer system. In Proceedings of the 1969 24th National Conference. Association for Computing Machinery, New York, NY, 379–385.
DOI: https://doi.org/10.1145/800195.805945 Google ScholarDigital Library - [61] . 2016. Hamburger Menus and Hidden Navigation Hurt UX Metrics. Retrieved on 01 Dec, 2020 from https://www.nngroup.com/articles/hamburger-menus/.Google Scholar
- [62] . 2015. Physical loci: Leveraging spatial, object and semantic memory for command selection. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 299–308.
DOI: https://doi.org/10.1145/2702123.2702126 Google ScholarDigital Library - [63] . 2020. Interacting with Signifier-Less Designs - The Case of Swhidgets. Master’sThesis. Université de Lille (2018-....). Retrieved on 01 Dec, 2020 from https://hal.inria.fr/tel-03133122.Google Scholar
- [64] . 2019. Awareness, usage and discovery of swipe-revealed hidden widgets in IOS. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces. Association for Computing Machinery, New York, NY, 193–204.
DOI: https://doi.org/10.1145/3343055.3359713 Google ScholarDigital Library - [65] . 2019. Measuring motivations of crowdworkers: The multidimensional crowdworker motivation scale. ACM Transactions on Social Computing 2, 2 (
Sept. 2019), Article 8, 34 pages.DOI: https://doi.org/10.1145/3335081 Google ScholarDigital Library - [66] . 2018. Can I use Ctrl + C, Ctrl + V and Other Shortcuts on Galaxy Tab S like I Would on a PC? Retrieved from https://www.samsung.com/ie/support/mobile-devices/can-i-use-ctrl-c-ctrl-v-and-other-shortcuts-on-galaxy-tab-s-like-i-would-on-a-pc/.Google Scholar
- [67] . 2012. Improving command selection with CommandMaps. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 257–266.
DOI: https://doi.org/10.1145/2207676.2207713 Google ScholarDigital Library - [68] . 2013. Testing the robustness and performance of spatially consistent interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 3139–3148.
DOI: https://doi.org/10.1145/2470654.2466430 Google ScholarDigital Library - [69] . 2011. Dips and ceilings: Understanding and supporting transitions to expertise in user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 2741–2750.
DOI: https://doi.org/10.1145/1978942.1979348 Google ScholarDigital Library - [70] . 2016. Supporting transitions to expertise in hidden toolbars. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 4687–4698.
DOI: https://doi.org/10.1145/2858036.2858412 Google ScholarDigital Library - [71] . 1992. Designing the User Interface (2nd Ed.): Strategies for Effective Human-Computer Interaction. Addison-Wesley Longman Publishing Co., Inc. Google ScholarDigital Library
- [72] . 1956. Rational choice and the structure of the environment. Psychological Review 63, 2 (1956), 129.Google ScholarCross Ref
- [73] . 1989. The Transfer of Cognitive Skill. Harvard University Press. Google ScholarDigital Library
- [74] . 2019. Optimus Popularis. Retrieved on 01 Dec, 2020 from https://www.artlebedev.com/optimus/popularis/.Google Scholar
- [75] . 2007. Shift: A technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 657–666.
DOI: https://doi.org/10.1145/1240624.1240727 Google ScholarDigital Library - [76] . 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 143–146.
DOI: https://doi.org/10.1145/1978942.1978963 Google ScholarDigital Library - [77] . 2004. Simple vs. compound mark hierarchical marking menus. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, 33–42.
DOI: https://doi.org/10.1145/1029632.1029639 Google ScholarDigital Library - [78] . 2018. M3 gesture menu: Design and experimental analyses of marking menus for touchscreen mobile interaction. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, Article 249, 14 pages.
DOI: https://doi.org/10.1145/3173574.3173823 Google ScholarDigital Library
Index Terms
- Towards a Unified and Efficient Command Selection Mechanism for Touch-Based Devices Using Soft Keyboard Hotkeys
Recommendations
Flickey: Flick-Based QWERTY Software Keyboard for Ultra-small Touch Screen Devices
Human-Computer Interaction. Interaction TechnologiesAbstractUltra-small touch screen devices (e.g., smartwatches) are required to be small and lightweight so that they can be worn on the body with no frustration. For this reason, users often have difficulties in selecting the correct keys, and thus ...
Promoting Hotkey use through rehearsal with ExposeHK
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsKeyboard shortcuts allow fast interaction, but they are known to be infrequently used, with most users relying heavily on traditional pointer-based selection for most commands. We describe the goals, design, and evaluation of ExposeHK, a new interface ...
Exploring callout design in selection task for ultra-small touch screen devices
OzCHI '16: Proceedings of the 28th Australian Conference on Computer-Human InteractionUltra-small touch screen devices tend to suffer from occlusion or the fat finger problem owing to their limited input area. A callout could solve these problems by displaying a copy of the occluded area in a non-occluded area. However, callout designs ...
Comments