Steven Houben
Nicolai Marquardt
ABSTRACT
People increasingly use smartwatches in tandem with other devices such as smartphones, laptops or tablets. This allows for novel cross-device applications that use the watch as both input device and output display. However, despite the increasing availability of smartwatches, prototyping cross-device watch-centric applications remains a challenging task. Developers are limited in the applications they can explore as available toolkits provide only limited access to different types of input sensors for cross-device interactions. To address this problem, we introduce WatchConnect, a toolkit for rapidly prototyping cross-device applications and interaction techniques with smartwatches. The toolkit provides developers with (i) an extendable hardware platform that emulates a smartwatch, (ii) a UI framework that integrates with an existing UI builder, and (iii) a rich set of input and output events using a range of built-in sensor mappings. We demonstrate the versatility and design space of the toolkit with five interaction techniques and applications.
Supplemental Material
- Ashbrook, D., Lyons, K., and Starner, T. An investigation into round touchscreen wristwatch interaction. Proc. of ACM MobileHCI'08. Google Scholar
Digital Library
- Badam, S.K. and Elmqvist, N. PolyChrome: A CrossDevice Framework for Collaborative Web Visualization. Proc. of ACM ITS'14. Google ScholarDigital Library
- Baudisch, P. and Chu, G. Back-of-device interaction allows creating very small touch devices. Proc. of ACM CHI'09. Google ScholarDigital Library
- Blasko, G. and Feiner, S. An interaction system for watch computers using tactile guidance and bidirectional segmented strokes. Proc. of IEEE ISWC 2004. Google ScholarDigital Library
- Butler, A., Izadi, S., and Hodges, S. SideSight: multitouch interaction around small devices. Proc. of ACM UIST'08. Google ScholarDigital Library
- Cao, X. and Balakrishnan, R. Interacting with dynamically defined information spaces using a handheld projector and a pen. Proc. of ACM UIST'09. Google ScholarDigital Library
- Chen, X., Grossman, T., Wigdor, D.J., and Fitzmaurice, G. Duet: exploring joint interactions on a smart phone and a smart watch. Proc. of ACM CHI'14. Google ScholarDigital Library
- Funk, M., Sahami, A., Henze, N., and Schmidt, A. Using a touch-sensitive wristband for text entry on smart watches. ACM CHI'14 EA. Google ScholarDigital Library
- Greenberg, S. Toolkits and interface creativity. Multimedia Tools and Applications, (2007). Google ScholarDigital Library
- Hamilton, P. and Wigdor, D.J. Conductor: enabling and understanding cross-device interaction. Proc. of ACM CHI'14. Google ScholarDigital Library
- Hardy, R. and Rukzio, E. Touch & interact: touch-based interaction of mobile phones with displays. Proc. of ACM MobileHCI'08. Google ScholarDigital Library
- Harrison, C. and Hudson, S.E. Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices. Proc. of ACM UIST'09. Google ScholarDigital Library
- Harrison, C., Schwarz, J., and Hudson, S.E. TapSense: enhancing finger interaction on touch surfaces. Proc. of ACM UIST'11. Google ScholarDigital Library
- Hartmann, B., Beaudouin-Lafon, M., and Mackay, W.E. HydraScope: creating multi-surface meta-applications through view synchronization and input multiplexing. Proc. of ACM PerDis'13. Google ScholarDigital Library
- Heikkinen, T., Goncalves, J., Kostakos, V., Elhart, I., and Ojala, T. Tandem Browsing Toolkit: Distributed MultiDisplay Interfaces with Web Technologies. Proc. of ACM PerDis'14. Google ScholarDigital Library
- Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P., and Smith, M. Stitching: pen gestures that span multiple displays. Proc. of ACM AVI'04. Google ScholarDigital Library
- Hinckley, K. Synchronous gestures for multiple persons and computers. Proc. of ACM UIST'03. Google ScholarDigital Library
- Huang, D.-Y., Tsai, M.-C., Tung, Y.-C., et al. TouchSense: expanding touchscreen input vocabulary using different areas of users' finger pads. Proc. of ACM CHI'14. Google ScholarDigital Library
- Kim, J., He, J., Lyons, K., and Starner, T. The gesture watch: A wireless contact-free gesture based wrist interface. Proc. of IEEE ISWC'07,. Google ScholarDigital Library
- Knibbe, J., Martinez Plasencia, D., Bainbridge, C., et al. Extending interaction for smart watches: enabling bimanual around device control. ACM CHI'14 EA. Google ScholarDigital Library
- Laput, G., Xiao, R., Chen, X., Hudson, S.E., and Harrison, C. Skin buttons: cheap, small, low-powered and clickable fixed-icon laser projectors. Proc. of ACM UIST'14. Google ScholarDigital Library
- Lyons, K., Nguyen, D., Ashbrook, D., and White, S. Facet: a multi-segment wrist worn system. Proc. of ACM UIST'12. Google ScholarDigital Library
- Marquardt, N., Ballendat, T., Boring, S., Greenberg, S., and Hinckley, K. Gradual engagement: facilitating information exchange between digital devices as a function of proximity. Proc. of ACM ITS'12. Google ScholarDigital Library
- Mayer, S. and Sörös, G. User Interface Beaming Seamless Interaction with Smart Things using Personal Wearable Computers". Proc. of IEEE BSN 2014. Google ScholarDigital Library
- Merrill, D., Kalanithi, J., and Maes, P. Siftables: towards sensor network user interfaces. Proc. of ACM TEI'07. Google ScholarDigital Library
- Myers, B., Hudson, S.E., and Pausch, R. Past, present, and future of user interface software tools. TOCHI 2000. Google ScholarDigital Library
- Nebeling, M., Mintsi, T., Husmann, M., and Norrie, M. Interactive development of cross-device user interfaces. Proc. of ACM CHI'14. Google ScholarDigital Library
- Nebeling, M., Teunissen, E., Husmann, M., and Norrie, M.C. XDKinect: development framework for crossdevice interaction using kinect. Proc. of ACM EICS'14. Google ScholarDigital Library
- Oakley, I. and Lee, D. Interaction on the edge: offset sensing for small devices. Proc. of ACM CHI'14. Google ScholarDigital Library
- Olberding, S., Yeo, K.P., Nanayakkara, S., and Steimle, J. AugmentedForearm: exploring the design space of a display-enhanced forearm. Proc. of ACM AH'13. Google ScholarDigital Library
- Olsen Jr, D.R. Evaluating user interface systems research. Proc. of ACM UIST'07. Google ScholarDigital Library
- Pasquero, J., Stobbe, S.J., and Stonehouse, N. A haptic wristwatch for eyes-free interactions. Proc. of ACM CHI'11. Google ScholarDigital Library
- Perrault, S.T., Lecolinet, E., Eagan, J., and Guiard, Y. Watchit: simple gestures and eyes-free interaction for wristwatches and bracelets. Proc. of ACM CHI'13. Google ScholarDigital Library
- Rekimoto, J. Pick-and-drop: a direct manipulation technique for multiple computer environments. Proc. of ACM UIST'97. Google ScholarDigital Library
- Rekimoto, J. Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. Proc. of IEEE ISWC'01. Google ScholarDigital Library
- Schmidt, D., Chehimi, F., Rukzio, E., and Gellersen, H. PhoneTouch: a technique for direct phone interaction on surfaces. Proc. of ACM UIST'10. Google ScholarDigital Library
- Schmidt, D., Seifert, J., Rukzio, E., and Gellersen, H. A cross-device interaction style for mobiles and surfaces. Proc. of ACM DIS'12. Google ScholarDigital Library
- Xiao, R., Laput, G., and Harrison, C. Expanding the input expressivity of smartwatches with mechanical pan, twist, tilt and click. Proc. of ACM CHI'14. Google ScholarDigital Library
- Yang, J. and Wigdor, D. Panelrama: enabling easy specification of cross-device web applications. Proc. of ACM CHI'14. Google ScholarDigital Library
- Von Zadow, U., Büschel, W., Langner, R., and Dachselt, Raimund. SleeD: Using a Sleeve Display to Interact with Touch-sensitive Display Walls. Proc. of ACM ITS'14. Google ScholarDigital Library
- Pebble. http://developer.getpebble.com.Google Scholar
- Sony Watch. http://developer.sony.com/.Google Scholar
- Apple Watch. http://apple.com/watch/.Google Scholar
- Phidgets. http://phidgets.com.Google Scholar
- Arduino. http://arduino.cc.Google Scholar
Index Terms
- WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications
Recommendations
WatchPen: Using Cross-Device Interaction Concepts to Augment Pen-Based Interaction
MobileHCI '19: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and ServicesPen-based input is often treated as auxiliary to mobile devices. We posit that cross-device interactions can inspire and extend the design space of pen-based interactions into new, expressive directions. We realize this through WatchPen, a smartwatch ...
WatchPen: Using Cross-Device Interaction Concepts to Augment Pen-Based Interaction
MobileHCI '19: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and ServicesWatchPen illustrates how cross-device interactions can inspire and extend the design space of pen-based interactions into new, expressive directions. We demonstrate WatchPen, a smartwatch mounted on a passive, capacitive stylus that: (1) senses the ...
ProjectorKit: easing rapid prototyping of interactive applications for mobile projectors
MobileHCI '13: Proceedings of the 15th international conference on Human-computer interaction with mobile devices and servicesResearchers have developed interaction concepts based on mobile projectors. Yet pursuing work in this area - particularly in building projector-based interactions techniques within an application - is cumbersome and time-consuming. To mitigate this ...
Comments