ABSTRACT
With the omnipresence of computing devices in our daily lives, interests in ubiquitous computing interfaces have grown. In response to this, various studies have introduced on-surface input techniques which use the surfaces of surrounding objects as a touch interface. However, these methods are yet struggling to support ubiquitous interaction due to their dependency on specific hardware or environments. In this paper, we propose UbiTap, an input method that turns solid surfaces into a touch input space, through the use of sound (i.e., with microphones already present in the commodity devices). More specifically, we develop a novel touch localization technique which leverages the physical phenomenon, referred to as dispersion, a characteristic of sound as it travels through solid surfaces, so as to address challenges which limit existing acoustic-based solutions in terms of portability, accuracy, usability, robustness, and responsiveness. Our extensive experiments with a prototype of UbiTap show that we can support sub-centimeter accuracy on various surfaces with minor user calibration effort. In our experience with real-world users, UbiTap significantly improves usability and robustness, thus enabling the emergence of more exciting applications.
- Stephen Butterworth. 1930. On the theory of filter amplifiers. Wireless Engineer 7, 6 (1930), 536--541.Google Scholar
- Florian Echtler, Andreas Dippon, Marcus Tönnis, and Gudrun Klinker. 2009. Inverted FTIR: easy multitouch sensing for flatscreens. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. Google ScholarDigital Library
- Chris Harrison, Hrvoje Benko, and Andrew D Wilson. 2011. OmniTouch: wearable multitouch interaction everywhere. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology. Google ScholarDigital Library
- Richard B Langley et al. 1999. Dilution of precision. GPS world 10, 5 (1999), 52--59.Google Scholar
- Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Yury Degtyarev, Sergey Grizan, Alec Wolman, and Jason Flinn. 2015. Outatime: Using speculation to enable low-latency continuous interaction for mobile cloud gaming. In Proceedings of the Annual International Conference on Mobile Systems, Applications, and Services. Google ScholarDigital Library
- Jian Liu, Yingying Chen, Marco Gruteser, and Yan Wang. 2017. VibSense: Sensing Touches on Ubiquitous Surfaces through Vibration. In Proceedings of the Annual IEEE International Conference on Sensing, Communication, and Networking.Google ScholarCross Ref
- Mostafa Mirshekari, Shijia Pan, Jonathon Fagert, Eve M Schooler, Pei Zhang, and Hae Young Noh. 2018. Occupant localization using footstep-induced structural vibration. Mechanical Systems and Signal Processing 112 (2018), 77--97.Google ScholarCross Ref
- Leap Motion. Retrieved Febuary 21, 2018 from https://www.leapmotion.com/.Google Scholar
- Samsung NEWSROOM. Retrieved Febuary 10, 2018 from https://news.samsung.com/global/share-the-fun-with-samsung-galaxy-beam.Google Scholar
- Tom Page. 2013. Usability of text input interfaces in smartphones. Journal of Design Research 11, 1 (2013), 39--56.Google ScholarCross Ref
- Shijia Pan, Ceferino Gabriel Ramirez, Mostafa Mirshekari, Jonathon Fagert, Albert Jin Chung, Chih Chi Hu, John Paul Shen, Hae Young Noh, and Pei Zhang. 2018. Surfacevibe: vibration-based tap & swipe tracking on ubiquitous surfaces. In Proceedings of the ACM/IEEE International Conference on Information Processing in Sensor Networks. Google ScholarDigital Library
- Joseph A Paradiso, Che King Leo, Nisha Checka, and Kaijen Hsiao. 2002. Passive acoustic knock tracking for interactive windows. In Proceedings of the SIGCHI Extended Abstracts on Human Factors in Computing Systems. Google ScholarDigital Library
- Duc Truong Pham, Ze Ji, Ming Yang, Zuobin Wang, and Mostafa Al-Kutubi. 2007. A novel human-computer interface based on passive acoustic localisation. In Proceedings of the International Conference on Human-Computer Interaction. Google ScholarDigital Library
- Jian Qiu, David Chu, Xiangying Meng, and Thomas Moscibroda. 2011. On the feasibility of real-time phone-to-phone 3d localization. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems. Google ScholarDigital Library
- Jun Rekimoto. 2002. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Google ScholarDigital Library
- Annie Ross and Germain Ostiguy. 2007. Propagation of the initial transient noise from an impacted plate. Journal of Sound and Vibration 301, 1-2 (2007), 28--42.Google ScholarCross Ref
- Munehiko Sato, Ivan Poupyrev, and Chris Harrison. 2012. Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Google ScholarDigital Library
- Julius O. Smith. 2011. Spectral Audio Signal Processing. W3K publishing.Google Scholar
- SONY. Retrieved Febuary 28, 2018 from https://www.sonymobile.com/global-en/products/smart-products/xperia- touch/#gref.Google Scholar
- Amir Sulaiman, Kirill Poletkin, and Andy WH Khong. 2010. Source localization in the presence of dispersion for next generation touch interface. In Cyberworlds (CW), 2010 International Conference on. IEEE, 82--86. Google ScholarDigital Library
- Yoshiki Takeoka, Takashi Miyaki, and Jun Rekimoto. 2010. Z-touch: an infrastructure for 3d gesture interaction in the proximity of tabletop surfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. Google ScholarDigital Library
- SMART Technologies. Retrieved Febuary 28, 2018 from https://education.smarttech.com/.Google Scholar
- Junjue Wang, Kaichen Zhao, Xinyu Zhang, and Chunyi Peng. 2014. Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In Proceedings of the Annual International Conference on Mobile Systems, Applications, and Services. Google ScholarDigital Library
- Andrew D Wilson. 2005. PlayAnywhere: a compact interactive tabletop projection-vision system. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology. Google ScholarDigital Library
- Robert Xiao, Chris Harrison, and Scott E Hudson. 2013. WorldKit: rapid and easy creation of ad-hoc interactive applications on everyday surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Google ScholarDigital Library
- Robert Xiao, Greg Lew, James Marsanico, Divya Hariharan, Scott Hudson, and Chris Harrison. 2014. Toffee: enabling ad hoc, around-device interaction with acoustic time-of-arrival correlation. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices & Services. Google ScholarDigital Library
- Yafeng Yin, Qun Li, Lei Xie, Shanhe Yi, Edmund Novak, and Sanglu Lu. 2018. CamK: Camera-based Keystroke Detection and Localization for Small Mobile Devices. IEEE Transactions on Mobile Computing PP, 99 (2018), 1--1.Google Scholar
- Chungkuk Yoo, Inseok Hwang, Eric Rozner, Yu Gu, and Robert F Dickerson. 2016. SymmetriSense: Enabling Near-Surface Interactivity on Glossy Surfaces using a Single Commodity Smartphone. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Google ScholarDigital Library
- Chi Zhang, Josh Tabor, Jialiang Zhang, and Xinyu Zhang. 2015. Extending mobile interaction through near-field visible light sensing. In Proceedings of the Annual International Conference on Mobile Computing and Networking. Google ScholarDigital Library
- Zengbin Zhang, David Chu, Xiaomeng Chen, and Thomas Moscibroda. 2012. Swordfight: Enabling a new class of phone-to-phone action games on commodity phones. In Proceedings of the International Conference on Mobile Systems, Applications, and Services. Google ScholarDigital Library
Index Terms
- UbiTap: Leveraging Acoustic Dispersion for Ubiquitous Touch Interface on Solid Surfaces
Recommendations
DESIGNING HUMAN-CENTRED UBIQUITOUS COMPUTING
We are entering the age of Ubiquitous Computing, and its most distinguished advantage is invisibility of computing. The essence of invisible computing is the invisibility of Human-computer Interaction, as people can focus their attention on the content ...
A Smart Home Experience Using Egocentric Interaction Design Principles
CSE '12: Proceedings of the 2012 IEEE 15th International Conference on Computational Science and EngineeringThe landscape of ubiquitous computing comprising of numerous interconnected computing devices seamlessly integrated within everyday environments introduces a need to do research beyond human-computer interaction: in particular incorporate human-...
Context-Awareness in Ambient Intelligence
Ambient Intelligence (AmI) deals with the issue of how we can create context-aware, electronic environments which foster seamless human-computer interaction. Ambient Intelligence encompasses the fields of ubiquitous computing, artificially intelligent ...
Comments