Li Sun
ABSTRACT
This paper demonstrates that it is possible to leverage WiFi signals from commodity mobile devices to enable hands-free drawing in the air. While prior solutions require the user to hold a wireless transmitter, or require custom wireless hardware, or can only determine a pre-defined set of hand gestures, this paper introduces WiDraw, the first hand motion tracking system using commodity WiFi cards, and without any user wearables. WiDraw harnesses the Angle-of-Arrival values of incoming wireless signals at the mobile device to track the user's hand trajectory. We utilize the intuition that whenever the user's hand occludes a signal coming from a certain direction, the signal strength of the angle representing the same direction will experience a drop. Our software prototype using commodity wireless cards can track the user's hand with a median error lower than 5 cm. We use WiDraw to implement an in-air handwriting application that allows the user to draw letters, words, and sentences, and achieves a mean word recognition accuracy of 91%.
- Microsoft kinect. http://www.microsoft.com/en-us/kinectforwindows/.Google Scholar
- Leap motion, inc. leap motion: Mac & pc gesture controller for game, design and more. https://www.leapmotion.com/.Google Scholar
- Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. Whole-home gesture recognition using wireless signals. In Proceedings of ACM Mobicom, 2013. Google ScholarDigital Library
- Fadel Adib and Dina Katabi. See through walls with WiFi! In Proceedings of ACM SIGCOMM, 2013. Google ScholarDigital Library
- Bryce Kellogg, Vamsi Talla, and Shyam Gollakota. Bringing gesture recognition to all devices. In Proceedings of USENIX NSDI, 2014. Google ScholarDigital Library
- Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 3D tracking via body radio reflections. In Proceedings of USENIX NSDI, 2014. Google ScholarDigital Library
- Jue Wang, Deepak Vasisht, and Dina Katabi. RF-IDraw: Virtual Touch Screen in the Air Using RF Signals. In Proceedings of ACM SIGCOMM, 2014. Google ScholarDigital Library
- Pedro Melgarejo, Xinyu Zhang, Parameswaran Ramanathan, and David Chu. Leveraging directional antenna capabilities for fine-grained gesture recognition. In Proceedings of ACM Ubicomp, 2014. Google ScholarDigital Library
- Ralph O Schmidt. Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas and Propagation, AP-34(3):276--280, 1986.Google ScholarCross Ref
- Jon Gjengset, Jie Xiong, Graeme McPhillips, and Kyle Jamieson. Phaser: enabling phased array signal processing on commodity wifi access points. In Proceedings of ACM Mobicom, 2014. Google ScholarDigital Library
- Souvik Sen, Jeongkeun Lee, Kyu-Han Kim, and Paul Congdon. Avoiding multipath to revive inbuilding wifi localization. In Proceeding of ACM MobiSys, 2013. Google ScholarDigital Library
- Kainam Thomas Wong and Michael D Zoltowski. Root-music-based azimuth-elevation angle-of-arrival estimation with uniformly spaced but arbitrarily oriented velocity hydrophones. Signal Processing, IEEE Transactions on, 47(12):3250--3260, 1999. Google ScholarDigital Library
- Li Sun, Souvik Sen, and Dimitrios Koutsonikolas. Bringing mobility-awareness to WLANs using PHY layer information. In Proceedings of the ACM CoNEXT, 2014. Google ScholarDigital Library
- Myscript. Myscript stylus app. http://myscript.com/.Google Scholar
- Wenwei Wang, Anja Brakensiek, and Gerhard Rigoll. Combination of multiple classifiers for handwritten word recognition. In Frontiers in Handwriting Recognition, 2002. Proceedings. Eighth International Workshop on, pages 117--122. IEEE, 2002. Google ScholarDigital Library
- Réjean Plamondon and Sargur N Srihari. On-line and off-line handwriting recognition: A comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1):63--84, 2000. Google ScholarDigital Library
- James M Rehg and Takeo Kanade. Visual tracking of high DOF articulated structures: an application to human hand tracking. In Proceedings of the European Conference on Computer Vision, 1994. Google ScholarDigital Library
- Thad Starner and Alex Pentland. Real-time american sign language recognition from video using hidden markov models. In Proceedings of the International Symposium on Computer Vision, 1995. Google ScholarDigital Library
- Nintendo wii. http://www.nintendo.com/wiiu.Google Scholar
- Sandip Agrawal, Ionut Constandache, Shravan Gaonkar, Romit Roy Choudhury, Kevin Caves, and Frank DeRuyter. Using mobile phones to write in air. In Proceedings of ACM MobiSys, 2011. Google ScholarDigital Library
- Taiwoo Park, Jinwon Lee, Inseok Hwang, Chungkuk Yoo, Lama Nachman, and Junehwa Song. E-Gesture: A Collaborative Architecture for Energy-efficient Gesture Recognition with Hand-worn Sensor and Mobile Devices. In Proceedings of ACM SenSys, 2011.Google Scholar
- Chao Xu, Parth H. Pathak, and Prasant Mohapatra. Finger-writing with smartwatch: A case for finger and hand gesture recognition using smartwatch. In Proceedings of ACM HotMobile, 2015. Google ScholarDigital Library
- Myo: Gesture control armband. https://www.thalmic.com/en/myo/.Google Scholar
- Abhinav Parate, Meng-Chieh Chiu, Chaniel Chadowitz, Deepak Ganesan, and Evangelos Kalogerakis. RisQ: recognizing smoking gestures with inertial sensors on a wristband. In Proceedings of ACM MobiSys, 2014. Google ScholarDigital Library
- Jeremy Gummeson, Bodhi Priyantha, and Jie Liu. An energy harvesting wearable ring platform for gestureinput on surfaces. In Proceedings of ACM MobiSys, 2014. Google ScholarDigital Library
- Shahriar Nirjon, Jeremy Gummeson, Dan Gelb, and Kyu-Han Kim. TypingRing: A Wearable Ring Platform for Text Input. In Proceedings of ACM MobiSys, 2015. Google ScholarDigital Library
- Fadel Adib, Zach Kabelac, and Dina Katabi. Multi-Person Localization via RF Body Reflections. In Proceedings of USENIX NSDI, 2015. Google ScholarDigital Library
- Rajalakshmi Nandakumar, Bryce Kellogg, and Shyamnath Gollakota. Wi-fi gesture recognition on existing devices. http://arxiv.org/ftp/arxiv/papers/1411/1411.5394.pdf.Google Scholar
- Heba Abdelnasser, Moustafa Youssef, and Khaled Harras. Wigest: A ubiquitous wifi-based gesture recognition system. In Proceedings of IEEE INFOCOM, 2015.Google ScholarCross Ref
- IEEE 802.11ac -- Amendment 4: Enhancements for Very High Throughput for Operation in Bands below 6 GHz. IEEE P802.11ac/D5.0, 2013.Google Scholar
- Ioannis Pefkianakis, Henrik Lundgren, Augustin Soule, Jaideep Chandrashekar, Pascal Le Guyadec, Christophe Diot, Martin May, Karel Van Doorselaer, and Koen Van Oost. Characterizing Home Wireless Performance: The Gateway View. In Proceedings of IEEE INFOCOM, 2015.Google Scholar
Index Terms
- WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices
Recommendations
SignFi: Sign Language Recognition Using WiFi
We propose SignFi to recognize sign language gestures using WiFi. SignFi uses Channel State Information (CSI) measured by WiFi packets as the input and a Convolutional Neural Network (CNN) as the classification algorithm. Existing WiFi-based sign gesture ...
Poster: Enabling Hands-free Drawing on WiFi Devices
S3 '15: Proceedings of the 2015 Workshop on Wireless of the Students, by the Students, & for the StudentsWe present the high-level design of WiDraw, the first 3D hand motion tracking system using COTS WiFi cards, proposed in [3]. WiDraw can track hand gestures without a priori learning or requiring the user to hold any hardware. It can be implemented on ...
Continuous recognition of one-handed and two-handed gestures using 3D full-body motion tracking sensors
IUI '12: Proceedings of the 2012 ACM international conference on Intelligent User InterfacesIn this paper we present a new bimanual markerless gesture interface for 3D full-body motion tracking sensors, such as the Kinect. Our interface uses a probabilistic algorithm to incrementally predict users' intended one-handed and twohanded gestures ...
Comments