skip to main content
10.1145/3307334.3326081acmconferencesArticle/Chapter ViewAbstractPublication PagesmobisysConference Proceedingsconference-collections
research-article
Public Access

Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi

Authors Info & Claims
Published:12 June 2019Publication History

ABSTRACT

Wi-Fi based sensing systems, although sound as being deployed almost everywhere there is Wi-Fi, are still practically difficult to be used without explicit adaptation efforts to new data domains. Various pioneering approaches have been proposed to resolve this contradiction by either translating features between domains or generating domain-independent features at a higher learning level. Still, extra training efforts are necessary in either data collection or model re-training when new data domains appear, limiting their practical usability. To advance cross-domain sensing and achieve fully zero-effort sensing, a domain-independent feature at the lower signal level acts as a key enabler. In this paper, we propose Widar3.0, a Wi-Fi based zero-effort cross-domain gesture recognition system. The key insight of Widar3.0 is to derive and estimate velocity profiles of gestures at the lower signal level, which represent unique kinetic characteristics of gestures and are irrespective of domains. On this basis, we develop a one-fits-all model that requires only one-time training but can adapt to different data domains. We implement this design and conduct comprehensive experiments. The evaluation results show that without re-training and across various domain factors (i.e. environments, locations and orientations of persons), Widar3.0 achieves 92.7% in-domain recognition accuracy and 82.6%-92.4% cross-domain recognition accuracy, outperforming the state-of-the-art solutions. To the best of our knowledge, Widar3.0 is the first zero-effort cross-domain gesture recognition work via Wi-Fi, a fundamental step towards ubiquitous sensing.

References

  1. Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. 2015. Wigest: A Ubiquitous Wifi-based Gesture Recognition System. In Proceedings of IEEE INFOCOM. Kowloon, Hong Kong.Google ScholarGoogle ScholarCross RefCross Ref
  2. Fadel Adib, Chen-Yu Hsu, Hongzi Mao, Dina Katabi, and Frédo Durand. 2015. Capturing the Human Figure Through a Wall. ACM Transactions on Graphics 34, 6 (November 2015), 219:1--219:13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Fadel Adib, Zachary Kabelac, and Dina Katabi. 2015. Multi-Person Localization via RF Body Reflections. In Proceedings of USENIX NSDI. Oakland, CA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 2014. 3d Tracking via Body Radio Reflections. In Proceedings of USENIX NSDI. Seattle, WA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Fadel Adib and Dina Katabi. 2013. See Through Walls with Wi-Fi!. In Proceedings of ACM SIGCOMM. Hong Kong, China. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad. 2017. Recognizing keystrokes using WiFi devices. IEEE Journal on Selected Areas in Communications 35, 5 (May 2017), 1175--1190.Google ScholarGoogle ScholarCross RefCross Ref
  7. Maurizio Bocca, Ossi Kaltiokallio, Neal Patwari, and Suresh Venkatasubramanian. 2014. Multiple Target Tracking with RF Sensor Networks. IEEE Transactions on Mobile Computing 13, 8 (August 2014), 1787--1800.Google ScholarGoogle ScholarCross RefCross Ref
  8. Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A Tutorial on Human Activity Recognition Using Body-Worn Inertial Sensors. ACM Comput. Surv. 46, 3 (January 2014), 33:1--33:33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kaixuan Chen, Lina Yao, Dalin Zhang, Xiaojun Chang, Guodong Long, and Sen Wang. 2018. Distributionally Robust Semi-Supervised Learning for People- Centric Sensing. In Proceedings of AAAI. New Orleans, LA, USA.Google ScholarGoogle Scholar
  10. François Chollet et al. 2015. Keras. https://github.com/fchollet/keras.Google ScholarGoogle Scholar
  11. Junyoung Chung, Sungjin Ahn, and Yoshua Bengio. 2016. Hierarchical Multiscale Recurrent Neural Networks. CoRR abs/1609.01704 (2016).Google ScholarGoogle Scholar
  12. Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR abs/1412.3555 (2014).Google ScholarGoogle Scholar
  13. David L Donoho. 2006. Compressed Sensing. IEEE Transactions on Information Theory 52, 4 (April 2006), 1289--1306. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Biyi Fang, Nicholas D Lane, Mi Zhang, Aidan Boran, and Fahim Kawsar. 2016. BodyScan: A Wearable Device for Contact-less Radio-based Sensing of Bodyrelated Activities. In Proceedings of ACM MobiSys. Singapore, Singapore.Google ScholarGoogle Scholar
  15. Biyi Fang, Nicholas D Lane, Mi Zhang, and Fahim Kawsar. 2016. HeadScan: A Wearable System for Radio-based Sensing of Head and Mouth-related Activities. In Proceedings of ACM/IEEE IPSN. Vienna, Austria.Google ScholarGoogle ScholarCross RefCross Ref
  16. Georgia Gkioxari, Ross Girshick, Piotr Dollár, and Kaiming He. 2018. Detecting and Recognizing Human-Object Interactions. In Proceedings of IEEE CVPR. Salt Lake City, UT, USA.Google ScholarGoogle ScholarCross RefCross Ref
  17. Yu Guan and Thomas Plötz. 2017. Ensembles of Deep LSTM Learners for Activity Recognition Using Wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (June 2017), 11:1--11:28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool Release: Gathering 802.11n Traces with Channel State Information. ACM SIGCOMM Computer Communication Review 41, 1 (January 2011), 53--53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Donny Huang, Rajalakshmi Nandakumar, and Shyamnath Gollakota. 2014. Feasibility and Limits of Wi-Fi Imaging. In Proceedings of ACM MobiSys. Bretton Woods, NH, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shuochao Yao, Yaqing Wang, Ye Yuan, Hongfei Xue, Chen Song, Xin Ma, Dimitrios Koutsonikolas, Wenyao Xu, and Lu Su. 2018. Towards Environment Independent Device Free Human Activity Recognition. In Proceedings of ACM MobiCom. New Delhi, India. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Kiran Joshi, Dinesh Bharadia, Manikanta Kotaru, and Sachin Katti. 2015. Wideo: Fine-grained Device-free Motion Tracing Using RF Backscatter. In Proceedings of USENIX NSDI. Oakland, CA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Kaustubh Kalgaonkar and Bhiksha Raj. 2009. One-Handed Gesture Recognition Using Ultrasonic Doppler Sonar. In Proceedings of IEEE ICASSP. Taipei, Taiwan. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Hong Li,Wei Yang, JianxinWang, Yang Xu, and Liusheng Huang. 2016. WiFinger: Talk to Your Smart Devices with Finger-grained Gesture. In Proceedings of ACM UbiComp. Heidelberg, Germany. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Tianxing Li, Qiang Liu, and Xia Zhou. 2016. Practical Human Sensing in the Light. In Proceedings of ACM MobiSys. Singapore, Singapore. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Xiang Li, Shengjie Li, Daqing Zhang, Jie Xiong, YashaWang, and Hong Mei. 2016. Dynamic-MUSIC: Accurate Device-Free Indoor Localization. In Proceedings of ACM UbiComp. Heidelberg, Germany. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Xiang Li, Daqing Zhang, Qin Lv, Jie Xiong, Shengjie Li, Yue Zhang, and Hong Mei. 2017. IndoTrack: Device-Free Indoor Human Tracking with Commodity Wi-Fi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (September 2017), 72:1--72:22.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Cihang Liu, Lan Zhang, Zongqian Liu, Kebin Liu, Xiangyang Li, and Yunhao Liu. 2016. Lasagna: Towards Deep Hierarchical Understanding and Searching over Mobile Sensing Data. In Proceedings of ACM MobiCom. New York City, NY, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Yongsen Ma, Gang Zhou, ShuangquanWang, Hongyang Zhao, andWoosub Jung. 2018. SignFi: Sign Language Recognition Using WiFi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (March 2018), 23:1--23:21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Rajalakshmi Nandakumar, Alex Takakuwa, Tadayoshi Kohno, and Shyamnath Gollakota. 2017. Covertband: Activity Information Leakage Using Music. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (September 2017), 87:1--87:24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Kai Niu, Fusang Zhang, Jie Xiong, Xiang Li, Enze Yi, and Daqing Zhang. 2018. Boosting Fine-grained Activity Sensing by Embracing Wireless Multipath Effects. In Proceedings of ACM CoNEXT. Heraklion/Crete, Greece. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole- Home Gesture Recognition Using Wireless Signals. In Proceedings of ACM Mobi- Com. Miami, FL, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kun Qian, ChenshuWu, Zheng Yang, Yunhao Liu, and Kyle Jamieson. 2017. Widar: Decimeter-Level Passive Tracking via Velocity Monitoring with Commodity Wi- Fi. In Proceedings of ACM MobiHoc. Chennai, India. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Kun Qian, ChenshuWu, Yi Zhang, Guidong Zhang, Zheng Yang, and Yunhao Liu. 2018. Widar2.0: Passive Human Tracking with a Single Wi-Fi Link. In Proceedings of ACM MobiSys. Munich, Germany. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kun Qian, Chenshu Wu, Zimu Zhou, Yue Zheng, Zheng Yang, and Yunhao Liu. 2017. Inferring Motion Direction Using Commodity Wi-Fi for Interactive Exergames. In Proceedings of ACM CHI. Denver, CO, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. 2000. The Earth Mover's Distance as a Metric for Image Retrieval. International Journal of Computer Vision 40, 2 (November 2000), 99--121. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Sheng Shen, HeWang, and Romit Roy Choudhury. 2016. I am a Smartwatch and I can Track my User's Arm. In Proceedings of ACM MobiSys. Singapore, Singapore. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. 2018. A DIRT-T Approach to Unsupervised Domain Adaptation. In Proceedings of ICLR. Vancouver, Canada.Google ScholarGoogle Scholar
  38. Raghav H. Venkatnarayan, Griffin Page, and Muhammad Shahzad. 2018. Multi- User Gesture Recognition Using WiFi. In Proceedings of ACM MobiSys. Munich, Germany.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Aditya Virmani and Muhammad Shahzad. 2017. Position and Orientation Agnostic Gesture Recognition Using WiFi. In Proceedings of ACM MobiSys. Niagara Falls, NY, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. JindongWang, Yiqiang Chen, Lisha Hu, Xiaohui Peng, and Philip S Yu. 2017. Stratified Transfer Learning for Cross-domain Activity Recognition. In Proceedings of IEEE PerCom. Big Island, HI, USA.Google ScholarGoogle Scholar
  41. Ju Wang, Hongbo Jiang, Jie Xiong, Kyle Jamieson, Xiaojiang Chen, Dingyi Fang, and Binbin Xie. 2016. LiFS: Low Human-effort, Device-free Localization with Fine-grained Subcarrier Information. In Proceedings of ACM MobiCom. New York City, NY, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Minsi Wang, Bingbing Ni, and Xiaokang Yang. 2017. Recurrent Modeling of Interaction Context for Collective Activity Recognition. In Proceedings of IEEE CVPR. Honolulu, HI, USA.Google ScholarGoogle ScholarCross RefCross Ref
  43. Wei Wang, Alex X Liu, and Muhammad Shahzad. 2016. Gait Recognition Using WiFi Signals. In Proceedings of ACM UbiComp. Heidelberg, Germany. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2017. Device-Free Human Activity Recognition Using Commercial WiFi Devices. IEEE Journal on Selected Areas in Communications 35, 5 (May 2017), 1118--1131.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: Device-free Location-oriented Activity Identification Using Fine-grained WiFi Signatures. In Proceedings of ACM MobiCom. Maui, HI, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Zheng Yang, Zimu Zhou, and Yunhao Liu. 2013. From RSSI to CSI: Indoor Localization via Channel Response. ACM Comput. Surv. 46, 2 (November 2013), 25:1--25:32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Shuochao Yao, Shaohan Hu, Yiran Zhao, Aston Zhang, and Tarek Abdelzaher. 2017. DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing. In Proceedings of ACM WWW. Perth, Australia.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Koji Yatani and Khai N Truong. 2012. BodyScope: A Wearable Acoustic Sensor for Activity Recognition. In Proceedings of ACM UbiComp. Pittsburgh, PA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Yunze Zeng, Parth H Pathak, and Prasant Mohapatra. 2016. WiWho: WiFi-Based Person Identification in Smart Spaces. In Proceedings of ACM/IEEE IPSN. Vienna, Austria. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Jie Zhang, Zhanyong Tang, Meng Li, Dingyi Fang, Petteri Tapio Nurmi, and Zheng Wang. 2018. CrossSense: Towards Cross-Site and Large-Scale WiFi Sensing. In Proceedings of ACM MobiCom. New Delhi, India.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi. 2018. Through-Wall Human Pose Estimation Using Radio Signals. In Proceedings of IEEE CVPR. Salt Lake City, UT, USA.Google ScholarGoogle ScholarCross RefCross Ref
  52. Mingmin Zhao, Yonglong Tian, Hang Zhao, Mohammad Abu Alsheikh, Tianhong Li, Rumen Hristov, Zachary Kabelac, Dina Katabi, and Antonio Torralba. 2018. RF-Based 3D Skeletons. In Proceedings of ACM SIGCOMM. Budapest, Hungary. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Zhongtang Zhao, Yiqiang Chen, Junfa Liu, Zhiqi Shen, and Mingjie Liu. 2011. Cross-People Mobile-Phone Based Activity Recognition. In Proceedings of IJCAI. Barcelona, Spain. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Yue Zheng, ChenshuWu, Kun Qian, Zheng Yang, and Yunhao Liu. 2017. Detecting Radio Frequency Interference for CSI Measurements on COTS WiFi Devices. In Proceedings of IEEE ICC. Paris, France.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      MobiSys '19: Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services
      June 2019
      736 pages
      ISBN:9781450366618
      DOI:10.1145/3307334

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 12 June 2019

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate274of1,679submissions,16%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader