ABSTRACT
Driven by a wide range of real-world applications, significant efforts have recently been made to explore device-free human activity recognition techniques that utilize the information collected by various wireless infrastructures to infer human activities without the need for the monitored subject to carry a dedicated device. Existing device free human activity recognition approaches and systems, though yielding reasonably good performance in certain cases, are faced with a major challenge. The wireless signals arriving at the receiving devices usually carry substantial information that is specific to the environment where the activities are recorded and the human subject who conducts the activities. Due to this reason, an activity recognition model that is trained on a specific subject in a specific environment typically does not work well when being applied to predict another subject's activities that are recorded in a different environment. To address this challenge, in this paper, we propose EI, a deep-learning based device free activity recognition framework that can remove the environment and subject specific information contained in the activity data and extract environment/subject-independent features shared by the data collected on different subjects under different environments. We conduct extensive experiments on four different device free activity recognition testbeds: WiFi, ultrasound, 60 GHz mmWave, and visible light. The experimental results demonstrate the superior effectiveness and generalizability of the proposed EI framework.
- Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. 2015. Wigest: A ubiquitous wifi-based gesture recognition system. In Computer Communications (INFOCOM), 2015 IEEE Conference on. IEEE, 1472--1480.Google ScholarCross Ref
- Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, and Mario Marchand. 2014. Domain-adversarial neural networks. arXiv preprint arXiv:1412.4446 (2014).Google Scholar
- Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using wifi signals. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 90--102. Google ScholarDigital Library
- Chuankai An, Tianxing Li, Zhao Tian, Andrew T Campbell, and Xia Zhou. 2015. Visible light knows who you are. In Proceedings of the 2nd International Workshop on Visible Light Communications Systems. ACM, 39--44. Google ScholarDigital Library
- Robert Bodor, Bennett Jackson, and Nikolaos Papanikolopoulos. 2003. Vision-based human tracking and activity recognition. In Proc. of the 11th Mediterranean Conf. on Control and Automation, Vol. 1.Google Scholar
- Ke-Yu Chen, Daniel Ashbrook, Mayank Goel, Sung-Hyuck Lee, and Shwetak Patel. 2014. AirLink: sharing files between multiple devices using in-air gestures. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing . ACM, 565--569. Google ScholarDigital Library
- Laurie Davies and Ursula Gather. 1993. The identification of multiple outliers. J. Amer. Statist. Assoc. , Vol. 88, 423 (1993), 782--792.Google ScholarCross Ref
- Shihong Duan, Tianqing Yu, and Jie He. 2018. WiDriver: Driver Activity Recognition System Based on WiFi CSI. International Journal of Wireless Information Networks (2018), 1--11.Google ScholarCross Ref
- Matthew Field, David Stirling, Zengxi Pan, Montserrat Ros, and Fazel Naghdy. 2015. Recognizing human motions through mixture modeling of inertial data. Pattern Recognition , Vol. 48, 8 (2015), 2394--2406. Google ScholarDigital Library
- Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning . 1180--1189. Google ScholarDigital Library
- Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research , Vol. 17, 1 (2016), 2096--2030. Google ScholarDigital Library
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680. Google ScholarDigital Library
- Yves Grandvalet and Yoshua Bengio. 2005. Semi-supervised learning by entropy minimization. In Advances in neural information processing systems. 529--536. Google ScholarDigital Library
- Xiaonan Guo, Bo Liu, Cong Shi, Hongbo Liu, Yingying Chen, and Mooi Choo Chuah. 2017. WiFi-Enabled Smart Human Dynamics Monitoring. In Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems. ACM, 16. Google ScholarDigital Library
- Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Soundwave: using the doppler effect to sense gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1911--1914. Google ScholarDigital Library
- Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool release: Gathering 802.11 n traces with channel state information. ACM SIGCOMM Computer Communication Review , Vol. 41, 1 (2011), 53--53. Google ScholarDigital Library
- Wenfeng He, Kaishun Wu, Yongpan Zou, and Zhong Ming. 2015. Wig: Wifi-based gesture recognition system. In Computer Communication and Networks (ICCCN), 2015 24th International Conference on. IEEE, 1--7.Google ScholarCross Ref
- National Instruments. 2017. Introduction to the NI mmWave Transceiver System Hardware - National Instruments. http://www.ni.com/white-paper/53095/en/. Accessed on 06/25/2017.Google Scholar
- Wenjun Jiang, Qi Li, Lu Su, Chenglin Miao, Quanquan Gu, and Wenyao Xu. 2018. Towards Personalized Learning in Mobile Sensing Systems. In Distributed Computing Systems (ICDCS), 2018 IEEE 38th International Conference on. IEEE.Google ScholarCross Ref
- Matthew Keally, Gang Zhou, Guoliang Xing, Jianxin Wu, and Andrew Pyles. 2011. Pbn: towards practical activity recognition using smartphone-based body sensor networks. In Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems. ACM, 246--259. Google ScholarDigital Library
- Adil Mehmood Khan, Ali Tufail, Asad Masood Khattak, and Teemu H Laine. 2014. Activity recognition on smartphones via sensor-fusion and kda-based svms. International Journal of Distributed Sensor Networks , Vol. 10, 5 (2014), 503291.Google ScholarCross Ref
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- B. Langen, G. Lober, and W. Herzig. 1994. Reflection and transmission behaviour of building materials at 60 GHz. In Personal, Indoor and Mobile Radio Communications, 1994. Wireless Networks-Catching the Mobile Future., 5th IEEE International Symposium on, Vol. 2. IEEE, 505--509.Google Scholar
- Hong Li, Wei Yang, Jianxin Wang, Yang Xu, and Liusheng Huang. 2016b. WiFinger: talk to your smart devices with finger-grained gesture. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing . ACM, 250--261. Google ScholarDigital Library
- Tianxing Li, Qiang Liu, and Xia Zhou. 2016a. Practical human sensing in the light. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 71--84. Google ScholarDigital Library
- Tianxing Li, Xi Xiong, Yifei Xie, George Hito, Xing-Dong Yang, and Xia Zhou. 2017. Reconstructing hand poses using visible light. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , Vol. 1, 3 (2017), 71. Google ScholarDigital Library
- Jaime Lien, Nicholas Gillian, M Emre Karagozler, Patrick Amihood, Carsten Schwesig, Erik Olson, Hakim Raja, and Ivan Poupyrev. 2016. Soli: Ubiquitous gesture sensing with millimeter wave radar. ACM Transactions on Graphics (TOG) , Vol. 35, 4 (2016), 142. Google ScholarDigital Library
- Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research , Vol. 9, Nov (2008), 2579--2605.Google Scholar
- Thomas Nitsche, Guillermo Bielsa, Irene Tejado, Adrian Loch, and Joerg Widmer. 2015. Boon and bane of 60 GHz networks: practical insights into beamforming, interference, and frame level operation. In Proceedings of the 11th ACM Conference on Emerging Networking Experiments and Technologies. ACM, 17. Google ScholarDigital Library
- Ronald Poppe. 2010. A survey on vision-based human action recognition. Image and vision computing , Vol. 28, 6 (2010), 976--990. Google ScholarDigital Library
- Yang Qifan, Tang Hao, Zhao Xuebing, Li Yin, and Zhang Sanfeng. 2014. Dolphin: Ultrasonic-based gesture recognition on smartphone platform. In Computational Science and Engineering (CSE), 2014 IEEE 17th International Conference on. IEEE, 1461--1468. Google ScholarDigital Library
- A Rodr'iguez Valiente, A Trinidad, JR Garc'ia Berrocal, C Górriz, and R Ram'irez Camacho. 2014. Extended high-frequency (9--20 kHz) audiometry reference thresholds in 645 healthy subjects. International journal of audiology , Vol. 53, 8 (2014), 531--545.Google Scholar
- Wenjie Ruan, Quan Z Sheng, Lei Yang, Tao Gu, Peipei Xu, and Longfei Shangguan. 2016. AudioGest: enabling fine-grained hand gesture detection by decoding echo signal. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 474--485. Google ScholarDigital Library
- Swetank Kumar Saha, Hany Assasa, Adrian Loch, Naveen Muralidhar Prakash, Roshan Shyamsunder, Shivang Aggarwal, Daniel Steinmetzer, Dimitrios Koutsonikolas, Joerg Widmer, and Matthias Hollick. 2018. Fast and infuriating: Performance and pitfalls of 60 ghz wlans based on consumer-grade hardware. In 2018 15th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON). IEEE.Google ScholarCross Ref
- Swetank Kumar Saha, Yasaman Ghasempour, Muhammad Kumail Haider, Tariq Siddiqui, Paulo De Melo, Neerad Somanchi, Luke Zakrajsek, Arjun Singh, Owen Torres, Daniel Uvaydov, Josep Miquel Jornet, Edward Knightly, Dimitrios Koutsonikolas, Dimitris Pados, and Zhi Sun. 2017. X60: A programmable testbed for wideband 60 ghz wlans with phased arrays. In Proceedings of the 11th Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization. ACM, 75--82. Google ScholarDigital Library
- Markus Scholz, Till Riedel, Mario Hock, and Michael Beigl. 2013. Device-free and device-bound activity recognition using radio signal strength. In Proceedings of the 4th Augmented Human International Conference. ACM, 100--107. Google ScholarDigital Library
- Jiacheng Shang and Jie Wu. 2017. A robust sign language recognition system with sparsely labeled instances using Wi-Fi signals. In Mobile Ad Hoc and Sensor Systems (MASS), 2017 IEEE 14th International Conference on. IEEE, 99--107.Google ScholarCross Ref
- Cong Shi, Jian Liu, Hongbo Liu, and Yingying Chen. 2017. Smart user authentication through actuation of daily activities leveraging WiFi-enabled IoT. In Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 5. Google ScholarDigital Library
- Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. 2018. A DIRT-T Approach to Unsupervised Domain Adaptation. In International Conference on Learning Representations .Google Scholar
- Stephan Sigg, Shuyu Shi, Felix Buesching, Yusheng Ji, and Lars Wolf. 2013. Leveraging RF-channel fluctuation for activity recognition: Active and passive systems, continuous and RSSI-based signal features. In Proceedings of International Conference on Advances in Mobile Computing & Multimedia. ACM, 43. Google ScholarDigital Library
- Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2818--2826.Google ScholarCross Ref
- Sheng Tan and Jie Yang. 2016. WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition. In Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 201--210. Google ScholarDigital Library
- Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko. 2015. Simultaneous deep transfer across domains and tasks. In Proceedings of the IEEE International Conference on Computer Vision. 4068--4076. Google ScholarDigital Library
- Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Computer Vision and Pattern Recognition (CVPR) , Vol. 1. 4.Google Scholar
- Raghav H Venkatnarayan, Griffin Page, and Muhammad Shahzad. 2018. Multi-User Gesture Recognition Using WiFi. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 401--413. Google ScholarDigital Library
- Aditya Virmani and Muhammad Shahzad. 2017. Position and orientation agnostic gesture recognition using wifi. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 252--264. Google ScholarDigital Library
- Guanhua Wang, Yongpan Zou, Zimu Zhou, Kaishun Wu, and Lionel M Ni. 2016b. We can hear you with wi-fi! IEEE Transactions on Mobile Computing , Vol. 15, 11 (2016), 2907--2920. Google ScholarDigital Library
- Jie Wang, Liming Zhang, Qinghua Gao, Miao Pan, and Hongyu Wang. 2018. Device-Free Wireless Sensing in Complex Scenarios Using Spatial Structural Information. IEEE Transactions on Wireless Communications , Vol. 17, 4 (2018), 2432--2442.Google ScholarCross Ref
- Jie Wang, Xiao Zhang, Qinhua Gao, Hao Yue, and Hongyu Wang. 2017. Device-free wireless localization and activity recognition: A deep learning approach. IEEE Transactions on Vehicular Technology , Vol. 66, 7 (2017), 6258--6267.Google ScholarCross Ref
- Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2015. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking . ACM, 65--76. Google ScholarDigital Library
- Wei Wang, Alex X Liu, and Ke Sun. 2016a. Device-free gesture tracking using acoustic signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 82--94. Google ScholarDigital Library
- Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In Proceedings of the 20th annual international conference on Mobile computing and networking. ACM, 617--628. Google ScholarDigital Library
- Teng Wei and Xinyu Zhang. 2015. mtrack: High-precision passive tracking using millimeter wave radios. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 117--129. Google ScholarDigital Library
- Jason Weston, Frédéric Ratle, Hossein Mobahi, and Ronan Collobert. 2012. Deep learning via semi-supervised embedding. In Neural Networks: Tricks of the Trade . Springer, 639--655. Google Scholar
- Dan Wu, Daqing Zhang, Chenren Xu, Yasha Wang, and Hao Wang. 2016. WiDir: walking direction estimation using wireless signals. In Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing. ACM, 351--362. Google ScholarDigital Library
- Lu Xia, Chia-Chih Chen, and JK Aggarwal. 2012. View invariant human action recognition using histograms of 3d joints. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on. IEEE, 20--27.Google ScholarCross Ref
- Zhicheng Yang, Parth H Pathak, Yunze Zeng, Xixi Liran, and Prasant Mohapatra. 2016. Monitoring vital signs using millimeter wave. In Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 211--220. Google ScholarDigital Library
- Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, and Joy Zhang. 2014. Convolutional neural networks for human activity recognition using mobile sensors. In Mobile Computing, Applications and Services (MobiCASE), 2014 6th International Conference on. IEEE, 197--205.Google Scholar
- Ouyang Zhang and Kannan Srinivasan. 2016. Mudra: User-friendly Fine-grained Gesture Recognition using WiFi Signals. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies. ACM, 83--96. Google ScholarDigital Library
- Mingmin Zhao, Shichao Yue, Dina Katabi, Tommi S Jaakkola, and Matt T Bianchi. 2017. Learning sleep stages from radio signals: A conditional adversarial architecture. In International Conference on Machine Learning . 4100--4109.Google Scholar
- Yibo Zhu, Yanzi Zhu, Zengbin Zhang, Ben Y Zhao, and Haitao Zheng. 2015a. 60GHz mobile imaging radar. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. ACM, 75--80. Google ScholarDigital Library
- Yanzi Zhu, Yibo Zhu, Ben Y Zhao, and Haitao Zheng. 2015b. Reusing 60ghz radios for mobile radar imaging. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 103--116. Google ScholarDigital Library
- Yan Zhuang, Chen Song, Aosen Wang, Feng Lin, Yiran Li, Changzhan Gu, Changzhi Li, and Wenyao Xu. 2015. SleepSense: Non-invasive sleep event recognition using an electromagnetic probe. In Wearable and Implantable Body Sensor Networks (BSN), 2015 IEEE 12th International Conference on. IEEE, 1--6.Google ScholarCross Ref
Index Terms
- Towards Environment Independent Device Free Human Activity Recognition
Recommendations
A spanning tree-based human activity prediction system using life logs from depth silhouette-based human activity recognition
CAIP'11: Proceedings of the 14th international conference on Computer analysis of images and patterns - Volume Part IIn this work, we propose a Human Activity Prediction (HAP) system using activity sequence spanning trees constructed from a life-log created by a video sensor-based daily Human Activity Recognition (HAR) system using time-sequential Independent ...
Human Activity Behavioural Pattern Recognition in Smart Home with Long-Hour Data Collection
AbstractThe research on human activity recognition has provided novel solutions to many applications like health care, sports, and user profiling. Considering the complex nature of human activities, it is still challenging even after effective and ...
User, device and orientation independent human activity recognition on mobile phones: challenges and a proposal
UbiComp '13 Adjunct: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publicationSmart phones equipped with a rich set of sensors are explored as alternative platforms for human activity recognition in the ubiquitous computing domain. However, there exist challenges that should be tackled before the successful acceptance of such ...
Comments