Martin Gjoreski
Stefan Kalabakov
ABSTRACT
Convolution Neural Network (CNN) filters learned on one domain can be used as feature extractors on another similar domain. Transferring filters allow reusing datasets across domains and reducing labelling costs. In this paper, four activity recognition datasets were analyzed to study the effects of transferring filters across the datasets. A spectro-temporal ResNet was implemented as a deep, end-to-end learning architecture. We analyzed the number of transferred CNN residual blocks with respect to the size of the target-adaptation data. The analysis showed that transfer learning using small adaptation subsets is more useful when the target domain contains a small number of different activities. Furthermore, the similarity between the domains, participating in the transfer learning scenario, seems to play a role in its success. The most successful transfer achieved an F1-score of 93%, which is an increase of 9 percentage points compared to a domain-specific baseline model.
- L. Deng, D. Yu. (2014). Deep Learning: Methods and Applications (PDF). Foundations and Trends in Signal Processing. 7 (3--4): 1--199.Google Scholar
- W. McCulloch, W. Pitts (1943). A Logical Calculus of Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics. 5 (4): 115--133.Google ScholarCross Ref
- Y. Bengio (2009). "Learning Deep Architectures for AI." Technical Report 1312. {Online. Accessible at: https://www.iro.umontreal.ca/~lisa/pointeurs/TR1312.pdf}Google Scholar
- A. Krizhevsky, I. Sutskever, G. E. Hinton (2012). ImageNet classification with deep convolutional neural networks. in International Conference on Neural Information Processing Systems, pp. 1097--1105. Google ScholarDigital Library
- H. Lee, R. Grosse, R. Ranganath, A.Y. Ng (2009). Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 609--616. ACM. Google ScholarDigital Library
- K. Jarrett, K. Kavukcuoglu, M. A. Ranzato, and Y. LeCun (2009). What is the best multi-stage architecture for object recognition? In International Conference on Computer V sion, pages 2146--2153. IEEE.Google ScholarCross Ref
- Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, vol. 86, no. 11, pp. 2278--2324.Google ScholarCross Ref
- H. Gjoreski, J. Bizjak, M. Gjoreski, M. Gams (2016). Comparing deep and classical machine learning methods for human activity recognition using wrist accelerometer, in: Proceedings of the IJCAI 2016 Workshop on Deep Learning for Artificial Intelligence, New York, NY, USA, Vol. 10.Google Scholar
- D. Ravi, C. Wong, B. Lo, G.-Z. Yang (2017). A deep learning approach to on-node sensor data analytics for mobile or wearable devices, IEEE journal of biomedical and health informatics 21 (1) 56--64.Google ScholarCross Ref
- S. Bhattacharya, N. D. Lane, Sparsification and separation of deep learning layers for constrained resource inference on wearables, in: Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD- ROM, ACM, 2016, pp. 176--189. Google ScholarDigital Library
- M. Zeng, L. T. Nguyen, B. Yu, O. J. Mengshoel, J. Zhu, P. Wu, J. Zhang (2014). Convolutional neural networks for human activity recognition using mobile sensors, in: 6th International Conference on Mobile Computing, Applications and Services, IEEE, pp. 197--205.Google ScholarCross Ref
- J. Yang, M. N. Nguyen, P. P. San, X. L. Li, S. Krishnaswamy (2015). Deep convolutional neural networks on multichannel time series for human activity recognition, in: Twenty-Fourth International Joint Conference on Artificial Intelligence. Google ScholarDigital Library
- F. Ordonez, D. Roggen (2016). Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition, Sensors 16 (1) 115.Google ScholarCross Ref
- S. Hochreiter, J. Schmidhuber (1997). Long short-term memory, Neural computation 9 (8) 1735--1780. Google ScholarDigital Library
- K. Krishna, D. Jain, S. V. Mehta, S. Choudhary (2018). An LSTM based system for prediction of human activities with durations, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1 (4) 147. Google ScholarDigital Library
- D. Cook, K. D. Feuz, N.C. Krishnan (2013). Transfer learning for activity recognition: A survey. Knowledge and information systems, 36(3), pp. 537--556. Google ScholarDigital Library
- S. Pan, Q. Yang (2010). A survey on transfer learning. Knowledge Data Entering IEEE Trans 22(10):1345--1359 Google ScholarDigital Library
- A. Calatroni, D. Roggen, and G. Tröster (2011). Automatic transfer of activity recognition capabilities between body-worn motion sensors: Training newcomers to recognize locomotion. In Proc. INSS, Vol. 6.Google Scholar
- M. Kurz, M. HÖlzle, A. Ferscha, A. Calatroni, and others (2011). Real-time transfer and evaluation of activity recognition capabilities in an opportunistic system. In Proc. ADAPTIVE. 73--78.Google Scholar
- R. Ding et al. (2019). Empirical Study and Improvement on Deep Transfer Learning for Human Activity Recognition. Sensors, 19(1), 57.Google ScholarCross Ref
- F. J. O. Morales, D. Roggen, (2016). Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (pp. 92--99). ACM. Google ScholarDigital Library
- D. Roggen, et al. (2010). Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh international conference on networked sensing systems (INSS) (pp. 233--240). IEEE.Google ScholarCross Ref
- P. Zappi et al. (2008). Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection. In European Conference on Wireless Sensor Networks (pp. 17--33). Springer, Berlin, Heidelberg. Google ScholarDigital Library
- H. Gjoreski, B. Kaluža, M. Gams, Radoje Milić, M. Luštrek (2015). Context-based Ensemble Method for Human Energy Expenditure Estimation. Applied Soft Computing, Pages 960--970. Google ScholarDigital Library
- S. Kozina, H. Gjoreski, M. Gams, M. Luštrek (2013). Three-layer activity recognition combining domain knowledge and meta-classification. Journal of Medical and Biological Engineering, vol. 33, no. 4. p.p. 406--414.Google ScholarCross Ref
- A. Reiss, D. Stricker (2012). Introducing a new benchmarked dataset for activity monitoring. In 2012 16th International Symposium on Wearable Computers (pp. 108--109). IEEE. Google ScholarDigital Library
- M. Gjoreski et al. (2019). Classical and deep learning methods for recognizing human activities and modes of transportation with smartphone sensors. Information Fusion, Under review.Google Scholar
- S. Hochreiter, Y. Bengio, P. Frasconi, J. Schmidhuber, et al. (2001). Gradient flow in recurrent nets: the difficulty of learning long-term dependenciesGoogle Scholar
- H. Gjoreski, M. Ciliberto, L. Wang, F. J. O. Morales, S. Mekki, S. Valentin, D. Roggen (2018). The University of Sussex-Huawei Locomotion and Transportation Dataset for Multimodal Analytics with Mobile Devices." IEEE Access, 2018Google Scholar
- S. Ioffe, C. Szegedy (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv preprint arXiv:1502.03167. Google ScholarDigital Library
- V. Nair, G. E. Hinton (2010). Rectified linear units improve restricted Boltzmann machines, in: Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 807--814. Google ScholarDigital Library
- S. Inoue, X Pan. (2016). Supervised and Unsupervised Transfer Learning for Activity Recognition from Simple In-home Sensors. MobiQuitous. Google ScholarDigital Library
Index Terms
- Cross-dataset deep transfer learning for activity recognition
Recommendations
Cross-Dataset Activity Recognition via Adaptive Spatial-Temporal Transfer Learning
Human activity recognition (HAR) aims at recognizing activities by training models on the large quantity of sensor data. Since it is time-consuming and expensive to acquire abundant labeled data, transfer learning becomes necessary for HAR by ...
Deep Transfer Learning for Cross-domain Activity Recognition
ICCSE'18: Proceedings of the 3rd International Conference on Crowd Science and EngineeringHuman activity recognition plays an important role in people's daily life. However, it is often expensive and time-consuming to acquire sufficient labeled activity data. To solve this problem, transfer learning leverages the labeled samples from the ...
Cross-domain activity recognition via transfer learning
In activity recognition, one major challenge is how to reduce the labeling effort one needs to make when recognizing a new set of activities. In this paper, we analyze the possibility of transferring knowledge from the available labeled data on a set of ...
Comments