Skip to main content
Erschienen in: Artificial Life and Robotics 2/2018

30.12.2017 | Original Article

Deep recurrent neural network for mobile human activity recognition with high throughput

verfasst von: Masaya Inoue, Sozo Inoue, Takeshi Nishida

Erschienen in: Artificial Life and Robotics | Ausgabe 2/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we propose a method of human activity recognition with high throughput from raw accelerometer data applying a deep recurrent neural network (DRNN), and investigate various architectures and its combination to find the best parameter values. The “high throughput” refers to short time at a time of recognition. We investigated various parameters and architectures of the DRNN by using the training dataset of 432 trials with 6 activity classes from 7 people. The maximum recognition rate was 95.42% and 83.43% against the test data of 108 segmented trials each of which has single activity class and 18 multiple sequential trials, respectively. Here, the maximum recognition rates by traditional methods were 71.65% and 54.97% for each. In addition, the efficiency of the found parameters was evaluated using additional dataset. Further, as for throughput of the recognition per unit time, the constructed DRNN was requiring only 1.347 ms, while the best traditional method required 11.031 ms which includes 11.027 ms for feature calculation. These advantages are caused by the compact and small architecture of the constructed real time oriented DRNN.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Amodei D, Ananthanarayanan S, Anubhai R, Bai J, Battenberg E, Case C, Casper J, Catanzaro B, Cheng Q, Chen G, Chen J, Chen J, Chen Z, Chrzanowski M, Coates A, Diamos G, Ding K, Du N, Elsen E, Engel J, Fang W, Fan L, Fougner C, Gao L, Gong C, Hannun A, Han T, Johannes L, Jiang B, Ju C, Jun B, LeGresley P, Lin L, Liu J, Liu Y, Li W, Li X, Ma D, Narang S, Ng A, Ozair S, Peng Y, Prenger R, Qian S, Quan Z, Raiman J, Rao V, Satheesh S, Seetapun D, Sengupta S, Srinet K, Sriram A, Tang H, Tang L, Wang C, Wang J, Wang K, Wang Y, Wang Z, Wang Z, Wu S, Wei L, Xiao B, Xie W, Xie Y, Yogatama D, Yuan B, Zhan J, Zhu Z (Jun 2016) Deep speech 2: end-to-end speech recognition in English and mandarin. In: Balcan MF, Weinberger KQ (eds) Proceedings of The 33rd international conference on machine learning, Proceedings of machine learning research, PMLR, New York, New York, USA, vol 48, 20–22, pp 173–182 Amodei D, Ananthanarayanan S, Anubhai R, Bai J, Battenberg E, Case C, Casper J, Catanzaro B, Cheng Q, Chen G, Chen J, Chen J, Chen Z, Chrzanowski M, Coates A, Diamos G, Ding K, Du N, Elsen E, Engel J, Fang W, Fan L, Fougner C, Gao L, Gong C, Hannun A, Han T, Johannes L, Jiang B, Ju C, Jun B, LeGresley P, Lin L, Liu J, Liu Y, Li W, Li X, Ma D, Narang S, Ng A, Ozair S, Peng Y, Prenger R, Qian S, Quan Z, Raiman J, Rao V, Satheesh S, Seetapun D, Sengupta S, Srinet K, Sriram A, Tang H, Tang L, Wang C, Wang J, Wang K, Wang Y, Wang Z, Wang Z, Wu S, Wei L, Xiao B, Xie W, Xie Y, Yogatama D, Yuan B, Zhan J, Zhu Z (Jun 2016) Deep speech 2: end-to-end speech recognition in English and mandarin. In: Balcan MF, Weinberger KQ (eds) Proceedings of The 33rd international conference on machine learning, Proceedings of machine learning research, PMLR, New York, New York, USA, vol 48, 20–22, pp 173–182
2.
Zurück zum Zitat Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL (2013) A public domain dataset for human activity recognition using Smartphones. In: ESANN Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL (2013) A public domain dataset for human activity recognition using Smartphones. In: ESANN
3.
Zurück zum Zitat Avci A, Bosch S, Marin-Perianu M, Marin-Perianu R, Havinga P (2010) Activity, recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey. In: 23rd international conference on Architecture of computing systems (ARCS), VDE 2010, pp 1–10 Avci A, Bosch S, Marin-Perianu M, Marin-Perianu R, Havinga P (2010) Activity, recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey. In: 23rd international conference on Architecture of computing systems (ARCS), VDE 2010, pp 1–10
4.
Zurück zum Zitat Bao L, Intille S (2004) Activity recognition from user-annotated acceleration data. In: Ferscha A., Mattern F (eds) Pervasive computing, Pervasive 2004. Lecture Notes in Computer Science, vol 3001. Springer, Berlin, Heidelberg, pp 1–17 Bao L, Intille S (2004) Activity recognition from user-annotated acceleration data. In: Ferscha A., Mattern F (eds) Pervasive computing, Pervasive 2004. Lecture Notes in Computer Science, vol 3001. Springer, Berlin, Heidelberg, pp 1–17
5.
Zurück zum Zitat Bhattacharya S, Nurmi P, Hammerla N, Plötz T (2014) Using unlabeled data in a sparse-coding framework for human activity recognition. Pervas Mobile Comput 15:242–262CrossRef Bhattacharya S, Nurmi P, Hammerla N, Plötz T (2014) Using unlabeled data in a sparse-coding framework for human activity recognition. Pervas Mobile Comput 15:242–262CrossRef
6.
Zurück zum Zitat Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Sur 46:1–33CrossRef Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Sur 46:1–33CrossRef
7.
Zurück zum Zitat Graves A, Fernández S, Gomez F, Schmidhuber J (2006) Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on machine learning. ACM, pp 369–376 Graves A, Fernández S, Gomez F, Schmidhuber J (2006) Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on machine learning. ACM, pp 369–376
8.
Zurück zum Zitat Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATH Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATH
9.
Zurück zum Zitat Hammerla NY, Halloran S, Ploetz T (2016) Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880 Hammerla NY, Halloran S, Ploetz T (2016) Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:​1604.​08880
10.
Zurück zum Zitat Haykin S (1994) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall PTR, Upper Saddle River, NJ, pp 664–682, 732–740 Haykin S (1994) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall PTR, Upper Saddle River, NJ, pp 664–682, 732–740
11.
Zurück zum Zitat Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv: 1207.0580, pp 1–18 Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:​ 1207.​0580, pp 1–18
12.
Zurück zum Zitat Hochreiter S, Hochreiter S, Schmidhuber J, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–80CrossRef Hochreiter S, Hochreiter S, Schmidhuber J, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–80CrossRef
13.
Zurück zum Zitat Inoue S, Ueda N, Nohara Y, Nakashima N (2015) mobile activity recognition for a whole day: recognizing real nursing activities with big dataset. In: ACM international conference on pervasive and ubiquitous computing (Ubicomp), Osaka Inoue S, Ueda N, Nohara Y, Nakashima N (2015) mobile activity recognition for a whole day: recognizing real nursing activities with big dataset. In: ACM international conference on pervasive and ubiquitous computing (Ubicomp), Osaka
14.
Zurück zum Zitat Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM (2001) Nurses’ recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med 161:2467–2473CrossRef Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM (2001) Nurses’ recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med 161:2467–2473CrossRef
15.
Zurück zum Zitat Kawaguchi N, Ogawa N, Iwasaki Y, Kaji, K, Terada T, Murao K, Inoue S, Kawahara Y, Sumi Y, Nishio N (2011) Hasc challenge: gathering large scale human activity corpus for the real-world activity understandings. In: Proceedings of the 2nd augmented human international conference. ACM, p 27 Kawaguchi N, Ogawa N, Iwasaki Y, Kaji, K, Terada T, Murao K, Inoue S, Kawahara Y, Sumi Y, Nishio N (2011) Hasc challenge: gathering large scale human activity corpus for the real-world activity understandings. In: Proceedings of the 2nd augmented human international conference. ACM, p 27
16.
Zurück zum Zitat Kim E, Helal S, Cook D (2010) Human activity recognition and pattern discovery. Pervas Comput IEEE 9:48–53CrossRef Kim E, Helal S, Cook D (2010) Human activity recognition and pattern discovery. Pervas Comput IEEE 9:48–53CrossRef
17.
Zurück zum Zitat Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: International conference on learning representations, pp 1–13 Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: International conference on learning representations, pp 1–13
18.
Zurück zum Zitat Krishnan NC, Cook DJ (2014) Activity recognition on streaming sensor data. Pervas Mobile Comput 10(PART B):138–154CrossRef Krishnan NC, Cook DJ (2014) Activity recognition on streaming sensor data. Pervas Mobile Comput 10(PART B):138–154CrossRef
19.
Zurück zum Zitat Kunze K, Barry M, Heinz E, Lukowicz P, Majoe D, Gutknecht J (2006) Towards, recognizing tai chi-an initial experiment using wearable sensors. In: 3rd international forum on applied wearable computing (IFAWC), VDE 2006, pp 1–6 Kunze K, Barry M, Heinz E, Lukowicz P, Majoe D, Gutknecht J (2006) Towards, recognizing tai chi-an initial experiment using wearable sensors. In: 3rd international forum on applied wearable computing (IFAWC), VDE 2006, pp 1–6
20.
Zurück zum Zitat Ladha C, Hammerla NY, Olivier P, Plötz T (2013) ClimbAX: skill assessment for climbing enthusiasts. In: Proceedings of the 2013 ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 235–244 Ladha C, Hammerla NY, Olivier P, Plötz T (2013) ClimbAX: skill assessment for climbing enthusiasts. In: Proceedings of the 2013 ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 235–244
21.
Zurück zum Zitat Lane ND, Miluzzo E, Lu H, Peebles D, Choudhury T, Campbell AT (2010) A survey of mobile phone sensing. IEEE Commun Mag 48:140–150CrossRef Lane ND, Miluzzo E, Lu H, Peebles D, Choudhury T, Campbell AT (2010) A survey of mobile phone sensing. IEEE Commun Mag 48:140–150CrossRef
22.
Zurück zum Zitat Le QV, Jaitly N, Hinton GE (2015) A simple way to initialize recurrent networks of rectified linear units. arXiv preprint arXiv:1504.00941 Le QV, Jaitly N, Hinton GE (2015) A simple way to initialize recurrent networks of rectified linear units. arXiv preprint arXiv:​1504.​00941
23.
Zurück zum Zitat Mazilu S, Blanke U, Dorfman M, Gazit E, Mirelman A, Hausdorff J, Tröster G (2015) A wearable assistant for gait training for Parkinsons disease with freezing of gait in out-of-the-lab environments. ACM Trans Interact Intell Syst (TiiS), 5 1:5 Mazilu S, Blanke U, Dorfman M, Gazit E, Mirelman A, Hausdorff J, Tröster G (2015) A wearable assistant for gait training for Parkinsons disease with freezing of gait in out-of-the-lab environments. ACM Trans Interact Intell Syst (TiiS), 5 1:5
24.
Zurück zum Zitat Ordóñez FJ, Roggen D (2016) Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1):115CrossRef Ordóñez FJ, Roggen D (2016) Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1):115CrossRef
25.
Zurück zum Zitat Pasa L, Sperduti A (2014) Pre-training of recurrent neural networks via linear autoencoders. Adv Neural Inf Process Syst 27:3572–3580 Pasa L, Sperduti A (2014) Pre-training of recurrent neural networks via linear autoencoders. Adv Neural Inf Process Syst 27:3572–3580
26.
Zurück zum Zitat Pascanu R, Mikolov T, Bengio Y (2012) On the difficulty of training recurrent neural networks. In: Proceedings of the 30th international conference on machine learning, vol 2, pp 1310–1318 Pascanu R, Mikolov T, Bengio Y (2012) On the difficulty of training recurrent neural networks. In: Proceedings of the 30th international conference on machine learning, vol 2, pp 1310–1318
27.
Zurück zum Zitat Saeedi R, Schimert B, Ghasemzadeh H (2014) Cost-sensitive feature selection for on-body sensor localization. In: 2nd international workshop on human activity sensing corpus and its application (HASCA2014) held at UbiComp 2014, pp 833–842 Saeedi R, Schimert B, Ghasemzadeh H (2014) Cost-sensitive feature selection for on-body sensor localization. In: 2nd international workshop on human activity sensing corpus and its application (HASCA2014) held at UbiComp 2014, pp 833–842
28.
Zurück zum Zitat Strohrmann C, Harms H, Tröster G (2011) What do sensors know about your running performance? In: Proceedings—international symposium on wearable computers, ISWC, pp 101–104 Strohrmann C, Harms H, Tröster G (2011) What do sensors know about your running performance? In: Proceedings—international symposium on wearable computers, ISWC, pp 101–104
29.
Zurück zum Zitat Sutskever I (2013) Training recurrent neural networks. Ph.D. thesis, p 101 Sutskever I (2013) Training recurrent neural networks. Ph.D. thesis, p 101
30.
Zurück zum Zitat Tokui S, Oono K, Hido S, Clayton J (2015) Chainer: a next-generation open source framework for deep learning. In: Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS) Tokui S, Oono K, Hido S, Clayton J (2015) Chainer: a next-generation open source framework for deep learning. In: Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS)
31.
Zurück zum Zitat Wu J, Yildirim I, Lim J, Freeman W, Tenenbaum J (2015) Galileo: perceiving physical object properties by integrating a physics engine with deep learning. In: Advances in neural information processing systems (NIPS 2015), vol 28, pp 1–9 Wu J, Yildirim I, Lim J, Freeman W, Tenenbaum J (2015) Galileo: perceiving physical object properties by integrating a physics engine with deep learning. In: Advances in neural information processing systems (NIPS 2015), vol 28, pp 1–9
32.
Zurück zum Zitat Yang JB, Nguyen MN, San PP, Li XL, Krishnaswamy S (2015) Deep convolutional neural networks on multichannel time series for human activity recognition. In: Proceedings of the 24th international joint conference on artificial intelligence (IJCAI), Buenos Aires, Argentina, pp 25–31 Yang JB, Nguyen MN, San PP, Li XL, Krishnaswamy S (2015) Deep convolutional neural networks on multichannel time series for human activity recognition. In: Proceedings of the 24th international joint conference on artificial intelligence (IJCAI), Buenos Aires, Argentina, pp 25–31
33.
Zurück zum Zitat Zhan K, Faux S, Ramos F (2014) Multi-scale conditional random fields for first-person activity recognition. In: IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 51–59 Zhan K, Faux S, Ramos F (2014) Multi-scale conditional random fields for first-person activity recognition. In: IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 51–59
34.
Zurück zum Zitat Zhang M, Sawchuk A (2012) Motion primitive-based human activity recognition using a bag-of-features approach. In: Proceedings of the 2nd ACM SIGHIT, vol 1, p 631 Zhang M, Sawchuk A (2012) Motion primitive-based human activity recognition using a bag-of-features approach. In: Proceedings of the 2nd ACM SIGHIT, vol 1, p 631
35.
Zurück zum Zitat Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. In: International conference on body area networks, pp 92–98 Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. In: International conference on body area networks, pp 92–98
Metadaten
Titel
Deep recurrent neural network for mobile human activity recognition with high throughput
verfasst von
Masaya Inoue
Sozo Inoue
Takeshi Nishida
Publikationsdatum
30.12.2017
Verlag
Springer Japan
Erschienen in
Artificial Life and Robotics / Ausgabe 2/2018
Print ISSN: 1433-5298
Elektronische ISSN: 1614-7456
DOI
https://doi.org/10.1007/s10015-017-0422-x

Weitere Artikel der Ausgabe 2/2018

Artificial Life and Robotics 2/2018 Zur Ausgabe

Neuer Inhalt