Skip to main content
Erschienen in: The Journal of Supercomputing 10/2020

14.07.2018

Transition activity recognition using fuzzy logic and overlapped sliding window-based convolutional neural networks

verfasst von: Jaewoong Kang, Jongmo Kim, Seongil Lee, Mye Sohn

Erschienen in: The Journal of Supercomputing | Ausgabe 10/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we propose a novel approach that can recognize transition activities (e.g., turn to left or right, stand up, and travel down the stairs). Unlike simple activities, the transition activities have unique characteristics that change continuously and occur instantaneously. To recognize the transition activities with these characteristics, we applied convolutional neural network (CNN) that is widely adopted to recognize images, voices, and human activities. In addition, to generate input instances for the CNN model, we developed the overlapped sliding window method, which can accurately recognize the transition activities occurring during a short time. To increase the accuracy of the activity recognition, we have learned CNN models by separating the simple activity and the transition activity. Finally, we adopt fuzzy logic that can be used to handle ambiguous activities. All the procedures of recognizing the elderly’s activities are performed using the data collected by the six sensors embedded in the smartphone. The effectiveness of the proposed approach is shown through experiments. We demonstrate that our approach can improve recognition accuracy of transition activities.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv (CSUR) 43(3):16CrossRef Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv (CSUR) 43(3):16CrossRef
2.
Zurück zum Zitat Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL (2012) Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: International Workshop on Ambient Assisted Living, pp 216–223 Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL (2012) Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: International Workshop on Ambient Assisted Living, pp 216–223
3.
Zurück zum Zitat Ann OC, Theng LB (2014) Human activity recognition: a review. In: 2014 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), pp 389–393 Ann OC, Theng LB (2014) Human activity recognition: a review. In: 2014 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), pp 389–393
4.
Zurück zum Zitat Attal F, Mohammed S, Dedabrishvili M, Chamroukhi F, Oukhellou L, Amirat Y (2015) Physical human activity recognition using wearable sensors. Sensors 15(12):31314–31338CrossRef Attal F, Mohammed S, Dedabrishvili M, Chamroukhi F, Oukhellou L, Amirat Y (2015) Physical human activity recognition using wearable sensors. Sensors 15(12):31314–31338CrossRef
5.
Zurück zum Zitat Baccouche M, Mamalet F, Wolf C, Garcia C, Baskurt A (2011) Sequential deep learning for human action recognition. In: International Workshop on Human Behavior Understanding, pp 29–39 Baccouche M, Mamalet F, Wolf C, Garcia C, Baskurt A (2011) Sequential deep learning for human action recognition. In: International Workshop on Human Behavior Understanding, pp 29–39
6.
Zurück zum Zitat Bayat A, Pomplun M, Tran DA (2014) A study on human activity recognition using accelerometer data from smartphones. Proc Comput Sci 34:450–457CrossRef Bayat A, Pomplun M, Tran DA (2014) A study on human activity recognition using accelerometer data from smartphones. Proc Comput Sci 34:450–457CrossRef
7.
Zurück zum Zitat Dernbach S, Das B, Krishnan NC, Thomas BL, Cook DJ (2012) Simple and complex activity recognition through smart phones. In: 2012 8th International Conference on Intelligent Environments (IE), pp 214–221 Dernbach S, Das B, Krishnan NC, Thomas BL, Cook DJ (2012) Simple and complex activity recognition through smart phones. In: 2012 8th International Conference on Intelligent Environments (IE), pp 214–221
8.
Zurück zum Zitat Dietterich TG (2002) Machine learning for sequential data: a review. In: Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), pp 15–30 Dietterich TG (2002) Machine learning for sequential data: a review. In: Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), pp 15–30
9.
Zurück zum Zitat Fong S, Liu K, Cho K, Wong R, Mohammed S, Fiaidhi J (2016) Improvised methods for tackling big data stream mining challenges: case study of human activity recognition. J Supercomput 72(10):3927–3959CrossRef Fong S, Liu K, Cho K, Wong R, Mohammed S, Fiaidhi J (2016) Improvised methods for tackling big data stream mining challenges: case study of human activity recognition. J Supercomput 72(10):3927–3959CrossRef
10.
Zurück zum Zitat Ghosh A, Riccardi G (2014) Recognizing human activities from smartphone sensor signals. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp 865–868 Ghosh A, Riccardi G (2014) Recognizing human activities from smartphone sensor signals. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp 865–868
11.
Zurück zum Zitat Goffe WL, Ferrier GD, Rogers J (1994) Global optimization of statistical functions with simulated annealing. J Econom 60(1–2):65–99MATHCrossRef Goffe WL, Ferrier GD, Rogers J (1994) Global optimization of statistical functions with simulated annealing. J Econom 60(1–2):65–99MATHCrossRef
12.
Zurück zum Zitat Guan D, Yuan W, Lee YK, Gavrilov A, Lee S (2007) Activity recognition based on semi-supervised learning. In: 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, 2007. RTCSA 2007, pp 469–475 Guan D, Yuan W, Lee YK, Gavrilov A, Lee S (2007) Activity recognition based on semi-supervised learning. In: 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, 2007. RTCSA 2007, pp 469–475
13.
Zurück zum Zitat Hammerla NY, Halloran S, Ploetz T (2016) Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv:1604.08880 Hammerla NY, Halloran S, Ploetz T (2016) Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv:​1604.​08880
14.
Zurück zum Zitat Huynh, T, Schiele, B (2005) Analyzing features for activity recognition. In: Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-Aware Services: Usages and Technologies, pp 159–163 Huynh, T, Schiele, B (2005) Analyzing features for activity recognition. In: Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-Aware Services: Usages and Technologies, pp 159–163
15.
Zurück zum Zitat Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp 1307–1310 Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp 1307–1310
16.
Zurück zum Zitat Kalita S, Karmakar A, Hazarika SM (2018) Efficient extraction of spatial relations for extended objects vis-à-vis human activity recognition in video. Appl Intell 48:204–219CrossRef Kalita S, Karmakar A, Hazarika SM (2018) Efficient extraction of spatial relations for extended objects vis-à-vis human activity recognition in video. Appl Intell 48:204–219CrossRef
17.
Zurück zum Zitat Kang J, Kim J, Lee S, Sohn M (2017) Recognition of transition activities of human using CNN-based on overlapped sliding window. In: The 5th International Conference on Big Data Applications and Services (BIGDAS), pp 143–148 Kang J, Kim J, Lee S, Sohn M (2017) Recognition of transition activities of human using CNN-based on overlapped sliding window. In: The 5th International Conference on Big Data Applications and Services (BIGDAS), pp 143–148
18.
Zurück zum Zitat Kavitha R (2017) Human activity recognition from sensor data using random forest algorithm. Int J Adv Res Comput Sci 8(3):334–337MathSciNet Kavitha R (2017) Human activity recognition from sensor data using random forest algorithm. Int J Adv Res Comput Sci 8(3):334–337MathSciNet
19.
Zurück zum Zitat Khan AM, Lee YK, Lee SY, Kim TS (2010) A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans Inf Technol Biomed 14(5):1166–1172CrossRef Khan AM, Lee YK, Lee SY, Kim TS (2010) A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans Inf Technol Biomed 14(5):1166–1172CrossRef
20.
Zurück zum Zitat Lee SM, Yoon SM, Cho H (2017) Human activity recognition from accelerometer data using convolutional neural network. In: 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), pp 131–134 Lee SM, Yoon SM, Cho H (2017) Human activity recognition from accelerometer data using convolutional neural network. In: 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), pp 131–134
21.
Zurück zum Zitat Lester J, Choudhury T, Kern N, Borriello G, Hannaford B (2005) A hybrid discriminative/generative approach for modeling human activities. In: IJCAI International Joint Conference on Artificial Intelligence (pp 766–772) Lester J, Choudhury T, Kern N, Borriello G, Hannaford B (2005) A hybrid discriminative/generative approach for modeling human activities. In: IJCAI International Joint Conference on Artificial Intelligence (pp 766–772)
22.
Zurück zum Zitat Lyu L, He X, Law YW, Palaniswami M (2017) Privacy-preserving collaborative deep learning with application to human activity recognition. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp 1219–1228 Lyu L, He X, Law YW, Palaniswami M (2017) Privacy-preserving collaborative deep learning with application to human activity recognition. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp 1219–1228
23.
Zurück zum Zitat Núñez JC, Cabido R, Pantrigo JJ, Montemayor AS, Vélez JF (2018) Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recogn 76:80–94CrossRef Núñez JC, Cabido R, Pantrigo JJ, Montemayor AS, Vélez JF (2018) Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recogn 76:80–94CrossRef
24.
Zurück zum Zitat Reyes-Ortiz JL, Oneto L, Samà A, Parra X, Anguita D (2016) Transition-aware human activity recognition using smartphones. Neurocomputing 171:754–767CrossRef Reyes-Ortiz JL, Oneto L, Samà A, Parra X, Anguita D (2016) Transition-aware human activity recognition using smartphones. Neurocomputing 171:754–767CrossRef
25.
Zurück zum Zitat Rodríguez ND, Cuéllar MP, Lilius J, Calvo-Flores MD (2014) A fuzzy ontology for semantic modelling and recognition of human behaviour. Knowl-Based Syst 66:46–60CrossRef Rodríguez ND, Cuéllar MP, Lilius J, Calvo-Flores MD (2014) A fuzzy ontology for semantic modelling and recognition of human behaviour. Knowl-Based Syst 66:46–60CrossRef
26.
Zurück zum Zitat Ronao CA, Cho SB (2014) Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models. In: 2014 10th International Conference on Natural Computation (ICNC), pp 681–686 Ronao CA, Cho SB (2014) Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models. In: 2014 10th International Conference on Natural Computation (ICNC), pp 681–686
27.
Zurück zum Zitat Roy N, Misra A, Cook D (2016) Ambient and smartphone sensor assisted ADL recognition in multi-inhabitant smart environments. J Ambient Intell Humaniz Comput 7(1):1–19CrossRef Roy N, Misra A, Cook D (2016) Ambient and smartphone sensor assisted ADL recognition in multi-inhabitant smart environments. J Ambient Intell Humaniz Comput 7(1):1–19CrossRef
28.
Zurück zum Zitat Sheng M, Jiang J, Su B, Tang Q, Yahya AA, Wang G (2016) Short-time activity recognition with wearable sensors using convolutional neural network. In: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume 1, pp 413–416 Sheng M, Jiang J, Su B, Tang Q, Yahya AA, Wang G (2016) Short-time activity recognition with wearable sensors using convolutional neural network. In: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume 1, pp 413–416
29.
Zurück zum Zitat Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A et al (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56(1):116–124CrossRef Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A et al (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56(1):116–124CrossRef
31.
Zurück zum Zitat Sousa W, Souto E, Rodrigres J, Sadarc P, Jalali R, El-Khatib K (2017) A comparative analysis of the impact of features on human activity recognition with smartphone sensors. In: Proceedings of the 23rd Brazilian Symposium on Multimedia and the Web, pp 397–404 Sousa W, Souto E, Rodrigres J, Sadarc P, Jalali R, El-Khatib K (2017) A comparative analysis of the impact of features on human activity recognition with smartphone sensors. In: Proceedings of the 23rd Brazilian Symposium on Multimedia and the Web, pp 397–404
32.
Zurück zum Zitat Tang W, Sazonov ES (2014) Highly accurate recognition of human postures and activities through classification with rejection. IEEE J Biomed Health Inform 18(1):309–315CrossRef Tang W, Sazonov ES (2014) Highly accurate recognition of human postures and activities through classification with rejection. IEEE J Biomed Health Inform 18(1):309–315CrossRef
33.
Zurück zum Zitat Vishwakarma DK, Kapoor R (2015) Hybrid classifier based human activity recognition using the silhouette and cells. Expert Syst Appl 42(20):6957–6965CrossRef Vishwakarma DK, Kapoor R (2015) Hybrid classifier based human activity recognition using the silhouette and cells. Expert Syst Appl 42(20):6957–6965CrossRef
34.
Zurück zum Zitat Vishwakarma DK, Kapoor R, Maheshwari R, Kapoor V, Raman S (2015) Recognition of abnormal human activity using the changes in orientation of silhouette in key frames. In: 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom), pp 336–341 Vishwakarma DK, Kapoor R, Maheshwari R, Kapoor V, Raman S (2015) Recognition of abnormal human activity using the changes in orientation of silhouette in key frames. In: 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom), pp 336–341
35.
Zurück zum Zitat Yang J, Lee J, Choi J (2011) Activity recognition based on RFID object usage for smart mobile devices. J Comput Sci Technol 26(2):239–246CrossRef Yang J, Lee J, Choi J (2011) Activity recognition based on RFID object usage for smart mobile devices. J Comput Sci Technol 26(2):239–246CrossRef
36.
Zurück zum Zitat Yazdansepas D, Niazi AH, Gay JL, Maier FW, Ramaswamy L, Rasheed K, Buman MP (2016) A multi-featured approach for wearable sensor-based human activity recognition. In: 2016 IEEE International Conference on Healthcare Informatics (ICHI), pp 423–431 Yazdansepas D, Niazi AH, Gay JL, Maier FW, Ramaswamy L, Rasheed K, Buman MP (2016) A multi-featured approach for wearable sensor-based human activity recognition. In: 2016 IEEE International Conference on Healthcare Informatics (ICHI), pp 423–431
37.
Zurück zum Zitat Zebin T, Scully PJ, Ozanyan KB (2016) Human activity recognition with inertial sensors using a deep learning approach. In: SENSORS, 2016. IEEE, pp 1–3 Zebin T, Scully PJ, Ozanyan KB (2016) Human activity recognition with inertial sensors using a deep learning approach. In: SENSORS, 2016. IEEE, pp 1–3
38.
Zurück zum Zitat Zeng M, Nguyen LT, Yu B, Mengshoel OJ, Zhu J, Wu P, Zhang J (2014) Convolutional neural networks for human activity recognition using mobile sensors. In: 2014 6th International Conference on Mobile Computing, Applications and Services (MobiCASE), pp 197–205 Zeng M, Nguyen LT, Yu B, Mengshoel OJ, Zhu J, Wu P, Zhang J (2014) Convolutional neural networks for human activity recognition using mobile sensors. In: 2014 6th International Conference on Mobile Computing, Applications and Services (MobiCASE), pp 197–205
Metadaten
Titel
Transition activity recognition using fuzzy logic and overlapped sliding window-based convolutional neural networks
verfasst von
Jaewoong Kang
Jongmo Kim
Seongil Lee
Mye Sohn
Publikationsdatum
14.07.2018
Verlag
Springer US
Erschienen in
The Journal of Supercomputing / Ausgabe 10/2020
Print ISSN: 0920-8542
Elektronische ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-018-2470-y

Weitere Artikel der Ausgabe 10/2020

The Journal of Supercomputing 10/2020 Zur Ausgabe