Skip to main content
Erschienen in: The Journal of Supercomputing 4/2021

30.07.2020

Suspicious activity detection using deep learning in secure assisted living IoT environments

verfasst von: G. Vallathan, A. John, Chandrasegar Thirumalai, SenthilKumar Mohan, Gautam Srivastava, Jerry Chun-Wei Lin

Erschienen in: The Journal of Supercomputing | Ausgabe 4/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Children who are left alone in environments such as daycares and crèches require constant monitoring and care to protect them from abuse. In this paper, we propose a novel deep learning-based method for predicting the occurrence of abnormal events using footage gathered from networked surveillance systems and notifying users of those events in an Internet of Things (IoT) environment. Sequences of images are converted to still frames and de-blurred using adaptive motion detection techniques. Then, abnormal activities are predicted using random forest differential evolution with kernel density (RFKD), and any abnormal activities that are detected cause signals to be sent to IoT devices via the MQTT protocol. The proposed work consists of a multi-classifier, deep neural network and kernel density functions. The multi-classifier is used for input classifications from the sequence of frames of videos. The deep neural network is used to learn and train the data and kernel density is used clustering and prediction of data. The novelty of the proposed work is in the dynamic nature of activity prediction. Most of the previous work in this research area concentrated on static activity prediction. The proposed work is able to support both static and dynamic activities of daycare environments. In our experimental trials, our novel method’s performance is shown to be superior to that of the ReHAR method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
3.
Zurück zum Zitat Marco Stolpe, Dortmund TU (2016) The internet of things: opportunities and challenges for distributed data analysis. Newsl ACM SIGKDD Explor Newsl Arch 18(1):15–34CrossRef Marco Stolpe, Dortmund TU (2016) The internet of things: opportunities and challenges for distributed data analysis. Newsl ACM SIGKDD Explor Newsl Arch 18(1):15–34CrossRef
4.
Zurück zum Zitat Verizon. State of the Market: The Internet of Things 2015 Verizon. State of the Market: The Internet of Things 2015
5.
Zurück zum Zitat Chen L, Hoey J, Nugent C, Cook D, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C 42(6):790–808CrossRef Chen L, Hoey J, Nugent C, Cook D, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C 42(6):790–808CrossRef
6.
Zurück zum Zitat Kidd CD, Orr R, Abowd GD, Atkeson CG, Essa IA, MacIntyre B, Mynatt ED, Starner T, Newstetter W. (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, CoBuild’99. Springer, London, pp 191–198 Kidd CD, Orr R, Abowd GD, Atkeson CG, Essa IA, MacIntyre B, Mynatt ED, Starner T, Newstetter W. (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, CoBuild’99. Springer, London, pp 191–198
7.
Zurück zum Zitat Kasteren TLM, Englebienne G, Krose BJA (2011) Human activity recognition from wireless sensor network data: bench-mark and software. In: Activity Recognition in Pervasive Intelligent Environments. Atlantis Press, Paris, pp 165–186 Kasteren TLM, Englebienne G, Krose BJA (2011) Human activity recognition from wireless sensor network data: bench-mark and software. In: Activity Recognition in Pervasive Intelligent Environments. Atlantis Press, Paris, pp 165–186
8.
Zurück zum Zitat Cook DJ, Augusto JC, Jakkula VR (2009) Ambient intelligence: technologies, applications, and opportunities. Pervasive Mob Comput 54:277–298CrossRef Cook DJ, Augusto JC, Jakkula VR (2009) Ambient intelligence: technologies, applications, and opportunities. Pervasive Mob Comput 54:277–298CrossRef
9.
Zurück zum Zitat Lloret J, Canovas A, Sendra S, Parra L (2015) Smart communication architecture for ambient assisted living. IEEE Commun Mag 53:26–33CrossRef Lloret J, Canovas A, Sendra S, Parra L (2015) Smart communication architecture for ambient assisted living. IEEE Commun Mag 53:26–33CrossRef
10.
Zurück zum Zitat Singh A, Bianchi-Berthouze N, Williams ACC (2017) Supporting everyday function in chronic pain using wearable technology. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11, pp 3903–3915 Singh A, Bianchi-Berthouze N, Williams ACC (2017) Supporting everyday function in chronic pain using wearable technology. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11, pp 3903–3915
11.
Zurück zum Zitat Ghayvat H, Mukhopadhyay S, Shenjie B, Chouhan A, Chen W (2018) Smart home based ambient assisted living: recognition of anomaly in the activity of daily living for an elderly living alone. In: Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA, pp 14–17 Ghayvat H, Mukhopadhyay S, Shenjie B, Chouhan A, Chen W (2018) Smart home based ambient assisted living: recognition of anomaly in the activity of daily living for an elderly living alone. In: Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA, pp 14–17
12.
Zurück zum Zitat Martinez-Hernandez U, Dehghani-Sanij AA (2018) Probabilistic identification of sit-to-stand and stand-to-sit with a wearable sensor. Pattern Recognit Lett 118:32–41CrossRef Martinez-Hernandez U, Dehghani-Sanij AA (2018) Probabilistic identification of sit-to-stand and stand-to-sit with a wearable sensor. Pattern Recognit Lett 118:32–41CrossRef
13.
Zurück zum Zitat Barger T, Alwan M, Kell S, Turner B, Wood S, Naidu A (2002) A objective remote assessment of activities of daily living: analysis of meal preparation patterns. Poster presentation, Medical Automation Research Center, University of Virginia Health System Barger T, Alwan M, Kell S, Turner B, Wood S, Naidu A (2002) A objective remote assessment of activities of daily living: analysis of meal preparation patterns. Poster presentation, Medical Automation Research Center, University of Virginia Health System
14.
Zurück zum Zitat Cook D, Schmitter-Edgecombe M, Crandall A, Sanders C, Thomas B (2009) Collecting and disseminating smart home sensor data in the CASAS project. In: Proceedings of CHI09 Workshop on Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research Cook D, Schmitter-Edgecombe M, Crandall A, Sanders C, Thomas B (2009) Collecting and disseminating smart home sensor data in the CASAS project. In: Proceedings of CHI09 Workshop on Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research
15.
Zurück zum Zitat Gnanavel VK, Srinivasan A (2014) Abnormal event detection in crowded video scenes. In: Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA), pp 441–448 Gnanavel VK, Srinivasan A (2014) Abnormal event detection in crowded video scenes. In: Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA), pp 441–448
16.
Zurück zum Zitat Marinho BL, Souza Junior AH, Rebouças FP (2016) A new approach to human activity recognition using machine learning techniques. In: International Conference on Intelligent Systems Design and Applications (ISDA) Intelligent Systems Design and Applications, pp 529–538 Marinho BL, Souza Junior AH, Rebouças FP (2016) A new approach to human activity recognition using machine learning techniques. In: International Conference on Intelligent Systems Design and Applications (ISDA) Intelligent Systems Design and Applications, pp 529–538
17.
Zurück zum Zitat Jingjing Z, Eryan Y, Xuewei C (2010) A novel frame rate conversion algorithm based on adaptive motion search. In: IEEE International Conference on Audio, Language and Image Processing Jingjing Z, Eryan Y, Xuewei C (2010) A novel frame rate conversion algorithm based on adaptive motion search. In: IEEE International Conference on Audio, Language and Image Processing
18.
Zurück zum Zitat Kevin H, Hyun WP (2000) Using motion-compensated frame-rate conversion for the correction of 3: 2 pulldown artifacts in video sequences. IEEE Trans Circuits Syst Video Technol 10(6):869–877CrossRef Kevin H, Hyun WP (2000) Using motion-compensated frame-rate conversion for the correction of 3: 2 pulldown artifacts in video sequences. IEEE Trans Circuits Syst Video Technol 10(6):869–877CrossRef
19.
Zurück zum Zitat Kumar V, Minz S (2014) Feature selection: a literature review. Smart Comput Rev 4(3):211–229CrossRef Kumar V, Minz S (2014) Feature selection: a literature review. Smart Comput Rev 4(3):211–229CrossRef
20.
Zurück zum Zitat Bobick Aaron F, Davis James W (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267CrossRef Bobick Aaron F, Davis James W (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267CrossRef
21.
Zurück zum Zitat Zhao T, Nevatia R (2004) Tracking multiple humans in complex situations. IEEE Trans Pattern Anal Mach Intell 26(9):1208–1221CrossRef Zhao T, Nevatia R (2004) Tracking multiple humans in complex situations. IEEE Trans Pattern Anal Mach Intell 26(9):1208–1221CrossRef
22.
Zurück zum Zitat Hariataoglu I, Harwood D, Davis L (2000) W4: a real-time surveillance of people and their activities. IEEE Trans Pattern Anal Mach Intell 22(8):809–830CrossRef Hariataoglu I, Harwood D, Davis L (2000) W4: a real-time surveillance of people and their activities. IEEE Trans Pattern Anal Mach Intell 22(8):809–830CrossRef
23.
Zurück zum Zitat Rmer R, Stan S (1999) 3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. vol 2 Rmer R, Stan S (1999) 3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. vol 2
24.
Zurück zum Zitat Alexei E, Alexander B, Greg M, Jitendra M (2003) Recognizing actions at a distance. In: IEEE International Conference on Computer Vision, Nice, France Alexei E, Alexander B, Greg M, Jitendra M (2003) Recognizing actions at a distance. In: IEEE International Conference on Computer Vision, Nice, France
25.
Zurück zum Zitat Liu Y, Kender JR (2003) Sort-merge feature selection for video data. In: SDM Liu Y, Kender JR (2003) Sort-merge feature selection for video data. In: SDM
26.
Zurück zum Zitat Ribeiro P, Santos-Victor J, Lisboa P (2005) Human activity recognition from video: modeling feature selection and classification architecture. In: Proceedings of International Workshop Human Activity Recognition Model. pp 1–10 Ribeiro P, Santos-Victor J, Lisboa P (2005) Human activity recognition from video: modeling feature selection and classification architecture. In: Proceedings of International Workshop Human Activity Recognition Model. pp 1–10
27.
Zurück zum Zitat Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. In: Proceedings of International Conference Body Area Networks, pp 92–98 Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. In: Proceedings of International Conference Body Area Networks, pp 92–98
28.
Zurück zum Zitat Nuno CS, João MM, Paulo M (2013) Features selection for human activity recognition with iPhone inertial sensors. In: 16th Portuguese Conference on Artificial Intelligence. EPIA 2013. Angra do Heroísmo, September 9–12 Nuno CS, João MM, Paulo M (2013) Features selection for human activity recognition with iPhone inertial sensors. In: 16th Portuguese Conference on Artificial Intelligence. EPIA 2013. Angra do Heroísmo, September 9–12
29.
Zurück zum Zitat Hussein M, Eid E, Hoda O (2015) Evaluation of feature selection on human activity recognition. In: IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS) Hussein M, Eid E, Hoda O (2015) Evaluation of feature selection on human activity recognition. In: IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS)
30.
Zurück zum Zitat Li X, Chuah MC (2018) 2018 ReHAR: robust and efficient human activity recognition. In: IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe Li X, Chuah MC (2018) 2018 ReHAR: robust and efficient human activity recognition. In: IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe
31.
Zurück zum Zitat Dong-GyuLee Seong-WhanLee (2019) Prediction of partially observed human activity based on pre-trained deep representation. Pattern Recogn 85:198–206CrossRef Dong-GyuLee Seong-WhanLee (2019) Prediction of partially observed human activity based on pre-trained deep representation. Pattern Recogn 85:198–206CrossRef
32.
Zurück zum Zitat Wang A, Chen G, Wu X, Liu L, An N, Chang C-Y, Wang A, Chen G, Wu X, Liu L (2018) Towards human activity recognition: a hierarchical feature selection framework. Sensors 18:3629CrossRef Wang A, Chen G, Wu X, Liu L, An N, Chang C-Y, Wang A, Chen G, Wu X, Liu L (2018) Towards human activity recognition: a hierarchical feature selection framework. Sensors 18:3629CrossRef
33.
Zurück zum Zitat Nunesa UM, Fariab DE, Peixoto P (2017) A human activity recognition framework using max–min features and key poses with differential evolution random forests classifier. Pattern Recogn Lett 99:21–31CrossRef Nunesa UM, Fariab DE, Peixoto P (2017) A human activity recognition framework using max–min features and key poses with differential evolution random forests classifier. Pattern Recogn Lett 99:21–31CrossRef
34.
Zurück zum Zitat Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359MathSciNetCrossRef Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359MathSciNetCrossRef
35.
Zurück zum Zitat Ratrout NT, Gazder U (2014) Factors affecting performance of parametric and non-parametric models for daily traffic forecasting. Procedia Comput Sci 32:285–292CrossRef Ratrout NT, Gazder U (2014) Factors affecting performance of parametric and non-parametric models for daily traffic forecasting. Procedia Comput Sci 32:285–292CrossRef
36.
Zurück zum Zitat Alpaydin E (2010) Introduction to machine learning, 2nd edn. MIT Press, CambridgeMATH Alpaydin E (2010) Introduction to machine learning, 2nd edn. MIT Press, CambridgeMATH
37.
Zurück zum Zitat Brox T, Rosenhahn B, Cremers D, Seidel HP (2007) Non-parametric density estimation with adaptive, anisotropic kernels for human motion tracking. In: Human MotionâĂŞUnderstanding, Modeling, Capture, and Animation. Springer, Berlin, pp 152–165 Brox T, Rosenhahn B, Cremers D, Seidel HP (2007) Non-parametric density estimation with adaptive, anisotropic kernels for human motion tracking. In: Human MotionâĂŞUnderstanding, Modeling, Capture, and Animation. Springer, Berlin, pp 152–165
38.
Zurück zum Zitat Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90:1151–1163CrossRef Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90:1151–1163CrossRef
39.
Zurück zum Zitat Allan S, Henrik B, Sourav B (2015) Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition. In: Proceedings of ACM SenSys Allan S, Henrik B, Sourav B (2015) Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition. In: Proceedings of ACM SenSys
40.
Zurück zum Zitat Timo S, Heiner S (2016) On-body localization of wearable devices: an investigation of position-aware activity recognition. In: 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 1–9 Timo S, Heiner S (2016) On-body localization of wearable devices: an investigation of position-aware activity recognition. In: 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 1–9
42.
Zurück zum Zitat Pawar K, Attar V (2019) Deep learning approaches for video-based anomalous activity detection. World Wide Web 22:571–601CrossRef Pawar K, Attar V (2019) Deep learning approaches for video-based anomalous activity detection. World Wide Web 22:571–601CrossRef
43.
Zurück zum Zitat Ramachandran S, Palivela LH (2019) An intelligent system to detect human suspicious activity using deep neural networks. J Intell Fuzzy Syst 18:4507–4518CrossRef Ramachandran S, Palivela LH (2019) An intelligent system to detect human suspicious activity using deep neural networks. J Intell Fuzzy Syst 18:4507–4518CrossRef
44.
Zurück zum Zitat Seyed YN, Yu C, Alexander A, Erik B, Timothy RF (2019) I-SAFE: instant suspicious activity identification at the edge using fuzzy decision making. arXiv Seyed YN, Yu C, Alexander A, Erik B, Timothy RF (2019) I-SAFE: instant suspicious activity identification at the edge using fuzzy decision making. arXiv
46.
Zurück zum Zitat Malina L, Srivastava G, Dzurenda P, Hajny J, Fujdiak R (2019) A secure publish/subscribe protocol for internet of things. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, Aug 26, pp 1–10 Malina L, Srivastava G, Dzurenda P, Hajny J, Fujdiak R (2019) A secure publish/subscribe protocol for internet of things. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, Aug 26, pp 1–10
47.
Zurück zum Zitat Dwivedi AD, Malina L, Dzurenda P, Srivastava G (2019) Optimized blockchain model for internet of things based healthcare applications. In: 2019 42nd International Conference on Telecommunications and Signal Processing (TSP) Jul 1, pp 135–139. IEEE Dwivedi AD, Malina L, Dzurenda P, Srivastava G (2019) Optimized blockchain model for internet of things based healthcare applications. In: 2019 42nd International Conference on Telecommunications and Signal Processing (TSP) Jul 1, pp 135–139. IEEE
48.
Zurück zum Zitat Srivastava G, Lin JC, Pirouz M, Li Y, Yun U (2020) A pre-large weighted-fusion system of sensed high-utility patterns. IEEE Sen J 1:1–10 Srivastava G, Lin JC, Pirouz M, Li Y, Yun U (2020) A pre-large weighted-fusion system of sensed high-utility patterns. IEEE Sen J 1:1–10
49.
Zurück zum Zitat Sharma B, Srivastava G, Lin JC (2020) A bidirectional congestion control transport protocol for the internet of drones. Comput Commun 1(153):102–16CrossRef Sharma B, Srivastava G, Lin JC (2020) A bidirectional congestion control transport protocol for the internet of drones. Comput Commun 1(153):102–16CrossRef
Metadaten
Titel
Suspicious activity detection using deep learning in secure assisted living IoT environments
verfasst von
G. Vallathan
A. John
Chandrasegar Thirumalai
SenthilKumar Mohan
Gautam Srivastava
Jerry Chun-Wei Lin
Publikationsdatum
30.07.2020
Verlag
Springer US
Erschienen in
The Journal of Supercomputing / Ausgabe 4/2021
Print ISSN: 0920-8542
Elektronische ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-020-03387-8

Weitere Artikel der Ausgabe 4/2021

The Journal of Supercomputing 4/2021 Zur Ausgabe

Premium Partner