Skip to main content
Top
Published in: The Journal of Supercomputing 4/2021

30-07-2020

Suspicious activity detection using deep learning in secure assisted living IoT environments

Authors: G. Vallathan, A. John, Chandrasegar Thirumalai, SenthilKumar Mohan, Gautam Srivastava, Jerry Chun-Wei Lin

Published in: The Journal of Supercomputing | Issue 4/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Children who are left alone in environments such as daycares and crèches require constant monitoring and care to protect them from abuse. In this paper, we propose a novel deep learning-based method for predicting the occurrence of abnormal events using footage gathered from networked surveillance systems and notifying users of those events in an Internet of Things (IoT) environment. Sequences of images are converted to still frames and de-blurred using adaptive motion detection techniques. Then, abnormal activities are predicted using random forest differential evolution with kernel density (RFKD), and any abnormal activities that are detected cause signals to be sent to IoT devices via the MQTT protocol. The proposed work consists of a multi-classifier, deep neural network and kernel density functions. The multi-classifier is used for input classifications from the sequence of frames of videos. The deep neural network is used to learn and train the data and kernel density is used clustering and prediction of data. The novelty of the proposed work is in the dynamic nature of activity prediction. Most of the previous work in this research area concentrated on static activity prediction. The proposed work is able to support both static and dynamic activities of daycare environments. In our experimental trials, our novel method’s performance is shown to be superior to that of the ReHAR method.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
3.
go back to reference Marco Stolpe, Dortmund TU (2016) The internet of things: opportunities and challenges for distributed data analysis. Newsl ACM SIGKDD Explor Newsl Arch 18(1):15–34CrossRef Marco Stolpe, Dortmund TU (2016) The internet of things: opportunities and challenges for distributed data analysis. Newsl ACM SIGKDD Explor Newsl Arch 18(1):15–34CrossRef
4.
go back to reference Verizon. State of the Market: The Internet of Things 2015 Verizon. State of the Market: The Internet of Things 2015
5.
go back to reference Chen L, Hoey J, Nugent C, Cook D, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C 42(6):790–808CrossRef Chen L, Hoey J, Nugent C, Cook D, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C 42(6):790–808CrossRef
6.
go back to reference Kidd CD, Orr R, Abowd GD, Atkeson CG, Essa IA, MacIntyre B, Mynatt ED, Starner T, Newstetter W. (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, CoBuild’99. Springer, London, pp 191–198 Kidd CD, Orr R, Abowd GD, Atkeson CG, Essa IA, MacIntyre B, Mynatt ED, Starner T, Newstetter W. (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, CoBuild’99. Springer, London, pp 191–198
7.
go back to reference Kasteren TLM, Englebienne G, Krose BJA (2011) Human activity recognition from wireless sensor network data: bench-mark and software. In: Activity Recognition in Pervasive Intelligent Environments. Atlantis Press, Paris, pp 165–186 Kasteren TLM, Englebienne G, Krose BJA (2011) Human activity recognition from wireless sensor network data: bench-mark and software. In: Activity Recognition in Pervasive Intelligent Environments. Atlantis Press, Paris, pp 165–186
8.
go back to reference Cook DJ, Augusto JC, Jakkula VR (2009) Ambient intelligence: technologies, applications, and opportunities. Pervasive Mob Comput 54:277–298CrossRef Cook DJ, Augusto JC, Jakkula VR (2009) Ambient intelligence: technologies, applications, and opportunities. Pervasive Mob Comput 54:277–298CrossRef
9.
go back to reference Lloret J, Canovas A, Sendra S, Parra L (2015) Smart communication architecture for ambient assisted living. IEEE Commun Mag 53:26–33CrossRef Lloret J, Canovas A, Sendra S, Parra L (2015) Smart communication architecture for ambient assisted living. IEEE Commun Mag 53:26–33CrossRef
10.
go back to reference Singh A, Bianchi-Berthouze N, Williams ACC (2017) Supporting everyday function in chronic pain using wearable technology. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11, pp 3903–3915 Singh A, Bianchi-Berthouze N, Williams ACC (2017) Supporting everyday function in chronic pain using wearable technology. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11, pp 3903–3915
11.
go back to reference Ghayvat H, Mukhopadhyay S, Shenjie B, Chouhan A, Chen W (2018) Smart home based ambient assisted living: recognition of anomaly in the activity of daily living for an elderly living alone. In: Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA, pp 14–17 Ghayvat H, Mukhopadhyay S, Shenjie B, Chouhan A, Chen W (2018) Smart home based ambient assisted living: recognition of anomaly in the activity of daily living for an elderly living alone. In: Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA, pp 14–17
12.
go back to reference Martinez-Hernandez U, Dehghani-Sanij AA (2018) Probabilistic identification of sit-to-stand and stand-to-sit with a wearable sensor. Pattern Recognit Lett 118:32–41CrossRef Martinez-Hernandez U, Dehghani-Sanij AA (2018) Probabilistic identification of sit-to-stand and stand-to-sit with a wearable sensor. Pattern Recognit Lett 118:32–41CrossRef
13.
go back to reference Barger T, Alwan M, Kell S, Turner B, Wood S, Naidu A (2002) A objective remote assessment of activities of daily living: analysis of meal preparation patterns. Poster presentation, Medical Automation Research Center, University of Virginia Health System Barger T, Alwan M, Kell S, Turner B, Wood S, Naidu A (2002) A objective remote assessment of activities of daily living: analysis of meal preparation patterns. Poster presentation, Medical Automation Research Center, University of Virginia Health System
14.
go back to reference Cook D, Schmitter-Edgecombe M, Crandall A, Sanders C, Thomas B (2009) Collecting and disseminating smart home sensor data in the CASAS project. In: Proceedings of CHI09 Workshop on Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research Cook D, Schmitter-Edgecombe M, Crandall A, Sanders C, Thomas B (2009) Collecting and disseminating smart home sensor data in the CASAS project. In: Proceedings of CHI09 Workshop on Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research
15.
go back to reference Gnanavel VK, Srinivasan A (2014) Abnormal event detection in crowded video scenes. In: Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA), pp 441–448 Gnanavel VK, Srinivasan A (2014) Abnormal event detection in crowded video scenes. In: Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA), pp 441–448
16.
go back to reference Marinho BL, Souza Junior AH, Rebouças FP (2016) A new approach to human activity recognition using machine learning techniques. In: International Conference on Intelligent Systems Design and Applications (ISDA) Intelligent Systems Design and Applications, pp 529–538 Marinho BL, Souza Junior AH, Rebouças FP (2016) A new approach to human activity recognition using machine learning techniques. In: International Conference on Intelligent Systems Design and Applications (ISDA) Intelligent Systems Design and Applications, pp 529–538
17.
go back to reference Jingjing Z, Eryan Y, Xuewei C (2010) A novel frame rate conversion algorithm based on adaptive motion search. In: IEEE International Conference on Audio, Language and Image Processing Jingjing Z, Eryan Y, Xuewei C (2010) A novel frame rate conversion algorithm based on adaptive motion search. In: IEEE International Conference on Audio, Language and Image Processing
18.
go back to reference Kevin H, Hyun WP (2000) Using motion-compensated frame-rate conversion for the correction of 3: 2 pulldown artifacts in video sequences. IEEE Trans Circuits Syst Video Technol 10(6):869–877CrossRef Kevin H, Hyun WP (2000) Using motion-compensated frame-rate conversion for the correction of 3: 2 pulldown artifacts in video sequences. IEEE Trans Circuits Syst Video Technol 10(6):869–877CrossRef
19.
go back to reference Kumar V, Minz S (2014) Feature selection: a literature review. Smart Comput Rev 4(3):211–229CrossRef Kumar V, Minz S (2014) Feature selection: a literature review. Smart Comput Rev 4(3):211–229CrossRef
20.
go back to reference Bobick Aaron F, Davis James W (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267CrossRef Bobick Aaron F, Davis James W (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267CrossRef
21.
go back to reference Zhao T, Nevatia R (2004) Tracking multiple humans in complex situations. IEEE Trans Pattern Anal Mach Intell 26(9):1208–1221CrossRef Zhao T, Nevatia R (2004) Tracking multiple humans in complex situations. IEEE Trans Pattern Anal Mach Intell 26(9):1208–1221CrossRef
22.
go back to reference Hariataoglu I, Harwood D, Davis L (2000) W4: a real-time surveillance of people and their activities. IEEE Trans Pattern Anal Mach Intell 22(8):809–830CrossRef Hariataoglu I, Harwood D, Davis L (2000) W4: a real-time surveillance of people and their activities. IEEE Trans Pattern Anal Mach Intell 22(8):809–830CrossRef
23.
go back to reference Rmer R, Stan S (1999) 3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. vol 2 Rmer R, Stan S (1999) 3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. vol 2
24.
go back to reference Alexei E, Alexander B, Greg M, Jitendra M (2003) Recognizing actions at a distance. In: IEEE International Conference on Computer Vision, Nice, France Alexei E, Alexander B, Greg M, Jitendra M (2003) Recognizing actions at a distance. In: IEEE International Conference on Computer Vision, Nice, France
25.
go back to reference Liu Y, Kender JR (2003) Sort-merge feature selection for video data. In: SDM Liu Y, Kender JR (2003) Sort-merge feature selection for video data. In: SDM
26.
go back to reference Ribeiro P, Santos-Victor J, Lisboa P (2005) Human activity recognition from video: modeling feature selection and classification architecture. In: Proceedings of International Workshop Human Activity Recognition Model. pp 1–10 Ribeiro P, Santos-Victor J, Lisboa P (2005) Human activity recognition from video: modeling feature selection and classification architecture. In: Proceedings of International Workshop Human Activity Recognition Model. pp 1–10
27.
go back to reference Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. In: Proceedings of International Conference Body Area Networks, pp 92–98 Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. In: Proceedings of International Conference Body Area Networks, pp 92–98
28.
go back to reference Nuno CS, João MM, Paulo M (2013) Features selection for human activity recognition with iPhone inertial sensors. In: 16th Portuguese Conference on Artificial Intelligence. EPIA 2013. Angra do Heroísmo, September 9–12 Nuno CS, João MM, Paulo M (2013) Features selection for human activity recognition with iPhone inertial sensors. In: 16th Portuguese Conference on Artificial Intelligence. EPIA 2013. Angra do Heroísmo, September 9–12
29.
go back to reference Hussein M, Eid E, Hoda O (2015) Evaluation of feature selection on human activity recognition. In: IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS) Hussein M, Eid E, Hoda O (2015) Evaluation of feature selection on human activity recognition. In: IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS)
30.
go back to reference Li X, Chuah MC (2018) 2018 ReHAR: robust and efficient human activity recognition. In: IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe Li X, Chuah MC (2018) 2018 ReHAR: robust and efficient human activity recognition. In: IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe
31.
go back to reference Dong-GyuLee Seong-WhanLee (2019) Prediction of partially observed human activity based on pre-trained deep representation. Pattern Recogn 85:198–206CrossRef Dong-GyuLee Seong-WhanLee (2019) Prediction of partially observed human activity based on pre-trained deep representation. Pattern Recogn 85:198–206CrossRef
32.
go back to reference Wang A, Chen G, Wu X, Liu L, An N, Chang C-Y, Wang A, Chen G, Wu X, Liu L (2018) Towards human activity recognition: a hierarchical feature selection framework. Sensors 18:3629CrossRef Wang A, Chen G, Wu X, Liu L, An N, Chang C-Y, Wang A, Chen G, Wu X, Liu L (2018) Towards human activity recognition: a hierarchical feature selection framework. Sensors 18:3629CrossRef
33.
go back to reference Nunesa UM, Fariab DE, Peixoto P (2017) A human activity recognition framework using max–min features and key poses with differential evolution random forests classifier. Pattern Recogn Lett 99:21–31CrossRef Nunesa UM, Fariab DE, Peixoto P (2017) A human activity recognition framework using max–min features and key poses with differential evolution random forests classifier. Pattern Recogn Lett 99:21–31CrossRef
34.
go back to reference Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359MathSciNetCrossRef Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359MathSciNetCrossRef
35.
go back to reference Ratrout NT, Gazder U (2014) Factors affecting performance of parametric and non-parametric models for daily traffic forecasting. Procedia Comput Sci 32:285–292CrossRef Ratrout NT, Gazder U (2014) Factors affecting performance of parametric and non-parametric models for daily traffic forecasting. Procedia Comput Sci 32:285–292CrossRef
36.
go back to reference Alpaydin E (2010) Introduction to machine learning, 2nd edn. MIT Press, CambridgeMATH Alpaydin E (2010) Introduction to machine learning, 2nd edn. MIT Press, CambridgeMATH
37.
go back to reference Brox T, Rosenhahn B, Cremers D, Seidel HP (2007) Non-parametric density estimation with adaptive, anisotropic kernels for human motion tracking. In: Human MotionâĂŞUnderstanding, Modeling, Capture, and Animation. Springer, Berlin, pp 152–165 Brox T, Rosenhahn B, Cremers D, Seidel HP (2007) Non-parametric density estimation with adaptive, anisotropic kernels for human motion tracking. In: Human MotionâĂŞUnderstanding, Modeling, Capture, and Animation. Springer, Berlin, pp 152–165
38.
go back to reference Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90:1151–1163CrossRef Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90:1151–1163CrossRef
39.
go back to reference Allan S, Henrik B, Sourav B (2015) Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition. In: Proceedings of ACM SenSys Allan S, Henrik B, Sourav B (2015) Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition. In: Proceedings of ACM SenSys
40.
go back to reference Timo S, Heiner S (2016) On-body localization of wearable devices: an investigation of position-aware activity recognition. In: 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 1–9 Timo S, Heiner S (2016) On-body localization of wearable devices: an investigation of position-aware activity recognition. In: 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 1–9
42.
go back to reference Pawar K, Attar V (2019) Deep learning approaches for video-based anomalous activity detection. World Wide Web 22:571–601CrossRef Pawar K, Attar V (2019) Deep learning approaches for video-based anomalous activity detection. World Wide Web 22:571–601CrossRef
43.
go back to reference Ramachandran S, Palivela LH (2019) An intelligent system to detect human suspicious activity using deep neural networks. J Intell Fuzzy Syst 18:4507–4518CrossRef Ramachandran S, Palivela LH (2019) An intelligent system to detect human suspicious activity using deep neural networks. J Intell Fuzzy Syst 18:4507–4518CrossRef
44.
go back to reference Seyed YN, Yu C, Alexander A, Erik B, Timothy RF (2019) I-SAFE: instant suspicious activity identification at the edge using fuzzy decision making. arXiv Seyed YN, Yu C, Alexander A, Erik B, Timothy RF (2019) I-SAFE: instant suspicious activity identification at the edge using fuzzy decision making. arXiv
46.
go back to reference Malina L, Srivastava G, Dzurenda P, Hajny J, Fujdiak R (2019) A secure publish/subscribe protocol for internet of things. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, Aug 26, pp 1–10 Malina L, Srivastava G, Dzurenda P, Hajny J, Fujdiak R (2019) A secure publish/subscribe protocol for internet of things. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, Aug 26, pp 1–10
47.
go back to reference Dwivedi AD, Malina L, Dzurenda P, Srivastava G (2019) Optimized blockchain model for internet of things based healthcare applications. In: 2019 42nd International Conference on Telecommunications and Signal Processing (TSP) Jul 1, pp 135–139. IEEE Dwivedi AD, Malina L, Dzurenda P, Srivastava G (2019) Optimized blockchain model for internet of things based healthcare applications. In: 2019 42nd International Conference on Telecommunications and Signal Processing (TSP) Jul 1, pp 135–139. IEEE
48.
go back to reference Srivastava G, Lin JC, Pirouz M, Li Y, Yun U (2020) A pre-large weighted-fusion system of sensed high-utility patterns. IEEE Sen J 1:1–10 Srivastava G, Lin JC, Pirouz M, Li Y, Yun U (2020) A pre-large weighted-fusion system of sensed high-utility patterns. IEEE Sen J 1:1–10
49.
go back to reference Sharma B, Srivastava G, Lin JC (2020) A bidirectional congestion control transport protocol for the internet of drones. Comput Commun 1(153):102–16CrossRef Sharma B, Srivastava G, Lin JC (2020) A bidirectional congestion control transport protocol for the internet of drones. Comput Commun 1(153):102–16CrossRef
Metadata
Title
Suspicious activity detection using deep learning in secure assisted living IoT environments
Authors
G. Vallathan
A. John
Chandrasegar Thirumalai
SenthilKumar Mohan
Gautam Srivastava
Jerry Chun-Wei Lin
Publication date
30-07-2020
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 4/2021
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-020-03387-8

Other articles of this Issue 4/2021

The Journal of Supercomputing 4/2021 Go to the issue

Premium Partner