Skip to main content
Erschienen in: Journal on Multimodal User Interfaces 4/2016

01.12.2016 | Original Paper

Pedestrian activity classification using patterns of motion and histogram of oriented gradient

verfasst von: Rifat Muhammad Mueid, Chandrama Ahmed, Md. Atiqur Rahman Ahad

Erschienen in: Journal on Multimodal User Interfaces | Ausgabe 4/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper presents a view-based spatio-temporal template formation scheme for pedestrian activity recognition and classification. It emphasizes on detailed classification of the entire wide range of pedestrian actions in the context of road crossing and their recognition from a viewpoint of a static car in the traffic signal. For this purpose, we develop a new pedestrian action dataset containing 160 videos of eight actions with twenty subjects per action. First, action sequences are converted into Motion History Image where silhouette sequence is condensed into gray scale images, while dominant motion information is preserved. Then, Histogram of Oriented Gradient of the Motion History Image is calculated from local appearance and shape within the image for feature extraction. Finally, Support Vector Machine is employed for pedestrian action classification. The demonstrated method successfully classified 91 % of the pedestrian action. To compare the method with other methods, we also implement some established methods with the developed dataset.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Ahad MAR, Tan JK, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23:255–281CrossRef Ahad MAR, Tan JK, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23:255–281CrossRef
2.
Zurück zum Zitat Rosales R (1998) Recognition of human action using moment-based features. In: Boston university computer science tech. report, BU 98–020, pp 1–19 Rosales R (1998) Recognition of human action using moment-based features. In: Boston university computer science tech. report, BU 98–020, pp 1–19
3.
Zurück zum Zitat Ahad MAR (2013) Motion history images for action recognition and understanding, 1st edn. Springer, BerlinCrossRefMATH Ahad MAR (2013) Motion history images for action recognition and understanding, 1st edn. Springer, BerlinCrossRefMATH
4.
Zurück zum Zitat Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Underst 73(3):428–440CrossRef Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Underst 73(3):428–440CrossRef
5.
Zurück zum Zitat Turaga P, Chellappa R, Subrahmanian VS, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video Technol 18(11):1473–1488CrossRef Turaga P, Chellappa R, Subrahmanian VS, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video Technol 18(11):1473–1488CrossRef
6.
Zurück zum Zitat Moeslund TB, Granum E (2001) A survey of computer vision-based human motion capture. Comput Vis Image Underst 106:90–126MATH Moeslund TB, Granum E (2001) A survey of computer vision-based human motion capture. Comput Vis Image Underst 106:90–126MATH
7.
Zurück zum Zitat Wang L, Hu W, Tan T (2003) Recent developments in human motion analysis. Pattern Recogn 36:585–601CrossRef Wang L, Hu W, Tan T (2003) Recent developments in human motion analysis. Pattern Recogn 36:585–601CrossRef
8.
Zurück zum Zitat Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Underst 108:4–18CrossRef Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Underst 108:4–18CrossRef
9.
Zurück zum Zitat Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28:976–990CrossRef Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28:976–990CrossRef
10.
Zurück zum Zitat Ahad MAR (2011) Computer vision and action recognition: a guide for image processing and computer vision community for action understanding, 1st edn. Springer, BerlinCrossRef Ahad MAR (2011) Computer vision and action recognition: a guide for image processing and computer vision community for action understanding, 1st edn. Springer, BerlinCrossRef
11.
Zurück zum Zitat Li W, Zhang Z, Liu Z (2010) Action recognition based on a bag of 3D points. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), pp 9–14 Li W, Zhang Z, Liu Z (2010) Action recognition based on a bag of 3D points. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), pp 9–14
12.
Zurück zum Zitat Satkin S, Hebert M (2010) Modeling the temporal extent of actions. In: European conference on computer vision Satkin S, Hebert M (2010) Modeling the temporal extent of actions. In: European conference on computer vision
14.
Zurück zum Zitat Rapantzikos K, Avrithis Y, Kollias S (2009) Dense saliency-based spatiotemporal feature points for action recognition. IEEE Comput Vis Pattern Recogn pp 1454–1461 Rapantzikos K, Avrithis Y, Kollias S (2009) Dense saliency-based spatiotemporal feature points for action recognition. IEEE Comput Vis Pattern Recogn pp 1454–1461
15.
Zurück zum Zitat Bregler C (1997) Learning and Recognizing Human Dynamics in Video Sequences. In: IEEE conf. comp, vision and pattern recognition Bregler C (1997) Learning and Recognizing Human Dynamics in Video Sequences. In: IEEE conf. comp, vision and pattern recognition
16.
Zurück zum Zitat Fathi A, Mori G (2008) Action recognition by learning mid-level motion features. IEEE Comput Vis Pattern Recogn pp 1–8 Fathi A, Mori G (2008) Action recognition by learning mid-level motion features. IEEE Comput Vis Pattern Recogn pp 1–8
17.
Zurück zum Zitat Bobick AF, Davis JW (1997) The representation and recognition of human movement using temporal templates. In: IEEE computer society conference on computer vision and pattern recognition, pp 928–934 Bobick AF, Davis JW (1997) The representation and recognition of human movement using temporal templates. In: IEEE computer society conference on computer vision and pattern recognition, pp 928–934
18.
Zurück zum Zitat Souvenir R, Babbs J (2008) Learning the viewpoint manifold for action recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–7 Souvenir R, Babbs J (2008) Learning the viewpoint manifold for action recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–7
19.
Zurück zum Zitat Gong D, Medioni G (2011) Dynamic manifold warping for view invariant action recognition. IEEE ICCV, pp 571–578 Gong D, Medioni G (2011) Dynamic manifold warping for view invariant action recognition. IEEE ICCV, pp 571–578
20.
Zurück zum Zitat Shen Y, Foroosh H (2009) View-invariant action recognition from point triplets. IEEE Trans PAMI 31(10):1898–1905 Shen Y, Foroosh H (2009) View-invariant action recognition from point triplets. IEEE Trans PAMI 31(10):1898–1905
21.
Zurück zum Zitat Luo Q, Kong X, Zeng G, Fan J (2010) Human action detection via boosted local motion histograms. Mach Vis Appl 21(3):377–389CrossRef Luo Q, Kong X, Zeng G, Fan J (2010) Human action detection via boosted local motion histograms. Mach Vis Appl 21(3):377–389CrossRef
22.
Zurück zum Zitat Ryoo MS, Aggarwal JK (2009) Spatio-temporal relationship match: video structure comparison for recognition of complex human activities. In: Proc. IEEE international conference on computer vision, ICCV Ryoo MS, Aggarwal JK (2009) Spatio-temporal relationship match: video structure comparison for recognition of complex human activities. In: Proc. IEEE international conference on computer vision, ICCV
23.
Zurück zum Zitat Kovashka A, Grauman K (2010) Learning a hierarchy of discriminative space-time neighborhood features for human action recognition. IEEE Comput Vis Pattern Recogn pp 2046–2053 Kovashka A, Grauman K (2010) Learning a hierarchy of discriminative space-time neighborhood features for human action recognition. IEEE Comput Vis Pattern Recogn pp 2046–2053
24.
Zurück zum Zitat Shao L, Ji L, Liu Y, Zhang J (2012) Human action segmentation and recognition via motion and shape analysis. Pattern Recogn Lett 33(4):438–445CrossRef Shao L, Ji L, Liu Y, Zhang J (2012) Human action segmentation and recognition via motion and shape analysis. Pattern Recogn Lett 33(4):438–445CrossRef
25.
Zurück zum Zitat Munder S, Gavrila DM (2006) An experimental study on pedestrian classification. IEEE Trans Pattern Anal Mach Intell 28(11):1863–1868CrossRef Munder S, Gavrila DM (2006) An experimental study on pedestrian classification. IEEE Trans Pattern Anal Mach Intell 28(11):1863–1868CrossRef
26.
Zurück zum Zitat Viola P, Jones M, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161CrossRef Viola P, Jones M, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161CrossRef
27.
Zurück zum Zitat Broggi A, Cerri P, Ghidoni S, Grisler P, Jung HG (2009) A new approach to urban pedestrian detection for automatic braking. IEEE Trans Intell Transp Syst 10(4):594–605CrossRef Broggi A, Cerri P, Ghidoni S, Grisler P, Jung HG (2009) A new approach to urban pedestrian detection for automatic braking. IEEE Trans Intell Transp Syst 10(4):594–605CrossRef
28.
Zurück zum Zitat Suard F, Rakotomamonjy A, Bensrhair A, Broggi A (2006) Pedestrian detection using infrared images and histograms of oriented gradients, IEEE Intelligent Vehicles Symposium, pp. 206–212 Suard F, Rakotomamonjy A, Bensrhair A, Broggi A (2006) Pedestrian detection using infrared images and histograms of oriented gradients, IEEE Intelligent Vehicles Symposium, pp. 206–212
29.
Zurück zum Zitat Bertozzi M, Broggi A, Fascioli A, Graf T, Marc-Michael Meinecke (2004) Pedestrian detection for driver assistance using multiresolution infrared vision. IEEE Trans Veh Technol 53(6):1666–1678CrossRef Bertozzi M, Broggi A, Fascioli A, Graf T, Marc-Michael Meinecke (2004) Pedestrian detection for driver assistance using multiresolution infrared vision. IEEE Trans Veh Technol 53(6):1666–1678CrossRef
30.
Zurück zum Zitat Nanda H, Davis L (2002) Probabilistic template based pedestrian detection in infrared videos. IEEE Intell Veh Symp 1:15–20 Nanda H, Davis L (2002) Probabilistic template based pedestrian detection in infrared videos. IEEE Intell Veh Symp 1:15–20
31.
Zurück zum Zitat Lin Z, Jiang Z, Davis LS (2009) Recognizing actions by shape-motion prototype trees. In: IEEE 12th international conference on computer vision, pp 444–451 Lin Z, Jiang Z, Davis LS (2009) Recognizing actions by shape-motion prototype trees. In: IEEE 12th international conference on computer vision, pp 444–451
32.
Zurück zum Zitat Broggi A, Bertozzi M, Fascioli A, Sechi M (2000) Shape-based pedestrian detection. IEEE Intell Veh Symp, pp 2046–2053 Broggi A, Bertozzi M, Fascioli A, Sechi M (2000) Shape-based pedestrian detection. IEEE Intell Veh Symp, pp 2046–2053
33.
Zurück zum Zitat Xu Y, Xu D, Lin S, Han TX, Cao X, Li X (2012) Detection of sudden pedestrian crossings for driving assistance systems. IEEE Trans Syst Man Cybern Part B Cybern 42(3):729–739CrossRef Xu Y, Xu D, Lin S, Han TX, Cao X, Li X (2012) Detection of sudden pedestrian crossings for driving assistance systems. IEEE Trans Syst Man Cybern Part B Cybern 42(3):729–739CrossRef
34.
Zurück zum Zitat Gerenimo D, Lopez AM, Sappa AD, Graf T (2010) Survey of pedestrian detection for advanced driver assistance systems. IEEE Trans Pattern Anal Mach Intell 32(7):1239–1258CrossRef Gerenimo D, Lopez AM, Sappa AD, Graf T (2010) Survey of pedestrian detection for advanced driver assistance systems. IEEE Trans Pattern Anal Mach Intell 32(7):1239–1258CrossRef
35.
Zurück zum Zitat Meng H, Pears N, Bailey C (2007) A human action recognition system for embedded computer vision application. In: Proc. workshop on embedded computer vision (with CVPR), pp 1–6 Meng H, Pears N, Bailey C (2007) A human action recognition system for embedded computer vision application. In: Proc. workshop on embedded computer vision (with CVPR), pp 1–6
36.
Zurück zum Zitat Ahad MAR, Ogata T, Tan JK, Kim H, Ishikawa S (2008) A complex motion recognition technique employing directional motion templates. Int J Innov Comput Inf Control 4(8):1943–1954 Ahad MAR, Ogata T, Tan JK, Kim H, Ishikawa S (2008) A complex motion recognition technique employing directional motion templates. Int J Innov Comput Inf Control 4(8):1943–1954
37.
Zurück zum Zitat Ahad MAR, Ogata T, Tan JK, Kim H, Ishikawa S (2008) Moment-based human motion recognition from the representation of DMHI templates, In: SICE annual conference, pp 578–583 Ahad MAR, Ogata T, Tan JK, Kim H, Ishikawa S (2008) Moment-based human motion recognition from the representation of DMHI templates, In: SICE annual conference, pp 578–583
38.
Zurück zum Zitat Bobick A, Davis J (1996) An appearance-based representation of action. In: International conference on pattern recognition, pp 307–312 Bobick A, Davis J (1996) An appearance-based representation of action. In: International conference on pattern recognition, pp 307–312
39.
Zurück zum Zitat Davis JW (1996) Appearance-based motion recognition of human actions. In: M.I.T. media lab perceptual computing group tech. report no. 387, p 51 Davis JW (1996) Appearance-based motion recognition of human actions. In: M.I.T. media lab perceptual computing group tech. report no. 387, p 51
40.
Zurück zum Zitat Anderson C, Bert P, Wal GV (1985) Change detection and tracking using pyramids transformation techniques. Proc SPIE-Intell Robot Comput Vis 579:72–78 Anderson C, Bert P, Wal GV (1985) Change detection and tracking using pyramids transformation techniques. Proc SPIE-Intell Robot Comput Vis 579:72–78
41.
Zurück zum Zitat Lipton AJ, Fujiyoshi H, Patil RS (1998) Moving target classification and tracking from real-time video. In: Proc. IEEE workshop on applications of computer vision, pp 8–14 Lipton AJ, Fujiyoshi H, Patil RS (1998) Moving target classification and tracking from real-time video. In: Proc. IEEE workshop on applications of computer vision, pp 8–14
42.
Zurück zum Zitat Ahad MAR, Tan JK, Kim H, Ishikawa S (2009) Temporal motion recognition and segmentation approach. Int J Imag Syst Technol 19:91–99CrossRef Ahad MAR, Tan JK, Kim H, Ishikawa S (2009) Temporal motion recognition and segmentation approach. Int J Imag Syst Technol 19:91–99CrossRef
43.
Zurück zum Zitat Bergen JR, Burt PJ, Hingorani R, Peleg S (1992) A three frame algorithm for estimating two-component image motion. IEEE Trans PAMI 14(9):886–896CrossRef Bergen JR, Burt PJ, Hingorani R, Peleg S (1992) A three frame algorithm for estimating two-component image motion. IEEE Trans PAMI 14(9):886–896CrossRef
44.
Zurück zum Zitat Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8:179–187MATH Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8:179–187MATH
45.
Zurück zum Zitat Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans PAMI 28(2):316–322CrossRef Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans PAMI 28(2):316–322CrossRef
46.
Zurück zum Zitat Chandrashekhar V, Venkatesh KS (2006) Action energy images for reliable human action recognition. In: Proc. of Asian symposium on information display (ASID), pp 484–487 Chandrashekhar V, Venkatesh KS (2006) Action energy images for reliable human action recognition. In: Proc. of Asian symposium on information display (ASID), pp 484–487
47.
Zurück zum Zitat Liu J, Zhang N (2007) Gait history image: a novel temporal template for gait recognition. In: Proc. IEEE international conference on multimedia and expo, pp 663–666 Liu J, Zhang N (2007) Gait history image: a novel temporal template for gait recognition. In: Proc. IEEE international conference on multimedia and expo, pp 663–666
48.
Zurück zum Zitat Yu C-C, Cheng H-Y, Cheng C-H, Fan K-C (2010) Efficient human action and gait analysis using multi resolution motion energy histogram. In: EURASIP journal on advances in signal processing, pp 1–13 Yu C-C, Cheng H-Y, Cheng C-H, Fan K-C (2010) Efficient human action and gait analysis using multi resolution motion energy histogram. In: EURASIP journal on advances in signal processing, pp 1–13
49.
Zurück zum Zitat Huang C, Hsieh C, Lai K, Huang W-Y (2011) Human action recognition using histogram of oriented gradient of motion history image. In: International conference on instrumentation, measurement, computer, communication and control, pp 353–356 Huang C, Hsieh C, Lai K, Huang W-Y (2011) Human action recognition using histogram of oriented gradient of motion history image. In: International conference on instrumentation, measurement, computer, communication and control, pp 353–356
50.
Zurück zum Zitat Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: International conference on computer vision and pattern recognition, pp 886–893 Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: International conference on computer vision and pattern recognition, pp 886–893
51.
Zurück zum Zitat Paisitkriangkrai S, Shen C, Zhang J (2008) Fast pedestrian detection using a cascade of boosted covariance features. IEEE Trans Circuits Syst Video Technol 18(8):1140–1151 Paisitkriangkrai S, Shen C, Zhang J (2008) Fast pedestrian detection using a cascade of boosted covariance features. IEEE Trans Circuits Syst Video Technol 18(8):1140–1151
52.
Zurück zum Zitat Bertozzi M, Broggi A, Del Rose M, Felisa M, Rakotomamonjy A, Suard F (2007) A pedestrian detector using histograms of oriented gradients and a support vector machine classifier. IEEE Intell Transp Syst Conf 2007:143–144 Bertozzi M, Broggi A, Del Rose M, Felisa M, Rakotomamonjy A, Suard F (2007) A pedestrian detector using histograms of oriented gradients and a support vector machine classifier. IEEE Intell Transp Syst Conf 2007:143–144
53.
Zurück zum Zitat Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: European conference on computer vision, pp 428–441 Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: European conference on computer vision, pp 428–441
54.
Zurück zum Zitat Mohan A, Papageorgiou C, Poggio T (2001) Example-based object detection in images by components. PAMI 23(4):349–361CrossRef Mohan A, Papageorgiou C, Poggio T (2001) Example-based object detection in images by components. PAMI 23(4):349–361CrossRef
55.
Zurück zum Zitat Cortes C, Vapnik V (1995) Support-vector netwoks. Mach Learn 20(3):273–297MATH Cortes C, Vapnik V (1995) Support-vector netwoks. Mach Learn 20(3):273–297MATH
56.
Zurück zum Zitat Vapnik Vladimir N (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999CrossRef Vapnik Vladimir N (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999CrossRef
57.
Zurück zum Zitat Sidenbladh H (2004) Detecting human motion with support vector machines. In: 17th IAPR, international conference on pattern recognition, vol 2, pp 188–191 Sidenbladh H (2004) Detecting human motion with support vector machines. In: 17th IAPR, international conference on pattern recognition, vol 2, pp 188–191
58.
Zurück zum Zitat Schüldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proc. of the 17th international conference on pattern recognition, vol 3, pp 32–36 Schüldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proc. of the 17th international conference on pattern recognition, vol 3, pp 32–36
59.
Zurück zum Zitat Wolf L, Shashua A (2003) Kernel principal angles for classification machines with applications to image sequence interpretation. In: Proc. of the computer vision and pattern recognition, pp I: 635–640 Wolf L, Shashua A (2003) Kernel principal angles for classification machines with applications to image sequence interpretation. In: Proc. of the computer vision and pattern recognition, pp I: 635–640
60.
Zurück zum Zitat Zelnik-Manor L, Irani M (2001) Event-based analysis of video. In: Proc. of the computer vision and pattern recognition, pp II: 123–130 Zelnik-Manor L, Irani M (2001) Event-based analysis of video. In: Proc. of the computer vision and pattern recognition, pp II: 123–130
62.
Zurück zum Zitat Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27CrossRefMATH Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27CrossRefMATH
Metadaten
Titel
Pedestrian activity classification using patterns of motion and histogram of oriented gradient
verfasst von
Rifat Muhammad Mueid
Chandrama Ahmed
Md. Atiqur Rahman Ahad
Publikationsdatum
01.12.2016
Verlag
Springer International Publishing
Erschienen in
Journal on Multimodal User Interfaces / Ausgabe 4/2016
Print ISSN: 1783-7677
Elektronische ISSN: 1783-8738
DOI
https://doi.org/10.1007/s12193-015-0178-3

Weitere Artikel der Ausgabe 4/2016

Journal on Multimodal User Interfaces 4/2016 Zur Ausgabe

Premium Partner