Skip to main content

2017 | OriginalPaper | Buchkapitel

People Detection and Tracking from an RGB-D Camera in Top-View Configuration: Review of Challenges and Applications

verfasst von : Daniele Liciotti, Marina Paolanti, Emanuele Frontoni, Primo Zingaretti

Erschienen in: New Trends in Image Analysis and Processing – ICIAP 2017

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper presents a literature review on the use of RGB-D camera for people detection and tracking. Our aim is to use this state-of-the-art report to demonstrate the potential of top-view configuration for people detection and tracking applications in several sub-domains, to outline key limitations and to indicate areas of technology, where solutions for remaining challenges may be found. The survey examines the success of RGB-D cameras because of their affordability and for the additional rough depth information coupled with visual images that provide. These cameras in configuration top-view have already been successfully applied in the several fields to univocally identify people and to analyse behaviours and interactions. From this report, it emerges that detecting and tracking people can be a valuable source of information for many fields and purposes.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Agusta, B.A.Y., Mittrapiyanuruk, P., Kaewtrakulpong, P.: Field seeding algorithm for people counting using kinect depth image. Indian J. Sci. Technol. 9(48) (2016) Agusta, B.A.Y., Mittrapiyanuruk, P., Kaewtrakulpong, P.: Field seeding algorithm for people counting using kinect depth image. Indian J. Sci. Technol. 9(48) (2016)
2.
Zurück zum Zitat Bednarık, J., Herman, D.: Human gesture recognition using top view depth data obtained from kinect sensor (2015) Bednarık, J., Herman, D.: Human gesture recognition using top view depth data obtained from kinect sensor (2015)
3.
Zurück zum Zitat Bevilacqua, A., Di Stefano, L., Azzari, P.: People tracking using a time-of-flight depth sensor. In: IEEE International Conference on Video and Signal Based Surveillance, AVSS 2006, pp. 89–89. IEEE (2006) Bevilacqua, A., Di Stefano, L., Azzari, P.: People tracking using a time-of-flight depth sensor. In: IEEE International Conference on Video and Signal Based Surveillance, AVSS 2006, pp. 89–89. IEEE (2006)
4.
Zurück zum Zitat Bonnin, A., Borràs, R., Vitrià, J.: A cluster-based strategy for active learning of rgb-d object detectors. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1215–1220. IEEE (2011) Bonnin, A., Borràs, R., Vitrià, J.: A cluster-based strategy for active learning of rgb-d object detectors. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1215–1220. IEEE (2011)
5.
Zurück zum Zitat Burbano, A., Bouaziz, S., Vasiliu, M.: 3D-sensing distributed embedded system for people tracking and counting. In: 2015 International Conference on Computational Science and Computational Intelligence (CSCI), pp. 470–475. IEEE (2015) Burbano, A., Bouaziz, S., Vasiliu, M.: 3D-sensing distributed embedded system for people tracking and counting. In: 2015 International Conference on Computational Science and Computational Intelligence (CSCI), pp. 470–475. IEEE (2015)
6.
Zurück zum Zitat Coşkun, A., Kara, A., Parlaktuna, M., Ozkan, M., Parlaktuna, O.: People counting system by using kinect sensor. In: 2015 International Symposium on Innovations in Intelligent SysTems and Applications (INISTA), pp. 1–7. IEEE (2015) Coşkun, A., Kara, A., Parlaktuna, M., Ozkan, M., Parlaktuna, O.: People counting system by using kinect sensor. In: 2015 International Symposium on Innovations in Intelligent SysTems and Applications (INISTA), pp. 1–7. IEEE (2015)
7.
Zurück zum Zitat Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005) Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)
8.
Zurück zum Zitat Del Pizzo, L., Foggia, P., Greco, A., Percannella, G., Vento, M.: Counting people by RGB or depth overhead cameras. Pattern Recogn. Lett. 81, 41–50 (2016)CrossRef Del Pizzo, L., Foggia, P., Greco, A., Percannella, G., Vento, M.: Counting people by RGB or depth overhead cameras. Pattern Recogn. Lett. 81, 41–50 (2016)CrossRef
9.
Zurück zum Zitat Dittrich, F., Woern, H., Sharma, V., Yayilgan, S.: Pixelwise object class segmentation based on synthetic data using an optimized training strategy. In: 2014 First International Conference on Networks & Soft Computing (ICNSC), pp. 388–394. IEEE (2014) Dittrich, F., Woern, H., Sharma, V., Yayilgan, S.: Pixelwise object class segmentation based on synthetic data using an optimized training strategy. In: 2014 First International Conference on Networks & Soft Computing (ICNSC), pp. 388–394. IEEE (2014)
10.
Zurück zum Zitat Felzenszwalb, P.F.: Learning models for object recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I–1056. IEEE (2001) Felzenszwalb, P.F.: Learning models for object recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I–1056. IEEE (2001)
11.
Zurück zum Zitat Fu, H., Ma, H., Xiao, H.: Scene-adaptive accurate and fast vertical crowd counting via joint using depth and color information. Multimedia Tools Appl. 73(1), 273 (2014)CrossRef Fu, H., Ma, H., Xiao, H.: Scene-adaptive accurate and fast vertical crowd counting via joint using depth and color information. Multimedia Tools Appl. 73(1), 273 (2014)CrossRef
12.
Zurück zum Zitat Gasparrini, S., Cippitelli, E., Spinsante, S., Gambi, E.: A depth-based fall detection system using a kinect® sensor. Sensors 14(2), 2756–2775 (2014)CrossRef Gasparrini, S., Cippitelli, E., Spinsante, S., Gambi, E.: A depth-based fall detection system using a kinect® sensor. Sensors 14(2), 2756–2775 (2014)CrossRef
13.
Zurück zum Zitat Heath, K., Guibas, L.: Multi-person tracking from sparse 3D trajectories in a camera sensor network. In: Second ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC 2008, pp. 1–9. IEEE (2008) Heath, K., Guibas, L.: Multi-person tracking from sparse 3D trajectories in a camera sensor network. In: Second ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC 2008, pp. 1–9. IEEE (2008)
14.
Zurück zum Zitat Hernandez, D., Castrillon, M., Lorenzo, J.: People counting with re-identification using depth cameras (2011) Hernandez, D., Castrillon, M., Lorenzo, J.: People counting with re-identification using depth cameras (2011)
15.
Zurück zum Zitat Kepski, M., Kwolek, B.: Detecting human falls with 3-axis accelerometer and depth sensor. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 770–773. IEEE (2014) Kepski, M., Kwolek, B.: Detecting human falls with 3-axis accelerometer and depth sensor. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 770–773. IEEE (2014)
16.
Zurück zum Zitat Kepski, M., Kwolek, B.: Fall detection using ceiling-mounted 3D depth camera. In: 2014 International Conference on Computer Vision Theory and Applications (VISAPP), vol. 2, pp. 640–647. IEEE (2014) Kepski, M., Kwolek, B.: Fall detection using ceiling-mounted 3D depth camera. In: 2014 International Conference on Computer Vision Theory and Applications (VISAPP), vol. 2, pp. 640–647. IEEE (2014)
17.
Zurück zum Zitat Kouno, D., Shimada, K., Endo, T.: Person identification using top-view image with depth information. In: 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel & Distributed Computing (SNPD), pp. 140–145. IEEE (2012) Kouno, D., Shimada, K., Endo, T.: Person identification using top-view image with depth information. In: 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel & Distributed Computing (SNPD), pp. 140–145. IEEE (2012)
18.
Zurück zum Zitat Liciotti, D., Contigiani, M., Frontoni, E., Mancini, A., Zingaretti, P., Placidi, V.: Shopper analytics: a customer activity recognition system using a distributed RGB-D camera network. In: Distante, C., Battiato, S., Cavallaro, A. (eds.) VAAM 2014. LNCS, vol. 8811, pp. 146–157. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-12811-5_11 Liciotti, D., Contigiani, M., Frontoni, E., Mancini, A., Zingaretti, P., Placidi, V.: Shopper analytics: a customer activity recognition system using a distributed RGB-D camera network. In: Distante, C., Battiato, S., Cavallaro, A. (eds.) VAAM 2014. LNCS, vol. 8811, pp. 146–157. Springer, Cham (2014). https://​doi.​org/​10.​1007/​978-3-319-12811-5_​11
19.
Zurück zum Zitat Liciotti, D., Ferroni, G., Frontoni, E., Squartini, S., Principi, E., Bonfigli, R., Zingaretti, P., Piazza, F.: Advanced integration of multimedia assistive technologies: a prospective outlook. In: 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2014) Liciotti, D., Ferroni, G., Frontoni, E., Squartini, S., Principi, E., Bonfigli, R., Zingaretti, P., Piazza, F.: Advanced integration of multimedia assistive technologies: a prospective outlook. In: 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2014)
20.
Zurück zum Zitat Liciotti, D., Frontoni, E., Mancini, A., Zingaretti, P.: Pervasive system for consumer behaviour analysis in retail environments. In: Nasrollahi, K., Distante, C., Hua, G., Cavallaro, A., Moeslund, T.B., Battiato, S., Ji, Q. (eds.) FFER/VAAM -2016. LNCS, vol. 10165, pp. 12–23. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56687-0_2 CrossRef Liciotti, D., Frontoni, E., Mancini, A., Zingaretti, P.: Pervasive system for consumer behaviour analysis in retail environments. In: Nasrollahi, K., Distante, C., Hua, G., Cavallaro, A., Moeslund, T.B., Battiato, S., Ji, Q. (eds.) FFER/VAAM -2016. LNCS, vol. 10165, pp. 12–23. Springer, Cham (2017). https://​doi.​org/​10.​1007/​978-3-319-56687-0_​2 CrossRef
21.
Zurück zum Zitat Liciotti, D., Frontoni, E., Zingaretti, P., Bellotto, N., Duckett, T.: Hmm-based activity recognition with a ceiling RGB-D camera. In: Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods, pp. 567–574 (2017) Liciotti, D., Frontoni, E., Zingaretti, P., Bellotto, N., Duckett, T.: Hmm-based activity recognition with a ceiling RGB-D camera. In: Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods, pp. 567–574 (2017)
22.
Zurück zum Zitat Liciotti, D., Massi, G., Frontoni, E., Mancini, A., Zingaretti, P.: Human activity analysis for in-home fall risk assessment. In: 2015 IEEE International Conference on Communication Workshop (ICCW), pp. 284–289. IEEE (2015) Liciotti, D., Massi, G., Frontoni, E., Mancini, A., Zingaretti, P.: Human activity analysis for in-home fall risk assessment. In: 2015 IEEE International Conference on Communication Workshop (ICCW), pp. 284–289. IEEE (2015)
23.
Zurück zum Zitat Liciotti, D., Paolanti, M., Frontoni, E., Mancini, A., Zingaretti, P.: Person re-identification dataset with RGB-D camera in a top-view configuration. In: Nasrollahi, K., Distante, C., Hua, G., Cavallaro, A., Moeslund, T.B., Battiato, S., Ji, Q. (eds.) FFER/VAAM -2016. LNCS, vol. 10165, pp. 1–11. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56687-0_1 CrossRef Liciotti, D., Paolanti, M., Frontoni, E., Mancini, A., Zingaretti, P.: Person re-identification dataset with RGB-D camera in a top-view configuration. In: Nasrollahi, K., Distante, C., Hua, G., Cavallaro, A., Moeslund, T.B., Battiato, S., Ji, Q. (eds.) FFER/VAAM -2016. LNCS, vol. 10165, pp. 1–11. Springer, Cham (2017). https://​doi.​org/​10.​1007/​978-3-319-56687-0_​1 CrossRef
24.
Zurück zum Zitat Liciotti, D., Zingaretti, P., Placidi, V.: An automatic analysis of shoppers behaviour using a distributed RGB-D cameras system. In: 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2014) Liciotti, D., Zingaretti, P., Placidi, V.: An automatic analysis of shoppers behaviour using a distributed RGB-D cameras system. In: 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2014)
25.
Zurück zum Zitat Lin, S.-C., Liu, A.-S., Hsu, T.-W., Fu, L.-C.: Representative body points on top-view depth sequences for daily activity recognition. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2968–2973. IEEE (2015) Lin, S.-C., Liu, A.-S., Hsu, T.-W., Fu, L.-C.: Representative body points on top-view depth sequences for daily activity recognition. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2968–2973. IEEE (2015)
26.
Zurück zum Zitat Liu, J., Liu, Y., Zhang, G., Zhu, P., Chen, Y.Q.: Detecting and tracking people in real time with RGB-D camera. Pattern Recogn. Lett. 53, 16–23 (2015)CrossRef Liu, J., Liu, Y., Zhang, G., Zhu, P., Chen, Y.Q.: Detecting and tracking people in real time with RGB-D camera. Pattern Recogn. Lett. 53, 16–23 (2015)CrossRef
28.
Zurück zum Zitat Malawski, F.: Top-view people counting in public transportation using kinect. Challenges Mod. Technol. 5 (2014) Malawski, F.: Top-view people counting in public transportation using kinect. Challenges Mod. Technol. 5 (2014)
29.
Zurück zum Zitat Marquardt, N., Hinckley, K., Greenberg, S.: Cross-device interaction via micro-mobility and f-formations. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, pp. 13–22. ACM (2012) Marquardt, N., Hinckley, K., Greenberg, S.: Cross-device interaction via micro-mobility and f-formations. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, pp. 13–22. ACM (2012)
30.
Zurück zum Zitat Migniot, C., Ababsa, F.: Hybrid 3D–2D human tracking in a top view. J. Real-Time Image Proc. 11(4), 769–784 (2016)CrossRef Migniot, C., Ababsa, F.: Hybrid 3D–2D human tracking in a top view. J. Real-Time Image Proc. 11(4), 769–784 (2016)CrossRef
31.
Zurück zum Zitat Migniot, C., Ababsa, F.: 3D Human Tracking in a Top View Using Depth Information Recorded by the Xtion Pro-Live Camera. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Li, B., Porikli, F., Zordan, V., Klosowski, J., Coquillart, S., Luo, X., Chen, M., Gotz, D. (eds.) ISVC 2013. LNCS, vol. 8034, pp. 603–612. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41939-3_59 CrossRef Migniot, C., Ababsa, F.: 3D Human Tracking in a Top View Using Depth Information Recorded by the Xtion Pro-Live Camera. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Li, B., Porikli, F., Zordan, V., Klosowski, J., Coquillart, S., Luo, X., Chen, M., Gotz, D. (eds.) ISVC 2013. LNCS, vol. 8034, pp. 603–612. Springer, Heidelberg (2013). https://​doi.​org/​10.​1007/​978-3-642-41939-3_​59 CrossRef
32.
Zurück zum Zitat Rauter, M.: Reliable human detection and tracking in top-view depth images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 529–534 (2013) Rauter, M.: Reliable human detection and tracking in top-view depth images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 529–534 (2013)
33.
Zurück zum Zitat Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. arXiv preprint arXiv:1505.04597 (2015) Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. arXiv preprint arXiv:​1505.​04597 (2015)
34.
Zurück zum Zitat Siegmund, D., Wainakh, A., Braun, A.: Verification of single-person access in a mantrap portal using RGB-D images. In: XII Workshop de Visao Computacional (WVC) (2016) Siegmund, D., Wainakh, A., Braun, A.: Verification of single-person access in a mantrap portal using RGB-D images. In: XII Workshop de Visao Computacional (WVC) (2016)
35.
Zurück zum Zitat Tian, Q., Zhou, B., Zhao, W.-H., Wei, Y., Fei, W.-W.: Human detection using hog features of head and shoulder based on depth map. JSW 8(9), 2223–2230 (2013)CrossRef Tian, Q., Zhou, B., Zhao, W.-H., Wei, Y., Fei, W.-W.: Human detection using hog features of head and shoulder based on depth map. JSW 8(9), 2223–2230 (2013)CrossRef
36.
Zurück zum Zitat Tseng, T.-E., Liu, A.-S., Hsiao, P.-H., Huang, C.-M., Fu, L.-C.: Real-time people detection and tracking for indoor surveillance using multiple top-view depth cameras. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 4077–4082. IEEE (2014) Tseng, T.-E., Liu, A.-S., Hsiao, P.-H., Huang, C.-M., Fu, L.-C.: Real-time people detection and tracking for indoor surveillance using multiple top-view depth cameras. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 4077–4082. IEEE (2014)
37.
Zurück zum Zitat Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int. J. Comput. Vision 75(2), 247–266 (2007)CrossRef Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int. J. Comput. Vision 75(2), 247–266 (2007)CrossRef
38.
Zurück zum Zitat Yahiaoui, T., Meurie, C., Khoudour, L., Cabestaing, F.: A people counting system based on dense and close stereovision. In: Image and Signal Processing, pp. 59–66 (2008) Yahiaoui, T., Meurie, C., Khoudour, L., Cabestaing, F.: A people counting system based on dense and close stereovision. In: Image and Signal Processing, pp. 59–66 (2008)
39.
Zurück zum Zitat Yamamoto, J., Inoue, K., Yoshioka, M.: Investigation of customer behavior analysis based on top-view depth camera. In: 2017 IEEE Winter Applications of Computer Vision Workshops (WACVW), pp. 67–74. IEEE (2017) Yamamoto, J., Inoue, K., Yoshioka, M.: Investigation of customer behavior analysis based on top-view depth camera. In: 2017 IEEE Winter Applications of Computer Vision Workshops (WACVW), pp. 67–74. IEEE (2017)
40.
Zurück zum Zitat Zhang, X., Yan, J., Feng, S., Lei, Z., Yi, D., Li, S.Z.: Water filling: unsupervised people counting via vertical kinect sensor. In: 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 215–220. IEEE (2012) Zhang, X., Yan, J., Feng, S., Lei, Z., Yi, D., Li, S.Z.: Water filling: unsupervised people counting via vertical kinect sensor. In: 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 215–220. IEEE (2012)
Metadaten
Titel
People Detection and Tracking from an RGB-D Camera in Top-View Configuration: Review of Challenges and Applications
verfasst von
Daniele Liciotti
Marina Paolanti
Emanuele Frontoni
Primo Zingaretti
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-70742-6_20

Premium Partner