Skip to main content

Advertisement

SpringerLink
Log in

Stereo Video Surveillance Multi-agent System: New Solutions for Human Motion Analysis

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

This article presents a distributed agent-based system that can process the visual information obtained by stereoscopic cameras. The system is embedded within a global project whose objective is to develop an intelligent environment for location and identification within dependent environments that merges with other types of technologies. In this kind of environments, vision algorithms are very costly and require a lot of time to produce a response, which is highly inconvenient since many applications can require action to be taken in real time. A multi-agent system (MAS) can automate the process of analyzing images obtained by cameras, and optimize the procedure. This study presents a MAS that can process stereoscopic images to detect and classify people by combining a series of novel techniques.

The article shows in detail the combination of techniques used to perform the detection process. The process can be subdivided into human detection, human tracking, and human behavior understanding. With the addition of a case-based reasoning (CBR) model, the system can also incorporate reasoning capabilities. The system was tested under different conditions and environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 7, 39–59 (1994)

    Google Scholar 

  2. Aggarwal, J.K., Cai, Q.: Human motion analysis: a review. Comput. Vis. Image Underst. 73(3), 428–440 (1999)

    Article  Google Scholar 

  3. Aguilar, J.J., Torres, F., Lope, M.A.: Stereo vision for 3D measurement: accuracy analysis, calibration and industrial applications. Measurement 18(4), 193–200 (1996)

    Article  Google Scholar 

  4. Aha, D., Kibler, D.: WInstance-based learning algorithms. Mach. Learn. 6, 37–66 (1991)

    Google Scholar 

  5. Alahi, A., Vandergheynst, P., Bierlaire, M., Kunt, M.: Cascade of descriptors to detect and track objects across any network of cameras. Comput. Vis. Image Underst. 114(6), 624–640 (2010)

    Article  Google Scholar 

  6. Ardissono, L., Petrone, G., Segnan, M.: A conversational approach to the interaction with Web Services. Comput. Intell. 20, 693–709 (2004)

    Article  MathSciNet  Google Scholar 

  7. Bahadori, S., Cesta, A., Grisetti, G., Iocchi, L., Leonel, R., Nardi, D., Oddi, A., Pecora, F., Rasconi, R.: RoboCare: pervasive intelligence for the domestic care of the elderly. Intell. Artif. 1(1), 16–21 (2004)

    Google Scholar 

  8. Bahadori, S., Iocchi, L., Leone, G.R., Nardi, D., Scozzafava, L.: Real-time people localization and tracking through fixed stereo vision. In: Lecture Notes on Artificial Intelligence (LNAI), vol. 3533, pp. 44–54 (2005)

    Google Scholar 

  9. Bajo, J., de Paz, J.F., de Paz, Y., Corchado, J.M.: Integrating case-based planning and RPTW neural networks to construct an intelligent environment for health care. Expert Syst. Appl. 36(3), Part 2, 5844–5858 (2009)

    Article  Google Scholar 

  10. Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 42–77 (1994)

    Article  Google Scholar 

  11. Breiman, L., Fried, J.H., Olshen, R.A., Stone, C.J.: Classification and regression trees. Wadsworth International Group (1984)

  12. Brenner, W., Wittig, H., Zarnekow, R.: Intelligent Software Agents: Foundations and Applications. Springer, New York (1998)

    Book  MATH  Google Scholar 

  13. Camarinha-Matos, L., Afsarmanesh, H.: Design of a virtual community infrastructure for elderly care. In: Proc. of 3rd IFIP Working Conference on Infrastructures for Virtual Enterprises: Collaborative Business Ecosystems and Virtual Enterprises, p. 635 (2002)

    Google Scholar 

  14. Camarinha-Matos, L.M., Afsarmanesh, H.: A comprehensive modeling framework for collaborative networked organizations. J. Intell. Manuf. 18(5), 529–542 (2007)

    Article  Google Scholar 

  15. Castanedo, F., García, J., Patricio, M.A., Molina, J.M.: Designing a visual sensor network using a multi-agent architecture. In: Advances in Soft Computing, vol. 55, pp. 430–439 (2009). doi:10.1007/978-3-642-00487-2_46

    Google Scholar 

  16. CIA: The World FactBook. ISSN 1553-8133 (2010)

  17. Cohen, W.W.: Fast effective rule induction. In: Proceedings of the 12th International Conference on Machine Learning. Morgan Kaufmann, San Mateo (1995), pp. 115–123

    Google Scholar 

  18. Collins, R.T., Lipton, A.J., Kanade, T., Fujiyoshi, H., Duggins, D., Tsin, Y., Tolliver, D., Enomoto, N., Hasegawa, O., Burt, P., Wixson, L.: A system for video surveillance and monitoring. Tech. Rep., CMU-RI-TR-00-12. Carnegie Mellon Univ., Pittsburgh, PA (2000)

  19. Comaniciu, D., Ramesh, V., Andmeer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25, 564–575 (2003)

    Article  Google Scholar 

  20. Corchado, J.M., Laza, R.: Constructing deliberative agents with case-based reasoning technology. Int. J. Intell. Syst. 18(12), 1227–1241 (2003)

    Article  Google Scholar 

  21. Corchado, J.M., Bajo, J., De Paz, Y., Tapia, D.I.: Intelligent Environment for Monitoring Alzheimer Patients, Agent Technology for Health Care. Decision Support Systems. Elsevier, Amsterdam (2006)

    Google Scholar 

  22. Corchado, J.M., Glez-Bedia, M., de Paz, Y., Bajo, J., de Paz, J.F.: Concept, formulation and mechanism for agent replanification: MRP architecture. In: Computational Intelligence. Blackwell Publishers, Malden (2008)

    Google Scholar 

  23. Corchado, J.M., Glez-Bedia, M., de Paz, Y., Bajo, J., de Paz, J.F.: Replanning mechanism for deliberative agents in dynamic changing environments. Comput. Intell. 24(2), 77–107 (2008)

    Article  Google Scholar 

  24. Cucchiara, R., Grana, C., Piccardi, M.: Prati, A. Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1337–1342 (2003)

    Article  Google Scholar 

  25. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference Computer Vision and Pattern Recognition, USA, pp. 886–893 (2005)

    Google Scholar 

  26. Davies, E.R.: Lateral histograms for efficient object location: speed versus ambiguity. Pattern Recognit. Lett. 6(3), 189–198 (1987)

    Article  Google Scholar 

  27. Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 1895–1923 (1998)

  28. Duda, R.O., Hart, P.: Pattern Classification and Scene Analysis. Wiley, New York (1973)

    MATH  Google Scholar 

  29. Eddy, S.R.: Hidden Markov models. Curr. Opin. Struct. Biol. 6(3), 361–365 (1996)

    Article  Google Scholar 

  30. Elgammal, H.M., Harwood, D., Davis, L.S.: Non-parametric model for background subtraction. In: Proc. of the 6th European Conference on Computer (ECCV). Springer, London (2000), pp. 751–767

    Google Scholar 

  31. Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Thirteenth International Conference on Machine Learning (1996), pp. 148–156

    Google Scholar 

  32. Gallagher, M., Downs, T.: Visualization of learning in multilayer perceptron networks using principal component analysis. IEEE Trans. Syst. Man Cybern., Part B, Cybern. 33, 28–34 (2003)

    Article  Google Scholar 

  33. Garvey, A.J., Lesser, V.R.: Design-to-time real-time scheduling. IEEE Trans. Syst. Man Cybern.. 23, 1491–1502 (1993)

    Article  Google Scholar 

  34. Gu, Z., Su, X., Liu, Y., Zhang, Q.: Local stereo matching with adaptive support-weight, rank transform and disparity calibration. Pattern Recognit. Lett. 29(9), 1230–1235 (2008)

    Article  Google Scholar 

  35. Haritaoglu, I., Harwood, D., Davis, L.S.: W4: real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 809–830 (2000)

    Article  Google Scholar 

  36. Ivanciuc, O.: Applications of support vector machines in chemistry. In: Lipkowitz, K.B., Cundari, T.R. (eds.) Reviews in Computational Chemistry, vol. 23, pp. 291–400. Wiley, New York (2007)

    Chapter  Google Scholar 

  37. Jabri, S., Duric, Z., Wechsler, H., Rosenfeld, A.: Detection and location of people in video images using adaptive fusion of color and edge information. In: Proc. of 15th International Conference on Pattern Recognition (ICPR’00), vol. 4, p. 4627 (2000)

    Google Scholar 

  38. Jeong, Y.-S., Jeong, M.K., Omitaomu, O.A.: Weighted dynamic time warping for time series classification. Pattern Recogn. (2011). doi:10.1016/j.patcog.2010.09.022

    Google Scholar 

  39. Kim, K., Chalidabhongse, T.H., Harwood, D. Davis, L.: Real-time foreground-background segmentation using codebook model. Real-Time Imaging 11(3), 172–185 (2005)

    Article  Google Scholar 

  40. Kohlrausch, J., Rohr, K., Stiehl, H.S.: A new class of elastic body splines for nonrigid registration of medical images. J. Math. Imaging Vis. 23(3), 253–280 (2005). doi:10.1007/s10851-005-0483-7

    Article  MathSciNet  Google Scholar 

  41. Lima, C.A.M., Coelho, A.L.V., Von Zuben, F.J.: Hybridizing mixtures of experts with support vector machines: investigation into nonlinear dynamic systems identification. Inf. Sci. 177, 2049–2074 (2007)

    Article  Google Scholar 

  42. Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30, 77–116 (1998)

    Google Scholar 

  43. Moeslund, T.B., Granum, E.: A survey of computer vision based human motion capture. Comput. Vis. Image Underst. 81(3), 231–268 (2001)

    Article  MATH  Google Scholar 

  44. Moeslund, T.B., Hilton, A., Kruger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2), 90–126 (2007)

    Article  Google Scholar 

  45. Mughadam, B., Pentland, A.: Probabilistic visual learning for object representation. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 696–710 (1997)

    Article  Google Scholar 

  46. Murray, D., Little, J.J.: Using real-time stereo vision for mobile robot navigation. Auton. Robots 8(2), 161–171 (2000). doi:10.1023/A:1008987612352

    Article  Google Scholar 

  47. Nguyena, M.H., Abbassa, H.A., Mckay, R.I.: A novel mixture of experts model based on cooperative coevolution. Neurocomputing 70, 155–163 (2006)

    Article  Google Scholar 

  48. Paragios, N., Deriche, R.: Geodesic active regions and level set methods for supervised texture segmentation. Int. J. Comput. Vis. 46(3), 223–247 (2002)

    Article  MATH  Google Scholar 

  49. Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods, pp. 185–208 (1999)

    Google Scholar 

  50. Platt, J.C.: Fast training of support vector machines using sequential minimal optimization, pp. 185–208 (1999)

  51. Point Grey Research Inc.: http://www.ptgrey.com/ (2009)

  52. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  53. Rangarajan, A., Hsiao, I.-T., Gindi, G.: A Bayesian joint mixture framework for the integration of anatomical information in functional image reconstruction. J. Math. Imaging Vis. 12(3), 199–217 (2000). doi:10.1023/A:1008314015446

    Article  MATH  Google Scholar 

  54. Rao, A.S., Georgeff, M.P.: BDI agents: from theory to practice. In: Proceedings of ICMAS’95. San Francisco, CA, USA (1995)

    Google Scholar 

  55. Rodríguez, S., De Paz, J.F., Bajo, J., Tapia, D.I., Pérez, B.: Stereo-MAS: multi-agent system for image stereo processing. In: Cabestany, J., et al. (eds.) IWANN’09. LNAI, vol. 5517, pp. 1256–1263. Springer, Berlin (2009). ISBN: 978-3-540-87655-7

    Google Scholar 

  56. Sato, K., Aggarwal, J.: Temporal spatio-velocity transform and its application to tracking and interaction. Comput. Vis. Image Underst. 96(2), 100–128 (2004)

    Article  Google Scholar 

  57. Serra, J.: Image Analysis and Mathematical Morphology. Academic Press, London (1982)

    MATH  Google Scholar 

  58. Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757 (2000)

    Article  Google Scholar 

  59. Subasi, A.: EEG signal classification using wavelet feature extraction and a mixture of expert model. Expert Syst. Appl. 32, 1084–1093 (2007)

    Article  Google Scholar 

  60. Vapnik, V., Lerner, A.: Pattern recognition using generalized portrait method, pp. 774–780 (1963)

  61. Veenman, C., Reinders, M., Backer, E.: Resolving motion correspondence for densely moving points. IEEE Trans. Pattern Anal. Mach. Intell. 23(1), 54–72 (2001)

    Article  Google Scholar 

  62. Wang, L., Hu, W., Tan, T.: Recent development in human motion analysis. Pattern Recognit. 36(3), 585–601 (2003)

    Article  Google Scholar 

  63. Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)

    Article  Google Scholar 

  64. Yilmaz, A., Li, X., Shah, M.: Contour based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1531–1536 (2004)

    Article  Google Scholar 

  65. Yilmaz, A., et al.: Object tracking: a survey. ACM J. Comput. Surv. 38(4) (2006)

  66. Zhanga, H., Lu, J.: Creating ensembles of classifiers via fuzzy clustering and deflection. 161(13), 1790–1802 (2010)

  67. Zitnick, C.L., Kang, S.B.: Stereo for image-based rendering using image over-segmentation. Int. J. Comput. Vis. 75(1), 49–65 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Digital and Information Technologies, University of Vic, C. de la Laura 13, 08500, Vic (Barcelona), Spain

    Pere Marti-Puig & Ramon Reig-Bolaño

  2. Department of Informatics, University of Salamanca, Plaza de la Merced s/n, 37008, Salamanca, Spain

    Sara Rodríguez & Juan F. De Paz

  3. Engineering Graphics Department, University of Salamanca, Avd. Requejo no 33, 49001, Zamora, Spain

    Manuel P. Rubio

  4. Department of Informatics, Pontifical University of Salamanca, Compañía, 5, 37002, Salamanca, Spain

    Javier Bajo

Authors
  1. Pere Marti-Puig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sara Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan F. De Paz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ramon Reig-Bolaño
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Manuel P. Rubio
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Javier Bajo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sara Rodríguez.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marti-Puig, P., Rodríguez, S., De Paz, J.F. et al. Stereo Video Surveillance Multi-agent System: New Solutions for Human Motion Analysis. J Math Imaging Vis 42, 176–195 (2012). https://doi.org/10.1007/s10851-011-0290-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-011-0290-2

Keywords

Search

Navigation