Skip to main content
Erschienen in: Machine Vision and Applications 1/2014

01.01.2014 | Original Paper

Summarizing high-level scene behavior

verfasst von: Kevin Streib, James W. Davis

Erschienen in: Machine Vision and Applications | Ausgabe 1/2014

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We present several novel techniques to summarize the high-level behavior in surveillance video. Our proposed methods can employ either optical flow or trajectories as input, and incorporate spatial and temporal information together, which improve upon existing approaches for summarization. To begin, we extract common pathway regions by performing graph-based clustering on similarity matrices describing the relationships between location/orientation states. We then employ the activities along the pathway regions to extract the aggregate behavioral patterns throughout scenes. We show how our summarization methods can be applied to detect anomalies, retrieve video clips of interest, and generate adaptive-speed summary videos. We examine our approaches on multiple complex urban scenes and present experimental results.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Cheung, V., Frey, B.J., Jojic, N.: Video epitomes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2005) Cheung, V., Frey, B.J., Jojic, N.: Video epitomes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2005)
2.
Zurück zum Zitat Grauman, K., Darrell, T.: The pyramid match kernel: discriminative classification with sets of image features. In: Proceedings of IEEE International Conferenc on Computer Vision (2005) Grauman, K., Darrell, T.: The pyramid match kernel: discriminative classification with sets of image features. In: Proceedings of IEEE International Conferenc on Computer Vision (2005)
3.
Zurück zum Zitat Hanjalic, A., Zhang, H.: An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE TCSVT 9(8), 1280–1289 (1999) Hanjalic, A., Zhang, H.: An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE TCSVT 9(8), 1280–1289 (1999)
4.
Zurück zum Zitat Hferlin, B., Hferlin, M., Weiskopf, D., Heidemann, G.: (2010) Information-based adaptive fast-forward for visual surveillance. Multimedia Tools Appl. 55(1), 1–24 Hferlin, B., Hferlin, M., Weiskopf, D., Heidemann, G.: (2010) Information-based adaptive fast-forward for visual surveillance. Multimedia Tools Appl. 55(1), 1–24
5.
Zurück zum Zitat Hospedales, T., Gong, S., Xiang, T.: A markov clustering topic model for mining behaviour in video. In: Proceedings of the IEEE International Conference on Computer Vision (2009) Hospedales, T., Gong, S., Xiang, T.: A markov clustering topic model for mining behaviour in video. In: Proceedings of the IEEE International Conference on Computer Vision (2009)
6.
Zurück zum Zitat Jojic, N., Frey, B.J., Kannan, A.: Epitomic analysis of appearance and shape. In: Proceedings of IEEE International Conference on Computer Vision (2003) Jojic, N., Frey, B.J., Kannan, A.: Epitomic analysis of appearance and shape. In: Proceedings of IEEE International Conference on Computer Vision (2003)
7.
Zurück zum Zitat Kang, H.W., Chen, X.Q., Matsushita, Y., Tang, X.: Space-time video montage. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2006) Kang, H.W., Chen, X.Q., Matsushita, Y., Tang, X.: Space-time video montage. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2006)
8.
Zurück zum Zitat Kasamwattanarote, S., Cooharojananone, N., Satoh, S., Lipikorn, R.: Real time tunnel based video summarization using direct shift collision detection. In: Advances in Multimedia Information Processing—PCM 2010, vol 6297, pp 136–147 (2010) Kasamwattanarote, S., Cooharojananone, N., Satoh, S., Lipikorn, R.: Real time tunnel based video summarization using direct shift collision detection. In: Advances in Multimedia Information Processing—PCM 2010, vol 6297, pp 136–147 (2010)
9.
Zurück zum Zitat Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2006) Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2006)
10.
Zurück zum Zitat Leung, Y., Zhang, J.S., Xu, Z.B.: Clustering by scale-space filtering. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1396–1410 (2000)CrossRef Leung, Y., Zhang, J.S., Xu, Z.B.: Clustering by scale-space filtering. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1396–1410 (2000)CrossRef
11.
Zurück zum Zitat Li, J., Gong, S., Xiang, T.: Scene segmentation for behaviour correlation. In: Proceedings of the European Conference on Computer Vision (2008) Li, J., Gong, S., Xiang, T.: Scene segmentation for behaviour correlation. In: Proceedings of the European Conference on Computer Vision (2008)
12.
Zurück zum Zitat Li, Z., Ishwar, P., Konrad, J.: Video condensation by ribbon carving. IEEE Trans. Image Proc. 18, 2572–2583 (2009)CrossRefMathSciNet Li, Z., Ishwar, P., Konrad, J.: Video condensation by ribbon carving. IEEE Trans. Image Proc. 18, 2572–2583 (2009)CrossRefMathSciNet
14.
Zurück zum Zitat Makris, D., Ellis, T.: Path detection in video surveillance. Image Vis. Comput. 20, 895–903 (2002)CrossRef Makris, D., Ellis, T.: Path detection in video surveillance. Image Vis. Comput. 20, 895–903 (2002)CrossRef
15.
Zurück zum Zitat Petrovic, N., Jojic, N.: Adaptive video fast forward. Multimedia Tools Appl. 26(2), 327–344 (2005)CrossRef Petrovic, N., Jojic, N.: Adaptive video fast forward. Multimedia Tools Appl. 26(2), 327–344 (2005)CrossRef
16.
Zurück zum Zitat Pop, I., Scuturici, M., Miguet, S.: Common motion map based on codebooks. In: 5th International Symposium, ISVC 2009, pp. 1181–1190. Las Vegas, NV, USA (2009) Pop, I., Scuturici, M., Miguet, S.: Common motion map based on codebooks. In: 5th International Symposium, ISVC 2009, pp. 1181–1190. Las Vegas, NV, USA (2009)
17.
Zurück zum Zitat Pritch, Y., Rav-Acha, A.: Nonchronological video synopsis and indexing. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1971–1984 (2008)CrossRef Pritch, Y., Rav-Acha, A.: Nonchronological video synopsis and indexing. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1971–1984 (2008)CrossRef
18.
Zurück zum Zitat Pritch, Y., Ratovich, S., Hendel, A., Peleg, S.: Clustered synopsis of surveillance video. Advanced Video and Signal Based Surveillance (2009) Pritch, Y., Ratovich, S., Hendel, A., Peleg, S.: Clustered synopsis of surveillance video. Advanced Video and Signal Based Surveillance (2009)
19.
Zurück zum Zitat Rav-Acha, A., Pritch, Y., Peleg, S.: Making a long video short: Dynamic video synopsis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2006) Rav-Acha, A., Pritch, Y., Peleg, S.: Making a long video short: Dynamic video synopsis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2006)
20.
Zurück zum Zitat Ren, X., Malik, J.: Learning a classification model for segmentation. In: Proceedings of the IEEE International Conference on Computer Vision (2003) Ren, X., Malik, J.: Learning a classification model for segmentation. In: Proceedings of the IEEE International Conference on Computer Vision (2003)
21.
Zurück zum Zitat Rodriguez, M.: CRAM: compact representation of actions in movies. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2010) Rodriguez, M.: CRAM: compact representation of actions in movies. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2010)
22.
Zurück zum Zitat Saleemi, I., Shafique, K., Shah, M.: Probabilistic modeling of scene dynamics for applications in visual surveillance. IEEE Trans. Pattern Anal. Mach. Intell. 31(8), 1472–1484 (2009)CrossRef Saleemi, I., Shafique, K., Shah, M.: Probabilistic modeling of scene dynamics for applications in visual surveillance. IEEE Trans. Pattern Anal. Mach. Intell. 31(8), 1472–1484 (2009)CrossRef
23.
Zurück zum Zitat Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern. Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRef Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern. Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRef
24.
Zurück zum Zitat Shi, J., Tomasi, C.: Good features to track. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (1994) Shi, J., Tomasi, C.: Good features to track. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (1994)
25.
Zurück zum Zitat Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2008) Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2008)
26.
Zurück zum Zitat Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–767 (2000)CrossRef Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–767 (2000)CrossRef
27.
Zurück zum Zitat Streib, K., Davis, J.W.: Improving graph-based clustering via Ripley’s K-function and local connection merging. In: Review in process, technical report pending (2012) Streib, K., Davis, J.W.: Improving graph-based clustering via Ripley’s K-function and local connection merging. In: Review in process, technical report pending (2012)
28.
Zurück zum Zitat Streib, K., Davis, J.W.: Extracting pathlets from weak tracking data. Advanced Video and Signal Based Surveillance (2010) Streib, K., Davis, J.W.: Extracting pathlets from weak tracking data. Advanced Video and Signal Based Surveillance (2010)
29.
Zurück zum Zitat Wang, X., Tieu, K., Grimson, W.E.L.: Learning semantic scene models by trajectory analysis. In: Proceedings of the European Conference on Computer Vision (2006) Wang, X., Tieu, K., Grimson, W.E.L.: Learning semantic scene models by trajectory analysis. In: Proceedings of the European Conference on Computer Vision (2006)
30.
Zurück zum Zitat Wang, X., Ma, X., Grimson, E.: Unsupervised activity perception in crowded and complicated scenes using hierarchical Bayesian models. IEEE Trans. Pattern. Anal. Mach. Intell. 31(3), 539–555 (2009)CrossRef Wang, X., Ma, X., Grimson, E.: Unsupervised activity perception in crowded and complicated scenes using hierarchical Bayesian models. IEEE Trans. Pattern. Anal. Mach. Intell. 31(3), 539–555 (2009)CrossRef
31.
Zurück zum Zitat Wilson, R., Spann, M.: A new approach to clustering. Pattern Recognit. 23(12), 1413–1425 (1990)CrossRef Wilson, R., Spann, M.: A new approach to clustering. Pattern Recognit. 23(12), 1413–1425 (1990)CrossRef
32.
Zurück zum Zitat Wang, X., Ma, K.T., Ng, W.G., Grimson, W.E.L.: Trajectory analysis and semantic region modeling using a nonparametric Bayesian model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2008) Wang, X., Ma, K.T., Ng, W.G., Grimson, W.E.L.: Trajectory analysis and semantic region modeling using a nonparametric Bayesian model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2008)
33.
Zurück zum Zitat Yang, Y., Liu, J., Shah, M.: Video scene understanding using multi-scale analysis. In: Proceedings of the IEEE International Conference on Computer Vision (2009) Yang, Y., Liu, J., Shah, M.: Video scene understanding using multi-scale analysis. In: Proceedings of the IEEE International Conference on Computer Vision (2009)
34.
Zurück zum Zitat Zhu, X., Wu, X., Fan, J.: Exploring video content structure for hierarchical summarization. Multimedia Syst. 10(3), 98–115 (2004)CrossRef Zhu, X., Wu, X., Fan, J.: Exploring video content structure for hierarchical summarization. Multimedia Syst. 10(3), 98–115 (2004)CrossRef
Metadaten
Titel
Summarizing high-level scene behavior
verfasst von
Kevin Streib
James W. Davis
Publikationsdatum
01.01.2014
Verlag
Springer Berlin Heidelberg
Erschienen in
Machine Vision and Applications / Ausgabe 1/2014
Print ISSN: 0932-8092
Elektronische ISSN: 1432-1769
DOI
https://doi.org/10.1007/s00138-013-0573-2

Weitere Artikel der Ausgabe 1/2014

Machine Vision and Applications 1/2014 Zur Ausgabe