Skip to main content
Top

2020 | OriginalPaper | Chapter

Fine-Grain Level Sports Video Search Engine

Authors : Zikai Song, Junqing Yu, Hengyou Cai, Yangliu Hu, Yi-Ping Phoebe Chen

Published in: MultiMedia Modeling

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

It becomes an urgent demand how to make people find relevant video content of interest from massive sports videos. We have designed and developed a sports video search engine based on distributed architecture, which aims to provide users with content-based video analysis and retrieval services. In sports video search engine, we focus on event detection, highlights analysis and image retrieval. Our work has several advantages: (I) CNN and RNN are used to extract features and integrate dynamic information and a new sliding window model are used for multi-length event detection. (II) For highlights analysis. An improved method based on self-adapting dual threshold and dominant color percentage are used to detect the shot boundary. Affect arousal method are used for highlights extraction. (III) For image’s indexing and retrieval. Hyper-spherical soft assignment method is proposed to generate image descriptor. Enhanced residual vector quantization is presented to construct multi-inverted index. Two adaptive retrieval methods based on hype-spherical filtration are used to improve the time efficient. (IV) All of previous algorithms are implemented in the distributed platform which we develop for massive video data processing.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Geetha, P., Narayanan, V.: A survey of content-based video retrieval. J. Comput. Sci. 4(6), 734 (2008) Geetha, P., Narayanan, V.: A survey of content-based video retrieval. J. Comput. Sci. 4(6), 734 (2008)
2.
go back to reference Chao, Y.W., Vijayanarasimhan, S., Seybold, B., et al.: Rethinking the faster R-CNN architecture for temporal action localization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1130–1139 (2018) Chao, Y.W., Vijayanarasimhan, S., Seybold, B., et al.: Rethinking the faster R-CNN architecture for temporal action localization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1130–1139 (2018)
3.
go back to reference Lin, T., Zhao, X., Su, H., et al.: BSN: boundary sensitive network for temporal action proposal generation. In: European Conference on Computer Vision (ECCV), pp. 3–19 (2018)CrossRef Lin, T., Zhao, X., Su, H., et al.: BSN: boundary sensitive network for temporal action proposal generation. In: European Conference on Computer Vision (ECCV), pp. 3–19 (2018)CrossRef
4.
go back to reference Ramanathan, V., Huang, J., Abu-El-Haija, S., et al.: Detecting events and key actors in multi-person videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3043–3053 (2016) Ramanathan, V., Huang, J., Abu-El-Haija, S., et al.: Detecting events and key actors in multi-person videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3043–3053 (2016)
5.
go back to reference Ibrahim, M.S., Muralidharan, S., Deng, Z., et al.: A hierarchical deep temporal model for group activity recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1971–1980 (2016) Ibrahim, M.S., Muralidharan, S., Deng, Z., et al.: A hierarchical deep temporal model for group activity recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1971–1980 (2016)
6.
go back to reference Lea, C., Flynn, M.D., Vidal, R., et al.: Temporal convolutional networks for action segmentation and detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1003–1012 (2017) Lea, C., Flynn, M.D., Vidal, R., et al.: Temporal convolutional networks for action segmentation and detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1003–1012 (2017)
7.
go back to reference Krishna, R., Hata, K., Ren, F., et al.: Dense-captioning events in videos. In: IEEE International Conference on Computer Vision (ICCV), pp. 706–715 (2017) Krishna, R., Hata, K., Ren, F., et al.: Dense-captioning events in videos. In: IEEE International Conference on Computer Vision (ICCV), pp. 706–715 (2017)
8.
go back to reference Hanjalic, A.: Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Trans. Multimedia 7(6), 1114–1122 (2005)CrossRef Hanjalic, A.: Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Trans. Multimedia 7(6), 1114–1122 (2005)CrossRef
9.
go back to reference Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in video. In: IEEE International Conference on Computer Vision (ICCV), pp. 1470–1477 (2003) Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in video. In: IEEE International Conference on Computer Vision (ICCV), pp. 1470–1477 (2003)
10.
go back to reference Jegou, H., Douze, M., Schmid, C., et al.: Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3304–3311 (2010) Jegou, H., Douze, M., Schmid, C., et al.: Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3304–3311 (2010)
11.
go back to reference Wengert, C., Douze, M., Jegou, H.: Bag-of-colors for improved image search. In: 19th ACM International Conference on Multimedia, pp. 1437–1440 (2011) Wengert, C., Douze, M., Jegou, H.: Bag-of-colors for improved image search. In: 19th ACM International Conference on Multimedia, pp. 1437–1440 (2011)
12.
go back to reference Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)CrossRef Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)CrossRef
13.
go back to reference Tavenard, R., Jegou, H., Amsaleg, L.: Balancing clusters to reduce response time variability in large scale image search. In: 9th International Workshop on Content-Based Multimedia Indexing (CBMI), pp. 19–24 (2011) Tavenard, R., Jegou, H., Amsaleg, L.: Balancing clusters to reduce response time variability in large scale image search. In: 9th International Workshop on Content-Based Multimedia Indexing (CBMI), pp. 19–24 (2011)
14.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556 (2014)
15.
go back to reference Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015) Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:​1502.​03167 (2015)
17.
go back to reference Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef
18.
go back to reference Xu, K., Ba, J., Kiros, R., et al.: Show, attend and tell: neural image caption generation with visual attention. arXiv preprint arXiv:1502.03044 (2015) Xu, K., Ba, J., Kiros, R., et al.: Show, attend and tell: neural image caption generation with visual attention. arXiv preprint arXiv:​1502.​03044 (2015)
19.
go back to reference Yu, J., Wang, N.: Shot classification for soccer video based on sub-window region. J. Image Graph. 1006–8961 (2008). 07-1347-06 Yu, J., Wang, N.: Shot classification for soccer video based on sub-window region. J. Image Graph. 1006–8961 (2008). 07-1347-06
20.
go back to reference Gan, C., et al.: DevNet: a deep event network for multimedia event detection and evidence recounting. In: IEEE Conference on Computer Vision and Pattern Recognition (2015) Gan, C., et al.: DevNet: a deep event network for multimedia event detection and evidence recounting. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)
21.
go back to reference Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: IEEE Conference on Computer Vision and Pattern Recognition (2015) Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)
22.
go back to reference Buch, S., et al.: SST: single-stream temporal action proposals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) Buch, S., et al.: SST: single-stream temporal action proposals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
23.
go back to reference Yu, J.Q., Lei, A.P., Song, Z.K., et al.: Comprehensive dataset of broadcast soccer videos. In: IEEE International Conference on Multimedia Information Processing and Retrieval (2018) Yu, J.Q., Lei, A.P., Song, Z.K., et al.: Comprehensive dataset of broadcast soccer videos. In: IEEE International Conference on Multimedia Information Processing and Retrieval (2018)
25.
go back to reference Herve, J., Matthijs, D., Cordelia, S., et al.: Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3304–3311 (2010) Herve, J., Matthijs, D., Cordelia, S., et al.: Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3304–3311 (2010)
Metadata
Title
Fine-Grain Level Sports Video Search Engine
Authors
Zikai Song
Junqing Yu
Hengyou Cai
Yangliu Hu
Yi-Ping Phoebe Chen
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-37731-1_42