Skip to main content
SpringerLink
Log in

Relative Flow Estimates for Shot Boundary Detection

  • Representation, Processing, Analysis, and Understanding of Images
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

This paper proposes a simple approach based on Relative Flow Estimates (RFE) for shot cut detection. The property of Relative flow estimates can be used for abrupt cut detection and a correction mechanism for gradual camera-shot transition detection (e.g., fade-in and fade-out, dissolves, wipes). The exacted feature vector in each frame can be mapped into a 3-D space along the continuous time axis, and these feature data can be treated as a virtually constructed pipe with fluid flowing in the 3-D axis. Compared with existing approaches, the new RFE-based algorithm can directly detect shot cut. A wide range of test videos are used to evaluate the performance of the proposed method. The experimental results show that the new scheme can produce promising results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Cooper, T. Liu, and E. Rieffel, “Video segmentation via temporal pattern classification,” IEEE Trans. Multimedia 9 (3), 610–618 (2007).

    Article  Google Scholar 

  2. S. Lefevre, J. Holler, and N. Vincent, “A review of realtime segmentation of uncompressed video sequences for content-based search and retrieval,” Real-Time Imag. 9, 73–98 (2003).

    Article  Google Scholar 

  3. Oguzhan Urhan, M. Kemal Güllü, and Sarp Ertürk, “Modified phase-correlation based robust hard-cut detection with application to archive film,” IEEE Trans. Circuits Syst. Video Technol. 16 (6) (2006).

    Google Scholar 

  4. A. Nagasaka and Y. Tanaka, “Automatic video indexing and full-video search for object appearances,” in Proc. IFIP Working Conf. Visual Database Systems (Budapest, Oct. 1991), pp. 113–127.

    Google Scholar 

  5. I. K. Sethi and N. Patel, “A statistical approach to scene change detection,” in SPIE Conf. Proc. Storage and Retrieval for Image and Video Databases III (La Jolla, CA, Feb. 1995), Vol. 2420, pp. 329–339.

    Chapter  Google Scholar 

  6. M. S. Lee, Y. M. Yang, and S. W. Lee, “Automatic video parsing using shot boundary detection and camera operation analysis,” Pattern Recogn. 34, 711–719 (2001).

    Article  MATH  Google Scholar 

  7. S. Lefevre, J. Holler, and N. Vincent, “Real time temporal segmentation of compressed and uncompressed dynamic colour image sequences,” in Proc. Int. Workshop on Real Time Image Sequence Analysis (Oulu, Aug. 2000), pp. 56–62.

    Google Scholar 

  8. T. Truong, S. Venkatesh, and C. Dorai, “Scene extraction in motion pictures,” IEEE Trans. Circuits Syst. Video Technol. 13 (1), 5–15 (2003).

    Article  Google Scholar 

  9. H. J. Heng and K. N. Ngan, “Integrated shot boundary detection using object-based technique,” in IEEE Int. Conf. Image Processing (Kobe, Oct. 1999), Vol. 3, pp. 289–293.

    Google Scholar 

  10. P. Bouthemy, M. Gelgon, and F. Ganansia, “A unified approach to shot change detection and camera motion characterization,” IEEE Trans. Circuits Syst. Video Technol. 9 (10), 1030–1044 (1999).

    Article  Google Scholar 

  11. T. Liu, X. Zhang, D. Wang, J. Feng, and K. T. Lo, “Inertia-based cut detection technique: a step to the integration of video coding and content-based retrieval,” in Proc. IEEE Int. Conf. Signal Processing (Beijing, Aug. 2000), pp. 1018–1025.

    Google Scholar 

  12. W. K. Li and S. H. Lai, “Integrated video shot segmentation algorithm,” in Proc. SPIE Conf. Storage and Retrieval for Media Databases (Santa Clara, CA, 2003), pp. 264–271.

    Google Scholar 

  13. M. Flickner, “Query by image content: the QBIC system,” IEEE Comput. Mag. 28 (9) (1995).

    Google Scholar 

  14. J. Huang, S. Kumar, M. Mitra, W. Zhu, and R. Zabih, “Image indexing using color correlograms,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (San Juan, 1997), pp. 762–768.

    Chapter  Google Scholar 

  15. G. Pass and R. Zabith, “Histogram refinement for content- based image retrieval,” in Proc. IEEE Workshop on Applications of Computer Vision (Sarasota, 1996), pp. 96–102.

    Chapter  Google Scholar 

  16. S. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. PAMI 11 (7), 674–693 (1989).

    Article  MATH  Google Scholar 

  17. B. S. Manjunath and W. Y. Ma, “Texture features for browsing and retrieval of image data,” IEEE Trans. Pattern Anal. Mach. Intellig. 18 (8), 837–842 (1996).

    Article  Google Scholar 

  18. B. Manjunath, P. Wu, S. Newsam, and H. Shin, “A texture descriptor for browsing and similarity retrieval,” J. Signal Processing: Image Commun. 16 (1), 3343 (2000).

    Google Scholar 

  19. J. Smith and S.-F. Chang, “Transform features for texture classification and discrimination in large image database,” in Proc. IEEE Int. Conf. on Image Processing (Austin, TX, 1994).

    Google Scholar 

  20. Muwei, Jian and Junyu Dong, “New perceptual texture features based on wavelet transform,” Int. J. Comput. Inf. Sci. 9 (1), 11–18 (2008).

    Google Scholar 

  21. R. L. Devalois, D.G. Albrecht, and L.G. Thorell, “Spatial -frequency selectivity of cells in acaque visual cortex,” Vision Res. 22, 545–559 (1982).

    Article  Google Scholar 

  22. T. Caelli, Visual Perception (Pergamon Press, 1981).

    Google Scholar 

  23. J. Yuan, H. Wang, L. Xiao, W. Zheng, J. Li, F. Lin, and B. Zhang, “A formal study of shot boundary detection,” IEEE Trans. Circuits Syst. Video Technology 17 (2), 168–186 (2007).

    Article  Google Scholar 

  24. G. G. Lakshmi Priya and S. Domnic, “Walsh-Hadamard transform kernel-based feature vector for shot boundary detection,” IEEE Trans. Image Processing 23 (12), 5187–5197 (2014).

    Article  MathSciNet  MATH  Google Scholar 

  25. Sawitchaya Tippaya, Tele Tan, Masood Khan, and Kosin Chamnongthai, “A study of discriminant visual descriptors for sport video shot boundary detection,” in Proc. 10th Asian Control Conf. (ASCC) (Kota Kinabalu, 2015), pp. 1–4.

    Google Scholar 

  26. B. H. Shekar, K. P. Uma, and K Raghurama Holla, “Shot boundary detection using correlation based spectral residual saliency map,” in Proc. Int. Conf. on Advances in Computing Communications and Informatics (ICACCI) (Jaipur, 2016), pp. 2242–2247.

    Google Scholar 

  27. Cai Pingping, Yue Guan, Xu Ding, and Zang Yu, “Shot boundary detection with sparse presentation,” in Proc. 13th IEEE Int. Conf. on Signal Processing (ICSP) (Buzios, 2016), pp. 900–904.

    Google Scholar 

  28. Dong-ju Jeong, Hyoung Jin Yoo, and Nam Ik Cho, “A static video summarization method based on the sparse coding of features and representativeness of frames,” EURASIP J. Image Video Processing (2017).

    Google Scholar 

  29. M. Jian and J. J. Ma, “Image retrieval using wavelet-based salient regions,” Imag. Sci. J. 59 (4), 219–231 (2011).

    Article  Google Scholar 

  30. http://www-nlpir.nist.gov/projects/trecvid/.

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China

    Muwei Jian & Yilong Yin

  2. Department of Computer Science and Technology, Ocean University of China, Qingdao, China

    Muwei Jian & Junyu Dong

Authors
  1. Muwei Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yilong Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyu Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muwei Jian.

Additional information

The article is published in the original.

Junyu Dong received his B.Sc. and M.Sc. in Applied Mathematics from the Ocean University of China (formerly called Ocean University of Qingdao) in 1993 and 1999, respectively. He won the Overseas Research Scholarship and James Watt Scholarship for his PhD study in 2000 and was awarded a PhD. degree in Image Processing in 2003 from the School of Mathematical and Computer Sciences, Heriot- Watt University, UK.

Dr. Junyu Dong joined Ocean University of China in 2004. From 2004 to 2010, Dr. Junyu Dong was an associate professor at the Department of Computer Science and Technology. He became a Professor in 2010 and is currently the Head of the Department of Computer Science and Technology. Prof. Dong was actively involved in professional activities. He has been a member of the program committee of several international conferences, including the 4th International Workshop on Texture Analysis and Synthesis (associated with ICCV2005), the 2006 British Machine Vision Conference (BMVC 2006) and the 3rd International Conference on Appearance (Predicting Perceptions 2012). Currently, Prof. Dong is the Chairman of Qingdao Young Computer Science and Engineering Forum (YOCSEF Qingdao). He is a member of ACM and IEEE. Prof. Dong’s research interest includes texture perception and analysis, 3D reconstruction, video analysis and underwater image processing.

Muwei Jian received the PhD degree from the Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, in October 2014. He was a Lecturer with the Department of Computer Science and Technology, Ocean University of China, from 2015 to 2017. Currently, Dr. Jian is a Professor and PhD Supervisor at the School of Computer Science and Technology, Shandong University of Finance and Economics. His current research interests include human face recognition, image and video processing, machine learning and computer vision. Prof. Jian was actively involved in professional activities. He has been a member of the Program Committee and Special Session Chair of several international conferences, such as SNPD 2007, ICIS 2008, APSIPA 2015, EEECS 2016 and ICTAI2016/2017. Dr. Jian has also served as a reviewer for several international SCI-indexed journals, including IEEE Trans., Pattern Recognition, Information Sciences, Computers in Industry, Machine Vision and Applications, Machine Learning and Cybernetics, The Imaging Science Journal, and Multimedia Tools and Applications. Prof. Jian holds 3 granted national patents and has published over 40 papers in refereed international leading journals/conferences such as IEEE Trans. on Cybernetics, IEEE Trans. on Circuits and Systems for Video Technology, Pattern Recognition, Information Sciences, Signal Processing, ISCAS, ICME and ICIP.

Yilong Yin received the Ph.D. degree from Jilin University, Changchun, China, in 2000. From 2000 to 2002, he was a Post-Doctoral Fellow with the Department of Electronics Science and Engineering, Nanjing University, Nanjing, China. He is currently the Director of the data Mining, Machine Learning, and their Applications Group and a Professor of the School of Computer Science and Technology, Shandong University, Jinan, China. His research interests include machine learning, data mining, and computational medicine.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jian, M., Yin, Y. & Dong, J. Relative Flow Estimates for Shot Boundary Detection. Pattern Recognit. Image Anal. 28, 53–58 (2018). https://doi.org/10.1134/S1054661818010121

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661818010121

Keywords

Search

Navigation