Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Theoretical and Practical Solutions in Remote Sensing Read chapter Read first chapter

2018 | OriginalPaper | Chapter

7. Warping Techniques in Video Stabilization

Authors : Margarita N. Favorskaya, Vladimir V. Buryachenko

Published in: Computer Vision in Control Systems-3

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Digital image and video stabilization are crucial issues in many surveillance systems. Good stabilization of the raw data provides a successful processing of visual materials. At present, the main approach directs on the search of the trade-offs between 2D and 3D stabilization methods in order to derive the benefits of both techniques. Our contribution is twofold. First, the multi-layered motion fields are applied in the warping during stabilization. For this purpose, the term “Structure-From-Layered-Motion” was introduced. Second, the warping and inpainting of the frame boundaries are executed using a pseudo-panoramic key frame and the multi-layered motion fields. Such inpainting permits to restore fast the cropped stabilized frames up to the sizes of the original non-stabilized frames. The dataset Sports Videos in the Wild, as well as the additional non-stationary video sequences, were used in experiments, which demonstrated good visibility results with a preserving of the frame sizes.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Automated Decision Making in Road Traffic Monitoring by On-board Unmanned Aerial Vehicle System
next chapter Image Deblurring Based on Physical Processes of Blur Impacts
Literature
1.
2.
Bechmann, D.: Space deformation models survey. Comput. Graph. 18(4), 571–586 (1994)CrossRef
3.
Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. Int. J. Comput. Vis. 2, 283–310 (1989)CrossRef
4.
Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)CrossRef
5.
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) Proceedings of the 8th European Conference on Computer Vision. Springer LNCS 3024, vol. 4, pp. 25–36. Springer, Berlin, Heidelberg (2004)
6.
Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Int. J. Comput. Vis. 67(2), 141–158 (2006)CrossRef
7.
Memin, E., Perez, P.: Hierarchical estimation and segmentation of dense motion fields. Int. J. Comput. Vis. 46(2), 129–155 (2002)CrossRefMATH
8.
Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacements. Int. J. Comput. Vis. 39(1), 41–56 (2000)CrossRefMATH
9.
Favorskaya, M., Jain, L.C., Buryachenko, V.: Digital video stabilization in static and dynamic scenes. In: Favorskaya, M.N,. Jain, L.C. (eds.) Computer Vision in Control Systems-1, ISRL, vol. 73, pp. 261–309. Springer International Publishing, Switzerland (2015)
10.
Favorskaya, M., Buryachenko, V.: Fuzzy-based digital video stabilization in static scenes. In: Tsihrintzis, G.A., Virvou, M., Jain, L.C., Howlett, R.J., Watanabe, T. (eds.) Intelligent Interactive Multimedia Systems and Services in Practice, SIST, vol. 36, pp. 63–83. Springer International Publishing, Switzerland (2015)
11.
Battiato, S., Gallo, G., Puglisi, G., Scellato, S.: SIFT features tracking for video stabilization. In: 14th IEEE International Conference on Image Analysis and Processing (ICIAP’2007), pp. 825–830 (2007)
12.
Cai, J., Walker, R.: Robust video stabilisation algorithm using feature point selection and delta optical flow. IET Comput. Vis. 3(4), 176–188 (2009)CrossRef
13.
Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3D video stabilization. ACM Trans. Graph. 28(3), 44.1–44.9 (2009)
14.
Wang, J.M., Chou, H.P., Chen, S.W., Fuh, C.S.: Video stabilization for a hand-held camera based on 3D motion model. In: IEEE International Conference on Image Processing (ICIP’2009), pp 3477–3480 (2009)
15.
Zhang, G., Hua, W., Qin, X., Shao, Y., Bao, H.: Video stabilization based on a 3D perspective camera model. Vis. Comput.: Int. J. Comput. Graph. 25(11), 997–1008 (2009)CrossRef
16.
Liu, F., Gleicher, M., Wang, J., Lin, H., Agarwala, A.: Subspace video stabilization. ACM Trans. Graph. 30(1), 4.1–4.10 (2011)
17.
Milliron, T., Jensen, R.J., Barzel, R., Finkelstein, A.: A framework for geometric warps and deformations. ACM Trans. Graph. 21(1), 20–51 (2002)CrossRef
18.
Alexa, M., Cohen-Or, D., Levin, D.: As-rigid-as-possible shape interpolation. In: 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’2000), pp. 157–164 (2000)
19.
Liu, L., Zhang, L., Xu, Y., Gotsman, C., Steven, S.J.: A local/global approach to mesh parameterization. In: Symposium on Geometry Processing (SGP’2008), pp. 1495–1504 (2008)
20.
Igarashi, T., Moscovich, T., Hughes, J.F.: As-rigid-as-possible shape manipulation. ACM Trans. Graph. 24(3), 1134–1141 (2005)CrossRef
21.
Schaefer, S., McPhail, T., Warren, J.: Image deformation using moving least squares. ACM Trans. Graph. 25(3), 533–540 (2006)CrossRef
22.
Zhou, K., Huang, J., Snyder, J., Liu, X., Bao, H., Guo, B., Shum, H.Y.: Large mesh deformation using the volumetric graph Laplacian. ACM Trans. Graph. 24(3), 496–503 (2005)CrossRef
23.
Sheffer, A., Kraevoy, V.: Pyramid coordinates for morphing and deformation. In: 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’2004), pp. 68–75 (2004)
24.
Weng, Y., Xu, W., Wu, Y., Zhou, K., Guo, B.: 2D shape deformation using nonlinear least squares optimization. Vis. Comput. 22(9), 653–660 (2006)CrossRef
25.
Gal, R., Sorkine, O., Cohen-Or, D.: Feature-aware texturing. In: 17th Eurographics Conference on Rendering Techniques (EGSR’2006), pp. 297–303 (2006)
26.
Litvin, A., Konrad, J., Karl, W.C.: Probabilistic video stabilization using Kalman filtering and mosaicking. In: IS&T/SPIE Symposium on Electronic Imaging, Image and Video Communications, Santa Clara, CA, USA, pp. 663–674 (2003)
27.
Wang, Y.S., Liu, F., Hsu, P.S., Lee, T.Y.: Spatially and temporally optimized video stabilization. IEEE Trans. Vis. Comput. Graph. 19(8), 1354–1361 (2013)CrossRef
28.
Liu, S., Yuan, L., Tan, P., Suny, J.: Bundled camera paths for video stabilization. ACM Trans. Graph. 32(4), 78.1–78.10 (2013)
29.
Liu, S., Yuan, L., Tan, P., Sun, T.: SteadyFlow: spatially smooth optical flow for video stabilization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2014), pp. 4209–4216 (2014)
30.
Smith, B.M., Zhang, L., Jin, H., Agarwala, A.: Light field video stabilization. In: IEEE International Conference on Computer Vision (ICCV’2009), pp. 341–348 (2009)
31.
Huynh, L., Choi, J., Medioni, G.: Aerial implicit 3D video stabilization using epipolar geometry constraint. In: 22nd International Conference on Pattern Recognition (ICPR’2014), pp. 3487–3492 (2014)
32.
Goldstein, A., Fattal, R.: Video stabilization using epipolar geometry. ACM Trans. Graph. 31(5), 126.1–126.10 (2012)
33.
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000)
34.
Lee, D.B., Choi, I.H., Song, B.C., Lee, T.H.: ROI-based video stabilization algorithm for hand-held cameras. In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW’2012), pp. 314–318 (2012)
35.
Matsushita, Y., Ofek, E., Ge, W., Tang, X., Shum, H.Y.: Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell. 28(7), 1150–1163 (2006)CrossRef
36.
Grundmann, M., Kwatra, V., Essa, I.: Auto-directed video stabilization with robust L1 optimal camera paths. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2011), pp. 225–232 (2011)
37.
Grundmann, M., Kwatra, V., Castro, D., Essa, I.: Calibration-free rolling shutter removal. In: IEEE International Conference on Computational Photography (ICCP’2012), pp. 4.1–4.8 (2012)
38.
Liu, S., Wang, Y., Yuan, L., Bu, J., Tan, P., Sun, J.: Video stabilization with a depth camera. IEEE Conf Computer Vision Pattern Recogn (CVPR’2012), pp. 89–95
39.
Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2004), vol. 1, pp. 120–127 (2004)
40.
Jia, J., Wu, T., Tai, Y., Tang, C.: Video repairing: inference of foreground and background under severe occlusion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2004), vol. 1, pp. 364–371 (2004)
41.
Bouguet, J.Y.: Pyramidal Implementation of the Lucas Kanade Feature Tracker: Description of the Algorithm. Intel Corporation, Microprocessor Research Labs, OpenCV Documents (2000)
42.
Telea, A.: An image inpainting technique based on the fast marching method. J. Graph. Tools 9(1), 25–36 (2004)CrossRef
43.
Shum, H.Y., Szeliski, R.: Construction of panoramic mosaics with global and local alignment. Int. J. Comput. Vis. 36(2), 101–130 (2000)CrossRef
44.
Liu, W.X., Chin, T.J.: Smooth globally warp locally: video stabilization using homography fields. In: International Conference on Digital Image Computing: Techniques and Applications (DICTA’2015), pp. 1–8 (2015)
45.
Lienhart, R.: Comparison of automatic shot boundary detection algorithms. In: SPIE Conference on Storage and Retrieval for Image and Video Databases VII, vol. 3656, pp. 290–301 (1999)
46.
Kelm, P., Schmiedekem, S., Sikora, T.: Feature-based video key frame extraction for low quality video sequences. In: 10th Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS’2009), pp. 25–28 (2009)
47.
Won, J.U., Chung, Y.S., Kim, I.S., Choi, J.G., Park, K.H.: Correlation based video-dissolve detection. Inf. Technol.: Res. Educ. 104–107 (2003)
48.
Karsch, K., Liu, C., Kang, S.B.: Depth extraction from video using non-parametric sampling. In: 12th European Conference on Computer Vision (ECCV’2012), vol. V, pp. 775–788 (2012)
49.
Liu, C., Yuen, J., Torralba, A.: SIFT flow: dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)CrossRef
50.
Saxena, A., Chung, S.H., Ng, A.Y.: 3-D Depth reconstruction from a single still image. Int. J. Comput. Vis. 76(1), 53–69 (2008)CrossRef
51.
Patil, S., Charles, P.: Review on 2D to 3D image and video conversion methods. In: International Conference on Computing Communication Control and Automation (ICCUBEA’2015), pp. 728–732 (2015)
52.
Dellaert, F., Seitz, S.M., Thorpe, C.E., Thrun, S.: Structure from motion without correspondence. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2000), pp. 557–564 (2000)
53.
Favorskaya, M., Buryachenko, V., Tomilina, A.: Global motion estimation using saliency maps in non-stationary videos with static scenes. In: De Pietro, G., Gallo, L., Howlett, R.J., Jain, L.C. (eds.) Intelligent Interactive Multimedia Systems and Services, SIST, vol. 55, pp. 133–144. Springer International Publishing, Switzerland (2016)
54.
Favorskaya, M., Buryachenko, V.: Fast salient object detection in non-stationary video sequences based on spatial saliency maps. In: De Pietro, G., Gallo, L., Howlett, R.J., Jain, L.C. (eds.) Intelligent Interactive Multimedia Systems and Services, SIST, vol. 55, pp. 121–132. Springer International Publishing, Switzerland (2016)
55.
Argyros, A.A., Lourakis, M.I.A., Trahanias, P.E., Orphanoudakis, S.C.: Independent 3D motion detection through robust regression in depth layers. In: British Machine Vision Conference (BMVC’1996), pp. 9–12 (1996)
56.
57.
58.
59.
Irani, M., Anandan, P.: A unified approach to moving objects detection in 2D and 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 6, 577–589 (1998)CrossRef
60.
Regan, M., Pose, R.: Priority rendering with a virtual reality address recalculation pipeline. In: Computer Graphics Annual Conference Series (SIGGRAPH’1994), pp. 155–162 (1994)
61.
Aliaga, D.G.: Visualization of complex models using dynamic texture-based simplification. In: IEEE 7th Conference on Visualization (VIS’1996), pp. 101–106 (1996)
62.
Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., Silva, C.T.: Computing and rendering point set surfaces. IEEE Trans. Vis. Comput. Graph. 9(1), 3–15 (2003)
63.
Zaragoza, J., Chin, T.J., Brown, M., Suter, D.: As-projective-as-possible image stitching with moving DLT. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’2013), pp. 2339–2346 (2013)
64.
Yu, S.X., Zhang, H., Malik, J.: Inferring spatial layout from a single image via depth-ordered grouping. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’2008), pp. 1–7 (2008)
65.
Kanatani, K.: Geometric Computation for Machine Vision. Clarendon Press (1993)
66.
Shi, J., Tomasi, C.: Good features to track. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’1994), pp. 593–600 (1994)
67.
Irani, M., Anandan, P.: Factorization with uncertainty. In: European Conference on Computer Vision (ECCV’2000), vol. 1, pp. 539–553 (2000)
68.
Koch, R.: Dynamic 3D scene analysis through synthesis feedback control. IEEE Trans. Pattern Anal. Mach. Intell. 15(6), 556–568 (1993)CrossRef
69.
Wang, J.Y.A., Adelson, E.H.: Representing moving images with layers. IEEE Trans. Image Process. 3(5), 625–638 (1994)CrossRef
70.
Jojic, N., Frey, B.: Learning flexible sprites in video layers. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’2001), vol. I, pp. 199–206 (2001)
71.
Kumar, M., Torr, P., Zisserman, A.: Learning layered motion segmentations of video. Int. J. Comput. Vis. 76(3), 301–319 (2008)CrossRef
72.
Longuet-Higgins, H.C., Prazdny, K.: The interpretation of a moving retinal image. Proc. R. Soc. Lond. Ser. B Biol. Sci. 208(1173), 385–397 (1980)
73.
Rouseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley, New York (1987)CrossRef
74.
Gleicher, M.L., Liu, F.: Re-cinematography: improving the camerawork of casual video. ACM Trans. Multimed. Comput. Commun. Appl. 5(1), 2:1–2:28 (2008)
75.
Weiss, Y.: Smoothness in layers: motion segmentation using nonparametric mixture estimation. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’1997), pp. 520–526 (1997)
76.
Sun, D., Sudderth, E.B., Black, M.J.: Layered segmentation and optical flow estimation over time. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’2012), pp. 1768–1775 (2012)
77.
Frey, B.J., Jojic, N., Kannan, A.: Learning appearance and transparency manifolds of occluded objects in layers. In: IEEE Conference on Computer Vision Pattern and Recognition (CVPR’2003), vol. 1, pp. 45–52 (2003)
78.
Sun, D., Sudderth, E.B., Black, M.J.: Layered image motion with explicit occlusions, temporal consistency, and depth ordering. In: Advances in Neural Information Processing Systems (NIPS’2010), vol. 23, pp. 2226–2234 (2010)
79.
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)CrossRef
80.
Matsushita, Y., Ofek, E., Tang, X., Shum, H.Y.: Full-frame video stabilization. In: IEEE Computer Vision and Pattern Recognition (CVPR’2005), vol. 1, pp. 50–57 (2005)
81.
Metadata
Title
Warping Techniques in Video Stabilization
Authors
Margarita N. Favorskaya
Vladimir V. Buryachenko
Copyright Year
2018
Book
Computer Vision in Control Systems-3

Print ISBN: 978-3-319-67515-2

Electronic ISBN: 978-3-319-67516-9

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-319-67516-9_7

Premium Partner

    Image Credits