Abstract
This paper presents a novel approach to stabilize video sequences based on a 3D perspective camera model. Compared to previous methods which are based on simplified models, our stabilization system can work in situations where significant depth variations exist in the scenes and the camera undergoes large translational movement. We formulate the stabilization problem as a quadratic cost function on smoothness and similarity constraints. This allows us to precisely control the smoothness by solving a sparse linear system of equations. By taking advantage of the sparseness, our optimization process is very efficient. Instead of recovering dense depths, we use approximate geometry representation and analyze the resulting warping errors. We show that by appropriately constraining warping error, visually plausible results can be achieved even using planar structures. A variety of experiments have been implemented, which demonstrates the robustness and efficiency of our approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
2d3: http://www.2d3.com
Barrett, R., Berry, M., Chan, T.F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., der Vorst, H.V.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd edn. SIAM, Philadelphia (1994)
Bhat, P., Zitnick, C.L., Snavely, N., Agarwala, A., Agrawala, M., Curless, B., Cohen, M., Kang, S.B.: Using photographs to enhance videos of a static scene. In: Kautz, J., Pattanaik, S. (eds.) Rendering Techniques 2007, Proceedings Eurographics Symposium on Rendering, pp. 327–338. Eurographics (2007)
Buehler, C., Bosse, M., McMillan, L.: Non-metric image-based rendering for video stabilization. In: CVPR, pp. 609–614 (2001)
Davis, L.S., Bajcsy, R., Herman, M.: RSTA on the move. In: Proceedings of the ARPA Image Understanding Workshop, pp. 435–456 (1994)
Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. In: CommACM, vol. 24, p. 381–395 (1981)
Fitzgibbon, A., Zisserman, A.: Automatic camera tracking. In: Shah, M., Kumar, R. (eds.) Video Registration, pp. 18–35. Kluwer, Dordrecht (2003)
Hansen, M., Anadan, P., Dana, K., van de Wal, G., Burt, P.: Realtime scene stabilization and mosaic construction. In: Proc. IEEE Image Understanding Workshop, pp. 457–463 (1994)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)
Litvin, A., Konrad, J., Karl, W.: Probabilistic video stabilization using Kalman filtering and mosaicking. In: IS&T/SPIE Symposium on Electronic Imaging, Image and Video Communications, pp. 663–674 (2003)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Mark, W.R., McMillan, L., Bishop, G.: Post-rendering 3D warping. In: SI3D, pp. 7–16, 180 (1997)
Matsushita, Y., Ofek, E., Tang, X., Shum, H.Y.: Full-frame video stabilization. In: CVPR (1), pp. 50–57 (2005)
Morimoto, C., Chellappa, R.: Fast 3D stabilization and mosaic construction. In: CVPR, pp. 660–665 (1997)
Morimoto, C., Chellappa, R.: Evaluation of image stabilization algorithms. In: International Conference on Acoustics, Speech and Signal Processing, vol. 5, pp. 2789–2792 (1998)
Nistér, D.: Preemptive ransac for live structure and motion estimation. Mach. Vis. Appl. 16(5), 321–329 (2005)
Pilu, M.: Video stabilization as a variational problem and numerical solution with the Viterbi method. In: CVPR (1), pp. 625–630 (2004)
Pollefeys, M., Gool, L.J.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004)
REALVIZ: http://www.realviz.com
Shade, J., Gortler, S.J., wei He, L., Szeliski, R.: Layered depth images. In: SIGGRAPH, pp. 231–242 (1998)
Snavely, N., Seitz, S.M., Szeliski, R.: Photo-tourism: exploring photo-collections in 3D. ACM Trans. Graph. 25(3), 835–846 (2006)
Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Workshop on Vision Algorithms, pp. 298–372 (1999)
Yao, Y., Burlina, P., Chellappa, R., Wu, T.: Electronic image stabilization using multiple visual cues. In: Proceedings of International Conference on Image Processing, pp. 191–194 (1995)
Zhang, G., Qin, X., Hua, W., Wong, T.T., Heng, P.A., Bao, H.: Robust metric reconstruction from challenging video sequences. In: CVPR (2007)
Zhang, Z.: Determining the epipolar geometry and its uncertainty: A review. Int. J. Comput. Vis. 27(2), 161–195 (1998)
Zhu, Z., Xu, G., Yang, Y., Jin, J.S.: Camera stabilization based on 2.5D motion model estimation and inertial motion filtering. In: IEEE Conference on Intelligent Vehicles, pp. 329–334 (1998)
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Zhang, G., Hua, W., Qin, X. et al. Video stabilization based on a 3D perspective camera model. Vis Comput 25, 997–1008 (2009). https://doi.org/10.1007/s00371-009-0310-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-009-0310-z