Neal Wadhwa
Abstract
We introduce a technique to manipulate small movements in videos based on an analysis of motion in complex-valued image pyramids. Phase variations of the coefficients of a complex-valued steerable pyramid over time correspond to motion, and can be temporally processed and amplified to reveal imperceptible motions, or attenuated to remove distracting changes. This processing does not involve the computation of optical flow, and in comparison to the previous Eulerian Video Magnification method it supports larger amplification factors and is significantly less sensitive to noise. These improved capabilities broaden the set of applications for motion processing in videos. We demonstrate the advantages of this approach on synthetic and natural video sequences, and explore applications in scientific analysis, visualization and video enhancement.
Supplemental Material
- Bai, J., Agarwala, A., Agrawala, M., and Ramamoorthi, R. 2012. Selectively de-animating video. ACM Transactions on Graphics. Google Scholar
Digital Library
- Bojanic, S., Simpson, T., and Bolger, C. 2001. Ocular microtremor: a tool for measuring depth of anaesthesia? British Journal of Anaesthesia 86, 4, 519--522.Google ScholarCross Ref
- Buades, A., Coll, B., and Morel, J.-M. 2008. Nonlocal image and movie denoising. International Journal of Computer Vision 76, 123--139. Google ScholarDigital Library
- Dabov, K., Foi, A., and Egiazarian, K. 2007. Video denoising by sparse 3d transform-domain collaborative filtering. In Proc. 15th European Signal Processing Conference, vol. 1, 7.Google Scholar
- Fleet, D. J., and Jepson, A. D. 1990. Computation of component image velocity from local phase information. Int. J. Comput. Vision 5, 1 (Sept.), 77--104. Google ScholarDigital Library
- Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1991. Motion without movement. SIGGRAPH Comput. Graph. 25 (Jul), 27--30. Google ScholarDigital Library
- Fuchs, M., Chen, T., Wang, O., Raskar, R., Seidel, H.-P., and Lensch, H. P. 2010. Real-time temporal shaping of high-speed video streams. Computers & Graphics 34, 5, 575--584. Google ScholarDigital Library
- Gautama, T., and Van Hulle, M. 2002. A phase-based approach to the estimation of the optical flow field using spatial filtering. Neural Networks, IEEE Transactions on 13, 5 (sep), 1127--1136. Google ScholarDigital Library
- Liu, C., and Freeman, W. 2010. A high-quality video denoising algorithm based on reliable motion estimation. In Computer Vision ECCV 2010, K. Daniilidis, P. Maragos, and N. Paragios, Eds., vol. 6313 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 706--719. Google ScholarDigital Library
- Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. Graph. 24 (Jul), 519--526. Google ScholarDigital Library
- Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vision 40, 1 (Oct.), 49--70. Google ScholarDigital Library
- Rolfs, M. 2009. Microsaccades: Small steps on a long way. Vision Research 49, 20, 2415--2441.Google ScholarCross Ref
- Rubinstein, M., Liu, C., Sand, P., Durand, F., and Freeman, W. T. 2011. Motion denoising with application to time-lapse photography. IEEE Computer Vision and Pattern Recognition (CVPR) (June), 313--320. Google ScholarDigital Library
- Simoncelli, E. P., and Freeman, W. T. 1995. The steerable pyramid: a flexible architecture for multi-scale derivative computation. In Proceedings of the 1995 International Conference on Image Processing (Vol. 3)-Volume 3 - Volume 3, IEEE Computer Society, Washington, DC, USA, ICIP '95, 3444--. Google ScholarDigital Library
- Simoncelli, E. P., Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1992. Shiftable multi-scale transforms. IEEE Trans. Info. Theory 2, 38, 587--607. Google ScholarDigital Library
- Wang, J., Drucker, S. M., Agrawala, M., and Cohen, M. F. 2006. The cartoon animation filter. ACM Trans. Graph. 25, 1169--1173. Google ScholarDigital Library
- Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proc. SIGGRAPH) 31 (aug). Google ScholarDigital Library
Index Terms
- Phase-based video motion processing
Recommendations
Motion magnification
We present motion magnification, a technique that acts like a microscope for visual motion. It can amplify subtle motions in a video sequence, allowing for visualization of deformations that would otherwise be invisible. To achieve motion magnification, ...
Motion magnification
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersWe present motion magnification, a technique that acts like a microscope for visual motion. It can amplify subtle motions in a video sequence, allowing for visualization of deformations that would otherwise be invisible. To achieve motion magnification, ...
Eulerian video magnification for revealing subtle changes in the world
Our goal is to reveal temporal variations in videos that are difficult or impossible to see with the naked eye and display them in an indicative manner. Our method, which we call Eulerian Video Magnification, takes a standard video sequence as input, ...
Comments