Sunghyun Cho
Abstract
Videos captured by hand-held Cameras often contain significant camera shake, causing many frames to be blurry. Restoring shaky videos not only requires smoothing the camera motion and stabilizing the content, but also demands removing blur from video frames. However, video blur is hard to remove using existing single or multiple image deblurring techniques, as the blur kernel is both spatially and temporally varying. This paper presents a video deblurring method that can effectively restore sharp frames from blurry ones caused by camera shake. Our method is built upon the observation that due to the nature of camera shake, not all video frames are equally blurry. The same object may appear sharp on some frames while blurry on others. Our method detects sharp regions in the video, and uses them to restore blurry regions of the same content in nearby frames. Our method also ensures that the deblurred frames are both spatially and temporally coherent using patch-based synthesis. Experimental results show that our method can effectively remove complex video blur under the presence of moving objects and other outliers, which cannot be achieved using previous deconvolution-based approaches.
Supplemental Material
Available for Download
Supplemental material.
- Agrawal, A., Xu, Y., and Raskar, R. 2009. Invertible motion blur in video. ACM Trans. Graphics 28, 3, 95:1--95:8. Google Scholar
Digital Library
- Baker, S., and Matthews, I. 2004. Lucas-kanade 20 years on: A unifying framework. International Journal of Computer Vision (IJCV) 56, 3, 221--255. Google ScholarDigital Library
- Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graphics 28, 3, 24:1--24:11. Google ScholarDigital Library
- Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized PatchMatch correspondence algorithm. In Proc. ECCV 2010, 29--43. Google ScholarDigital Library
- Buades, A., and Coll, B. 2005. A non-local algorithm for image denoising. In Proc. CVPR 2006, 60--65. Google ScholarDigital Library
- Cai, J.-F., Ji, H., Liu, C., and Shen, Z. 2009. Blind motion deblurring using multiple images. J. Comput. Phys. 228, 5057--5071. Google ScholarDigital Library
- Chen, J., Yuan, L., Tang, C.-K., and Quan, L. 2008. Robust dual motion deblurring. In Proc. CVPR 2008, 1--8.Google Scholar
- Cho, S., and Lee, S. 2009. Fast motion deblurring. ACM Trans. Graphics 28, 5, 145:1--145:8. Google ScholarDigital Library
- Cho, S., Matsushita, Y., and Lee, S. 2007. Removing non-uniform motion blur from images. In Proc. ICCV 2007, 1--8.Google Scholar
- Cho, S., Wang, J., and Lee, S. 2011. Handling outliers in non-blind image deconvolution. In Proc. ICCV 2011, 495--502. Google ScholarDigital Library
- Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. Proc. ACM SIGGRAPH 2001, 341--346. Google ScholarDigital Library
- Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., and Freeman, W. T. 2006. Removing camera shake from a single photograph. ACM Trans. Graphics 25, 3, 787--794. Google ScholarDigital Library
- Freedman, G., and Fattal, R. 2011. Image and video up-scaling from local self-examples. ACM Trans. Graphics 30, 2, 12:1--12:11. Google ScholarDigital Library
- Fried, D. L. 1978. Probability of getting a lucky short-exposure image through turbulence. J. Opt. Soc. Am. 68, 12, 1651--1657.Google ScholarCross Ref
- Grundmann, M., Kwatra, V., and Essa, I. 2011. Auto-directed video stabilization with robust L1 optimal camera paths. In Proc. CVPR 2011, 225--232. Google ScholarDigital Library
- Gupta, A., Joshi, N., Zitnick, C. L., Cohen, M., and Curless, B. 2010. Single image deblurring using motion density functions. In Proc. ECCV 2010, 171--184. Google ScholarDigital Library
- HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graphics 30, 4, 70:1--70:10. Google ScholarDigital Library
- Hirsch, M., Schuler, C. J., Harmeling, S., and Schölkopf, B. 2011. Fast removal of non-uniform camera shake. In Proc. ICCV 2011, 463--470. Google ScholarDigital Library
- Joshi, N., and Cohen, M. 2010. Seeing mt. rainier: Lucky imaging for multi-image denoising, sharpening, and haze removal. In Proc. ICCP 2010, 1--8.Google Scholar
- Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Trans. Graphics 24, 3, 795--802. Google ScholarDigital Library
- Levin, A., Weiss, Y., Durand, F., and Freeman, W. T. 2011. Efficient marginal likelihood optimization in blind deconvolution. In Proc. CVPR 2011, 2657--2664. Google ScholarDigital Library
- Li, Y., Kang, S. B., Joshi, N., Seitz, S. M., and Huttenlocher, D. P. 2010. Generating sharp panoramas from motion-blurred videos. In Proc. CVPR 2010, 2424--2431.Google Scholar
- Liu, C., and Freeman, W. T. 2010. A high-quality video denoising algorithm based on reliable motion estimation. In Proc. ECCV 2010, 706--719. Google ScholarDigital Library
- Liu, F., Gleicher, M., Wang, J., Jin, H., and Agarwala, A. 2011. Subspace video stabilization. ACM Trans. Graphics 30, 1, 4:1--4:10. Google ScholarDigital Library
- Matsushita, Y., Ofek, E., Ge, W., Tang, X., and Shum, H.-Y. 2006. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Analysis Machine Intelligence 28, 7, 1150--1163. Google ScholarDigital Library
- Osher, S., and Rudin, L. I. 1990. Feature-oriented image enhancement using shock filters. SIAM Journal on Numerical Analysis 27, 4, 919--940. Google ScholarDigital Library
- Shan, Q., Jia, J., and Agarwala, A. 2008. High-quality motion deblurring from a single image. ACM Trans. Graphics 27, 3, 73:1--73:10. Google ScholarDigital Library
- Shi, J., and Tomasi, C. 1994. Good features to track. In Proc. CVPR 1994, 593--600.Google Scholar
- Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In Proc. CVPR 2008, 1--8.Google Scholar
- Tai, Y.-W., Tan, P., and Brown, M. S. 2011. Richardson-lucy deblurring for scenes under a projective motion path. IEEE Trans. Pattern Analysis Machine Intelligence 33, 8, 1603--1618. Google ScholarDigital Library
- Whyte, O., Sivic, J., Zisserman, A., and Ponce, J. 2010. Non-uniform deblurring for shaken images. In Proc. CVPR 2010, 491--498.Google Scholar
Index Terms
- Video deblurring for hand-held cameras using patch-based synthesis
Recommendations
A Hybrid Motion Deblurring Strategy Using Patch Based Edge Restoration and Bilateral Filter
Motion blur is a common problem in digital photography. In the dim light, a long exposure time is needed to acquire a satisfactory photograph, and if the camera shakes during exposure, a motion blur is captured. Image deblurring has become a crucial ...
Video deblurring via motion compensation and adaptive information fusion
AbstractNon-uniform motion blur caused by camera shake or object motion is a common artifact in videos captured by hand-held devices. Recent advances in video deblurring have shown that convolutional neural networks (CNNs) are able to ...
High-resolution optical flow and frame-recurrent network for video super-resolution and deblurring
AbstractOver the last years, advances in deep learning have brought huge developments to the studying of super-resolution reconstruction. However, most super-resolution methods only deal with simply down-sampled sharp images, which may lose ...
Comments