Skip to main content
SpringerLink
Log in

A Linear Systems Approach to Imaging Through Turbulence

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

In this paper we address the problem of recovering an image from a sequence of distorted versions of it, where the distortion is caused by what is commonly referred to as ground-level turbulence. In mathematical terms, such distortion can be described as the cumulative effect of a blurring kernel and a time-dependent deformation of the image domain. We introduce a statistical dynamic model for the generation of turbulence based on linear dynamical systems (LDS). We expand the model to include the unknown image as part of the unobserved state and apply Kalman filtering to estimate such state. This operation yields a blurry image where the blurring kernel is effectively isoplanatic. Applying blind nonlocal Total Variation (NL-TV) deconvolution yields a sharp final result.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. We remind the reader that the Horn-Schunck method minimizes a data fidelity term for the flow plus (as regularizing term) the H 1-seminorm of the flow itself times a smoothing parameter; see [22] for details.

References

  1. Andrews, H.C., Hunt, B.R.: Digital Image Restoration. Prentice Hall, Englewood Cliffs (1977)

    Google Scholar 

  2. Aubailly, M., Vorontsov, M.A., Carhart, G.W., Valley, M.T.: Automated video enhancement from a stream of atmospherically-distorted images: the lucky-region fusion approach. In: SPIE Atmospheric Optics: Models, Measurements, and Target-in-the-Loop Propagation, San Diego, CA, Aug. (2009)

    Google Scholar 

  3. Bascle, B., Blake, A., Zisserman, A.: Motion deblurring and super-resolution from an image sequence. In: Proceedings of the Fourth European Conference on Computer Vision, pp. 573–582. Springer, Berlin (1996)

    Google Scholar 

  4. Beg, M.F., Miller, M.I., Trouvé, A., Younes, L.: Computing large deformation metric mappings via geodesic flows of diffeomorphisms. Int. J. Comput. Vis. 61(2), 139–157 (2005)

    Article  Google Scholar 

  5. Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)

    Article  Google Scholar 

  6. Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA (2005)

    Google Scholar 

  7. Buades, A., Coll, B., Morel, J.-M.: A review of image denoising methods, with a new one. Multiscale Model. Simul. 4(3), 490–530 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  8. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004)

    Article  MathSciNet  Google Scholar 

  9. Chan, A.B., Mahadevan, V., Vasconcelos, N.: Generalized Stauffer–Grimson background subtraction for dynamic scenes. Mach. Vis. Appl. 22(5), 751–766 (2011)

    Article  Google Scholar 

  10. Chan, T.F., Wong, C.-K.: Total variation blind deconvolution. IEEE Trans. Image Process. 7(3), 370–375 (1998)

    Article  Google Scholar 

  11. Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003)

    Article  MATH  Google Scholar 

  12. Fried, D.L.: Statistics of a geometric representation of wavefront distortion. J. Opt. Soc. Am. 55(11), 1427–1435 (1965)

    Article  MathSciNet  Google Scholar 

  13. Fried, D.L.: Optical resolution through a randomly inhomogeneous medium for very long and very short exposures. J. Opt. Soc. Am. 56(10), 1372–1379 (1966)

    Article  MathSciNet  Google Scholar 

  14. Fried, D.L.: Probability of getting a lucky short-exposure image through turbulence. J. Opt. Soc. Am. 68, 1651–1657 (1978)

    Article  Google Scholar 

  15. Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7(3), 1005–1028 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  16. Gilles, J., Dagobert, T., De Franchis, C.: Atmospheric turbulence restoration by diffeomorphic image registration and blind deconvolution. In: Advanced Concepts for Intelligent Vision Systems. Lecture Notes in Computer Science, vol. 5259, pp. 400–409. Springer, Berlin (2008)

    Chapter  Google Scholar 

  17. Gilles, J., Osher, S.: Fried deconvolution. CAM Technical Report 11-62, UCLA Department of Mathematics, Dec. (2011)

  18. Goldstein, T., Osher, S.: The split Bregman method for L 1 regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  19. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  20. He, L., Marquina, A., Osher, S.: Blind deconvolution using TV regularization and Bregman iteration. Int. J. Imaging Syst. Technol. 15(1), 74–83 (2005)

    Article  Google Scholar 

  21. Hirsch, M., Sra, S., Schölkopf, B., Harmeling, S.: Efficient filter flow for space-variant multiframe blind deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2010), San Francisco, CA, June, pp. 607–614 (2010)

    Google Scholar 

  22. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)

    Article  Google Scholar 

  23. Kumar, P.R., Varaiya, P.: Stochastic Systems: Estimation, Identification, and Adaptive Control. Prentice Hall, Englewood Cliffs (1986)

    MATH  Google Scholar 

  24. Li, D., Mersereau, R.M., Simske, S.: Atmospheric turbulence-degraded image restoration using principal components analysis. IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007)

    Article  Google Scholar 

  25. Lin, H.T., Tai, Y.-W., Brown, M.S.: Motion regularization for matting motion blurred objects. IEEE Trans. Pattern Anal. Mach. Intell. (2011, to appear)

  26. Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice Hall, Englewood Cliffs (1999)

    Google Scholar 

  27. Lou, Y., Zhang, X., Osher, S., Bertozzi, A.: Image recovery via nonlocal operators. Journal of Scientific Computing 2(42) (2010)

  28. Mao, Y., Gilles, J.: Non-rigid geometric distortions corrections – application to atmospheric turbulence stabilization. Inverse Probl. Imaging 6(3) (2012, to appear)

  29. Metari, S., Deschênes, F.: A new convolution kernel for atmospheric point spread function applied to computer vision. In: Proceedings of the IEEE 11th International Conference on Computer Vision (ICCV 2007), Rio de Janeiro, Brazil, Oct. 2007

    Google Scholar 

  30. Miller, M.I., Younes, L.: Group actions, homeomorphisms, and matching: a general framework. Int. J. Comput. Vis. 41(1/2), 61–84 (2001)

    Article  MATH  Google Scholar 

  31. Paxman, R.G., Seldin, J.H., Löfdahl, M.G., Scharmer, G.B., Keller, C.U.: Evaluation of phase-diversity techniques for solar-image restoration. Astrophys. J. 466, 1087–1099 (1996)

    Article  Google Scholar 

  32. Paxman, R.G., Thelen, B.J., Seldin, J.H.: Phase-diversity correction of turbulence-induced space-variant blur. Opt. Lett. 19(16), 1231–1233 (1994)

    Article  Google Scholar 

  33. Pennec, X.: Probabilities and Statistics on Riemannian Manifolds: a Geometric Approach. Research Report 5093, INRIA, January 2004

  34. Roddier, F.: Adaptive Optics in Astronomy. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  35. Roggemann, M.C., Stoudt, C.A., Welsh, B.M.: Image-spectrum signal-to-noise-ratio improvements by statistical frame selection for adaptive-optics imaging through atmospheric turbulence. Opt. Eng. 33(10), 3254–3264 (1994)

    Article  Google Scholar 

  36. Roggemann, M.C., Welsh, B.M.: Imaging Through Turbulence. CRC Press, Boca Raton (1996)

    Google Scholar 

  37. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MATH  Google Scholar 

  38. Shimizu, M., Yoshimura, S., Tanaka, M., Okutomi, M.: Super-resolution from image sequence under influence of hot-air optical turbulence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, Alaska, June 2008

    Google Scholar 

  39. Shumway, R.H., Stoffer, D.S.: An approach to time series smoothing and forecasting using the EM algorithm. J. Time Ser. Anal. 3(4), 253–264 (1982)

    Article  MATH  Google Scholar 

  40. Stone, J., Hu, P.H., Mills, S.P., Ma, S.: Anisoplanatic effects in finite-aperture optical systems. J. Opt. Soc. Am. A 11(1), 347–357 (1994)

    Article  Google Scholar 

  41. Tahtali, M., Lambert, A., Fraser, D.: Self-tuning Kalman filter estimation of atmospheric warp. In: SPIE Image Reconstruction from Incomplete Data, San Diego, CA, Aug. 2008

    Google Scholar 

  42. Tai, Y.-W., Tan, P., Brown, M.S.: Richardson-Lucy deblurring for scenes under a projective motion path. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1603–1618 (2011)

    Article  Google Scholar 

  43. Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problems, Winston, Washington DC (1977)

    Google Scholar 

  44. Van Overschee, P., De Moor, B.: N4sid: subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica 30(1), 75–93 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  45. Welsh, B.M., Gardner, C.S.: Effects of turbulence-induced anisoplanatism on the imaging performance of adaptive-astronomical telescopes using laser guide stars. J. Opt. Soc. Am. A 8(1), 69–81 (1991)

    Article  Google Scholar 

  46. Younes, L.: Shapes and Diffeomorphisms. Applied Mathematical Sciences, vol. 171. Springer, Berlin (2010)

    Book  MATH  Google Scholar 

  47. Zhu, X., Milanfar, P.: Image reconstruction from videos distorted by atmospheric turbulence. In: SPIE Electronic Imaging, San Jose, CA, Jan. 2010

    Google Scholar 

  48. Zhu, X., Milanfar, P.: Stabilizing and deblurring atmospheric turbulence. In: Proceedings of the of the IEEE International Conference on Computational Photography (ICCP 2011), Carnegie Mellon University, Pittsburgh (2011)

    Google Scholar 

Download references

Acknowledgements

Mario Micheli’s research was partially supported by ONR grant N000140910256. Yifei Lou and Andrea Bertozzi were partially supported by ONR grants N00014101022, N000141210040 and by NSF grants DMS-0914856, DMS-1118971. Stefano Soatto was partially supported by ONR grant N0001411100863. The authors would like to thank Dr. Alan Van Nevel at the U.S. Naval Air Warfare Center (China Lake, California) for providing the image data. We are also deeply grateful to Stanley Osher and Jérôme Gilles of UCLA, and to Angelo Cenedese of Università di Padova (Italy) for the insightful discussions on the topic. We would also like to thank Xiaoqun Zhang, formerly at UCLA and now at Shanghai Jiao Tong University, for providing the nonlocal Total Variation (NL-TV) deconvolution Matlab code.

Author information

Authors and Affiliations

  1. Department of Mathematics, UCLA, Los Angeles, CA, 90095-1555, USA

    Mario Micheli & Andrea L. Bertozzi

  2. School of Electrical and Computer Engineering, Georgia Inst. of Technology, Atlanta, GA, 30332, USA

    Yifei Lou

  3. Department of Computer Science, UCLA, Los Angeles, CA, 90095-1596, USA

    Stefano Soatto

Authors
  1. Mario Micheli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yifei Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Soatto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrea L. Bertozzi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mario Micheli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Micheli, M., Lou, Y., Soatto, S. et al. A Linear Systems Approach to Imaging Through Turbulence. J Math Imaging Vis 48, 185–201 (2014). https://doi.org/10.1007/s10851-012-0410-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-012-0410-7

Keywords

Search

Navigation