Skip to main content
Top
Published in: Journal of Scientific Computing 3/2019

01-10-2018

Variational Models for Joint Subsampling and Reconstruction of Turbulence-Degraded Images

Authors: Chun Pong Lau, Yu Hin Lai, Lok Ming Lui

Published in: Journal of Scientific Computing | Issue 3/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Turbulence-degraded image frames are distorted by both turbulent deformations and space–time-varying blurs. To suppress these effects, we propose a multi-frame reconstruction scheme to recover a latent image from the observed distorted image sequence. Recent approaches are commonly based on registering each frame to a reference image, by which geometric turbulent deformations can be estimated and a sharp image can be restored. A major challenge is that a fine reference image is usually unavailable, as every turbulence-degraded frame is distorted. A high-quality reference image is crucial for the accurate estimation of geometric deformations and fusion of frames. Besides, it is unlikely that all frames from the image sequence are useful, and thus frame selection is necessary and highly beneficial. In this work, we propose a variational model for joint subsampling of frames and extraction of a clear image. A fine image and a suitable choice of subsample are simultaneously obtained by iteratively reducing an energy functional. The energy consists of a fidelity term measuring the discrepancy between the extracted image and the subsampled frames, as well as regularization terms on the extracted image and the subsample. Different choices of fidelity and regularization terms are explored. By carefully selecting suitable frames and extracting the image, the quality of the reconstructed image can be significantly improved. Extensive experiments have been carried out, which demonstrate the efficacy of our proposed model. In addition, the extracted subsamples and images can be put in existing algorithms to produce improved results.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Anantrasirichai, N., Achim, A., Kingsbury, N.G., Bull, D.R.: Atmospheric turbulence mitigation using complex wavelet-based fusion. IEEE Trans. Image Process. 22(6), 2398–2408 (2013)MathSciNetCrossRefMATH Anantrasirichai, N., Achim, A., Kingsbury, N.G., Bull, D.R.: Atmospheric turbulence mitigation using complex wavelet-based fusion. IEEE Trans. Image Process. 22(6), 2398–2408 (2013)MathSciNetCrossRefMATH
2.
go back to reference Aubailly, M., Vorontsov, M.A., Carhart, G.W., Valley, M.T.: Automated video enhancement from a stream of atmospherically-distorted images: the lucky-region fusion approach. Proc. SPIE 7463, 74630C (2009)CrossRef Aubailly, M., Vorontsov, M.A., Carhart, G.W., Valley, M.T.: Automated video enhancement from a stream of atmospherically-distorted images: the lucky-region fusion approach. Proc. SPIE 7463, 74630C (2009)CrossRef
4.
go back to reference Frakes, D.H., Monaco, J.W., Smith, M.J.T.: Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach. In: 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221), vol. 3, pp. 1881–1884 vol. 3 (2001). https://doi.org/10.1109/ICASSP.2001.941311 Frakes, D.H., Monaco, J.W., Smith, M.J.T.: Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach. In: 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221), vol. 3, pp. 1881–1884 vol. 3 (2001). https://​doi.​org/​10.​1109/​ICASSP.​2001.​941311
5.
go back to reference Fried, D.L.: Probability of getting a lucky short-exposure image through turbulence. JOSA 68(12), 1651–1658 (1978)CrossRef Fried, D.L.: Probability of getting a lucky short-exposure image through turbulence. JOSA 68(12), 1651–1658 (1978)CrossRef
6.
go back to reference Furhad, M.H., Tahtali, M., Lambert, A.: Restoring atmospheric-turbulence-degraded images. Appl. Opt. 55(19), 5082–5090 (2016)CrossRef Furhad, M.H., Tahtali, M., Lambert, A.: Restoring atmospheric-turbulence-degraded images. Appl. Opt. 55(19), 5082–5090 (2016)CrossRef
8.
go back to reference He, R., Wang, Z., Fan, Y., Fengg, D.: Atmospheric turbulence mitigation based on turbulence extraction. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1442–1446. IEEE (2016) He, R., Wang, Z., Fan, Y., Fengg, D.: Atmospheric turbulence mitigation based on turbulence extraction. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1442–1446. IEEE (2016)
9.
go back to reference Hirsch, M., Sra, S., Schölkopf, B., Harmeling, S.: Efficient filter flow for space-variant multiframe blind deconvolution. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 607–614. IEEE (2010) Hirsch, M., Sra, S., Schölkopf, B., Harmeling, S.: Efficient filter flow for space-variant multiframe blind deconvolution. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 607–614. IEEE (2010)
10.
go back to reference Hufnagel, R., Stanley, N.: Modulation transfer function associated with image transmission through turbulent media. JOSA 54(1), 52–61 (1964)CrossRef Hufnagel, R., Stanley, N.: Modulation transfer function associated with image transmission through turbulent media. JOSA 54(1), 52–61 (1964)CrossRef
12.
go back to reference Li, D., Mersereau, R.M., Simske, S.: Atmospheric turbulence-degraded image restoration using principal components analysis. IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007)CrossRef Li, D., Mersereau, R.M., Simske, S.: Atmospheric turbulence-degraded image restoration using principal components analysis. IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007)CrossRef
13.
go back to reference Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055 (2010) Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:​1009.​5055 (2010)
14.
go back to reference Liu, C.: Beyond pixels: Exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009) Liu, C.: Beyond pixels: Exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009)
15.
go back to reference Lou, Y., Kang, S.H., Soatto, S., Bertozzi, A.L.: Video stabilization of atmospheric turbulence distortion. Inverse Probl. Imag. 7(3), 839–861 (2013)MathSciNetCrossRefMATH Lou, Y., Kang, S.H., Soatto, S., Bertozzi, A.L.: Video stabilization of atmospheric turbulence distortion. Inverse Probl. Imag. 7(3), 839–861 (2013)MathSciNetCrossRefMATH
16.
go back to reference Mao, Y., Gilles, J.: Non rigid geometric distortions correction–application to atmospheric turbulence stabilization. Inverse Probl. Imag. 6, 531–546 (2012) Mao, Y., Gilles, J.: Non rigid geometric distortions correction–application to atmospheric turbulence stabilization. Inverse Probl. Imag. 6, 531–546 (2012)
17.
go back to reference Meinhardt-Llopis, E., Micheli, M.: Implementation of the centroid method for the correction of turbulence. Image Process. On Line 4, 187–195 (2014)CrossRef Meinhardt-Llopis, E., Micheli, M.: Implementation of the centroid method for the correction of turbulence. Image Process. On Line 4, 187–195 (2014)CrossRef
18.
go back to reference Micheli, M., Lou, Y., Soatto, S., Bertozzi, A.L.: A linear systems approach to imaging through turbulence. J. Math. Imaging Vis. 48(1), 185–201 (2014)MathSciNetCrossRefMATH Micheli, M., Lou, Y., Soatto, S., Bertozzi, A.L.: A linear systems approach to imaging through turbulence. J. Math. Imaging Vis. 48(1), 185–201 (2014)MathSciNetCrossRefMATH
19.
go back to reference Pearson, J.E.: Atmospheric turbulence compensation using coherent optical adaptive techniques. Appl. Opt. 15(3), 622–631 (1976)CrossRef Pearson, J.E.: Atmospheric turbulence compensation using coherent optical adaptive techniques. Appl. Opt. 15(3), 622–631 (1976)CrossRef
20.
go back to reference Roggemann, M.C., Stoudt, C.A., Welsh, B.M.: Image-spectrum signal-to-noise-ratio improvements by statistical frame selection for adaptive-optics imaging through atmospheric turbulence. Opt. Eng. 33(10), 3254–3265 (1994)CrossRef Roggemann, M.C., Stoudt, C.A., Welsh, B.M.: Image-spectrum signal-to-noise-ratio improvements by statistical frame selection for adaptive-optics imaging through atmospheric turbulence. Opt. Eng. 33(10), 3254–3265 (1994)CrossRef
21.
go back to reference Roggemann, M.C., Welsh, B.M., Hunt, B.R.: Imaging Through Turbulence. CRC Press, Boca Raton (1996) Roggemann, M.C., Welsh, B.M., Hunt, B.R.: Imaging Through Turbulence. CRC Press, Boca Raton (1996)
22.
go back to reference Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1–4), 259–268 (1992)MathSciNetCrossRefMATH Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1–4), 259–268 (1992)MathSciNetCrossRefMATH
23.
go back to reference Seitz, S.M., Baker, S.: Filter flow. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 143–150. IEEE (2009) Seitz, S.M., Baker, S.: Filter flow. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 143–150. IEEE (2009)
25.
go back to reference Shimizu, M., Yoshimura, S., Tanaka, M., Okutomi, M.: Super-resolution from image sequence under influence of hot-air optical turbulence. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008) Shimizu, M., Yoshimura, S., Tanaka, M., Okutomi, M.: Super-resolution from image sequence under influence of hot-air optical turbulence. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)
26.
go back to reference Sobral, A., Bouwmans, T., Zahzah, E.H.: LRSLibrary: low-rank and sparse tools for background modeling and subtraction in videos. In: Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing. CRC Press, Taylor and Francis Group (2015) Sobral, A., Bouwmans, T., Zahzah, E.H.: LRSLibrary: low-rank and sparse tools for background modeling and subtraction in videos. In: Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing. CRC Press, Taylor and Francis Group (2015)
27.
go back to reference Tyson, R.K.: Principles of Adaptive Optics. CRC Press, Boca Raton (2015) Tyson, R.K.: Principles of Adaptive Optics. CRC Press, Boca Raton (2015)
28.
go back to reference Vorontsov, M.A.: Parallel image processing based on an evolution equation with anisotropic gain: integrated optoelectronic architectures. JOSA A 16(7), 1623–1637 (1999)CrossRef Vorontsov, M.A.: Parallel image processing based on an evolution equation with anisotropic gain: integrated optoelectronic architectures. JOSA A 16(7), 1623–1637 (1999)CrossRef
29.
go back to reference Vorontsov, M.A., Carhart, G.W.: Anisoplanatic imaging through turbulent media: image recovery by local information fusion from a set of short-exposure images. JOSA A 18(6), 1312–1324 (2001)CrossRef Vorontsov, M.A., Carhart, G.W.: Anisoplanatic imaging through turbulent media: image recovery by local information fusion from a set of short-exposure images. JOSA A 18(6), 1312–1324 (2001)CrossRef
30.
go back to reference Xie, Y., Zhang, W., Tao, D., Hu, W., Qu, Y., Wang, H.: Removing turbulence effect via hybrid total variation and deformation-guided kernel regression. IEEE Trans. Image Process. 25(10), 4943–4958 (2016)MathSciNetCrossRefMATH Xie, Y., Zhang, W., Tao, D., Hu, W., Qu, Y., Wang, H.: Removing turbulence effect via hybrid total variation and deformation-guided kernel regression. IEEE Trans. Image Process. 25(10), 4943–4958 (2016)MathSciNetCrossRefMATH
31.
go back to reference Zhu, X., Milanfar, P.: Removing atmospheric turbulence via space-invariant deconvolution. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 157–170 (2013)CrossRef Zhu, X., Milanfar, P.: Removing atmospheric turbulence via space-invariant deconvolution. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 157–170 (2013)CrossRef
Metadata
Title
Variational Models for Joint Subsampling and Reconstruction of Turbulence-Degraded Images
Authors
Chun Pong Lau
Yu Hin Lai
Lok Ming Lui
Publication date
01-10-2018
Publisher
Springer US
Published in
Journal of Scientific Computing / Issue 3/2019
Print ISSN: 0885-7474
Electronic ISSN: 1573-7691
DOI
https://doi.org/10.1007/s10915-018-0833-4

Other articles of this Issue 3/2019

Journal of Scientific Computing 3/2019 Go to the issue

Premium Partner