Skip to main content
SpringerLink
Log in

Real-time stabilization of long range observation system turbulent video

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The paper presents a real-time algorithm that compensates image distortions due to atmospheric turbulence in video sequences, while keeping the real moving objects in the video unharmed. The algorithm involves (1) generation of a “reference” frame, (2) estimation, for each incoming video frame, of a local image displacement map with respect to the reference frame, (3) segmentation of the displacement map into two classes: stationary and moving objects; (4) turbulence compensation of stationary objects. Experiments with both simulated and real-life sequences have shown that the restored videos, generated in real-time using standard computer hardware, exhibit excellent stability for stationary objects while retaining real motion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Roggermann, M.C., Welsh, B.: Imaging Through Turbulence, Chap. 3, CRC Press, USA, pp. 57–115 (1996)

  2. Farmer, W.M.: The Atmospheric Filter, vol. I Sources. JCD Publishing, (2001)

  3. Welsh, B.M., Gardner, C.S.: Performance analysis of adaptive optics systems using slope sensors. J. Opt. Soc. Am. A 6:1913–1923 (1989)

    Google Scholar 

  4. Thorpe, G., Lambert, A., Fraser, D.: Atmospheric turbulence visualization through image time-sequence registration. In: Proceedings International Conference on Pattern Recognition, vol. 2, pp. 1768–1770. IEEE Computer Society, Brisbane (1998)

  5. Ellerbroek, B.: First-order performance evaluation of adaptive-optics systems for atmospheric-turbulence compensation in extended-field-of-view astronomical telescopes. J. Opt. Soc. Am. A, 11(2):783 (1994)

    Article  MathSciNet  Google Scholar 

  6. Farmer, W.M.: The Atmospheric Filter, vol. II Sources. JCD Publishing, Winter Park (2001)

  7. Sheppard, D.G., Hunt, B.R., Marcellin, M.W.: Iterative multiframe super-resolution algorithms for atmospheric turbulence-degraded imagery. J. Opt. Soc. Am. A, 15(4):972–992 (1998)

    Article  Google Scholar 

  8. Sadot, D., Kopeika, N.: Imaging through the atmosphere: practical instrumentation-based theory and verification of aerosol modulation transfer function. J. Opt. Soc. Am. A, 10(1):172 (1993)

    Google Scholar 

  9. Cohen, B., Avrin, V., Belitsky, M., Dinstein, I.: Generation of a restored image from a videosequence recorded under turbulence effects. Opt. Eng. 36(12):3312–3317 (1997)

    Article  Google Scholar 

  10. Frieden, B.R.: Turbulent image reconstruction using object power spectrum information. Opt. Commun. 109(3–4): 227–230 (1994)

    Article  Google Scholar 

  11. Wang, Y., Frieden, B.R.: Minimum entropy-neural network approach to turbulent-image reconstruction. Appl. Opt. 34(26):5938–5944 (1995)

    Google Scholar 

  12. Mohammed, A.T., Burge, R.E.: Short-exposure turbulent image reconstructions. J. Phys. D Appl. Phys. 21(7):1067–1077 (1988)

    Article  Google Scholar 

  13. David, H.F., Joseph, W.M., Mark J.T.S.: Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach. In: Wee, S.J., Apostolopoulos J.G. (eds.) IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP01), vol. 3, pp. 1881–1884. IEEE International Society, Salt Lake City (2001)

  14. van der Elst, H., van Schalkwyk, J.J.D.: Modelling and restoring images distorted by atmospheric turbulence. South African Symposium on Communications and Signal Processing (COMSIG-94), pp. 162–167. IEEE International Society, Stellenbosch (1994)

  15. Gepshtein, Sh., Shteinman, A., Fishbain, B., Yaroslavsky, L.: Restoration of atmospheric turbulent video containing real motion using elastic image registration. In: The 2004 European Signal Processing Conference (EUSIPCO-2004), John Platt, pp. 477–480. Vienna, Austria, (2004)

  16. Yaroslavsky, L., Fishbain, B., Shteinman, A., Gepshtein, Sh.: Processing and fusion of thermal and video sequences for terrestrial long range observation systems. In: Johan, S (ed.) The 7th International Conference on Information Fusion, pp. 848–855, International Society of Information Fusion, Stockholm, Sweden, June 2004

  17. Yaroslavsky, L.P., Fishbain, B., Ideses, I., Slasky, D., Hadas, Z.: Simple methods for real-time stabilization of turbulent video. In: Calvo, M.L., Pavlov, A.V., Jahns, J. (eds.) Proceeding of ICO topical meeting on optoinformatics/information photonics, ITMO, pp. 138–140. St Petersburg, Russia (2006)

  18. Fishbain, B., Yaroslavsky, L.P., Ideses, I.A., Shtern, A., Ben-Zvi, O.: Real-time stabilization of long-range observation system turbulent video. In: Proceedings of Real-Time Image Processing/Electronic Imaging 2007, SPIE Vol. 6496, San-Jose, CA, USA, 28 January–1 February 2007

  19. Yaroslavsky, L.P.: Digital Holography and Digital Image Processing. Kluwer, Boston (2003)

    Google Scholar 

  20. Bondeau, C., Bourennane, E.: Restoration of images degraded by the atmospheric turbulence. In: Proceedings of the 4th International Conference on Signal Processing (ICSP), vol.2, pp. 1056–1059. Beijing China (1998)

  21. Glick, Y., Baram, A., Loebenstein, H.M., Azar Z:. Restoration of turbulence-degraded images by the most-common method. Appl. Opt. 30(27):3924–3929 (1991)

    Google Scholar 

  22. Cheung, S.C., Kamath, C.: Robust techniques for background subtraction in urban traffic video. Video Communications and Image Processing, SPIE Electronic Imaging, San Jose (2004)

    Google Scholar 

  23. http://www.eng.tau.ac.il/∼barak/RealTimeTurbulenceCompensation

  24. Huang, T.S., Yang, G.J., Yang, G.Y.: A Fast Two-Dimensional Median Filtering Algorithm. IEEE Trans. Acoust. Speech Signal Process. ASSP-27, 13 (1979)

    Google Scholar 

  25. Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. J Imaging Vis. 14(3), 245–255 (2001)

    Article  MATH  Google Scholar 

  26. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings Of the 7th International Joint Conference on Artificial Intelligence (IJCAI), pp. 674–679. Vancouver BC (1981)

  27. Horn, B., Schunck, B.: Determining optical flow. Art. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  28. Mitiche, A., Bouthemy, P.: Computation and analysis of image motion: a synopsis of current problems and methods. Int. J. Comput. Vis. 19(1), 29–55 (1996)

    Article  Google Scholar 

  29. Barron, L.J., Fleet, D.J., Beachemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)

    Article  Google Scholar 

  30. Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacement. Int. J. Comput. Vis. 39(1), 41–56 (2000)

    Article  MATH  Google Scholar 

  31. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation based on Theory for Wrapping. European Conference on Computer Vision (ECCV04), Vaclav Hlavac, vol. 4, pp. 25–36. Springer LNCS, Prague Czech Republic (2004)

  32. Ben-Ari, R., Sochen, N.: A general framework for regularization in PDE based computation of optical flow via embedded maps and minimal surfaces. In: Proceedings of IEEE Computer Vision and Pattern Recognition Conference, Dan Huttenlocher and David Forsyth, pp. 529–536. IEEE Computer Society, New-York (2006)

  33. Nagel, H.H., Enkelman, W.: An investigation of smoothness constrains for estimation of displacement vector fields from image sequences. IEEE Trans. Patt. Anal. Mach. Intell. 8(5), 565–593 (1986)

    Google Scholar 

  34. Deriche, R., Kornprobst, P., Aubert, G.: Optical flow estimation while preserving its discontinuities: a variational approach. Lecture Notes in Computer Science, vol. 1035. pp. 71–80 (1996)

  35. Yaroslavsky, L.P., Eden, M.: Fundamental of Digital Optics. Birkhäuser, Boston (1996)

    Google Scholar 

  36. Stefan W.: Digital Video Quality—Vision Models and Metrics. Wiley, New York (2005)

  37. Recommendation H.261 of the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T), Line Transmission of Non-Telephone Signals—Video Codec for Audiovisual Services at p x 64 kb (1993)

Download references

Acknowledgments

The authors appreciate the contribution to this research of Ofer Ben-Zvi and Alon Shtern, Faculty of Engineering, Tel-Aviv University, for their useful suggestions and their help with the C++ implementation of the algorithm. The video database was acquired with the kind help of Elbit Systems Electro-Optics—ELOP Ltd, Israel and the Israeli Army R&D branch.

Author information

Authors and Affiliations

  1. Department of Interdisciplinary Studies, the Iby and Aladar Fleischman Faculty of Engineering, Tel-Aviv University, Tel Aviv, 69978, Israel

    Barak Fishbain, Leonid P. Yaroslavsky & Ianir A. Ideses

Authors
  1. Barak Fishbain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leonid P. Yaroslavsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ianir A. Ideses
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barak Fishbain.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fishbain, B., Yaroslavsky, L.P. & Ideses, I.A. Real-time stabilization of long range observation system turbulent video. J Real-Time Image Proc 2, 11–22 (2007). https://doi.org/10.1007/s11554-007-0037-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-007-0037-x

Keywords

Search

Navigation