nach oben

Erschienen in:

2017 | OriginalPaper | Buchkapitel

End-to-End Learning for Image Burst Deblurring

verfasst von : Patrick Wieschollek, Bernhard Schölkopf, Hendrik P. A. Lensch, Michael Hirsch

Erschienen in: Computer Vision – ACCV 2016

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We present a neural network model approach for multi-frame blind deconvolution. The discriminative approach adopts and combines two recent techniques for image deblurring into a single neural network architecture. Our proposed hybrid-architecture combines the explicit prediction of a deconvolution filter and non-trivial averaging of Fourier coefficients in the frequency domain. In order to make full use of the information contained in all images in one burst, the proposed network embeds smaller networks, which explicitly allow the model to transfer information between images in early layers. Our system is trained end-to-end using standard backpropagation on a set of artificially generated training examples, enabling competitive performance in multi-frame blind deconvolution, both with respect to quality and runtime.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Sparse Coding on Cascaded Residuals

Nächstes Kapitel Spectral Reflectance Recovery with Interreflection Using a Hyperspectral Image

Nur mit Berechtigung zugänglich

Provided by [29].

http://webdav.is.mpg.de/pixel/benchmark4camerashake/

Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: can plain neural networks compete with BM3D? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2012)

Schuler, C., Burger, H., Harmeling, S., Scholköpf, B.: A machine learning approach for non-blind image deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2013)

Delbracio, M., Sapiro, G.: Burst deblurring: removing camera shake through fourier burst accumulation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2015)

Wang, R., Tao, D.: Recent progress in image deblurring. arXiv preprint (2014). arXiv:1409.6838

Sun, L., Cho, S., Wang, J., Hays, J.: Edge-based blur kernel estimation using patch priors. In: IEEE International Conference in Computational Photography (ICCP), pp. 1–8. IEEE (2013)

Michaeli, T., Irani, M.: Blind deblurring using internal patch recurrence. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 783–798. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10578-9_51

Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q., (eds.): Advances in Neural Information Processing Systems (NIPS), pp. 1790–1798. Curran Associates, Inc. (2014)

Rosenbaum, D., Weiss, Y.: The return of the gating network: combining generative models and discriminative training in natural image priors. In: Advances in Neural Information Processing Systems (NIPS), pp. 2665–2673 (2015)

Schuler, C.J., Hirsch, M., Harmeling, S., Schölkopf, B.: Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence (2015)

10.

Sun, J., Cao, W., Xu, Z., Ponce, J.: Learning a convolutional neural network for non-uniform motion blur removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2015)

11.

Chakrabarti, A.: A neural approach to blind motion deblurring. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 221–235. Springer, Heidelberg (2016). doi:10.1007/978-3-319-46487-9_14 CrossRef

12.

Hradiš, M., Kotera, J., Zemcík, P., Šroubek, F.: Convolutional neural networks for direct text deblurring. In: Proceedings of BMVC, vol. 10 (2015)

13.

Svoboda, P., Hradi, M., Mark, L., Zemck, P.: CNN for license plate motion deblurring. In: IEEE International Conference on Image Processing (ICIP), pp. 3832–3836 (2016)

14.

Loktyushin, A., Schuler, C., Scheffler, K., Schölkopf, B.: Retrospective motion correction of magnitude-input MR images. In: Bhatia, K.K., Lombaert, H. (eds.) MLMMI 2015. LNCS, vol. 9487, pp. 3–12. Springer, Cham (2015). doi:10.1007/978-3-319-27929-9_1 CrossRef

15.

Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 479–486. IEEE Computer Society (2011)

16.

Hasinoff, S.W., Kutulakos, K.N., Durand, F., Freeman, W.T.: Time-constrained photography. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 333–340. IEEE (2009)

17.

Rav-Acha, A., Peleg, S.: Two motion-blurred images are better than one. Pattern Recogn. Lett. 26, 311–317 (2005)CrossRef

18.

Zhang, H., Wipf, D.P., Zhang, Y.: Multi-observation blind deconvolution with an adaptive sparse prior. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1628–1643 (2014)CrossRef

19.

Chen, J., Yuan, L., Tang, C.K., Quan, L.: Robust dual motion deblurring. In: IEEE Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE Computer Society (2008)

20.

Cai, J.F., Ji, H., Liu, C., Shen, Z.: Blind motion deblurring using multiple images. J. Comput. Phys. 228, 5057–5071 (2009)MathSciNetCrossRefMATH

21.

Šroubek, F., Milanfar, P.: Robust multichannel blind deconvolution via fast alternating minimization. IEEE Trans. Image Process. 21, 1687–1700 (2012)MathSciNetCrossRef

22.

Zhu, X., Šroubek, F., Milanfar, P.: Deconvolving PSFs for a better motion deblurring using multiple images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 636–647. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33715-4_46 CrossRef

23.

Zhang, H., Carin, L.: Multi-shot imaging: joint alignment, deblurring and resolution-enhancement. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2014)

24.

Zhang, H., Yang, J.: Intra-frame deblurring by leveraging inter-frame camera motion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2015)

25.

Kim, T.H., Nah, S., Lee, K.M.: Dynamic scene deblurring using a locally adaptive linear blur model. arXiv preprint (2016). arXiv:1603.04265

26.

Ito, A., Sankaranarayanan, A.C., Veeraraghavan, A., Baraniuk, R.G.: BlurBurst: removing blur due to camera shake using multiple images. ACM Trans. Graph. 3 (2014). Submitted

27.

Garrel, V., Guyon, O., Baudoz, P.: A highly efficient lucky imaging algorithm: image synthesis based on fourier amplitude selection. Publ. Astron. Soc. Pac. 124, 861–867 (2012)CrossRef

28.

Delbracio, M., Sapiro, G.: Hand-held video deblurring via efficient fourier aggregation. IEEE Trans. Comput. Imaging 1, 270–283 (2015)MathSciNetCrossRef

29.

Lin, T., Maire, M., Belongie, S.J., Bourdev, L.D., Girshick, R.B., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. arXiv preprint (2014). arXiv:1405.0312

30.

Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint (2014). arXiv:1412.6980

31.

Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)MathSciNetMATH

32.

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). Software. tensorflow.org

33.

Zhang, H., Wipf, D., Zhang, Y.: Multi-image blind deblurring using a coupled adaptive sparse prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1051–1058. IEEE Computer Society (2013)

34.

Köhler, R., Hirsch, M., Mohler, B., Schölkopf, B., Harmeling, S.: Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 27–40. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33786-4_3 CrossRef

35.

Hirsch, M., Sra, S., Schölkopf, B., Harmeling, S.: Efficient filter flow for space-variant multiframe blind deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2010)

36.

Pătrăucean, V., Handa, A., Cipolla, R.: Spatio-temporal video autoencoder with differentiable memory. arXiv preprint (2015). arXiv:1511.06309

37.

Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 2008–2016 (2015)

Titel: End-to-End Learning for Image Burst Deblurring
verfasst von: Patrick Wieschollek
Bernhard Schölkopf
Hendrik P. A. Lensch
Michael Hirsch
Verlag: Springer International Publishing
Buch: Computer Vision – ACCV 2016
Print ISBN: 978-3-319-54189-1

Electronic ISBN: 978-3-319-54190-7

Copyright-Jahr: 2017
DOI: https://doi.org/10.1007/978-3-319-54190-7_3

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner