Skip to main content
Erschienen in: Neural Processing Letters 1/2021

13.11.2020

Context Module Based Multi-patch Hierarchical Network for Motion Deblurring

verfasst von: Kang Tang, Dahong Xu, Hong Liu, Zhixin Zeng

Erschienen in: Neural Processing Letters | Ausgabe 1/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Single image blind motion deblurring refers to transferring a blurred motion image into a corresponding clear image, which is a challenging and classic problem in the field of computer vision. The spatially variant blur is usually caused by many factors, such as camera jitter and object motion. Since image deblurring can be regarded as the task of image transformation, deep learning methods based on coarse-to-fine scheme, especially those using multi-scale architectures become popular. However, they have the disadvantages of unsatisfactory image quality and time-consuming running caused by large kernel size. In this paper, we propose a novel end-to-end network structure based on Deep Hierarchical Multi-patch network architecture integrated with Context Module and additional ResBlocks in order to tackle deblurring problem. Compared with recently proposed networks, it generates images with better visual effect as well as higher image quality index. Besides, our model significantly reduces the test time. We evaluate the proposed network structure on public GoPro dataset, a large-scale image dataset with complex synthetic blur. The experiments on the benchmark dataset prove that our effective method outperforms other state-of-the-art blind deblurring algorithms both qualitatively and quantitatively, which demonstrates the effectiveness of Context Module in the task of single image blur removal.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Hyun Kim T, Ahn B, Mu Lee K (2013) Dynamic scene deblurring. In: ICCV, pp 3160–3167 Hyun Kim T, Ahn B, Mu Lee K (2013) Dynamic scene deblurring. In: ICCV, pp 3160–3167
2.
Zurück zum Zitat Sun J, Cao W, Xu Z, Ponce J (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: CVPR, pp 769–777 Sun J, Cao W, Xu Z, Ponce J (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: CVPR, pp 769–777
3.
Zurück zum Zitat Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 257–265 Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 257–265
4.
Zurück zum Zitat Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: CVPR, pp 8174–8182 Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: CVPR, pp 8174–8182
5.
Zurück zum Zitat Mei J, Wu Z, Chen X et al (2019) DeepDeblur: text image recovery from blur to sharp. Multim Tools Appl 78(13):18869–18885CrossRef Mei J, Wu Z, Chen X et al (2019) DeepDeblur: text image recovery from blur to sharp. Multim Tools Appl 78(13):18869–18885CrossRef
7.
Zurück zum Zitat Ren W, Zhang J, Ma L, Pan J, Cao X, Zuo W, LiuW, Yang M-S (2018) Deep non-blind deconvolution via generalized low-rank approximation. In: Advances in neural information processing systems, pp 295–305 Ren W, Zhang J, Ma L, Pan J, Cao X, Zuo W, LiuW, Yang M-S (2018) Deep non-blind deconvolution via generalized low-rank approximation. In: Advances in neural information processing systems, pp 295–305
8.
Zurück zum Zitat Xiangyu Xu, Jinshan Pan, Yu-Jin Zhang, and Ming-HsuanYang. Motion blur kernel estimation via deep learning. IEEE Transactions on Image Processing, 27(1):194–205, 2018 Xiangyu Xu, Jinshan Pan, Yu-Jin Zhang, and Ming-HsuanYang. Motion blur kernel estimation via deep learning. IEEE Transactions on Image Processing, 27(1):194–205, 2018
9.
Zurück zum Zitat Li L, Pan J, Lai W-S, Gao X, Sang N, Yang M-H (2018) Learning a discriminative prior for blind image deblurring. In: The IEEE conference on computer vision and pattern recognition (CVPR), June 2018 Li L, Pan J, Lai W-S, Gao X, Sang N, Yang M-H (2018) Learning a discriminative prior for blind image deblurring. In: The IEEE conference on computer vision and pattern recognition (CVPR), June 2018
10.
Zurück zum Zitat Chang C-F, Wu J-L (2014) A new single image deblurring algorithm using hyper laplacian priors. In: ICS, pp 1015–1022 Chang C-F, Wu J-L (2014) A new single image deblurring algorithm using hyper laplacian priors. In: ICS, pp 1015–1022
11.
Zurück zum Zitat Xu L, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring. In: CVPR Xu L, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring. In: CVPR
12.
Zurück zum Zitat Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: CVPR Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: CVPR
13.
Zurück zum Zitat Cao Y, Wang S, Guo Z et al (2017) Synchronization of memristive neural networks with leakage delay and parameters mismatch via event-triggered control. Neural Netw 2019:119 Cao Y, Wang S, Guo Z et al (2017) Synchronization of memristive neural networks with leakage delay and parameters mismatch via event-triggered control. Neural Netw 2019:119
14.
Zurück zum Zitat Wang Shengbo, Cao Yuting, Huang Tingwen, Chen Yiran, Wen Shiping (2020) Event-triggered distributed control for synchronization of multiple memristive neural networks under cyber-physical attacks. Inf Sci 518:361–375MathSciNetCrossRef Wang Shengbo, Cao Yuting, Huang Tingwen, Chen Yiran, Wen Shiping (2020) Event-triggered distributed control for synchronization of multiple memristive neural networks under cyber-physical attacks. Inf Sci 518:361–375MathSciNetCrossRef
15.
Zurück zum Zitat Xu L, Ren JSJ, Liu C, Jia J (2014) Deep convolutional neural network for image deconvolution. In: Proceedings of advance neural information processing system, pp 1790–1798 Xu L, Ren JSJ, Liu C, Jia J (2014) Deep convolutional neural network for image deconvolution. In: Proceedings of advance neural information processing system, pp 1790–1798
16.
Zurück zum Zitat Gong D, Yang J, Liu L, Zhang Y, Reid ID, Shen C, Van Den Hengel A, Shi Q (2017) From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. In: CVPR, 2017 Gong D, Yang J, Liu L, Zhang Y, Reid ID, Shen C, Van Den Hengel A, Shi Q (2017) From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. In: CVPR, 2017
17.
Zurück zum Zitat Schuler CJ, Hirsch M, Harmeling S, Schölkopf B (2016) Learning to deblur. IEEE Trans Pattern Anal Mach Intell 38(7):1439–1451CrossRef Schuler CJ, Hirsch M, Harmeling S, Schölkopf B (2016) Learning to deblur. IEEE Trans Pattern Anal Mach Intell 38(7):1439–1451CrossRef
18.
Zurück zum Zitat Chakrabarti A (2016) A neural approach to blind motion deblurring. In: ECCV. Springer, pp 221–235 Chakrabarti A (2016) A neural approach to blind motion deblurring. In: ECCV. Springer, pp 221–235
19.
Zurück zum Zitat Gao H, Tao X, Shen X, Jia J (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. CVPR 2019, pp 3848–3856 Gao H, Tao X, Shen X, Jia J (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. CVPR 2019, pp 3848–3856
20.
Zurück zum Zitat Ramakrishnan S, Pachori S, Gangopadhyay A, Raman S (2017) Deep generative filter for motion deblurring. In: 2017 IEEE international conference on computer vision workshops (ICCVW), pp 2993–3000 Ramakrishnan S, Pachori S, Gangopadhyay A, Raman S (2017) Deep generative filter for motion deblurring. In: 2017 IEEE international conference on computer vision workshops (ICCVW), pp 2993–3000
21.
Zurück zum Zitat Isola P, Zhu J-Y, Zhou T (2016) Image-to-image translation with conditional adversarial networks Isola P, Zhu J-Y, Zhou T (2016) Image-to-image translation with conditional adversarial networks
22.
Zurück zum Zitat Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192 Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192
24.
Zurück zum Zitat Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: Advances in neural information processing systems, pp 5767–5777 Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: Advances in neural information processing systems, pp 5767–5777
25.
Zurück zum Zitat Kupyn O, Martyniuk T, Wu J, Wang Z (2019) DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better. CoRR arXiv:1908.03826 Kupyn O, Martyniuk T, Wu J, Wang Z (2019) DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better. CoRR arXiv:​1908.​03826
26.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp 770–778
27.
Zurück zum Zitat Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 2169–2178 Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 2169–2178
28.
Zurück zum Zitat Zhang Hongguang, Dai Yuchao, Li Hongdong (2019) PiotrKoniusz. Deep stacked hierarchical multi-patch network for image deblurring with, IEEE Comput Vis Pattern Recognit (CVPR) Zhang Hongguang, Dai Yuchao, Li Hongdong (2019) PiotrKoniusz. Deep stacked hierarchical multi-patch network for image deblurring with, IEEE Comput Vis Pattern Recognit (CVPR)
29.
Zurück zum Zitat Chen LC, Papandreou G, Kokkinos I et al (2018) DeepLab: semantic Image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834CrossRef Chen LC, Papandreou G, Kokkinos I et al (2018) DeepLab: semantic Image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834CrossRef
30.
Zurück zum Zitat He K, Zhang X, Ren S et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intel 37(9):1904–1916CrossRef He K, Zhang X, Ren S et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intel 37(9):1904–1916CrossRef
31.
Zurück zum Zitat Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild. In: GCPR, 2017 Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild. In: GCPR, 2017
32.
Zurück zum Zitat Zhou S, Zhang J, Zuo W, Xie H, Pan J, Ren JS (2019) DAVANet: stereo deblurring with view aggregation. In: CVPR 2019, pp 10996–11005 Zhou S, Zhang J, Zuo W, Xie H, Pan J, Ren JS (2019) DAVANet: stereo deblurring with view aggregation. In: CVPR 2019, pp 10996–11005
33.
Zurück zum Zitat Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. Eur Conf Comput Vis 1(4):5 Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. Eur Conf Comput Vis 1(4):5
34.
Zurück zum Zitat Zhao H, Gallo O, Frosio I et al (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57CrossRef Zhao H, Gallo O, Frosio I et al (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57CrossRef
35.
Zurück zum Zitat Kohler R, Hirsch M, Mohler B, Scholkopf B, Harmeling S (2012) Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: ECCV. Springer, pp 27–40 Kohler R, Hirsch M, Mohler B, Scholkopf B, Harmeling S (2012) Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: ECCV. Springer, pp 27–40
37.
Zurück zum Zitat Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Computer Science Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Computer Science
38.
Zurück zum Zitat Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feed forward neural networks. Aistats 9:249–256 Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feed forward neural networks. Aistats 9:249–256
39.
Zurück zum Zitat Zhang J, Pan J, Ren J, Song Y, Bao L, Lau RWH, Yang M-H (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of IEEE conference on computer vision pattern recognition, pp 2521–2529 Zhang J, Pan J, Ren J, Song Y, Bao L, Lau RWH, Yang M-H (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of IEEE conference on computer vision pattern recognition, pp 2521–2529
Metadaten
Titel
Context Module Based Multi-patch Hierarchical Network for Motion Deblurring
verfasst von
Kang Tang
Dahong Xu
Hong Liu
Zhixin Zeng
Publikationsdatum
13.11.2020
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 1/2021
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-020-10370-0

Weitere Artikel der Ausgabe 1/2021

Neural Processing Letters 1/2021 Zur Ausgabe

Neuer Inhalt