Skip to main content
Top
Published in: Neural Processing Letters 1/2021

13-11-2020

Context Module Based Multi-patch Hierarchical Network for Motion Deblurring

Authors: Kang Tang, Dahong Xu, Hong Liu, Zhixin Zeng

Published in: Neural Processing Letters | Issue 1/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Single image blind motion deblurring refers to transferring a blurred motion image into a corresponding clear image, which is a challenging and classic problem in the field of computer vision. The spatially variant blur is usually caused by many factors, such as camera jitter and object motion. Since image deblurring can be regarded as the task of image transformation, deep learning methods based on coarse-to-fine scheme, especially those using multi-scale architectures become popular. However, they have the disadvantages of unsatisfactory image quality and time-consuming running caused by large kernel size. In this paper, we propose a novel end-to-end network structure based on Deep Hierarchical Multi-patch network architecture integrated with Context Module and additional ResBlocks in order to tackle deblurring problem. Compared with recently proposed networks, it generates images with better visual effect as well as higher image quality index. Besides, our model significantly reduces the test time. We evaluate the proposed network structure on public GoPro dataset, a large-scale image dataset with complex synthetic blur. The experiments on the benchmark dataset prove that our effective method outperforms other state-of-the-art blind deblurring algorithms both qualitatively and quantitatively, which demonstrates the effectiveness of Context Module in the task of single image blur removal.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Hyun Kim T, Ahn B, Mu Lee K (2013) Dynamic scene deblurring. In: ICCV, pp 3160–3167 Hyun Kim T, Ahn B, Mu Lee K (2013) Dynamic scene deblurring. In: ICCV, pp 3160–3167
2.
go back to reference Sun J, Cao W, Xu Z, Ponce J (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: CVPR, pp 769–777 Sun J, Cao W, Xu Z, Ponce J (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: CVPR, pp 769–777
3.
go back to reference Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 257–265 Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 257–265
4.
go back to reference Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: CVPR, pp 8174–8182 Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: CVPR, pp 8174–8182
5.
go back to reference Mei J, Wu Z, Chen X et al (2019) DeepDeblur: text image recovery from blur to sharp. Multim Tools Appl 78(13):18869–18885CrossRef Mei J, Wu Z, Chen X et al (2019) DeepDeblur: text image recovery from blur to sharp. Multim Tools Appl 78(13):18869–18885CrossRef
7.
go back to reference Ren W, Zhang J, Ma L, Pan J, Cao X, Zuo W, LiuW, Yang M-S (2018) Deep non-blind deconvolution via generalized low-rank approximation. In: Advances in neural information processing systems, pp 295–305 Ren W, Zhang J, Ma L, Pan J, Cao X, Zuo W, LiuW, Yang M-S (2018) Deep non-blind deconvolution via generalized low-rank approximation. In: Advances in neural information processing systems, pp 295–305
8.
go back to reference Xiangyu Xu, Jinshan Pan, Yu-Jin Zhang, and Ming-HsuanYang. Motion blur kernel estimation via deep learning. IEEE Transactions on Image Processing, 27(1):194–205, 2018 Xiangyu Xu, Jinshan Pan, Yu-Jin Zhang, and Ming-HsuanYang. Motion blur kernel estimation via deep learning. IEEE Transactions on Image Processing, 27(1):194–205, 2018
9.
go back to reference Li L, Pan J, Lai W-S, Gao X, Sang N, Yang M-H (2018) Learning a discriminative prior for blind image deblurring. In: The IEEE conference on computer vision and pattern recognition (CVPR), June 2018 Li L, Pan J, Lai W-S, Gao X, Sang N, Yang M-H (2018) Learning a discriminative prior for blind image deblurring. In: The IEEE conference on computer vision and pattern recognition (CVPR), June 2018
10.
go back to reference Chang C-F, Wu J-L (2014) A new single image deblurring algorithm using hyper laplacian priors. In: ICS, pp 1015–1022 Chang C-F, Wu J-L (2014) A new single image deblurring algorithm using hyper laplacian priors. In: ICS, pp 1015–1022
11.
go back to reference Xu L, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring. In: CVPR Xu L, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring. In: CVPR
12.
go back to reference Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: CVPR Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: CVPR
13.
go back to reference Cao Y, Wang S, Guo Z et al (2017) Synchronization of memristive neural networks with leakage delay and parameters mismatch via event-triggered control. Neural Netw 2019:119 Cao Y, Wang S, Guo Z et al (2017) Synchronization of memristive neural networks with leakage delay and parameters mismatch via event-triggered control. Neural Netw 2019:119
14.
go back to reference Wang Shengbo, Cao Yuting, Huang Tingwen, Chen Yiran, Wen Shiping (2020) Event-triggered distributed control for synchronization of multiple memristive neural networks under cyber-physical attacks. Inf Sci 518:361–375MathSciNetCrossRef Wang Shengbo, Cao Yuting, Huang Tingwen, Chen Yiran, Wen Shiping (2020) Event-triggered distributed control for synchronization of multiple memristive neural networks under cyber-physical attacks. Inf Sci 518:361–375MathSciNetCrossRef
15.
go back to reference Xu L, Ren JSJ, Liu C, Jia J (2014) Deep convolutional neural network for image deconvolution. In: Proceedings of advance neural information processing system, pp 1790–1798 Xu L, Ren JSJ, Liu C, Jia J (2014) Deep convolutional neural network for image deconvolution. In: Proceedings of advance neural information processing system, pp 1790–1798
16.
go back to reference Gong D, Yang J, Liu L, Zhang Y, Reid ID, Shen C, Van Den Hengel A, Shi Q (2017) From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. In: CVPR, 2017 Gong D, Yang J, Liu L, Zhang Y, Reid ID, Shen C, Van Den Hengel A, Shi Q (2017) From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur. In: CVPR, 2017
17.
go back to reference Schuler CJ, Hirsch M, Harmeling S, Schölkopf B (2016) Learning to deblur. IEEE Trans Pattern Anal Mach Intell 38(7):1439–1451CrossRef Schuler CJ, Hirsch M, Harmeling S, Schölkopf B (2016) Learning to deblur. IEEE Trans Pattern Anal Mach Intell 38(7):1439–1451CrossRef
18.
go back to reference Chakrabarti A (2016) A neural approach to blind motion deblurring. In: ECCV. Springer, pp 221–235 Chakrabarti A (2016) A neural approach to blind motion deblurring. In: ECCV. Springer, pp 221–235
19.
go back to reference Gao H, Tao X, Shen X, Jia J (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. CVPR 2019, pp 3848–3856 Gao H, Tao X, Shen X, Jia J (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. CVPR 2019, pp 3848–3856
20.
go back to reference Ramakrishnan S, Pachori S, Gangopadhyay A, Raman S (2017) Deep generative filter for motion deblurring. In: 2017 IEEE international conference on computer vision workshops (ICCVW), pp 2993–3000 Ramakrishnan S, Pachori S, Gangopadhyay A, Raman S (2017) Deep generative filter for motion deblurring. In: 2017 IEEE international conference on computer vision workshops (ICCVW), pp 2993–3000
21.
go back to reference Isola P, Zhu J-Y, Zhou T (2016) Image-to-image translation with conditional adversarial networks Isola P, Zhu J-Y, Zhou T (2016) Image-to-image translation with conditional adversarial networks
22.
go back to reference Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192 Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192
24.
go back to reference Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: Advances in neural information processing systems, pp 5767–5777 Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: Advances in neural information processing systems, pp 5767–5777
25.
26.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp 770–778
27.
go back to reference Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 2169–2178 Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 2169–2178
28.
go back to reference Zhang Hongguang, Dai Yuchao, Li Hongdong (2019) PiotrKoniusz. Deep stacked hierarchical multi-patch network for image deblurring with, IEEE Comput Vis Pattern Recognit (CVPR) Zhang Hongguang, Dai Yuchao, Li Hongdong (2019) PiotrKoniusz. Deep stacked hierarchical multi-patch network for image deblurring with, IEEE Comput Vis Pattern Recognit (CVPR)
29.
go back to reference Chen LC, Papandreou G, Kokkinos I et al (2018) DeepLab: semantic Image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834CrossRef Chen LC, Papandreou G, Kokkinos I et al (2018) DeepLab: semantic Image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834CrossRef
30.
go back to reference He K, Zhang X, Ren S et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intel 37(9):1904–1916CrossRef He K, Zhang X, Ren S et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intel 37(9):1904–1916CrossRef
31.
go back to reference Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild. In: GCPR, 2017 Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild. In: GCPR, 2017
32.
go back to reference Zhou S, Zhang J, Zuo W, Xie H, Pan J, Ren JS (2019) DAVANet: stereo deblurring with view aggregation. In: CVPR 2019, pp 10996–11005 Zhou S, Zhang J, Zuo W, Xie H, Pan J, Ren JS (2019) DAVANet: stereo deblurring with view aggregation. In: CVPR 2019, pp 10996–11005
33.
go back to reference Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. Eur Conf Comput Vis 1(4):5 Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. Eur Conf Comput Vis 1(4):5
34.
go back to reference Zhao H, Gallo O, Frosio I et al (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57CrossRef Zhao H, Gallo O, Frosio I et al (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57CrossRef
35.
go back to reference Kohler R, Hirsch M, Mohler B, Scholkopf B, Harmeling S (2012) Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: ECCV. Springer, pp 27–40 Kohler R, Hirsch M, Mohler B, Scholkopf B, Harmeling S (2012) Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: ECCV. Springer, pp 27–40
37.
go back to reference Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Computer Science Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Computer Science
38.
go back to reference Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feed forward neural networks. Aistats 9:249–256 Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feed forward neural networks. Aistats 9:249–256
39.
go back to reference Zhang J, Pan J, Ren J, Song Y, Bao L, Lau RWH, Yang M-H (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of IEEE conference on computer vision pattern recognition, pp 2521–2529 Zhang J, Pan J, Ren J, Song Y, Bao L, Lau RWH, Yang M-H (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of IEEE conference on computer vision pattern recognition, pp 2521–2529
Metadata
Title
Context Module Based Multi-patch Hierarchical Network for Motion Deblurring
Authors
Kang Tang
Dahong Xu
Hong Liu
Zhixin Zeng
Publication date
13-11-2020
Publisher
Springer US
Published in
Neural Processing Letters / Issue 1/2021
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-020-10370-0

Other articles of this Issue 1/2021

Neural Processing Letters 1/2021 Go to the issue