Skip to main content
Erschienen in: Neural Processing Letters 6/2021

15.09.2021

Stack-based Scale-recurrent Network for Face Image Deblurring

verfasst von: Yanqiu Wu, Chaoqun Hong, Xuebai Zhang, Yifan He

Erschienen in: Neural Processing Letters | Ausgabe 6/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In recent years, the image deblurring task has attracted more and more researchers’ attention. Many researchers are devoted to eliminating motion blur by using a “coarse-to-fine” architecture. This architecture can effectively eliminate motion blur caused by a simple relative displacement. But when using the architecture directly for the face image deblurring task, there would exist some problems. For example, complex network structures make the model difficult to be trained, and a large number of parameters need to be calculated would result in expensive runtime. Since details of images can’t be completely restored, the quality of images will deteriorate, and deblurring visual effect is poor, so in order to solve these issues, combining the “coarse-to-fine” architecture with the “stacked” architecture, we propose a new network architecture-“Stack-based Scale-recurrent Network” for the face image deblurring task. The ConvLSTM network is employed to build a model. We achieves the purpose of improving the visual clarity and the quality of restored face images. And we use the improved encoder-decoder to enhance network performance. Compared with the other deblurring methods, our method can restore clear face images with higher-quality and clearer vision.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Wang Z, Yao Z, Wang Q (2017) Improved scheme of estimating motion blur parameters for image restoration[J]. Digit Sign Process 65:11–18MathSciNetCrossRef Wang Z, Yao Z, Wang Q (2017) Improved scheme of estimating motion blur parameters for image restoration[J]. Digit Sign Process 65:11–18MathSciNetCrossRef
2.
Zurück zum Zitat Hirsch M, Schuler C J, Harmeling S et al (2012) Fast removal of non-uniform camera shake[C]. In: IEEE International conference on computer vision. IEEE Hirsch M, Schuler C J, Harmeling S et al (2012) Fast removal of non-uniform camera shake[C]. In: IEEE International conference on computer vision. IEEE
3.
Zurück zum Zitat Harmeling S, Hirsch M, Schlkopf B (2010) Space-variant single-image blind deconvolution for removing camera shake[C]. Adv Neural Inf Process Syst 23:829–837 Harmeling S, Hirsch M, Schlkopf B (2010) Space-variant single-image blind deconvolution for removing camera shake[C]. Adv Neural Inf Process Syst 23:829–837
4.
Zurück zum Zitat Perrone D, Favaro P (2016) A logarithmic image prior for blind deconvolution [J]. Int J Comput Vis 117(2):159–172MathSciNetCrossRef Perrone D, Favaro P (2016) A logarithmic image prior for blind deconvolution [J]. Int J Comput Vis 117(2):159–172MathSciNetCrossRef
5.
Zurück zum Zitat Jiang X, Yao H, Zhao S (2017) Text image deblurring via two-tone prior [J]. Neurocomputing 242:1–14CrossRef Jiang X, Yao H, Zhao S (2017) Text image deblurring via two-tone prior [J]. Neurocomputing 242:1–14CrossRef
6.
Zurück zum Zitat Xu Z, Chen H, Li Z (2021) Fast blind deconvolution using a deeper sparse patch-wise maximum gradient prior[J]. Signal Process Image Commun 90:1114–1122 Xu Z, Chen H, Li Z (2021) Fast blind deconvolution using a deeper sparse patch-wise maximum gradient prior[J]. Signal Process Image Commun 90:1114–1122
7.
Zurück zum Zitat Pan J, Sun D, Pfifister H et al (2016) Blind image deblurring using dark channel prior[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE, pp. 1628–1636 Pan J, Sun D, Pfifister H et al (2016) Blind image deblurring using dark channel prior[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE, pp. 1628–1636
8.
Zurück zum Zitat Yi O (2019) Total variation constraint GAN for dynamic scene deblurring[J]. Image Vis Comput 88:113–119CrossRef Yi O (2019) Total variation constraint GAN for dynamic scene deblurring[J]. Image Vis Comput 88:113–119CrossRef
9.
Zurück zum Zitat Kja B, Ying SC, Ql C et al (2021) Image restoration using overlapping group sparsity on hyper-Laplacian prior of image gradient[J]. Neurocomputing 420:57–69CrossRef Kja B, Ying SC, Ql C et al (2021) Image restoration using overlapping group sparsity on hyper-Laplacian prior of image gradient[J]. Neurocomputing 420:57–69CrossRef
10.
Zurück zum Zitat Zhou K, Zhuang P, Xiong J et al (2020) Blind image deblurring with joint extreme channels and L0-regularized intensity and gradient priors[C]. In: IEEE International conference on image processing (ICIP). IEEE Zhou K, Zhuang P, Xiong J et al (2020) Blind image deblurring with joint extreme channels and L0-regularized intensity and gradient priors[C]. In: IEEE International conference on image processing (ICIP). IEEE
11.
Zurück zum Zitat Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super–resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 105–114 Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super–resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 105–114
12.
Zurück zum Zitat Yda B, Zd C, Shuai YC et al (2020) Kernel-attended residual network for single image super-resolution - ScienceDirect[J]. Knowledge-Based Syst 213:52–60 Yda B, Zd C, Shuai YC et al (2020) Kernel-attended residual network for single image super-resolution - ScienceDirect[J]. Knowledge-Based Syst 213:52–60
13.
Zurück zum Zitat Yu J, Tan M, Zhang H et al (2019) Hierarchical deep click feature prediction for fine-grained image recognition[J]. IEEE Trans Pattern Anal Mach Intell 99:1 Yu J, Tan M, Zhang H et al (2019) Hierarchical deep click feature prediction for fine-grained image recognition[J]. IEEE Trans Pattern Anal Mach Intell 99:1
14.
Zurück zum Zitat Isola P, Zhu J Y, Zhou T et al (2016) Image-to-image translation with conditional adversarial networks[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE Isola P, Zhu J Y, Zhou T et al (2016) Image-to-image translation with conditional adversarial networks[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE
15.
Zurück zum Zitat Yu J, Kuang Z, Zhang B et al (2018) Leveraging content sensitiveness and user trustworthiness to recommend fine-grained privacy settings for social image sharing[J]. IEEE Trans Inf Forensics Secur 13:1317–1332CrossRef Yu J, Kuang Z, Zhang B et al (2018) Leveraging content sensitiveness and user trustworthiness to recommend fine-grained privacy settings for social image sharing[J]. IEEE Trans Inf Forensics Secur 13:1317–1332CrossRef
16.
Zurück zum Zitat Li Y, Huang JB, Ahuja N et al (2016) Deep joint image filtering[J]. In: European conference on computer vision. pp. 771–779 Li Y, Huang JB, Ahuja N et al (2016) Deep joint image filtering[J]. In: European conference on computer vision. pp. 771–779
17.
Zurück zum Zitat Khan AT, Cao X, Li S et al (2021) Quantum beetle antennae search: a novel technique for the constrained portfolio optimization problem[J]. Sci China Inf Sci 64(5):152204 Khan AT, Cao X, Li S et al (2021) Quantum beetle antennae search: a novel technique for the constrained portfolio optimization problem[J]. Sci China Inf Sci 64(5):152204
18.
Zurück zum Zitat Atk A, Shuai LA, Xc B (2021) Control framework for cooperative robots in smart home using bio-inspired neural network - ScienceDirect[J]. Measurement 167:108253CrossRef Atk A, Shuai LA, Xc B (2021) Control framework for cooperative robots in smart home using bio-inspired neural network - ScienceDirect[J]. Measurement 167:108253CrossRef
19.
Zurück zum Zitat Gampala V, Kumar M, Sushama C, Sehar E, Raj FI (2020) Deep learning based image processing approaches for image deblurring. Mater Today Proc 2020:601–609 Gampala V, Kumar M, Sushama C, Sehar E, Raj FI (2020) Deep learning based image processing approaches for image deblurring. Mater Today Proc 2020:601–609
20.
Zurück zum Zitat Zhang J, Pan J, Ren J et al (2018) Dynamic scene deblurring using spatially variant recurrent neural networks[C]. In: IEEE/CVF Conference on computer vision and pattern recognition. IEEE Zhang J, Pan J, Ren J et al (2018) Dynamic scene deblurring using spatially variant recurrent neural networks[C]. In: IEEE/CVF Conference on computer vision and pattern recognition. IEEE
21.
Zurück zum Zitat Huang L, Xia Y (2019) Joint blur Kernel estimation and CNN for blind image restoration[J]. Neurocomputing 396:562–570 Huang L, Xia Y (2019) Joint blur Kernel estimation and CNN for blind image restoration[J]. Neurocomputing 396:562–570
22.
Zurück zum Zitat Singhal J, DeblurRL Narang P (2021) Image deblurring with deep reinforcement learning[M]. Electronic Industry Press, Beijing Singhal J, DeblurRL Narang P (2021) Image deblurring with deep reinforcement learning[M]. Electronic Industry Press, Beijing
23.
Zurück zum Zitat Wang M, Hou S, Li H et al (2019) Generative image deblurring based on multi-scaled residual adversary network driven by composed prior-posterior loss[J]. J Vis Commun Image Represent 65:1621–1629 Wang M, Hou S, Li H et al (2019) Generative image deblurring based on multi-scaled residual adversary network driven by composed prior-posterior loss[J]. J Vis Commun Image Represent 65:1621–1629
24.
Zurück zum Zitat Xu L, Ren JS, Liu C et al (2014) Deep convolutional neural network for image deconvolution[C]. NIPS 27:1790–1798 Xu L, Ren JS, Liu C et al (2014) Deep convolutional neural network for image deconvolution[C]. NIPS 27:1790–1798
25.
Zurück zum Zitat Gong D, Yang J, Liu L et al (2017) From motion blur to motion flflow: a deep learning solution for removing heterogeneous motion blur[C]. In: CVPR Gong D, Yang J, Liu L et al (2017) From motion blur to motion flflow: a deep learning solution for removing heterogeneous motion blur[C]. In: CVPR
26.
Zurück zum Zitat Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild[C]. In: German conference on pattern recognition Noroozi M, Chandramouli P, Favaro P (2017) Motion deblurring in the wild[C]. In: German conference on pattern recognition
27.
Zurück zum Zitat Sun J, Cao W, Xu Z et al (2015) Learning a convolutional neural network for non-uniform motion blur removal[J]. In: IEEE Conference on computer vision and pattern recognition pp. 235–243 Sun J, Cao W, Xu Z et al (2015) Learning a convolutional neural network for non-uniform motion blur removal[J]. In: IEEE Conference on computer vision and pattern recognition pp. 235–243
28.
Zurück zum Zitat Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 257–265 Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 257–265
29.
Zurück zum Zitat Chakrabarti A (2016) A neural approach to blind motion deblurring[C]. In: European conference on computer vision. Springer, Cham, pp 221–235 Chakrabarti A (2016) A neural approach to blind motion deblurring[C]. In: European conference on computer vision. Springer, Cham, pp 221–235
30.
Zurück zum Zitat Dong G, Jie Y, Liu L et al (2017) From Motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE Dong G, Jie Y, Liu L et al (2017) From Motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE
31.
Zurück zum Zitat Seungjun Nah, Tae Hyun Kim, Kyoung Mu Lee (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 257–265 Seungjun Nah, Tae Hyun Kim, Kyoung Mu Lee (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 257–265
32.
Zurück zum Zitat Mao XJ, Shen C, Yang YB (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[J]. In: Conference on neural information processing systems pp. 2802–2810 Mao XJ, Shen C, Yang YB (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[J]. In: Conference on neural information processing systems pp. 2802–2810
33.
Zurück zum Zitat Hongguang Zhang, Yuchao Dai, Hongdong Li et al (2019) Deep stacked hierarchical multi-patch network for image deblurring[C]. In: CVPR, pp.5979-5986 Hongguang Zhang, Yuchao Dai, Hongdong Li et al (2019) Deep stacked hierarchical multi-patch network for image deblurring[C]. In: CVPR, pp.5979-5986
34.
Zurück zum Zitat Kupyn O, Budzan V, Mykhailych M et al (2018) DeblurGAN: Blind motion deblurring using conditional adversarial networks[C]. In: IEEE/CVF Conference on computer vision and pattern recognition. IEEE Kupyn O, Budzan V, Mykhailych M et al (2018) DeblurGAN: Blind motion deblurring using conditional adversarial networks[C]. In: IEEE/CVF Conference on computer vision and pattern recognition. IEEE
35.
Zurück zum Zitat Yu J, Zhu C, Zhang J et al (2019) Spatial pyramid-enhanced NetVLAD with weighted triplet loss for place recognition[J]. IEEE Trans Neural Netw Learn Syst 31:1–14 Yu J, Zhu C, Zhang J et al (2019) Spatial pyramid-enhanced NetVLAD with weighted triplet loss for place recognition[J]. IEEE Trans Neural Netw Learn Syst 31:1–14
36.
Zurück zum Zitat Eigen D, Fergus R (2014) Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]. In: IEEE. IEEE Eigen D, Fergus R (2014) Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]. In: IEEE. IEEE
37.
Zurück zum Zitat Mathieu MF, Couprie C, Cun Y (2018) Deep multi-scale video prediction. US20180137389[P] Mathieu MF, Couprie C, Cun Y (2018) Deep multi-scale video prediction. US20180137389[P]
38.
Zurück zum Zitat Dosovitskiy A, Fischer P, Ilg E et al (2015) Flownet: Learning optical flflow with convolutional networks. In: CVPR, pp. 2758–2766 Dosovitskiy A, Fischer P, Ilg E et al (2015) Flownet: Learning optical flflow with convolutional networks. In: CVPR, pp. 2758–2766
39.
Zurück zum Zitat Khan AH, Li S, Luo X (2020) Obstacle avoidance and tracking control of redundant robotic manipulator: an RNN-based metaheuristic approach[J]. IEEE Trans Ind Inform 16(7):4670–4680CrossRef Khan AH, Li S, Luo X (2020) Obstacle avoidance and tracking control of redundant robotic manipulator: an RNN-based metaheuristic approach[J]. IEEE Trans Ind Inform 16(7):4670–4680CrossRef
40.
Zurück zum Zitat Khan AH, Li S, Cao X (2021) Tracking control of redundant manipulator under active remote center-of-motion constraints: an RNN-based metaheuristic approach[J]. Sci China Inf Sci 64(3):1–18MathSciNetCrossRef Khan AH, Li S, Cao X (2021) Tracking control of redundant manipulator under active remote center-of-motion constraints: an RNN-based metaheuristic approach[J]. Sci China Inf Sci 64(3):1–18MathSciNetCrossRef
41.
Zurück zum Zitat Ahk A, Shuai LB, Dc C et al (2020) Tracking control of redundant mobile manipulator: an RNN based metaheuristic approach[J]. Neurocomputing 400:272–284CrossRef Ahk A, Shuai LB, Dc C et al (2020) Tracking control of redundant mobile manipulator: an RNN based metaheuristic approach[J]. Neurocomputing 400:272–284CrossRef
42.
Zurück zum Zitat Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8174–8182 Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8174–8182
43.
Zurück zum Zitat Liu Z, Yeh R, Tang X et al (2017) Video frame synthesis using deep voxel flow[J]. In: IEEE Liu Z, Yeh R, Tang X et al (2017) Video frame synthesis using deep voxel flow[J]. In: IEEE
44.
Zurück zum Zitat Su S, Delbracio M, Wang J et al. (2016) Deep video deblurring [J] Su S, Delbracio M, Wang J et al. (2016) Deep video deblurring [J]
45.
Zurück zum Zitat Tao X, Gao H, Liao R et al (2017) Detail-revealing deep video super-resolution. In: IEEE Computer society Tao X, Gao H, Liao R et al (2017) Detail-revealing deep video super-resolution. In: IEEE Computer society
46.
Zurück zum Zitat Xu N, Price B, Cohen S et al (2017) Deep Image Matting[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE Computer society, pp. 311–320 Xu N, Price B, Cohen S et al (2017) Deep Image Matting[C]. In: IEEE Conference on computer vision and pattern recognition. IEEE Computer society, pp. 311–320
47.
Zurück zum Zitat He K, Zhang X, Ren S et al (2016) Deep Residual Learning for Image Recognition[C]. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE, 2016 He K, Zhang X, Ren S et al (2016) Deep Residual Learning for Image Recognition[C]. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE, 2016
48.
Zurück zum Zitat Kingma D, Adam Ba J (2014) A method for stochastic optimization[J]. Computer. Science Kingma D, Adam Ba J (2014) A method for stochastic optimization[J]. Computer. Science
Metadaten
Titel
Stack-based Scale-recurrent Network for Face Image Deblurring
verfasst von
Yanqiu Wu
Chaoqun Hong
Xuebai Zhang
Yifan He
Publikationsdatum
15.09.2021
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 6/2021
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-021-10604-9

Weitere Artikel der Ausgabe 6/2021

Neural Processing Letters 6/2021 Zur Ausgabe

Neuer Inhalt