Skip to main content
Erschienen in: Neural Computing and Applications 18/2020

17.05.2019 | Deep Learning Approaches for RealTime Image Super Resolution (DLRSR)

Perceptual image quality using dual generative adversarial network

verfasst von: Masoumeh Zareapoor, Huiyu Zhou, Jie Yang

Erschienen in: Neural Computing and Applications | Ausgabe 18/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Generative adversarial networks have received a remarkable success in many computer vision applications for their ability to learn from complex data distribution. In particular, they are capable to generate realistic images from latent space with a simple and intuitive structure. The main focus of existing models has been improving the performance; however, there is a little attention to make a robust model. In this paper, we investigate solutions to the super-resolution problems—in particular perceptual quality—by proposing a robust GAN. The proposed model unlike the standard GAN employs two generators and two discriminators in which, a discriminator determines that the samples are from real data or generated one, while another discriminator acts as classifier to return the wrong samples to its corresponding generators. Generators learn a mixture of many distributions from prior to the complex distribution. This new methodology is trained with the feature matching loss and allows us to return the wrong samples to the corresponding generators, in order to regenerate the real-look samples. Experimental results in various datasets show the superiority of the proposed model compared to the state of the art methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Zareapoor M, Shamsolmoali P, Yang J (2019) Learning depth super-resolution by using multi-scale convolutional neural network. J Intell Fuzzy Syst 36(2):1773–1783CrossRef Zareapoor M, Shamsolmoali P, Yang J (2019) Learning depth super-resolution by using multi-scale convolutional neural network. J Intell Fuzzy Syst 36(2):1773–1783CrossRef
3.
Zurück zum Zitat Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceeding of advances in neural information processing systems, pp 2672–2680 Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceeding of advances in neural information processing systems, pp 2672–2680
4.
Zurück zum Zitat Ledig C, Theis L, Huszar F, Caballero J, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2016) Photo-realistic single image super-resolution using a generative adversarial network. CoRR, vol. abs/1609.04802, 2016. [Online]. http://arxiv.org/abs/1609.04802 Ledig C, Theis L, Huszar F, Caballero J, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2016) Photo-realistic single image super-resolution using a generative adversarial network. CoRR, vol. abs/1609.04802, 2016. [Online]. http://​arxiv.​org/​abs/​1609.​04802
5.
Zurück zum Zitat Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H (2016) Generative adversarial text-to-image synthesis. In: Proceedings of ICML, pp 1060–1069 Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H (2016) Generative adversarial text-to-image synthesis. In: Proceedings of ICML, pp 1060–1069
6.
Zurück zum Zitat Zhang H, Xu T, Li H, Zhang S, Huang X, Wang X, Metaxas DN (2017) StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. In: Proceeding of the ICCV, pp 5907–5915 Zhang H, Xu T, Li H, Zhang S, Huang X, Wang X, Metaxas DN (2017) StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. In: Proceeding of the ICCV, pp 5907–5915
7.
Zurück zum Zitat Durugkar IP, Gemp I, Mahadevan S (2016) Generative multi-adversarial networks. ICLR. CoRR, abs/1611.01673 Durugkar IP, Gemp I, Mahadevan S (2016) Generative multi-adversarial networks. ICLR. CoRR, abs/1611.01673
10.
Zurück zum Zitat Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: ICCV Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: ICCV
11.
Zurück zum Zitat Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRef Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307CrossRef
12.
Zurück zum Zitat Zareapoor M, Jain DK, Yang J (2018) Local spatial information for image super-resolution. Cogn Syst Res 52:49–57CrossRef Zareapoor M, Jain DK, Yang J (2018) Local spatial information for image super-resolution. Cogn Syst Res 52:49–57CrossRef
13.
Zurück zum Zitat Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. In: Proceeding of international conference on learning representations arXiv:1511.06434 Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. In: Proceeding of international conference on learning representations arXiv:​1511.​06434
14.
Zurück zum Zitat Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training GANs. In: Proceeding of the NIPS, pp 2234–2242 Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training GANs. In: Proceeding of the NIPS, pp 2234–2242
15.
Zurück zum Zitat Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier GANs. In: International conference on machine learning (PMLR), pp 2642–2651 Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier GANs. In: International conference on machine learning (PMLR), pp 2642–2651
16.
Zurück zum Zitat Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Advances in neural information processing systems Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Advances in neural information processing systems
17.
Zurück zum Zitat Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Proceedings of the 34th international conference on machine learning, pp 214–223 Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Proceedings of the 34th international conference on machine learning, pp 214–223
18.
Zurück zum Zitat Nguyen TD, Le T, Vu H, Phung D (2017) Dual discriminator generative adversarial nets. In: Advances in neural information processing systems 29 (NIPS) (accepted) Nguyen TD, Le T, Vu H, Phung D (2017) Dual discriminator generative adversarial nets. In: Advances in neural information processing systems 29 (NIPS) (accepted)
19.
Zurück zum Zitat Arora S, Ge R, Liang Y, Ma T, Zhang Y (2017) Generalization and equilibrium in generative adversarial nets (gans). arXiv preprint arXiv:1703.00573 Arora S, Ge R, Liang Y, Ma T, Zhang Y (2017) Generalization and equilibrium in generative adversarial nets (gans). arXiv preprint arXiv:​1703.​00573
20.
Zurück zum Zitat Tolstikhin I, Gelly S, Bousquet O, Simon-Gabriel C-J, Sch¨olkopf B (2017) Adagan: boosting generative models. arXiv preprint arXiv:1701.02386 Tolstikhin I, Gelly S, Bousquet O, Simon-Gabriel C-J, Sch¨olkopf B (2017) Adagan: boosting generative models. arXiv preprint arXiv:​1701.​02386
21.
Zurück zum Zitat Ghosh A, Kulharia V, Namboodiri VP, Torr PHS, Dokania PK (2017) Multi-agent diverse generative adversarial networks. In: Proceeding of the CVPR, pp 8513–8521 Ghosh A, Kulharia V, Namboodiri VP, Torr PHS, Dokania PK (2017) Multi-agent diverse generative adversarial networks. In: Proceeding of the CVPR, pp 8513–8521
22.
23.
Zurück zum Zitat Yang J, Kannan A, Batra D, Parikh D (2017) Lr-gan: layered recursive generative adversarial networks for image generation. arXiv preprint arXiv:1703.01560 Yang J, Kannan A, Batra D, Parikh D (2017) Lr-gan: layered recursive generative adversarial networks for image generation. arXiv preprint arXiv:​1703.​01560
24.
Zurück zum Zitat Denton E, Chintala S, Szlam A, Fergus R (2015) Deep generative image models using a Laplacian pyramid of adversarial networks. In: Proceeding the NIPS, pp 1486–1494 Denton E, Chintala S, Szlam A, Fergus R (2015) Deep generative image models using a Laplacian pyramid of adversarial networks. In: Proceeding the NIPS, pp 1486–1494
25.
Zurück zum Zitat Burt PJ, Adelson EH (1987) The Laplacian pyramid as a compact image code. In: Readings in computer vision. Elsevier, pp 671–679 Burt PJ, Adelson EH (1987) The Laplacian pyramid as a compact image code. In: Readings in computer vision. Elsevier, pp 671–679
27.
Zurück zum Zitat Liu M-Y, Tuzel O (2016) Coupled generative adversarial networks. In: Proceedings of the advances in neural information processing systems (NIPS 2016), Barcelona, Spain, pp 469–477 Liu M-Y, Tuzel O (2016) Coupled generative adversarial networks. In: Proceedings of the advances in neural information processing systems (NIPS 2016), Barcelona, Spain, pp 469–477
29.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: ICCV He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: ICCV
30.
Zurück zum Zitat Maas A, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models Maas A, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models
31.
Zurück zum Zitat Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) TensorFlow: a system for large-scale machine learning. In: OSDI, vol 16, pp 265–283 Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) TensorFlow: a system for large-scale machine learning. In: OSDI, vol 16, pp 265–283
32.
Zurück zum Zitat Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. CoRR, vol. abs/1412.6980 Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. CoRR, vol. abs/1412.6980
33.
Zurück zum Zitat Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the CVPR, pp 1646–1654 Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the CVPR, pp 1646–1654
34.
Zurück zum Zitat Lai WS, Huang J-B, Ahuja N, Yang M-H (2017) Deep Laplacian pyramid networks for fast and accurate superresolution. In: CVPR, pp 624–632 Lai WS, Huang J-B, Ahuja N, Yang M-H (2017) Deep Laplacian pyramid networks for fast and accurate superresolution. In: CVPR, pp 624–632
35.
Zurück zum Zitat Wang Y, Perazzi F, Williams BM, Hornung AS, Hornung OS, Schroers C (2017) A fully progressive approach to single-image super-resolution. arXiv:1804.02900v2 Wang Y, Perazzi F, Williams BM, Hornung AS, Hornung OS, Schroers C (2017) A fully progressive approach to single-image super-resolution. arXiv:​1804.​02900v2
36.
Zurück zum Zitat Berthelot D, Schumm T, Metz L (2017) Began: boundary equilibrium generative adversarial networks. CoRR, abs/1703.10717 Berthelot D, Schumm T, Metz L (2017) Began: boundary equilibrium generative adversarial networks. CoRR, abs/1703.10717
37.
Zurück zum Zitat Juefei-Xu F, Boddeti VN, Savvides M (2017) Gang of gans: generative adversarial networks with maximum margin ranking. arXiv preprint arXiv:1704.04865 Juefei-Xu F, Boddeti VN, Savvides M (2017) Gang of gans: generative adversarial networks with maximum margin ranking. arXiv preprint arXiv:​1704.​04865
38.
39.
Zurück zum Zitat Wang R, Cully A, Chang HJ, Demiris Y (2017) Magan: Margin adaptation for generative adversarial networks. arXiv preprint arXiv:1704.03817 Wang R, Cully A, Chang HJ, Demiris Y (2017) Magan: Margin adaptation for generative adversarial networks. arXiv preprint arXiv:​1704.​03817
40.
Zurück zum Zitat Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: ECCV Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: ECCV
41.
Zurück zum Zitat Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: CVPR Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: CVPR
42.
Zurück zum Zitat Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the CVPR, pp 2790–2798 Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the CVPR, pp 2790–2798
44.
Zurück zum Zitat Wang X, Yu K, Dong C, Loy CC (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform. In: CVPR. arXiv:1804.02815v1 Wang X, Yu K, Dong C, Loy CC (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform. In: CVPR. arXiv:​1804.​02815v1
45.
Zurück zum Zitat Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision (ECCV), pp 391–407 Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision (ECCV), pp 391–407
46.
Zurück zum Zitat Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: CVPR Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: CVPR
47.
Zurück zum Zitat Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: ICCV Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: ICCV
Metadaten
Titel
Perceptual image quality using dual generative adversarial network
verfasst von
Masoumeh Zareapoor
Huiyu Zhou
Jie Yang
Publikationsdatum
17.05.2019
Verlag
Springer London
Erschienen in
Neural Computing and Applications / Ausgabe 18/2020
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-019-04239-0

Weitere Artikel der Ausgabe 18/2020

Neural Computing and Applications 18/2020 Zur Ausgabe

Extreme Learning Machine and Deep Learning Networks

Hierarchical attentive Siamese network for real-time visual tracking

Premium Partner