Skip to main content
Top
Published in: Neural Computing and Applications 1/2022

05-08-2021 | Original Article

Multidomain image-to-image translation model based on hidden space sharing

Authors: Ding Yuxin, Wang Longfei

Published in: Neural Computing and Applications | Issue 1/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Image-to-image translation translates an image from one domain to another. The goal is to learn the translation relationship between different image domains. Compared with the translation models to be trained using paired training data, CycleGAN has the advantage of learning to translate between domains without paired input–output training examples. However, when using CycleGAN to translate images among multiple domains, the complexity of the model increases nonlinearly with the number of domains. To reduce the model complexity of CycleGAN-based translation models, we assume that there is a hidden space shared by different domains, and this space stores the common features of images. Then, we design a common encoder to learn image features in the hidden space. Based on the hidden space, we propose a translation model that scales linearly with the number of domains. To further improve the common feature representation accuracy, we introduce the adversarial component in the hidden space to learn the common features. We test the proposed models on different datasets, including painting style and season transfer datasets and achieve good results.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial nets, In Proceedings of the International Conference on Neural Information Processing Systems, pp. 2672–2680, Montreal, MIT Press, Canada Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial nets, In Proceedings of the International Conference on Neural Information Processing Systems, pp. 2672–2680, Montreal, MIT Press, Canada
2.
go back to reference Zhu J-Y, Park T, Isola P, et al. (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of International Conference on Computer Vision, Pages 2223–2230, Venice, Italy, IEEE. Zhu J-Y, Park T, Isola P, et al. (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of International Conference on Computer Vision, Pages 2223–2230, Venice, Italy, IEEE.
3.
go back to reference Isola P, Zhu J-Y, Zhou T, et al (2017) Image-to-image translation with conditional adversarial networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Pages 5967–5976, Hawaii, USA. IEEE. Isola P, Zhu J-Y, Zhou T, et al (2017) Image-to-image translation with conditional adversarial networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Pages 5967–5976, Hawaii, USA. IEEE.
4.
go back to reference Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Pages 2414–2423, Las Vegas, USA.IEEE. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Pages 2414–2423, Las Vegas, USA.IEEE.
6.
go back to reference Ulyanov D, Lebedev V, Vedaldi A, et al. (2016) Texture Networks: Feed-forward Synthesis of Textures and Stylized Images. In the Proceedings of International Conference on Machine Learning. Pages 1349–1357, NY, USA. IMLS. Ulyanov D, Lebedev V, Vedaldi A, et al. (2016) Texture Networks: Feed-forward Synthesis of Textures and Stylized Images. In the Proceedings of International Conference on Machine Learning. Pages 1349–1357, NY, USA. IMLS.
7.
go back to reference Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In the proc. of IEEE Conference on Computer Vision and Pattern Recognition. Pages 3431–3440, Boston, USA. IEEE. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In the proc. of IEEE Conference on Computer Vision and Pattern Recognition. Pages 3431–3440, Boston, USA. IEEE.
8.
go back to reference Xu S, Zhu Q, Wang J (2020) Generative image completion with image-to-image translation. Neural Comput & Applic 32:7333–7345CrossRef Xu S, Zhu Q, Wang J (2020) Generative image completion with image-to-image translation. Neural Comput & Applic 32:7333–7345CrossRef
9.
go back to reference Karatsiolis S, Schizas CN, Petkov N (2020) Modular domain-to-domain translation network. Neural Comput & Applic 32:6779–6791CrossRef Karatsiolis S, Schizas CN, Petkov N (2020) Modular domain-to-domain translation network. Neural Comput & Applic 32:6779–6791CrossRef
10.
go back to reference Pathak D, Krahenbuhl P, Donahue J, et al.(2016) Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Pages 2536–2544, Las Vegas , USA. IEEE. Pathak D, Krahenbuhl P, Donahue J, et al.(2016) Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Pages 2536–2544, Las Vegas , USA. IEEE.
11.
go back to reference Zhang R, Isola P, Efros A A (2016) Colorful image colorization. In Proceedings of European Conference on Computer Vision. Pages 649–666, Amsterdam, Netherlands. Springer. Zhang R, Isola P, Efros A A (2016) Colorful image colorization. In Proceedings of European Conference on Computer Vision. Pages 649–666, Amsterdam, Netherlands. Springer.
12.
go back to reference Hertzmann A, Jacobs C E, Oliver N, et al (2001) Image analogies.p. In Proceedings of the ACM SIGGRAPH, Pages 327–340. Los Angeles, USA. ACM. Hertzmann A, Jacobs C E, Oliver N, et al (2001) Image analogies.p. In Proceedings of the ACM SIGGRAPH, Pages 327–340. Los Angeles, USA. ACM.
13.
go back to reference Choi Y, Choi M, Kim M, et al. (2017) StarGAN: Unified Generative Adversarial Networks forMulti-Domain Image-to-Image Translation. ArXiv e-prints. arXiv:1711.09020v1. Choi Y, Choi M, Kim M, et al. (2017) StarGAN: Unified Generative Adversarial Networks forMulti-Domain Image-to-Image Translation. ArXiv e-prints. arXiv:​1711.​09020v1.
14.
go back to reference Hsu SY,Yang CY, Huang CC,et al. (2018) SemiStarGAN: Semi-supervised Generative Adversarial Networks for Multi-domain Image-to-Image Translation. Asian Conference on Computer Vision, pages 338–353. Hsu SY,Yang CY, Huang CC,et al. (2018) SemiStarGAN: Semi-supervised Generative Adversarial Networks for Multi-domain Image-to-Image Translation. Asian Conference on Computer Vision, pages 338–353.
15.
go back to reference Anoosheh A, Agustsson E, Timofte R, Van Gool L (2018) ComboGAN: unrestrained scalability for image domain translation. IEEE/CVF Con Comput Vis Pattern Recognit Workshops (CVPRW) 2018:896–8967 Anoosheh A, Agustsson E, Timofte R, Van Gool L (2018) ComboGAN: unrestrained scalability for image domain translation. IEEE/CVF Con Comput Vis Pattern Recognit Workshops (CVPRW) 2018:896–8967
17.
go back to reference Salimans T, Goodfellow I, Zaremba W, et al. (2016) Improved techniques for training gans. In Proc. of International Conference on Neural Information Processing Systems. Pages 2234–2242, Barcelona, Spain. MIT Press. Salimans T, Goodfellow I, Zaremba W, et al. (2016) Improved techniques for training gans. In Proc. of International Conference on Neural Information Processing Systems. Pages 2234–2242, Barcelona, Spain. MIT Press.
20.
go back to reference Szegedy C., Vanhoucke V., Ioffe S., et al. (2016) Rethinking the Inception Architecture for Computer Vision. IEEE Conference on Computer Vision and Pattern Recognition, Pages 1–10, Las Vegas, USA, IEEE. Szegedy C., Vanhoucke V., Ioffe S., et al. (2016) Rethinking the Inception Architecture for Computer Vision. IEEE Conference on Computer Vision and Pattern Recognition, Pages 1–10, Las Vegas, USA, IEEE.
21.
go back to reference Efros AA, Freeman WT (2001) Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and Interactive Techniques. Pages 341–346, Los Angeles, USA. ACM. Efros AA, Freeman WT (2001) Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and Interactive Techniques. Pages 341–346, Los Angeles, USA. ACM.
22.
go back to reference Justin J, Alexandre A, Li Fei-Fei (2016) Perceptual losses for real-time style transfer and super-resolution. In the Proceedings of European Conference on Computer Vision. Pages 694–711, Amsterdam, Netherlands. Springer. Justin J, Alexandre A, Li Fei-Fei (2016) Perceptual losses for real-time style transfer and super-resolution. In the Proceedings of European Conference on Computer Vision. Pages 694–711, Amsterdam, Netherlands. Springer.
23.
go back to reference Li C, Wand M (2016) Precomputed real-time texture synthesis with markovian generative adversarial networks. In Proc. of European Conference on Computer Vision. Pages 702–716, Amsterdam, Netherlands. Springer. Li C, Wand M (2016) Precomputed real-time texture synthesis with markovian generative adversarial networks. In Proc. of European Conference on Computer Vision. Pages 702–716, Amsterdam, Netherlands. Springer.
24.
go back to reference Liu M-Y, Tuzel O(2016) Coupled generative adversarial networks. In Proc. of International Conference on Neural Information Processing Systems. Pages 469–477, Barcelona, Spain. MIT Press. Liu M-Y, Tuzel O(2016) Coupled generative adversarial networks. In Proc. of International Conference on Neural Information Processing Systems. Pages 469–477, Barcelona, Spain. MIT Press.
25.
go back to reference Mikołaj B., Danica J. , Michael A., et al. (2018) Demystifying MMD GANs. International Conference on Learning Representations, Pages 1–36, Vancouver, Canada. Mikołaj B., Danica J. , Michael A., et al. (2018) Demystifying MMD GANs. International Conference on Learning Representations, Pages 1–36, Vancouver, Canada.
26.
go back to reference Taigman Y, Polyak A, Wolf L (2017) Unsupervised cross-domain image generation. In Proc. 5th International Conference on Learning Representations, Pages 1–6, Toulon, France. ICLR. Taigman Y, Polyak A, Wolf L (2017) Unsupervised cross-domain image generation. In Proc. 5th International Conference on Learning Representations, Pages 1–6, Toulon, France. ICLR.
Metadata
Title
Multidomain image-to-image translation model based on hidden space sharing
Authors
Ding Yuxin
Wang Longfei
Publication date
05-08-2021
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 1/2022
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-021-06386-9

Other articles of this Issue 1/2022

Neural Computing and Applications 1/2022 Go to the issue

Premium Partner