Top

Published in:

2021 | OriginalPaper | Chapter

Dissecting Neural Networks Filter Responses for Artistic Style Transfer

Authors : Florian Uhde, Sanaz Mostaghim

Published in: Artificial Intelligence in Music, Sound, Art and Design

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Current developments in the field of Artistic Style Transfer use the information encoded in pre-trained neural networks to extract properties from images in an unsupervised process. This neural style transfer works well with art and paintings but only produces limited results when dealing with highly structured data. Characteristics of the extracted information directly define the quality of the generated artifact and traditionally require the user to do manual fine-tuning. This paper uses current methods of deep learning to analyze the properties embedded in the network, group filter responses into semantic classes and extract an optimized layer set for artistic style transfer, to improve the artifact generation with a potentially unsupervised preprocessing step.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Incremental Evolution of Stylized Images

next chapter A Fusion of Deep and Shallow Learning to Predict Genres Based on Instrument and Timbre Features

Available only for authorised users

Chen, T.Q., Schmidt, M.: Fast Patch-based Style Transfer of Arbitrary Style. arXiv preprint arXiv:1612.04337 (2016). (NIPS) (2016). http://arxiv.org/abs/1612.04337

Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. In: Proceedings of ICLR (2017). http://arxiv.org/abs/1610.07629

Elad, M., Milanfar, P.: Style-transfer via texture-synthesis. IEEE Trans. Image Process. 26(5) (2017). http://arxiv.org/abs/1609.03057

Gatys, L.A., Ecker, A.S., Bethge, M.: A Neural Algorithm of Artistic Style, pp. 3–7 (2015)

Li, S., Xu, X., Nie, L., Chua, T.S.: Laplacian-Steered Neural Style Transfer (2017). ArXiv:1707.01253

Uhde, F.: Applicability of Convolutional Neural Network Artistic Style Transfer Algorithms (2019)

Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)

Umetani, N.: Exploring generative 3D shapes using autoencoder networks. In: SIGGRAPH Asia 2017 Technical Briefs on - SA’17, pp. 1–4 (2017). http://dl.acm.org/citation.cfm?doid=3145749.3145758

Efros, A., Freeman, W.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (August), pp. 1–6 (2001)

10.

Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image analogies. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques SIGGRAPH’01 2001(August), pp. 327–340 (2001)

11.

Kuri, D., Root, E., Theisel, H.: Hexagonal Image Quilting for Texture Synthesis, pp. 67–76 (2014)

12.

Gatys, L.A., Ecker, A.S., Bethge, M.: Texture Synthesis Using Convolutional Neural Networks. In: Neural Image Processing Systems, pp. 1–10 (2015)

13.

Uhde, F., Mostaghim, S.: Towards a general framework for artistic style transfer. In: Liapis, A., Romero Cardalda, J.J., Ekárt, A. (eds.) Computational Intelligence in Music, Sound, Art and Design. LNCS, pp. 177–193. Springer, Cham (2018)CrossRef

14.

Li, C., Wand, M.: Precomputed real-time texture synthesis with Markovian generative adversarial networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision - ECCV 2016. Lecture Notes in Computer Science(Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9907, pp. 702–716. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_43CrossRef

15.

Lim, I., Gehre, A., Kobbelt, L.: Identifying style of 3D shapes using deep metric learning. Comput. Graph. Forum 35(5), 207–215 (2016)CrossRef

16.

Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep Painterly Harmonization. Comput. Graph. Forum 37(4), 95–106 (2018)CrossRef

17.

Li, C., Wand, M.: Combining Markov random fields and convolutional neural networks for image synthesis. CVPR 2016, 9 (2016)

18.

Mordvintsev, A., Christopher Olah, M.T.: Google AI Blog: Inceptionism: Going Deeper into Neural Networks (2015). https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html

19.

Zeiler, M.D., Fergus, R.: Visualizing and Understanding Convolutional Networks. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). LNCS, PART 1, vol. 8689, pp. 818–833 . Springer, Cham (2014)

20.

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. ImageNet Challenge 96(2), 1–10 (2014)

21.

Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRef

22.

Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3D object representations for fine-grained categorization. In: 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia (2013)

23.

Kuznetsova, A., et al.: The Open Images Dataset V4: unified image classification, object detection, and visual relationship detection at scale, pp. 1–20 (2018). http://arxiv.org/abs/1811.00982

24.

Mordvintsev, A., Olah, C., Tyka, M.: Research blog: DeepDream - a code example for visualizing Neural Networks (2015). http://googleresearch.blogspot.ie/2015/07/deepdream-code-example-for-visualizing.html

25.

Chollet, F.: Deep Learning with Python. Manning Publications Company (2017). https://books.google.de/books?id=Yo3CAQAACAAJ

26.

Hoshen, Y., Malik, J.: Non-Adversarial Image Synthesis with Generative Latent Nearest Neighbors. Technical report, Facebook AI Research (2018). https://arxiv.org/abs/1812.08985

27.

Schmidhuber, J.: Deep Learning in Neural Networks : An Overview, pp. 1–88 (2014)

Title: Dissecting Neural Networks Filter Responses for Artistic Style Transfer
Authors: Florian Uhde
Sanaz Mostaghim
Publisher: Springer International Publishing
Book: Artificial Intelligence in Music, Sound, Art and Design
Print ISBN: 978-3-030-72913-4

Electronic ISBN: 978-3-030-72914-1

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-72914-1_20

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner