Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Multimedia Systems
Published in: Multimedia Systems 2/2024

01-04-2024 | Regular Paper

Arbitrary style transfer method with attentional feature distribution matching

Authors: Bin Ge, Zhenshan Hu, Chenxing Xia, Junming Guan

Published in: Multimedia Systems | Issue 2/2024

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Most arbitrary style transfer methods only consider transferring the features of the style and content images. Although the pixel-wise style transfer is achieved. It is limited to preserving the content structure, the model tends to transfer the style features, and the loss of image information occurs during the transfer process. The model incline to transfer the style features and preservation of the content structure is weak. The generated pictures will produce artifacts and patterns of style pictures. In this paper, an attention feature distribution matching method for arbitrary style transfer is proposed. In network architecture, a combination of the self-attention mechanism and second-order statistics is used to perform style transfer, and the style strengthen block enhances the style features of generated images. In the loss function, the traditional content loss is not used. We integrate the attention mechanism and feature distribution matching to construct the loss function. The constraints are strengthened to avoid artifacts in the generated image. Qualitative and quantitative experiments demonstrate the effectiveness of our method compared with state-of-the-art arbitrary style transfer in improving arbitrary style transfer quality.
previous article Design of integrated interactive system for pre-diagnosis of breast cancer pathological images based on CNN and PyQt5
next article Multiscale image denoising algorithm based on UNet3+

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Gatys, L. A., Ecker, A. S., Bethge, M.: Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2414–2423) (2016)
2.
Wu, H., Sun, Z., Yuan, W.: Direction-aware neural style transfer. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 1163–1171 (2018)
3.
Ulyanov, D., Vedaldi, A., Lempitsky, V.:. Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6924–6932, (2017)
4.
Kotovenko, D., Sanakoyeu, A., Lang, S., Ommer, B.: Content and style disentanglement for artistic style transfer. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 4422–4431 (2019)
5.
Chen H, Zhao L, Zhang H, et al. Diverse image style transfer via invertible cross-space mapping[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, 2021: 14860–14869.
6.
Risser E.: Optimal textures: Fast and robust texture synthesis and style transfer through optimal transport[J] (2020). arXiv preprint arXiv:​2010.​14702
7.
Kotovenko D, Wright M, Heimbrecht A, et al. Rethinking style transfer: from pixels to parameterized brushstrokes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12196–12205 (2021)
8.
Lin T, Ma Z, Li F, He D, Li X, Ding E, Wang N, Li J, Gao X:. Drafting and revision: Laplacian pyramid network for fast high-quality artistic style transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5141–5150, (2021)
9.
Chen, D., Yuan, L., Liao, J., Yu, N., StyleBank, H.G.: An explicit representation for neural image style transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–10 (2017)
10.
11.
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Diversified texture synthesis with feed-forward networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3920–3928 (2017)
12.
Zhang, H., Dana, K.: Multi-style generative network for real-time transfer. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0–0 (2018)
13.
Chen, H., Wang, Z., Zhang, H., Zuo, Z., Li, A., Xing, W., Lu, D.: Artistic style transfer with internal-external learning and contrastive learning. Adv. Neural. Inf. Process. Syst. 34, 26561–26573 (2021)
14.
15.
Deng, Y., Tang, F., Dong, W., Huang, H., Ma, C., Xu, C.: Arbitrary video style transfer via multi-channel correlation. In: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, No. 2, pp. 1210–1217 (2021)
16.
Deng, Y., Tang, F., Dong, W., Sun, W., Huang, F., Xu, C.: Arbitrary style transfer via multi-adaptation network. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 2719–2727 (2020)
17.
Zhang, Y., Tian, Y., Hou, J.: CSAST: Content self-supervised and style contrastive learning for arbitrary style transfer. Neural Netw. 164, 146–155 (2023)CrossRef
18.
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)
19.
Li, X., Liu, S., Kautz, J., Yang, M.H.: Learning linear transformations for fast arbitrary style transfer (2018). arXiv preprint arXiv:​1808.​04537.
20.
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Ang, M.H.: Universal style transfer via feature transforms. In: Advances in neural information processing systems, 30 (2017)
21.
Park, D.Y., Lee, K.H.: Arbitrary style transfer with style-attentional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5880–5888 (2019)
22.
Sheng, L., Lin, Z., Shao, J., Wang, X.: Avatar-net: Multi-scale zero-shot style transfer by feature decoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8242–8250 (2018)
23.
Wang Z, Zhao L, Zuo Z, et al. MicroAST: towards super-fast ultra-resolution arbitrary style transfer. In: Proceedings of the AAAI Conference on Artificial Intelligence. 37(3): 2742–2750 (2023)
24.
25.
Liu, S., Lin, T., He, D., Li, F., Wang, M., Li, X., Ding, E.: Adaattn: Revisit attention mechanism in arbitrary neural style transfer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6649–6658 (2021)
26.
Deng, Y., Tang, F., Dong, W., Ma, C., Pan, X., Wang, L., Xu, C.: Stytr2: Image style transfer with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11326–11336 (2022)
27.
An, J., Huang, S., Song, Y., Dou, D., Liu, W., Luo, J.: Artflow: Unbiased image style transfer via reversible neural flows. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 862–871 (2021)
28.
Yao, Y., Ren, J., Xie, X., Liu, W., Liu, Y. J., Wang, J.: Attention-aware multi-stroke style transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1467–1475 (2019)
29.
Wu, Z., Zhu, Z., Du, J., & Bai, X: CCPL: contrastive coherence preserving loss for versatile style transfer. In: European Conference on Computer Vision (pp. 189–206). Cham: Springer Nature Switzerland (2022)
30.
Zhang Y, Li M, Li R, et al. Exact feature distribution matching for arbitrary style transfer and domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8035–8045 (2022)
31.
Hong, K., Jeon, S., Lee, J., Ahn, N., Kim, K., Lee, P., Byun, H.: AesPA-Net: aesthetic pattern-aware style transfer networks (2023). arXiv preprint arXiv:​2307.​09724.
32.
33.
Wang, X., Wang, W., Yang, S., et al.: CLAST: contrastive learning for arbitrary style transfer[J]. IEEE Trans. Image Process. 31, 6761–6772 (2022)CrossRef
34.
Wang Z, Zhang Z, Zhao L, et al.: AesUST: towards aesthetic-enhanced universal style transfer. In: Proceedings of the 30th ACM International Conference on Multimedia. pp. 1095–1106 (2022)
35.
Wu Z, Song C, Zhou Y, et al. Efanet: Exchangeable feature alignment network for arbitrary style transfer[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(07): 12305–12312.
36.
Xin, H., Li, L.: Arbitrary style transfer with fused convolutional block attention modules[J]. IEEE Access 11, 4497 (2023)CrossRef
37.
38.
Chen, H., Zhao, L., Wang, Z., Zhang, H., Zuo, Z., Li, A., Lu, D.: Dualast: Dual style-learning networks for artistic style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 872–881 (2021)
39.
Lin, T.Y., Maire, M., Belongie, S., et al.: Microsoft coco: Common objects in context. In: European conference on computer vision, pp. 740–755. Springer, Cham (2014)
40.
Metadata
Title
Arbitrary style transfer method with attentional feature distribution matching
Authors
Bin Ge
Zhenshan Hu
Chenxing Xia
Junming Guan
Publication date
01-04-2024
Publisher
Springer Berlin Heidelberg
Published in
Multimedia Systems / Issue 2/2024
Print ISSN: 0942-4962
Electronic ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-024-01300-4

Other articles of this Issue 2/2024

Multimedia Systems 2/2024 Go to the issue

Regular Paper

Domain-adaptive person re-identification via domain alignment and mutual pseudo-label refinement

Regular Paper

SMA-GCN: a fall detection method based on spatio-temporal relationship

Regular Paper

Mmy-net: a multimodal network exploiting image and patient metadata for simultaneous segmentation and diagnosis

Regular Paper

Dual-band low-light image enhancement

Regular Paper

Integrating user-side information into matrix factorization to address data sparsity of collaborative filtering

Regular Paper

Assessing the adoption of the Yavuz Battleship application in the mixed reality environment using the technology acceptance model