Top

Published in:

2021 | OriginalPaper | Chapter

A Brief Overview of Deep Learning Approaches to Pattern Extraction and Recognition in Paintings and Drawings

Authors : Giovanna Castellano, Gennaro Vessio

Published in: Pattern Recognition. ICPR International Workshops and Challenges

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This paper provides a brief overview of some of the most relevant deep learning approaches to visual art pattern extraction and recognition, particularly painting and drawing. Indeed, recent advances in deep learning and computer vision, coupled with the growing availability of large digitized visual art collections, have opened new opportunities for computer science researchers to assist the art community with automatic tools to analyze and further understand visual arts. Among other benefits, a deeper understanding of visual arts has the potential to make them more accessible to a wider population, both in terms of fruition and creation, thus supporting the spread of culture.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Investigating the Effect of Inter-letter Spacing Modulation on Data-Driven Detection of Developmental Dyslexia Based on Eye-Movement Correlates of Reading: A Machine Learning Approach

next chapter Iconographic Image Captioning for Artworks

https://www.wikiart.org.

https://www.metmuseum.org.

Aggarwal, G., Parikh, D.: Neuro-symbolic generative art: a preliminary study. arXiv preprint arXiv:2007.02171 (2020)

Amizadeh, S., Palangi, H., Polozov, O., Huang, Y., Koishida, K.: Neuro-symbolic visual reasoning: Disentangling “visual” from “reasoning”. arXiv preprint arXiv:2006.11524 (2020)

Bar, Y., Levy, N., Wolf, L.: Classification of artistic styles using binarized features derived from a deep neural network. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 71–84. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_5CrossRef

Belhi, A., Bouras, A., Foufou, S.: Leveraging known data for missing label prediction in cultural heritage context. Appl. Sci. 8(10), 1768 (2018)CrossRef

Cai, H., Wu, Q., Corradi, T., Hall, P.: The cross-depiction problem: computer vision algorithms for recognising objects in artwork and in photographs. arXiv preprint arXiv:1505.00110 (2015)

Cai, H., Wu, Q., Hall, P.: Beyond photo-domain object recognition: Benchmarks for the cross-depiction problem. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–6 (2015)

Carneiro, G., da Silva, N.P., Del Bue, A., Costeira, J.P.: Artistic image classification: an analysis on the PRINTART database. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 143–157. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33765-9_11 CrossRef

Castellano, G., Lella, E., Vessio, G.: Visual link retrieval and knowledge discovery in painting datasets. Multimedia Tools Appl. (2020, in press)

Castellano, G., Vessio, G.: Towards a tool for visual link retrieval and knowledge discovery in painting datasets. In: Ceci, M., Ferilli, S., Poggi, A. (eds.) IRCDL 2020. CCIS, vol. 1177, pp. 105–110. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39905-4_11CrossRef

10.

Cetinic, E., Lipic, T., Grgic, S.: Fine-tuning convolutional neural networks for fine art classification. Expert Syst. Appl. 114, 107–118 (2018)CrossRef

11.

Chen, L., Yang, J.: Recognizing the style of visual arts via adaptive cross-layer correlation. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2459–2467 (2019)

12.

Crowley, E.J., Zisserman, A.: In search of art. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 54–70. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_4CrossRef

13.

Crowley, E.J., Zisserman, A.: The art of detection. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 721–737. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_50CrossRef

14.

Elgammal, A., Liu, B., Elhoseiny, M., Mazzone, M.: CAN: creative adversarial networks, generating “art” by learning about styles and deviating from style norms. arXiv preprint arXiv:1706.07068 (2017)

15.

Garcia, N., Renoust, B., Nakashima, Y.: ContextNet: representation and exploration for painting classification and retrieval in context. Int. J. Multimedia Inf. Retr. 9(1), 17–30 (2020). https://doi.org/10.1007/s13735-019-00189-4CrossRef

16.

Garcia, N., Vogiatzis, G.: How to read paintings: semantic art understanding with multi-modal retrieval. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11130, pp. 676–691. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11012-3_52CrossRef

17.

Ginosar, S., Haas, D., Brown, T., Malik, J.: Detecting people in cubist art. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 101–116. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_7CrossRef

18.

Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

19.

Gonthier, N., Gousseau, Y., Ladjal, S., Bonfait, O.: Weakly supervised object detection in artworks. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11130, pp. 692–709. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11012-3_53CrossRef

20.

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

21.

van de Kamp, M.T., Admiraal, W., van Drie, J., Rijlaarsdam, G.: Enhancing divergent thinking in visual arts education: effects of explicit instruction of meta-cognition. Br. J. Educ. Psychol. 85(1), 47–58 (2015)CrossRef

22.

Karayev, S., et al.: Recognizing image style. arXiv preprint arXiv:1311.3715 (2013)

23.

Khan, F.S., Beigpour, S., Van de Weijer, J., Felsberg, M.: Painting-91: a large scale database for computational painting categorization. Mach. Vis. Appl. 25(6), 1385–1397 (2014)CrossRef

24.

Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

25.

Leavy, P.: Handbook of Arts-Based Research. Guilford Publications, New York (2017)

26.

LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRef

27.

Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., Alsaadi, F.E.: A survey of deep neural network architectures and their applications. Neurocomputing 234, 11–26 (2017)CrossRef

28.

Mao, H., Cheung, M., She, J.: DeepArt: learning joint representations of visual arts. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1183–1191. ACM (2017)

29.

Mercuriali, G.: Digital art history and the computational imagination. Int. J. Digit. Art Hist. Issue 3 2018: Digit. Space Architect. 3, 141 (2019)

30.

Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

31.

Saleh, B., Abe, K., Arora, R.S., Elgammal, A.: Toward automated discovery of artistic influence. Multimedia Tools Appl. 75(7), 3565–3591 (2014). https://doi.org/10.1007/s11042-014-2193-xCrossRef

32.

Saleh, B., Elgammal, A.: Large-scale classification of fine-art paintings: learning the right metric on the right feature. arXiv preprint arXiv:1505.00855 (2015)

33.

Sandoval, C., Pirogova, E., Lech, M.: Two-stage deep learning approach to the classification of fine-art paintings. IEEE Access 7, 41770–41781 (2019)CrossRef

34.

Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. ACM Trans. Appl. Percept. (TAP) 7(2), 8 (2010)

35.

Shen, X., Efros, A.A., Aubry, M.: Discovering visual patterns in art collections with spatially-consistent feature learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9278–9287 (2019)

36.

Stefanini, M., Cornia, M., Baraldi, L., Corsini, M., Cucchiara, R.: Artpedia: a new visual-semantic dataset with visual and contextual sentences in the artistic domain. In: Ricci, E., Rota Bulò, S., Snoek, C., Lanz, O., Messelodi, S., Sebe, N. (eds.) ICIAP 2019. LNCS, vol. 11752, pp. 729–740. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30645-8_66CrossRef

37.

Strezoski, G., Worring, M.: OmniArt: multi-task deep learning for artistic data analysis. arXiv preprint arXiv:1708.00684 (2017)

38.

Tan, W.R., Chan, C.S., Aguirre, H.E., Tanaka, K.: Ceci n’est pas une pipe: a deep convolutional network for fine-art paintings classification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3703–3707. IEEE (2016)

39.

Tan, W.R., Chan, C.S., Aguirre, H.E., Tanaka, K.: Improved ArtGan for conditional synthesis of natural image and artwork. IEEE Trans. Image Process. 28(1), 394–409 (2018)MathSciNetCrossRef

40.

Tomei, M., Cornia, M., Baraldi, L., Cucchiara, R.: Art2Real: unfolding the reality of artworks via semantically-aware image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5849–5859 (2019)

41.

Van Noord, N., Hendriks, E., Postma, E.: Toward discovery of the artist’s style: learning to recognize artists by their artworks. IEEE Signal Process. Mag. 32(4), 46–54 (2015)CrossRef

42.

Westlake, N., Cai, H., Hall, P.: Detecting people in artwork with CNNs. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 825–841. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_57CrossRef

43.

Wilber, M.J., Fang, C., Jin, H., Hertzmann, A., Collomosse, J., Belongie, S.: BAM! The behance artistic media dataset for recognition beyond photography. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1202–1211 (2017)

Title: A Brief Overview of Deep Learning Approaches to Pattern Extraction and Recognition in Paintings and Drawings
Authors: Giovanna Castellano
Gennaro Vessio
Publisher: Springer International Publishing
Book: Pattern Recognition. ICPR International Workshops and Challenges
Print ISBN: 978-3-030-68795-3

Electronic ISBN: 978-3-030-68796-0

Copyright Year: 2021
DOI: https://doi.org/10.1007/978-3-030-68796-0_35

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner