Skip to main content
main-content

Tipp

Weitere Kapitel dieses Buchs durch Wischen aufrufen

2020 | OriginalPaper | Buchkapitel

Demographic Influences on Contemporary Art with Unsupervised Style Embeddings

verfasst von: Nikolai Huckle, Noa Garcia, Yuta Nakashima

Erschienen in: Computer Vision – ECCV 2020 Workshops

Verlag: Springer International Publishing

share
TEILEN

Abstract

Computational art analysis has, through its reliance on classification tasks, prioritised historical datasets in which the artworks are already well sorted with the necessary annotations. Art produced today, on the other hand, is numerous and easily accessible, through the internet and social networks that are used by professional and amateur artists alike to display their work. Although this art—yet unsorted in terms of style and genre—is less suited for supervised analysis, the data sources come with novel information that may help frame the visual content in equally novel ways. As a first step in this direction, we present contempArt, a multi-modal dataset of exclusively contemporary artworks. contempArt is a collection of paintings and drawings, a detailed graph network based on social connections on Instagram and additional socio-demographic information; all attached to 442 artists at the beginning of their career. We evaluate three methods suited for generating unsupervised style embeddings of images and correlate them with the remaining data. We find no connections between visual style on the one hand and social proximity, gender, and nationality on the other.
Fußnoten
6
Images available on artists dedicated webpages are generally of high resolution and only depict their work. Contrary to Instagram, which limits the image resolution by default to \(1080\times 1080\) pixels and where the images uploaded by the artists were often noisy; e.g. taken from a larger distance or of artwork surrounded by objects. Cropping away unnecessary content further reduced the image size.
 
7
Two Instagram accounts were deleted or renamed during the data collection process so only their image data is available.
 
8
Definition of archetype: the original pattern or model of which all things of the same type are representations or copies [39].
 
9
Included styles: Abstract Art, Abstract Expressionism, Art Informel, Art Nouveau (Modern), Baroque, Cubism, Early Renaissance, Expressionism, High Renaissance, Impressionism, Naïve Art (Primitivism), Neoclassicism, Northern Renaissance, Post-Impressionism, Realism, Rococo, Romanticism, Surrealism, Symbolism, Ukiyo-e.
 
Literatur
1.
Zurück zum Zitat Abe, S., Elsner, J.: Introduction: Some Stakes of Comparison. In: Comparativism in Art History, pp. 1–15. Routledge (2017) Abe, S., Elsner, J.: Introduction: Some Stakes of Comparison. In: Comparativism in Art History, pp. 1–15. Routledge (2017)
2.
Zurück zum Zitat Badea, M., Florea, C., Florea, L., Vertan, C.: Can we teach computers to understand art? Domain adaptation for enhancing deep networks capacity to de-abstract art. Image Vis. Comput. 77, 21–32 (2018) CrossRef Badea, M., Florea, C., Florea, L., Vertan, C.: Can we teach computers to understand art? Domain adaptation for enhancing deep networks capacity to de-abstract art. Image Vis. Comput. 77, 21–32 (2018) CrossRef
4.
Zurück zum Zitat Bianco, S., Mazzini, D., Napoletano, P., Schettini, R.: Multitask painting categorization by deep multibranch neural network. Expert Syst. Appl. 135, 90–101 (2019) CrossRef Bianco, S., Mazzini, D., Napoletano, P., Schettini, R.: Multitask painting categorization by deep multibranch neural network. Expert Syst. Appl. 135, 90–101 (2019) CrossRef
5.
Zurück zum Zitat Bishop, C.: Against digital art history. Int. J. Dig. Art History (3), 122–131 (2018) Bishop, C.: Against digital art history. Int. J. Dig. Art History (3), 122–131 (2018)
6.
Zurück zum Zitat Brachmann, A., Barth, E., Redies, C.: Using CNN features to better understand what makes visual artworks special. Front. Psychol. 8, 830 (2017) CrossRef Brachmann, A., Barth, E., Redies, C.: Using CNN features to better understand what makes visual artworks special. Front. Psychol. 8, 830 (2017) CrossRef
8.
Zurück zum Zitat Cetinic, E., Lipic, T., Grgic, S.: A deep learning perspective on beauty, sentiment, and remembrance of art. IEEE Access 7, 73694–73710 (2019) CrossRef Cetinic, E., Lipic, T., Grgic, S.: A deep learning perspective on beauty, sentiment, and remembrance of art. IEEE Access 7, 73694–73710 (2019) CrossRef
9.
Zurück zum Zitat Chen, Y., Mairal, J., Harchaoui, Z.: Fast and robust archetypal analysis for representation learning. In: Proceedings of CVPR, pp. 1478–1485 (2014) Chen, Y., Mairal, J., Harchaoui, Z.: Fast and robust archetypal analysis for representation learning. In: Proceedings of CVPR, pp. 1478–1485 (2014)
10.
Zurück zum Zitat Chu, W.T., Wu, Y.L.: Image style classification based on learnt deep correlation features. IEEE Trans. Multimedia 20(9), 2491–2502 (2018) CrossRef Chu, W.T., Wu, Y.L.: Image style classification based on learnt deep correlation features. IEEE Trans. Multimedia 20(9), 2491–2502 (2018) CrossRef
12.
Zurück zum Zitat Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Proceedings of CVPR, pp. 248–255 (2009) Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Proceedings of CVPR, pp. 248–255 (2009)
13.
Zurück zum Zitat Drucker, J.: Is there a “digital” art history? Vis. Resour. 29(1–2), 5–13 (2013) CrossRef Drucker, J.: Is there a “digital” art history? Vis. Resour. 29(1–2), 5–13 (2013) CrossRef
14.
Zurück zum Zitat Elgammal, A., Liu, B., Kim, D., Elhoseiny, M., Mazzone, M.: The shape of art history in the eyes of the machine. In: Proceedings of AAAI (2018) Elgammal, A., Liu, B., Kim, D., Elhoseiny, M., Mazzone, M.: The shape of art history in the eyes of the machine. In: Proceedings of AAAI (2018)
15.
Zurück zum Zitat Elgammal, A., Saleh, B.: Quantifying creativity in art networks. In: Proceedings of ICCC, p. 39 (2015) Elgammal, A., Saleh, B.: Quantifying creativity in art networks. In: Proceedings of ICCC, p. 39 (2015)
16.
Zurück zum Zitat Garcia, N., Renoust, B., Nakashima, Y.: Context-aware embeddings for automatic art analysis. In: Proceedings of ICMR, pp. 25–33 (2019) Garcia, N., Renoust, B., Nakashima, Y.: Context-aware embeddings for automatic art analysis. In: Proceedings of ICMR, pp. 25–33 (2019)
18.
Zurück zum Zitat Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Proceedings of NeurIPS, pp. 262–270 (2015) Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Proceedings of NeurIPS, pp. 262–270 (2015)
19.
Zurück zum Zitat Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of CVPR, pp. 2414–2423 (2016) Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of CVPR, pp. 2414–2423 (2016)
20.
Zurück zum Zitat Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of SIGKDD, pp. 855–864 (2016) Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of SIGKDD, pp. 855–864 (2016)
21.
Zurück zum Zitat Gultepe, E., Conturo, T.E., Makrehchi, M.: Predicting and grouping digitized paintings by style using unsupervised feature learning. J. Cultur. Heritage 31, 13–23 (2018) CrossRef Gultepe, E., Conturo, T.E., Makrehchi, M.: Predicting and grouping digitized paintings by style using unsupervised feature learning. J. Cultur. Heritage 31, 13–23 (2018) CrossRef
22.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of CVPR, pp. 770–778 (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of CVPR, pp. 770–778 (2016)
23.
Zurück zum Zitat Jaskot, P.B.: Digital art history as the social history of art: towards the disciplinary relevance of digital methods. Vis. Resour. 35(1–2), 21–33 (2019) CrossRef Jaskot, P.B.: Digital art history as the social history of art: towards the disciplinary relevance of digital methods. Vis. Resour. 35(1–2), 21–33 (2019) CrossRef
24.
Zurück zum Zitat Jaskot, P.B.: Digital methods and the historiography of art. The Routledge Companion to Digital Humanities and Art History (2020) Jaskot, P.B.: Digital methods and the historiography of art. The Routledge Companion to Digital Humanities and Art History (2020)
25.
Zurück zum Zitat Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., Song, M.: Neural style transfer: a review. Trans. Vis. Comput. Graph. 26, 3365–3385 (2019) CrossRef Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., Song, M.: Neural style transfer: a review. Trans. Vis. Comput. Graph. 26, 3365–3385 (2019) CrossRef
26.
Zurück zum Zitat Karayev, S., et al.: Recognizing image style. In: Proceedings of BMVC (2013) Karayev, S., et al.: Recognizing image style. In: Proceedings of BMVC (2013)
27.
Zurück zum Zitat Khan, F.S., Beigpour, S., Van de Weijer, J., Felsberg, M.: Painting-91: a large scale database for computational painting categorization. Mach. Vis. Appl. 25(6), 1385–1397 (2014) CrossRef Khan, F.S., Beigpour, S., Van de Weijer, J., Felsberg, M.: Painting-91: a large scale database for computational painting categorization. Mach. Vis. Appl. 25(6), 1385–1397 (2014) CrossRef
28.
Zurück zum Zitat Kim, D., Son, S.W., Jeong, H.: Large-scale quantitative analysis of painting arts. Sci. Rep. 4, 7370 (2014) CrossRef Kim, D., Son, S.W., Jeong, H.: Large-scale quantitative analysis of painting arts. Sci. Rep. 4, 7370 (2014) CrossRef
30.
Zurück zum Zitat Kotovenko, D., Sanakoyeu, A., Lang, S., Ommer, B.: Content and style disentanglement for artistic style transfer. In: Proceedings of ICCV, pp. 4422–4431 (2019) Kotovenko, D., Sanakoyeu, A., Lang, S., Ommer, B.: Content and style disentanglement for artistic style transfer. In: Proceedings of ICCV, pp. 4422–4431 (2019)
31.
Zurück zum Zitat Lee, B., Kim, D., Sun, S., Jeong, H., Park, J.: Heterogeneity in chromatic distance in images and characterization of massive painting data set. PloS ONE 13(9), 1–16 (2018) Lee, B., Kim, D., Sun, S., Jeong, H., Park, J.: Heterogeneity in chromatic distance in images and characterization of massive painting data set. PloS ONE 13(9), 1–16 (2018)
32.
Zurück zum Zitat Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Diversified texture synthesis with feed-forward networks. In: Proceedings of CVPR, pp. 3920–3928 (2017) Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Diversified texture synthesis with feed-forward networks. In: Proceedings of CVPR, pp. 3920–3928 (2017)
34.
Zurück zum Zitat Maaten, L.v.d., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008) Maaten, L.v.d., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)
35.
Zurück zum Zitat Mao, H., Cheung, M., She, J.: DeepArt: learning joint representations of visual arts. In: Proceedings of ACMMM, pp. 1183–1191 (2017) Mao, H., Cheung, M., She, J.: DeepArt: learning joint representations of visual arts. In: Proceedings of ACMMM, pp. 1183–1191 (2017)
36.
Zurück zum Zitat Matsuo, S., Yanai, K.: CNN-based style vector for style image retrieval. In: Proceedings of ICMR, pp. 309–312 (2016) Matsuo, S., Yanai, K.: CNN-based style vector for style image retrieval. In: Proceedings of ICMR, pp. 309–312 (2016)
37.
Zurück zum Zitat Mensink, T., Van Gemert, J.: The Rijksmuseum challenge: museum-centered visual recognition. In: Proceedings of ICMR, pp. 451–454 (2014) Mensink, T., Van Gemert, J.: The Rijksmuseum challenge: museum-centered visual recognition. In: Proceedings of ICMR, pp. 451–454 (2014)
38.
Zurück zum Zitat Mercuriali, G.: Digital art history and the computational imagination. In: International Journal for Digital Art History: Issue 3, 2018: Digital Space and Architecture, vol. 3, p. 141 (2019) Mercuriali, G.: Digital art history and the computational imagination. In: International Journal for Digital Art History: Issue 3, 2018: Digital Space and Architecture, vol. 3, p. 141 (2019)
40.
Zurück zum Zitat Salah, A.A., Manovich, L., Salah, A.A., Chow, J.: Combining cultural analytics and networks analysis: studying a social network site with user-generated content. J. Broadcast. Electron. Med. 57(3), 409–426 (2013) CrossRef Salah, A.A., Manovich, L., Salah, A.A., Chow, J.: Combining cultural analytics and networks analysis: studying a social network site with user-generated content. J. Broadcast. Electron. Med. 57(3), 409–426 (2013) CrossRef
41.
Zurück zum Zitat Salah, A.A., Salah, A.A., Buter, B., Dijkshoorn, N., Modolo, D., Nguyen, Q., van Noort, S., van de Poel, B.: DeviantArt in spotlight: a network of artists. Leonardo 45(5), 486–487 (2012) CrossRef Salah, A.A., Salah, A.A., Buter, B., Dijkshoorn, N., Modolo, D., Nguyen, Q., van Noort, S., van de Poel, B.: DeviantArt in spotlight: a network of artists. Leonardo 45(5), 486–487 (2012) CrossRef
42.
Zurück zum Zitat Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. Trans. Appl. Percept. 7(2), 1–17 (2010) CrossRef Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. Trans. Appl. Percept. 7(2), 1–17 (2010) CrossRef
43.
Zurück zum Zitat Shamir, L., Tarakhovsky, J.A.: Computer analysis of art. J. Comput. Cultur. Heritag. 5(2), 1–11 (2012) CrossRef Shamir, L., Tarakhovsky, J.A.: Computer analysis of art. J. Comput. Cultur. Heritag. 5(2), 1–11 (2012) CrossRef
44.
Zurück zum Zitat Sigaki, H.Y., Perc, M., Ribeiro, H.V.: History of art paintings through the lens of entropy and complexity. Proc. Natl. Acad. Sci. 115(37), E8585–E8594 (2018) CrossRef Sigaki, H.Y., Perc, M., Ribeiro, H.V.: History of art paintings through the lens of entropy and complexity. Proc. Natl. Acad. Sci. 115(37), E8585–E8594 (2018) CrossRef
45.
Zurück zum Zitat Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of ICLR (2015) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of ICLR (2015)
46.
Zurück zum Zitat Spehr, M., Wallraven, C., Fleming, R.W.: Image statistics for clustering paintings according to their visual appearance. In: Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging, pp. 57–64 (2009) Spehr, M., Wallraven, C., Fleming, R.W.: Image statistics for clustering paintings according to their visual appearance. In: Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging, pp. 57–64 (2009)
47.
Zurück zum Zitat Strezoski, G., Worring, M.: OmniArt: a large-scale artistic benchmark. TOMM 14(4), 1–21 (2018) CrossRef Strezoski, G., Worring, M.: OmniArt: a large-scale artistic benchmark. TOMM 14(4), 1–21 (2018) CrossRef
48.
Zurück zum Zitat Taylor, R.P., Micolich, A.P., Jonas, D.: Fractal analysis of pollock’s drip paintings. Nature 399(6735), 422–422 (1999) CrossRef Taylor, R.P., Micolich, A.P., Jonas, D.: Fractal analysis of pollock’s drip paintings. Nature 399(6735), 422–422 (1999) CrossRef
49.
Zurück zum Zitat Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, 2837–2854 (2010) MathSciNetMATH Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, 2837–2854 (2010) MathSciNetMATH
50.
Zurück zum Zitat Wynen, D., Schmid, C., Mairal, J.: Unsupervised learning of artistic styles with archetypal style analysis. In: Proceedings of NeurIPS, pp. 6584–6593 (2018) Wynen, D., Schmid, C., Mairal, J.: Unsupervised learning of artistic styles with archetypal style analysis. In: Proceedings of NeurIPS, pp. 6584–6593 (2018)
51.
Zurück zum Zitat Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:​1708.​07747 (2017) Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:​1708.​07747 (2017)
52.
Zurück zum Zitat Yazdani, M., Chow, J., Manovich, L.: Quantifying the development of user-generated art during 2001–2010. PloS ONE 12(8), 1–24 (2017) Yazdani, M., Chow, J., Manovich, L.: Quantifying the development of user-generated art during 2001–2010. PloS ONE 12(8), 1–24 (2017)
53.
Zurück zum Zitat Yeh, M.C., Tang, S., Bhattad, A., Zou, C., Forsyth, D.: Improving style transfer with calibrated metrics. In: Proceedings of WACV, pp. 3160–3168 (2020) Yeh, M.C., Tang, S., Bhattad, A., Zou, C., Forsyth, D.: Improving style transfer with calibrated metrics. In: Proceedings of WACV, pp. 3160–3168 (2020)
54.
Zurück zum Zitat Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of ICCV, pp. 2223–2232 (2017) Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of ICCV, pp. 2223–2232 (2017)
Metadaten
Titel
Demographic Influences on Contemporary Art with Unsupervised Style Embeddings
verfasst von
Nikolai Huckle
Noa Garcia
Yuta Nakashima
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-66096-3_10

Premium Partner