Skip to main content
Erschienen in: Machine Vision and Applications 6/2014

01.08.2014 | Original Paper

Painting-91: a large scale database for computational painting categorization

verfasst von: Fahad Shahbaz Khan, Shida Beigpour, Joost van de Weijer, Michael Felsberg

Erschienen in: Machine Vision and Applications | Ausgabe 6/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Computer analysis of visual art, especially paintings, is an interesting cross-disciplinary research domain. Most of the research in the analysis of paintings involve medium to small range datasets with own specific settings. Interestingly, significant progress has been made in the field of object and scene recognition lately. A key factor in this success is the introduction and availability of benchmark datasets for evaluation. Surprisingly, such a benchmark setup is still missing in the area of computational painting categorization. In this work, we propose a novel large scale dataset of digital paintings. The dataset consists of paintings from 91 different painters. We further show three applications of our dataset namely: artist categorization, style classification and saliency detection. We investigate how local and global features popular in image classification perform for the tasks of artist and style categorization. For both categorization tasks, our experimental results suggest that combining multiple features significantly improves the final performance. We show that state-of-the-art computer vision methods can correctly classify 50 % of unseen paintings to its painter in a large dataset and correctly attribute its artistic style in over 60 % of the cases. Additionally, we explore the task of saliency detection on paintings and show experimental findings using state-of-the-art saliency estimation algorithms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Arora, R., Elgammal, A.: Towards automated classification of fine-art painting style: a comparative study. In: Proceedings of the ICPR (2012) Arora, R., Elgammal, A.: Towards automated classification of fine-art painting style: a comparative study. In: Proceedings of the ICPR (2012)
2.
Zurück zum Zitat Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the CIVR (2007) Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the CIVR (2007)
3.
Zurück zum Zitat Carneiro, G., da Silva, N.P., Bue, A.D., Costeira, J.P.: Artistic image classification: an analysis on the printart database. In: Proceedings of the ECCV (2012) Carneiro, G., da Silva, N.P., Bue, A.D., Costeira, J.P.: Artistic image classification: an analysis on the printart database. In: Proceedings of the ECCV (2012)
4.
Zurück zum Zitat Cinzia Di Dio, G.R., Macaluso, E.: The golden beauty: brain response to classical and renaissance sculptures. Plos One 2(11), e1201 (2007) Cinzia Di Dio, G.R., Macaluso, E.: The golden beauty: brain response to classical and renaissance sculptures. Plos One 2(11), e1201 (2007)
5.
Zurück zum Zitat Condorovici, R.G., Vranceanu, R., Vertan, C.: Saliency map retrieval for artistic paintings inspired from human understanding. In: Proceedings of the SPAMEC (2011) Condorovici, R.G., Vranceanu, R., Vertan, C.: Saliency map retrieval for artistic paintings inspired from human understanding. In: Proceedings of the SPAMEC (2011)
6.
Zurück zum Zitat Csurka, G., Bray, C., Dance, C., Fan, L.: Visual categorization with bags of keypoints. In: Proceedings of the Workshop on Statistical Learning in Computer Vision, ECCV (2004) Csurka, G., Bray, C., Dance, C., Fan, L.: Visual categorization with bags of keypoints. In: Proceedings of the Workshop on Statistical Learning in Computer Vision, ECCV (2004)
7.
Zurück zum Zitat Danelljan, M., Khan, F.S., Felsberg, M., van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: Proceedings of the CVPR (2014) Danelljan, M., Khan, F.S., Felsberg, M., van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: Proceedings of the CVPR (2014)
8.
Zurück zum Zitat Deng, J., Dong, W., Socher, R., Jia, L., Kai, L., Li, F.F.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the CVPR (2009) Deng, J., Dong, W., Socher, R., Jia, L., Kai, L., Li, F.F.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the CVPR (2009)
9.
Zurück zum Zitat Elfiky, N., Khan, F.S., van de Weijer, J., Gonzalez, J.: Discriminative compact pyramids for object and scene recognition. Pattern Recognit. 45(4), 1627–1636 (2012)CrossRefMATH Elfiky, N., Khan, F.S., van de Weijer, J., Gonzalez, J.: Discriminative compact pyramids for object and scene recognition. Pattern Recognit. 45(4), 1627–1636 (2012)CrossRefMATH
10.
Zurück zum Zitat Everingham, M., Gool, L.J.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRef Everingham, M., Gool, L.J.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRef
11.
Zurück zum Zitat Gehler, P.V., Nowozin, S.: On feature combination for multiclass object classification. In: Proceedings of the ICCV (2009) Gehler, P.V., Nowozin, S.: On feature combination for multiclass object classification. In: Proceedings of the ICCV (2009)
12.
Zurück zum Zitat Goguen, J.: Art and the Brain. Imprint Academic, UK (1999) Goguen, J.: Art and the Brain. Imprint Academic, UK (1999)
13.
Zurück zum Zitat Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. Trans. Image Process. 19(6), 1657–1663 (2010)CrossRefMathSciNet Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. Trans. Image Process. 19(6), 1657–1663 (2010)CrossRefMathSciNet
14.
Zurück zum Zitat Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Proceedings of the NIPS (2006) Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Proceedings of the NIPS (2006)
15.
Zurück zum Zitat Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the CVPR (2007) Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the CVPR (2007)
16.
Zurück zum Zitat Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. Pattern Anal. Mach. Intell. 34(1), 194–201 (2012)CrossRef Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. Pattern Anal. Mach. Intell. 34(1), 194–201 (2012)CrossRef
17.
Zurück zum Zitat Itti, L., Cristof, K., Ernst, N.: A model of saliency-based visual attention for rapid scene analysis. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRef Itti, L., Cristof, K., Ernst, N.: A model of saliency-based visual attention for rapid scene analysis. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRef
18.
Zurück zum Zitat Jiang, S., Huang, Q., Ye, Q., Gao, W.: An effective method to detect and categorize digitized traditional chinese paintings. Pattern Recognit. Lett. 27(7), 734–746 (2006)CrossRef Jiang, S., Huang, Q., Ye, Q., Gao, W.: An effective method to detect and categorize digitized traditional chinese paintings. Pattern Recognit. Lett. 27(7), 734–746 (2006)CrossRef
19.
Zurück zum Zitat Johnson, R., Hendriks, E., Berezhnoy, I.J., Brevdo, E., Hughes, S.M., Daubechies, I., Li, J., Postma, E., Wang, J.Z.: Image processing for artist identification. IEEE Signal Process. Mag. 25(4), 37–48 (2008)CrossRef Johnson, R., Hendriks, E., Berezhnoy, I.J., Brevdo, E., Hughes, S.M., Daubechies, I., Li, J., Postma, E., Wang, J.Z.: Image processing for artist identification. IEEE Signal Process. Mag. 25(4), 37–48 (2008)CrossRef
20.
Zurück zum Zitat Judd, T., Ehinger, K.A., Durand, F., Torralba, A.: Learning to predict where humans look. In: Proceedings of the ICCV (2009) Judd, T., Ehinger, K.A., Durand, F., Torralba, A.: Learning to predict where humans look. In: Proceedings of the ICCV (2009)
21.
Zurück zum Zitat Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A.D., Vanrell, M., Lopez, A.M.: Color attributes for object detection. In: Proceedings of the CVPR (2012a) Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A.D., Vanrell, M., Lopez, A.M.: Color attributes for object detection. In: Proceedings of the CVPR (2012a)
22.
Zurück zum Zitat Khan, F.S., van de Weijer, J., Ali, S., Felsberg, M.: Evaluating the impact of color on texture recognition. In: Proceedings of the CAIP (2013b) Khan, F.S., van de Weijer, J., Ali, S., Felsberg, M.: Evaluating the impact of color on texture recognition. In: Proceedings of the CAIP (2013b)
23.
Zurück zum Zitat Khan, F.S., van de Weijer, J., Bagdanov, A.D., Vanrell, M.: Portmanteau vocabularies for multi-cue image representations. In: Proceedings of the NIPS (2011) Khan, F.S., van de Weijer, J., Bagdanov, A.D., Vanrell, M.: Portmanteau vocabularies for multi-cue image representations. In: Proceedings of the NIPS (2011)
24.
Zurück zum Zitat Khan, F.S., van de Weijer, J., Vanrell, M.: Top-down color attention for object recognition. In: Proceedings of the ICCV (2009) Khan, F.S., van de Weijer, J., Vanrell, M.: Top-down color attention for object recognition. In: Proceedings of the ICCV (2009)
25.
Zurück zum Zitat Khan, F.S., van de Weijer, J., Vanrell, M.: Modulating shape features by color attention for object recognition. Int. J. Comput. Vis. 98(1), 49–64 (2012b)CrossRef Khan, F.S., van de Weijer, J., Vanrell, M.: Modulating shape features by color attention for object recognition. Int. J. Comput. Vis. 98(1), 49–64 (2012b)CrossRef
26.
Zurück zum Zitat Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A., Lopez, A., Felsberg, M.: Coloring action recognition in still images. Int. J. Comput. Vis. 105(3), 205–221 (2013a)CrossRef Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A., Lopez, A., Felsberg, M.: Coloring action recognition in still images. Int. J. Comput. Vis. 105(3), 205–221 (2013a)CrossRef
27.
Zurück zum Zitat Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the CVPR (2006) Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the CVPR (2006)
28.
Zurück zum Zitat Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)CrossRef Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)CrossRef
29.
Zurück zum Zitat Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. Pattern Anal. Mach. Intell. 32(2), 353–367 (2011) Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. Pattern Anal. Mach. Intell. 32(2), 353–367 (2011)
30.
Zurück zum Zitat Lowe, D.G.: Distinctive image features from scale-invariant points. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef Lowe, D.G.: Distinctive image features from scale-invariant points. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef
31.
Zurück zum Zitat Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRef Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRef
32.
Zurück zum Zitat Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: Proceedings of the CVPR (2012) Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: Proceedings of the CVPR (2012)
33.
Zurück zum Zitat Mutch, J., Lowe, D.G.: Object class recognition and localization using sparse features with limited receptive fields. Int. J. Comput. Vis. 80(1), 45–57 (2008)CrossRef Mutch, J., Lowe, D.G.: Object class recognition and localization using sparse features with limited receptive fields. Int. J. Comput. Vis. 80(1), 45–57 (2008)CrossRef
34.
Zurück zum Zitat Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRef Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRef
35.
Zurück zum Zitat Oliva, A., Torralba, A.B.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)CrossRefMATH Oliva, A., Torralba, A.B.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)CrossRefMATH
36.
Zurück zum Zitat Quiroga, R.Q., Pedreira, C.: How do we see art: an eye-tracker study. Front. Hum. Neurosci. 5, 98 (2011) Quiroga, R.Q., Pedreira, C.: How do we see art: an eye-tracker study. Front. Hum. Neurosci. 5, 98 (2011)
37.
Zurück zum Zitat Ramachandran, V., Hirstein, W.: The science of art: a neurological theory of aesthetic experience. J. Conscious. Stud. 6(7), 15–51 (1999) Ramachandran, V., Hirstein, W.: The science of art: a neurological theory of aesthetic experience. J. Conscious. Stud. 6(7), 15–51 (1999)
38.
Zurück zum Zitat Rojas, D., Khan, F.S., van de Weijer, J., Gevers, T.: The impact of color on bag-of-words based object recognition. In: Proceedings of the ICPR (2010) Rojas, D., Khan, F.S., van de Weijer, J., Gevers, T.: The impact of color on bag-of-words based object recognition. In: Proceedings of the ICPR (2010)
39.
Zurück zum Zitat Sablatnig, R., Kammerer, P., Zolda, E.: Hierarchical classification of paintings using face- and brush-stroke models. In: Proceedings of the ICPR (1998) Sablatnig, R., Kammerer, P., Zolda, E.: Hierarchical classification of paintings using face- and brush-stroke models. In: Proceedings of the ICPR (1998)
40.
Zurück zum Zitat Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. ACM Trans. Appl. Percept. 7(2), 8–16 (2010)CrossRef Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. ACM Trans. Appl. Percept. 7(2), 8–16 (2010)CrossRef
41.
Zurück zum Zitat Shamir, L.: Computer analysis reveals similarities between the artistic styles of Van Gogh and Pollock. Leonardo 45(2), 149–154 (2012)CrossRef Shamir, L.: Computer analysis reveals similarities between the artistic styles of Van Gogh and Pollock. Leonardo 45(2), 149–154 (2012)CrossRef
42.
Zurück zum Zitat Shamir, L., Tarakhovsky, J.A.: Computer analysis of art. J. Comput. Cult. Herit. 5(2), 1–7 (2012)CrossRef Shamir, L., Tarakhovsky, J.A.: Computer analysis of art. J. Comput. Cult. Herit. 5(2), 1–7 (2012)CrossRef
43.
Zurück zum Zitat Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: Proceedings of the CVPR (2007) Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: Proceedings of the CVPR (2007)
44.
Zurück zum Zitat Shen, J.: Stochastic modeling western paintings for effective classification. Pattern Recognit. 42(2), 293–301 (2009)CrossRefMATH Shen, J.: Stochastic modeling western paintings for effective classification. Pattern Recognit. 42(2), 293–301 (2009)CrossRefMATH
45.
Zurück zum Zitat Siddiquie, B., Vitaladevuni, S.N.P., Davis, L.S.: Combining multiple kernels for efficient image classification. In: Proceedings of the WACV (2009) Siddiquie, B., Vitaladevuni, S.N.P., Davis, L.S.: Combining multiple kernels for efficient image classification. In: Proceedings of the WACV (2009)
46.
Zurück zum Zitat Vikram, T., Tscherepanow, M., Wrede, B.: A saliency map based on sampling an image into random rectangular regions of interest. Pattern Recognit. 45(9), 3114–3124 (2012) Vikram, T., Tscherepanow, M., Wrede, B.: A saliency map based on sampling an image into random rectangular regions of interest. Pattern Recognit. 45(9), 3114–3124 (2012)
47.
Zurück zum Zitat van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2010)CrossRef van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2010)CrossRef
48.
Zurück zum Zitat van de Weijer, J., Schmid, C., Verbeek, J.J., Larlus, D.: Learning color names for real-world applications. Trans. Image Process. 18(7), 1512–1524 (2009)CrossRefMathSciNet van de Weijer, J., Schmid, C., Verbeek, J.J., Larlus, D.: Learning color names for real-world applications. Trans. Image Process. 18(7), 1512–1524 (2009)CrossRefMathSciNet
49.
Zurück zum Zitat Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: Proceedings of the CVPR (2010) Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: Proceedings of the CVPR (2010)
50.
Zurück zum Zitat Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object catergories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–218 (2007)CrossRef Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object catergories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–218 (2007)CrossRef
51.
Zurück zum Zitat Zujovic, J., Gandy, L., Friedman, S., Pardo, B., Pappas, T.: Classifying paintings by artistic genre: an analysis of features and classifiers. In: Proceedings of the MMSP (2009) Zujovic, J., Gandy, L., Friedman, S., Pardo, B., Pappas, T.: Classifying paintings by artistic genre: an analysis of features and classifiers. In: Proceedings of the MMSP (2009)
Metadaten
Titel
Painting-91: a large scale database for computational painting categorization
verfasst von
Fahad Shahbaz Khan
Shida Beigpour
Joost van de Weijer
Michael Felsberg
Publikationsdatum
01.08.2014
Verlag
Springer Berlin Heidelberg
Erschienen in
Machine Vision and Applications / Ausgabe 6/2014
Print ISSN: 0932-8092
Elektronische ISSN: 1432-1769
DOI
https://doi.org/10.1007/s00138-014-0621-6

Weitere Artikel der Ausgabe 6/2014

Machine Vision and Applications 6/2014 Zur Ausgabe

Premium Partner