Skip to main content
Top
Published in: Machine Vision and Applications 6/2014

01-08-2014 | Original Paper

Painting-91: a large scale database for computational painting categorization

Authors: Fahad Shahbaz Khan, Shida Beigpour, Joost van de Weijer, Michael Felsberg

Published in: Machine Vision and Applications | Issue 6/2014

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Computer analysis of visual art, especially paintings, is an interesting cross-disciplinary research domain. Most of the research in the analysis of paintings involve medium to small range datasets with own specific settings. Interestingly, significant progress has been made in the field of object and scene recognition lately. A key factor in this success is the introduction and availability of benchmark datasets for evaluation. Surprisingly, such a benchmark setup is still missing in the area of computational painting categorization. In this work, we propose a novel large scale dataset of digital paintings. The dataset consists of paintings from 91 different painters. We further show three applications of our dataset namely: artist categorization, style classification and saliency detection. We investigate how local and global features popular in image classification perform for the tasks of artist and style categorization. For both categorization tasks, our experimental results suggest that combining multiple features significantly improves the final performance. We show that state-of-the-art computer vision methods can correctly classify 50 % of unseen paintings to its painter in a large dataset and correctly attribute its artistic style in over 60 % of the cases. Additionally, we explore the task of saliency detection on paintings and show experimental findings using state-of-the-art saliency estimation algorithms.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Arora, R., Elgammal, A.: Towards automated classification of fine-art painting style: a comparative study. In: Proceedings of the ICPR (2012) Arora, R., Elgammal, A.: Towards automated classification of fine-art painting style: a comparative study. In: Proceedings of the ICPR (2012)
2.
go back to reference Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the CIVR (2007) Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the CIVR (2007)
3.
go back to reference Carneiro, G., da Silva, N.P., Bue, A.D., Costeira, J.P.: Artistic image classification: an analysis on the printart database. In: Proceedings of the ECCV (2012) Carneiro, G., da Silva, N.P., Bue, A.D., Costeira, J.P.: Artistic image classification: an analysis on the printart database. In: Proceedings of the ECCV (2012)
4.
go back to reference Cinzia Di Dio, G.R., Macaluso, E.: The golden beauty: brain response to classical and renaissance sculptures. Plos One 2(11), e1201 (2007) Cinzia Di Dio, G.R., Macaluso, E.: The golden beauty: brain response to classical and renaissance sculptures. Plos One 2(11), e1201 (2007)
5.
go back to reference Condorovici, R.G., Vranceanu, R., Vertan, C.: Saliency map retrieval for artistic paintings inspired from human understanding. In: Proceedings of the SPAMEC (2011) Condorovici, R.G., Vranceanu, R., Vertan, C.: Saliency map retrieval for artistic paintings inspired from human understanding. In: Proceedings of the SPAMEC (2011)
6.
go back to reference Csurka, G., Bray, C., Dance, C., Fan, L.: Visual categorization with bags of keypoints. In: Proceedings of the Workshop on Statistical Learning in Computer Vision, ECCV (2004) Csurka, G., Bray, C., Dance, C., Fan, L.: Visual categorization with bags of keypoints. In: Proceedings of the Workshop on Statistical Learning in Computer Vision, ECCV (2004)
7.
go back to reference Danelljan, M., Khan, F.S., Felsberg, M., van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: Proceedings of the CVPR (2014) Danelljan, M., Khan, F.S., Felsberg, M., van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: Proceedings of the CVPR (2014)
8.
go back to reference Deng, J., Dong, W., Socher, R., Jia, L., Kai, L., Li, F.F.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the CVPR (2009) Deng, J., Dong, W., Socher, R., Jia, L., Kai, L., Li, F.F.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the CVPR (2009)
9.
go back to reference Elfiky, N., Khan, F.S., van de Weijer, J., Gonzalez, J.: Discriminative compact pyramids for object and scene recognition. Pattern Recognit. 45(4), 1627–1636 (2012)CrossRefMATH Elfiky, N., Khan, F.S., van de Weijer, J., Gonzalez, J.: Discriminative compact pyramids for object and scene recognition. Pattern Recognit. 45(4), 1627–1636 (2012)CrossRefMATH
10.
go back to reference Everingham, M., Gool, L.J.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRef Everingham, M., Gool, L.J.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRef
11.
go back to reference Gehler, P.V., Nowozin, S.: On feature combination for multiclass object classification. In: Proceedings of the ICCV (2009) Gehler, P.V., Nowozin, S.: On feature combination for multiclass object classification. In: Proceedings of the ICCV (2009)
12.
go back to reference Goguen, J.: Art and the Brain. Imprint Academic, UK (1999) Goguen, J.: Art and the Brain. Imprint Academic, UK (1999)
13.
go back to reference Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. Trans. Image Process. 19(6), 1657–1663 (2010)CrossRefMathSciNet Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. Trans. Image Process. 19(6), 1657–1663 (2010)CrossRefMathSciNet
14.
go back to reference Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Proceedings of the NIPS (2006) Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Proceedings of the NIPS (2006)
15.
go back to reference Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the CVPR (2007) Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the CVPR (2007)
16.
go back to reference Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. Pattern Anal. Mach. Intell. 34(1), 194–201 (2012)CrossRef Hou, X., Harel, J., Koch, C.: Image signature: highlighting sparse salient regions. Pattern Anal. Mach. Intell. 34(1), 194–201 (2012)CrossRef
17.
go back to reference Itti, L., Cristof, K., Ernst, N.: A model of saliency-based visual attention for rapid scene analysis. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRef Itti, L., Cristof, K., Ernst, N.: A model of saliency-based visual attention for rapid scene analysis. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRef
18.
go back to reference Jiang, S., Huang, Q., Ye, Q., Gao, W.: An effective method to detect and categorize digitized traditional chinese paintings. Pattern Recognit. Lett. 27(7), 734–746 (2006)CrossRef Jiang, S., Huang, Q., Ye, Q., Gao, W.: An effective method to detect and categorize digitized traditional chinese paintings. Pattern Recognit. Lett. 27(7), 734–746 (2006)CrossRef
19.
go back to reference Johnson, R., Hendriks, E., Berezhnoy, I.J., Brevdo, E., Hughes, S.M., Daubechies, I., Li, J., Postma, E., Wang, J.Z.: Image processing for artist identification. IEEE Signal Process. Mag. 25(4), 37–48 (2008)CrossRef Johnson, R., Hendriks, E., Berezhnoy, I.J., Brevdo, E., Hughes, S.M., Daubechies, I., Li, J., Postma, E., Wang, J.Z.: Image processing for artist identification. IEEE Signal Process. Mag. 25(4), 37–48 (2008)CrossRef
20.
go back to reference Judd, T., Ehinger, K.A., Durand, F., Torralba, A.: Learning to predict where humans look. In: Proceedings of the ICCV (2009) Judd, T., Ehinger, K.A., Durand, F., Torralba, A.: Learning to predict where humans look. In: Proceedings of the ICCV (2009)
21.
go back to reference Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A.D., Vanrell, M., Lopez, A.M.: Color attributes for object detection. In: Proceedings of the CVPR (2012a) Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A.D., Vanrell, M., Lopez, A.M.: Color attributes for object detection. In: Proceedings of the CVPR (2012a)
22.
go back to reference Khan, F.S., van de Weijer, J., Ali, S., Felsberg, M.: Evaluating the impact of color on texture recognition. In: Proceedings of the CAIP (2013b) Khan, F.S., van de Weijer, J., Ali, S., Felsberg, M.: Evaluating the impact of color on texture recognition. In: Proceedings of the CAIP (2013b)
23.
go back to reference Khan, F.S., van de Weijer, J., Bagdanov, A.D., Vanrell, M.: Portmanteau vocabularies for multi-cue image representations. In: Proceedings of the NIPS (2011) Khan, F.S., van de Weijer, J., Bagdanov, A.D., Vanrell, M.: Portmanteau vocabularies for multi-cue image representations. In: Proceedings of the NIPS (2011)
24.
go back to reference Khan, F.S., van de Weijer, J., Vanrell, M.: Top-down color attention for object recognition. In: Proceedings of the ICCV (2009) Khan, F.S., van de Weijer, J., Vanrell, M.: Top-down color attention for object recognition. In: Proceedings of the ICCV (2009)
25.
go back to reference Khan, F.S., van de Weijer, J., Vanrell, M.: Modulating shape features by color attention for object recognition. Int. J. Comput. Vis. 98(1), 49–64 (2012b)CrossRef Khan, F.S., van de Weijer, J., Vanrell, M.: Modulating shape features by color attention for object recognition. Int. J. Comput. Vis. 98(1), 49–64 (2012b)CrossRef
26.
go back to reference Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A., Lopez, A., Felsberg, M.: Coloring action recognition in still images. Int. J. Comput. Vis. 105(3), 205–221 (2013a)CrossRef Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A., Lopez, A., Felsberg, M.: Coloring action recognition in still images. Int. J. Comput. Vis. 105(3), 205–221 (2013a)CrossRef
27.
go back to reference Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the CVPR (2006) Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the CVPR (2006)
28.
go back to reference Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)CrossRef Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)CrossRef
29.
go back to reference Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. Pattern Anal. Mach. Intell. 32(2), 353–367 (2011) Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. Pattern Anal. Mach. Intell. 32(2), 353–367 (2011)
30.
go back to reference Lowe, D.G.: Distinctive image features from scale-invariant points. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef Lowe, D.G.: Distinctive image features from scale-invariant points. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef
31.
go back to reference Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRef Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRef
32.
go back to reference Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: Proceedings of the CVPR (2012) Murray, N., Marchesotti, L., Perronnin, F.: Ava: a large-scale database for aesthetic visual analysis. In: Proceedings of the CVPR (2012)
33.
go back to reference Mutch, J., Lowe, D.G.: Object class recognition and localization using sparse features with limited receptive fields. Int. J. Comput. Vis. 80(1), 45–57 (2008)CrossRef Mutch, J., Lowe, D.G.: Object class recognition and localization using sparse features with limited receptive fields. Int. J. Comput. Vis. 80(1), 45–57 (2008)CrossRef
34.
go back to reference Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRef Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRef
35.
go back to reference Oliva, A., Torralba, A.B.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)CrossRefMATH Oliva, A., Torralba, A.B.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)CrossRefMATH
36.
go back to reference Quiroga, R.Q., Pedreira, C.: How do we see art: an eye-tracker study. Front. Hum. Neurosci. 5, 98 (2011) Quiroga, R.Q., Pedreira, C.: How do we see art: an eye-tracker study. Front. Hum. Neurosci. 5, 98 (2011)
37.
go back to reference Ramachandran, V., Hirstein, W.: The science of art: a neurological theory of aesthetic experience. J. Conscious. Stud. 6(7), 15–51 (1999) Ramachandran, V., Hirstein, W.: The science of art: a neurological theory of aesthetic experience. J. Conscious. Stud. 6(7), 15–51 (1999)
38.
go back to reference Rojas, D., Khan, F.S., van de Weijer, J., Gevers, T.: The impact of color on bag-of-words based object recognition. In: Proceedings of the ICPR (2010) Rojas, D., Khan, F.S., van de Weijer, J., Gevers, T.: The impact of color on bag-of-words based object recognition. In: Proceedings of the ICPR (2010)
39.
go back to reference Sablatnig, R., Kammerer, P., Zolda, E.: Hierarchical classification of paintings using face- and brush-stroke models. In: Proceedings of the ICPR (1998) Sablatnig, R., Kammerer, P., Zolda, E.: Hierarchical classification of paintings using face- and brush-stroke models. In: Proceedings of the ICPR (1998)
40.
go back to reference Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. ACM Trans. Appl. Percept. 7(2), 8–16 (2010)CrossRef Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. ACM Trans. Appl. Percept. 7(2), 8–16 (2010)CrossRef
41.
go back to reference Shamir, L.: Computer analysis reveals similarities between the artistic styles of Van Gogh and Pollock. Leonardo 45(2), 149–154 (2012)CrossRef Shamir, L.: Computer analysis reveals similarities between the artistic styles of Van Gogh and Pollock. Leonardo 45(2), 149–154 (2012)CrossRef
42.
go back to reference Shamir, L., Tarakhovsky, J.A.: Computer analysis of art. J. Comput. Cult. Herit. 5(2), 1–7 (2012)CrossRef Shamir, L., Tarakhovsky, J.A.: Computer analysis of art. J. Comput. Cult. Herit. 5(2), 1–7 (2012)CrossRef
43.
go back to reference Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: Proceedings of the CVPR (2007) Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: Proceedings of the CVPR (2007)
44.
go back to reference Shen, J.: Stochastic modeling western paintings for effective classification. Pattern Recognit. 42(2), 293–301 (2009)CrossRefMATH Shen, J.: Stochastic modeling western paintings for effective classification. Pattern Recognit. 42(2), 293–301 (2009)CrossRefMATH
45.
go back to reference Siddiquie, B., Vitaladevuni, S.N.P., Davis, L.S.: Combining multiple kernels for efficient image classification. In: Proceedings of the WACV (2009) Siddiquie, B., Vitaladevuni, S.N.P., Davis, L.S.: Combining multiple kernels for efficient image classification. In: Proceedings of the WACV (2009)
46.
go back to reference Vikram, T., Tscherepanow, M., Wrede, B.: A saliency map based on sampling an image into random rectangular regions of interest. Pattern Recognit. 45(9), 3114–3124 (2012) Vikram, T., Tscherepanow, M., Wrede, B.: A saliency map based on sampling an image into random rectangular regions of interest. Pattern Recognit. 45(9), 3114–3124 (2012)
47.
go back to reference van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2010)CrossRef van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2010)CrossRef
48.
go back to reference van de Weijer, J., Schmid, C., Verbeek, J.J., Larlus, D.: Learning color names for real-world applications. Trans. Image Process. 18(7), 1512–1524 (2009)CrossRefMathSciNet van de Weijer, J., Schmid, C., Verbeek, J.J., Larlus, D.: Learning color names for real-world applications. Trans. Image Process. 18(7), 1512–1524 (2009)CrossRefMathSciNet
49.
go back to reference Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: Proceedings of the CVPR (2010) Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: Proceedings of the CVPR (2010)
50.
go back to reference Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object catergories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–218 (2007)CrossRef Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object catergories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–218 (2007)CrossRef
51.
go back to reference Zujovic, J., Gandy, L., Friedman, S., Pardo, B., Pappas, T.: Classifying paintings by artistic genre: an analysis of features and classifiers. In: Proceedings of the MMSP (2009) Zujovic, J., Gandy, L., Friedman, S., Pardo, B., Pappas, T.: Classifying paintings by artistic genre: an analysis of features and classifiers. In: Proceedings of the MMSP (2009)
Metadata
Title
Painting-91: a large scale database for computational painting categorization
Authors
Fahad Shahbaz Khan
Shida Beigpour
Joost van de Weijer
Michael Felsberg
Publication date
01-08-2014
Publisher
Springer Berlin Heidelberg
Published in
Machine Vision and Applications / Issue 6/2014
Print ISSN: 0932-8092
Electronic ISSN: 1432-1769
DOI
https://doi.org/10.1007/s00138-014-0621-6

Other articles of this Issue 6/2014

Machine Vision and Applications 6/2014 Go to the issue

Premium Partner