Victoria Yanulevskaya
ABSTRACT
Most artworks are explicitly created to evoke a strong emotional response. During the centuries there were several art movements which employed different techniques to achieve emotional expressions conveyed by artworks. Yet people were always consistently able to read the emotional messages even from the most abstract paintings. Can a machine learn what makes an artwork emotional? In this work, we consider a set of 500 abstract paintings from Museum of Modern and Contemporary Art of Trento and Rovereto (MART), where each painting was scored as carrying a positive or negative response on a Likert scale of 1-7. We employ a state-of-the-art recognition system to learn which statistical patterns are associated with positive and negative emotions. Additionally, we dissect the classification machinery to determine which parts of an image evokes what emotions. This opens new opportunities to research why a specific painting is perceived as emotional. We also demonstrate how quantification of evidence for positive and negative emotions can be used to predict the way in which people observe paintings.
- R. Arnheim. Art and Visual Perception: A Psychology of the Creative Eye. University of California Press, Nov. 2004.Google Scholar
- D. H. Brainard. The psychophysics toolbox. Spatial Vision, 10(4), 1997.Google Scholar
- N. Bruce and J. Tsotsos. Saliency, attention, and visual search: An information theoretic approach. Journal of Vision, 9(3), 2009.Google ScholarCross Ref
- G. C. Cupchik, O. Vartanian, A. Crawley, and D. J. Mikulis. Viewing artworks: Contributions of cognitive control and perceptual facilitation to aesthetic experience. Brain and Cognition, 70:84--91, 2009.Google ScholarCross Ref
- M. Dellagiacoma, P. Zontone, G. Boato, and L. Albertazzi. Emotion based classification of natural images. In Int. workshop on Detecting and Exploiting Cultural Diversity on the Social Web, pages 17--22, 2011. Google ScholarDigital Library
- K. Grauman and B. Leibe. Visual Object Recognition. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2011. Google ScholarDigital Library
- H. Hagtvedt, R. Hagtvedt, and V. Patrick. The perception and evaluation of visual art. Empirical Studies of the Arts, 26(2):197--218, 2008.Google ScholarCross Ref
- P. Isola, J. Xiao, A. Torralba, and A. Oliva. What makes an image memorable? In IEEE Conference on Computer Vision and Pattern Recognition, 2011. Google ScholarDigital Library
- D. Joshi, R. Datta, E. A. Fedorovskaya, Q.-T. Luong, J. Z. Wang, J. Li, and J. Luo. Aesthetics and emotions in images. IEEE Signal Process. Mag., 28(5):94--115, 2011.Google ScholarCross Ref
- D. Joshi, R. Datta, E. A. Fedorovskaya, Q.-T. Luong, J. Z. Wang, J. Li, and J. Luo. Aesthetic judgement of orientation in modern art. i-Perception, 3:18--24, 2012.Google Scholar
- P. J. Lang, M. M. Bradley, and B. N. Cuthbert. International affective picture system (iaps): Technical manual and affective ratings. 1999.Google Scholar
- P. Locher, E. A. Krupinski, C. Mello-Thoms, and C. F. Nodine. Visual interest in pictorial art during an aesthetic experience. Spatial Vision, 21(1--2):55--77, 2007.Google Scholar
- D. G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vision, 60:91--110, 2004. Google ScholarDigital Library
- J. Machajdik and A. Hanbury. Affective image classification using features inspired by psychology and art theory. In ACM Multimedia, pages 83--92, 2010. Google ScholarDigital Library
- S. Maji, A. C. Berg, and J. Malik. Classification using intersection kernel support vector machines is efficient. In IEEE Conference on Computer Vision and Pattern Recognition, 2008.Google ScholarCross Ref
- L. Marchesotti, F. Perronnin, D. Larlus, and G. Csurka. Assessing the aesthetic quality of photographs using generic image descriptors. In International Conference on Computer Vision, 2011. Google ScholarDigital Library
- I. C. McManus, K. Stöver, and D. Kim. Arnheim's gestalt theory of visual balance: Examining the compositional structure of art photographs and abstract images. i-Perception, 2:615--647, 2011.Google Scholar
- M. Pelowski and F. Akiba. A model of art perception, evaluation and emotion in transformative aesthetic experience. New Ideas in Psychology, 29:80--97, 2011.Google ScholarCross Ref
- E. Pihko, A. Virtanen, V. Saarinen, S. Pannasch, L. Hirvenkari, T. Tossavainen, A. Haapala, and R. Hari. Experiencing art: The influence of expertise and painting abstraction level. Front Hum Neurosci, 5, 2011.Google Scholar
- E. Pihko, A. Virtanen, V. M. Saarinen, S. Pannasch, L. Hirvenkari, T. Tossavainen, A. Haapala, and R. Hari. Experiencing art: the influence of expertise and painting abstraction level. Frontiers in human neuroscience, 5, 2011.Google Scholar
- P. Reinagel and A. Zador. Natural scene statistics at the centre of gaze. Network: Computation in Neural Systems, 10(4), 1999.Google Scholar
- P. J. Silvia. Emotional responses to art: From collation and arousal to cognition and emotion. Review of general psychology, 9(4):342--357, 2005.Google Scholar
- J. Sivic and A. Zisserman. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the International Conference on Computer Vision, pages 1470--1477, Oct. 2003. Google ScholarDigital Library
- R. Subramanian, H. Katti, N. Sebe, M. Kankanhalli, and T. S. Chua. An eye-fixation database for saliency detection in images. European Conference on Computer Vision, 2010. Google ScholarDigital Library
- R. Subramanian, V. Yanulevskaya, and N. Sebe. Can computers learn from humans to see better? Inferring scene semantics from viewers' eye movements. In ACM Multimedia, pages 33--42, 2011. Google ScholarDigital Library
- R. Szeliski. Computer vision: Algorithms and applications. 2010. Google ScholarDigital Library
- B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist. Visual correlates of fixation selection: Effects of scale and time. Vision Research, 45(5), 2005.Google Scholar
- J. R. R. Uijlings, A. W. M. Smeulders, and R. J. H. Scha. Real-time Visual Concept Classification. IEEE Transactions on Multimedia, 12, 2010. Google ScholarDigital Library
- J. R. R. Uijlings, A. W. M. Smeulders, and R. J. H. Scha. The Visual Extent of an Object. International Journal of Computer Vision, 2012. Google ScholarDigital Library
- K. E. A. van de Sande, T. Gevers, and C. G. M. Snoek. Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 2010. Google ScholarDigital Library
- A. Vedaldi and B. Fulkerson. VLFeat -- an open and portable library of computer vision algorithms. In ACM Multimedia, 2010. Google ScholarDigital Library
- L. Veronesi. Luigi Veronesi. Con una antologia di scritti. Torino, 1968.Google Scholar
- O. Wu, Y. Chen, B. Li, and W. Hu. Evaluating the visual quality of web pages using a computational aesthetic approach. In Association of Computing Machinery conference on Web search and data mining, pages 337--346, 2011. Google ScholarDigital Library
- V. Yanulevskaya, J. V. Gemert, K. Roth, A. Herbold, N. Sebe, and J. Geusebroek. Emotional valence categorization using holistic image features. In IEEE International Conference on Image Processing, pages 101--104, 2008.Google ScholarCross Ref
- V. Yanulevskaya, J. B. Marsman, F. Cornelissen, and J. M. Geusebroek. An image statistics based model for fixation prediction. Cognitive Computation, 3(1), 2011.Google Scholar
- Q. Zhao and C. Koch. Learning a saliency map using fixated locations in natural scenes. Journal of vision, 11(3), 2011.Google Scholar
Index Terms
- In the eye of the beholder: employing statistical analysis and eye tracking for analyzing abstract paintings
Recommendations
Who's Afraid of Itten: Using the Art Theory of Color Combination to Analyze Emotions in Abstract Paintings
MM '15: Proceedings of the 23rd ACM international conference on MultimediaColor plays an essential role in everyday life and is one of the most important visual cues in human perception. In abstract art, color is one of the essential means to convey the artist's intention and to affect the viewer emotionally. However, colors ...
Affective Analysis of Professional and Amateur Abstract Paintings Using Statistical Analysis and Art Theory
Special Issue on Behavior Understanding for Arts and Entertainment (Part 1 of 2)When artists express their feelings through the artworks they create, it is believed that the resulting works transform into objects with “emotions” capable of conveying the artists' mood to the audience. There is little to no dispute about this belief: ...
Affective Analysis of Abstract Paintings Using Statistical Analysis and Art Theory
ICMI '14: Proceedings of the 16th International Conference on Multimodal InteractionA novel approach to the emotion classification of abstract paintings is proposed. Based on a user study, we employ computer vision techniques to understand what makes an abstract artwork emotional. Our aim is to identify and quantify which are the ...
Comments