Sicheng Zhao
ABSTRACT
Emotions can be evoked in humans by images. Most previous works on image emotion analysis mainly used the elements-of-art-based low-level visual features. However, these features are vulnerable and not invariant to the different arrangements of elements. In this paper, we investigate the concept of principles-of-art and its influence on image emotions. Principles-of-art-based emotion features (PAEF) are extracted to classify and score image emotions for understanding the relationship between artistic principles and emotions. PAEF are the unified combination of representation features derived from different principles, including balance, emphasis, harmony, variety, gradation, and movement. Experiments on the International Affective Picture System (IAPS), a set of artistic photography and a set of peer rated abstract paintings, demonstrate the superiority of PAEF for affective image classification and regression (with about 5% improvement on classification accuracy and 0.2 decrease in mean squared error), as compared to the state-of-the-art approaches. We then utilize PAEF to analyze the emotions of master paintings, with promising results.
- R. Arnheim. Art and visual perception: A psychology of the creative eye. University of California Press, 1954.Google Scholar
- D. H. Ballard. Generalizing the hough transform to detect arbitrary shapes. Pattern Recognition, 13(2):111--122, 1981.Google ScholarCross Ref
- S. Benini, L. Canini, and R. Leonardi. A connotative space for supporting movie affective recommendation. IEEE Transactions on Multimedia, 13(6):1356--1370, 2011. Google ScholarDigital Library
- D. Borth, R. Ji, T. Chen, T. Breuel, and S.-F. Chang. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In ACM MM, 2013. Google ScholarDigital Library
- S. Calahan. Storytelling through lighting: a computer graphics perspective. SIGGRAPH course notes, 96, 1996.Google Scholar
- R. G. Collingwood. The principles of art, volume 62. Oxford University Press, USA, 1958.Google Scholar
- R. Datta, J. Li, and J. Z. Wang. Algorithmic inferencing of aesthetics and emotion in natural images: An exposition. In ICIP, 2008.Google ScholarCross Ref
- S. Dhar, V. Ordonez, and T. Berg. High level describable attributes for predicting aesthetics and interestingness. In CVPR, 2011. Google ScholarDigital Library
- Y. Gao, M. Wang, Z.-J. Zha, J. Shen, X. Li, and X. Wu. Visual-textual joint relevance learning for tag-based social image search. IEEE Transactions on Image Processing, 22(1):363--376, 2013. Google ScholarDigital Library
- A. Hanjalic. Extracting moods from pictures and sounds: Towards truly personalized tv. IEEE Signal Processing Magazine, 23(2):90--100, 2006.Google ScholarCross Ref
- A. Hanjalic and L.-Q. Xu. Affective video content representation and modeling. IEEE Transactions on Multimedia, 7(1):143--154, 2005. Google ScholarDigital Library
- J. Hobbs, R. Salome, and K. Vieth. The visual experience. Davis Publications, 1995.Google Scholar
- G. Irie, T. Satou, A. Kojima, T. Yamasaki, and K. Aizawa. Affective audio-visual words and latent topic driving model for realizing movie affective scene classification. IEEE Transactions on Multimedia, 12(6):523--535, 2010. Google ScholarDigital Library
- J. Itten. The art of color: the subjective experience and objective rationale of color. Wiley, 1974.Google Scholar
- J. Jia, S. Wu, X. Wang, P. Hu, L. Cai, and J. Tang. Can we understand van gogh's mood? learning to infer affects from images in social networks. In ACM MM, 2012. Google ScholarDigital Library
- D. Joshi, R. Datta, E. Fedorovskaya, Q. Luong, J. Z. Wang, J. Li, and J. Luo. Aesthetics and emotions in images. IEEE Signal Processing Magazine, 28(5):94--115, 2011.Google ScholarCross Ref
- H. Kang. Affective content detection using hmms. In ACM MM, 2003. Google ScholarDigital Library
- M. Kass and J. Solomon. Smoothed local histogram filters. ACM Transactions on Graphics, 29(4):100, 2010. Google ScholarDigital Library
- P. Lang, M. Bradley, B. Cuthbert, et al. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. NIMH, Center for the Study of Emotion & Attention, 2005.Google Scholar
- B. Li, W. Xiong, W. Hu, and X. Ding. Context-aware affective images classification based on bilayer sparse representation. In ACM MM, 2012. Google ScholarDigital Library
- L. Liu, R. Chen, L. Wolf, and D. Cohen-Or. Optimizing photo composition. In Computer Graphics Forum, 2010.{22} G. Loy and J. Eklundh. Detecting symmetry and symmetric constellations of features. In ICCV, 2006.Google Scholar
- G. Loy and A. Zelinsky. Fast radial symmetry for detecting points of interest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8):959--973, 2003. Google ScholarDigital Library
- X. Lu, P. Suryanarayan, R. B. Adams Jr, J. Li, M. G. Newman, and J. Z. Wang. On shape and the computability of emotions. In ACM MM, 2012. Google ScholarDigital Library
- Y. Luo and X. Tang. Photo and video quality evaluation: Focusing on the subject. In ECCV, 2008. Google ScholarDigital Library
- J. Machajdik and A. Hanbury. Affective image classification using features inspired by psychology and art theory. In ACM MM, 2010. Google ScholarDigital Library
- J. Mikels, B. Fredrickson, G. Larkin, C. Lindberg, S. Maglio, and P. Reuter-Lorenz. Emotional category data on images from the international affective picture system. Behavior research methods, 37(4):626--630, 2005.Google Scholar
- J. Ni, M. Singh, and C. Bahlmann. Fast radial symmetry detection under affine transformations. In CVPR, 2012.Google Scholar
- M. A. Nicolaou, H. Gunes, and M. Pantic. A multi-layer hybrid framework for dimensional emotion classification. In ACM MM, 2011. Google ScholarDigital Library
- M. Redi and B. Merialdo. Enhancing semantic features with compositional analysis for scene recognition. In ECCV Workshop, 2012. Google ScholarDigital Library
- J. Ruskan. Emotion and Art: Mastering the Challenges of the Artist's Path. R. Wyler & Co., 2012.Google Scholar
- H. Schlosberg. Three dimensions of emotion. Psychological review, 61(2):81, 1954.Google ScholarCross Ref
- M. Solli and R. Lenz. Color based bags-of-emotions. In CAIP, 2009. Google ScholarDigital Library
- X. Sun, H. Yao, and R. Ji. What are we looking for: Towards statistical modeling of saccadic eye movements and visual saliency. In CVPR, 2012.Google Scholar
- X. Sun, H. Yao, R. Ji, and S. Liu. Photo assessment based on computational visual attention model. In ACM MM, 2009. Google ScholarDigital Library
- J. Van De W., C. Schmid, and J. Verbeek. Learning color names from real-world images. In CVPR, 2007.Google Scholar
- J. C. van Gemert. Exploiting photographic style for category-level image classification by generalizing the spatial pyramid. In ICMR, 2011. Google ScholarDigital Library
- W. Wang, Y. Yu, and S. Jiang. Image retrieval by emotional semantics: A study of emotional space and feature extraction. In IEEE SMC, 2006.Google Scholar
- X. Xiang and M. Kankanhalli. Affect-based adaptive presentation of home videos. In ACM MM, 2011. Google ScholarDigital Library
- L. Xu, Q. Yan, Y. Xia, and J. Jia. Structure extraction from texture via relative total variation. ACM Transactions on Graphics, 31(6):139, 2012. Google ScholarDigital Library
- M. Xu, J. S. Jin, S. Luo, and L. Duan. Hierarchical movie affective content analysis based on arousal and valence features. In ACM MM, 2008. Google ScholarDigital Library
- V. Yanulevskaya, J. Van Gemert, K. Roth, A. Herbold, N. Sebe, and J. Geusebroek. Emotional valence categorization using holistic image features. In ICIP, 2008.Google ScholarCross Ref
- S. Zhao, H. Yao, X. Sun, P. Xu, X. Liu, and R. Ji. Video indexing and recommendation based on affective analysis of viewers. In ACM MM, 2011. Google ScholarDigital Library
- S. Zhao, H. Yao, F. Wang, X. Jiang, and W. Zhang. Emotion based image musicalization. In ICMEW, 2014.Google ScholarCross Ref
- S. Zhao, H. Yao, Y. Yang, and Y. Zhang. Affective image retrieval via multi-graph learning. In ACM MM, 2014. Google ScholarDigital Library
Index Terms
- Exploring Principles-of-Art Features For Image Emotion Recognition
Recommendations
Predicting Personalized Emotion Perceptions of Social Images
MM '16: Proceedings of the 24th ACM international conference on MultimediaImages can convey rich semantics and induce various emotions to viewers. Most existing works on affective image analysis focused on predicting the dominant emotions for the majority of viewers. However, such dominant emotion is often insufficient in ...
Affective image classification using features inspired by psychology and art theory
MM '10: Proceedings of the 18th ACM international conference on MultimediaImages can affect people on an emotional level. Since the emotions that arise in the viewer of an image are highly subjective, they are rarely indexed. However there are situations when it would be helpful if images could be retrieved based on their ...
Image Emotion Computing
MM '16: Proceedings of the 24th ACM international conference on MultimediaImages can convey rich semantics and induce strong emotions in viewers. My research aims to predict image emotions from different aspects with respect to two main challenges: affective gap and subjective evaluation. To bridge the affective gap, we ...
Comments