Abstract
Many images carry a strong emotional semantic. These last years, some investigations have been driven to automatically identify induced emotions that may arise in viewers when looking at images, based on low-level image properties. Since these features can only catch the image atmosphere, they may fail when the emotional semantic is carried by objects. Therefore additional information is needed, and we propose in this paper to make use of textual information describing the image, such as tags. Thus, we have developed two textual features to catch the text emotional meaning: one is based on the semantic distance matrix between the text and an emotional dictionary, and the other one carries the valence and arousal meanings of words. Experiments have been driven on two datasets to evaluate visual and textual features and their fusion. The results have shown that our textual features can improve the classification accuracy of affective images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Smeulders, A.W.M., et al.: Content-based Image Retrieval: the end of the early years. IEEE Trans. PAMI 22(12), 1349–1380 (2000)
Zeng, Z., et al.: A survey of affect recognition methods: audio, visual and spontaneous expressions. IEEE Transactions PAMI 31(1), 39–58 (2009)
Wang, W., He, Q.: A survey on emotional semantic image retrieval. In: ICIP, pp. 117–120 (2008)
Wang, S., Wang, X.: Emotion semantics image retrieval: a brief overview. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 490–497. Springer, Heidelberg (2005)
Al-Ani, A., Deriche, M.: A new technique for combing multiple classifiers using the Dempster Shafer theory of evidence. J. Artif. Intell. Res. 17, 333–361 (2002)
Columbo, C., Del Bimbo, A., Pala, P.: Semantics in visual information retrieval. IEEE Multimedia 6(3), 38–53 (1999)
Itten, J.: The art of colour. Otto Maier Verlab, Ravensburg, Germany (1961)
Dellandréa, E., Liu, N., Chen, L.: Classification of affective semantics in images based on discrete and dimensional models of emotions. In: CBMI, pp. 99–104 (2010)
Yanulevskaya, V., et al.: Emotional valence categorization using holistic image features. In: ICIP, pp. 101–104 (2008)
Weining, W., Yinlin, Y., Shengming, J.: Image retrieval by emotional semantics: A study of emotional space and feature extraction. ICSMC 4, 3534–3539 (2006)
Machajdik, J., Hanbury, A.: Affective image classification using features inspired by psychology and art theory. ACM Multimedia (2010)
Wang, G., Hoiem, D., Forsyth, D.: Building text features for object image classification. In: CVPR, pp. 1367–1374 (2009)
Hevner, K.: Experimental studies of the elements of expression in music. American Journal of Psychology 48(2), 246–268 (1936)
Natural language toolkit, http://www.nltk.org
Bradley, M.M., Lang, P.J.: Affective norms for English words (ANEW). Tech. Rep C-1, GCR in Psychophysiology, University of Florida (1999)
Tamura, H., Mori, S., Yamawaki, T.: Textural features corresponding to visual perception. IEEE Transactions on SMC 8(6), 460–473 (1978)
Liu, N., Dellandréa, E., Tellez, B., Chen, L.: Evaluation of Features and Combination Approaches for the Classification of Emotional Semantics in Images. VISAPP (2011)
Datta, R., Li, J., Wang, J.Z.: Content-based image retrieval: approaches and trends of the new age. In: ACM Workshop MIR (2005)
Ke, Y., Tang, X., Jing, F.: The Design of High-Level Features for Photo Quality Assessment. In: CVPR (2006)
Dunker, P., Nowak, S., Begau, A., Lanz, C.: Content-based mood classification for photos and music. In: ACM MIR, pp. 97–104 (2008)
Lang, P.J., Bradley, M.M., Cuthbert, B.N.: The IAPS: Technical manual and affective ratings. Tech. Rep A-8., GCR in Psychophysiology, Unv. of Florida (2008)
Huiskes, M.J., Lew, M.S.: The MIR Flickr Retrieval Evaluation. In: ACM Multimedia Information Retrieval, MIR 2008 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, N., Dellandréa, E., Tellez, B., Chen, L. (2011). Associating Textual Features with Visual Ones to Improve Affective Image Classification. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-24600-5_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24599-2
Online ISBN: 978-3-642-24600-5
eBook Packages: Computer ScienceComputer Science (R0)