Skip to main content
SpringerLink
Log in

Facial expression recognition in peripheral versus central vision: role of the eyes and the mouth

  • Original Article
  • Published:
Psychological Research Aims and scope Submit manuscript

Abstract

This study investigated facial expression recognition in peripheral relative to central vision, and the factors accounting for the recognition advantage of some expressions in the visual periphery. Whole faces or only the eyes or the mouth regions were presented for 150 ms, either at fixation or extrafoveally (2.5° or 6°), followed by a backward mask and a probe word. Results indicated that (a) all the basic expressions were recognized above chance level, although performance in peripheral vision was less impaired for happy than for non-happy expressions, (b) the happy face advantage remained when only the mouth region was presented, and (c) the smiling mouth was the most visually salient and most distinctive facial feature of all expressions. This suggests that the saliency and the diagnostic value of the smile account for the advantage in happy face recognition in peripheral vision. Because of saliency, the smiling mouth accrues sensory gain and becomes resistant to visual degradation due to stimulus eccentricity, thus remaining accessible extrafoveally. Because of diagnostic value, the smile provides a distinctive single cue of facial happiness, thus bypassing integration of face parts and reducing susceptibility to breakdown of configural processing in peripheral vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. In the Calvo et al. (2010) study using a 150-ms display, saccades were initiated from the central fixation point towards the target face on 5.8 % of trials. This implies that most saccade latencies were longer than 150 ms. Crucially, the probability that a saccade actually landed on the face was negligible (0.2 % of trials), and there were no differences as a function of expression. This allows us to rule out the hypothesis that the effects found in the current study could be due to foveal fixations on the faces.

  2. We used relatively simple visual stimuli such as faces from which, in addition, some aspects—not relevant to expression— such as hair, etc., had been removed. With more complex stimuli and naturalistic scenes, the predictive power of saliency as a purely sensory-driven factor in guiding attention may be limited (Tatler, Hayhoe, Land, & Ballard, 2011), and task demands can override the effects of saliency (Einhäuser, Rutishauser, & Koch 2008). Consequently, saliency models probably work best on simple stimuli with clear saliency peaks and when top-down goals are minimal.

References

  • Adolphs, R. (2002). Recognizing emotion from facial expressions: psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews, 1, 21–62.

    Article  PubMed  Google Scholar 

  • Bayle, D. J., Henaff, M. A., & Krolak-Salmon, P. (2009). Unconsciously perceived fear in peripheral vision alerts the limbic system: a MEG study. PLoS ONE, 4(12), e8207. doi:10.1371/journal.pone.0008207.

    Article  PubMed Central  PubMed  Google Scholar 

  • Bayle, D. J., Schoendorff, B., Henaff, M. A., & Krolak-Salmon, P. (2011). Emotional facial expression detection in the peripheral visual field. PLoS ONE, 6(6), e21584. doi:10.1371/journal.pone.0021584.

    Article  PubMed Central  PubMed  Google Scholar 

  • Blais, C., Roy, C., Fiset, D., Arguin, M., & Gosselin, F. (2012). The eyes are not the window to basic emotions. Neuropsychologia, 50, 2830–2838.

    Article  PubMed  Google Scholar 

  • Borji, A., & Itti, L. (2013). State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 185–207.

    Google Scholar 

  • Calder, A. J., Young, A. W., Keane, J., & Dean, M. (2000). Configural information in facial expression perception. Journal of Experimental Psychology: Human Perception and Performance, 26, 527–551.

    PubMed  Google Scholar 

  • Calvo, M. G., Fernández-Martín, A., & Nummenmaa, L. (2012). Perceptual, categorical, and affective processing of ambiguous smiling facial expressions. Cognition, 125, 273–293.

    Article  Google Scholar 

  • Calvo, M. G., & Lundqvist, D. (2008). Facial expressions of emotion (KDEF): identification under different display-duration conditions. Behavior Research Methods, 40, 109–115.

    Article  PubMed  Google Scholar 

  • Calvo, M. G., & Marrero, H. (2009). Visual search of emotional faces: the role of affective content and featural distinctiveness. Cognition and Emotion, 23, 782–806.

    Article  Google Scholar 

  • Calvo, M. G., & Nummenmaa, L. (2008). Detection of emotional faces: salient physical features guide effective visual search. Journal of Experimental Psychology: General, 137, 471–494.

    Article  Google Scholar 

  • Calvo, M. G., & Nummenmaa, L. (2009). Eye-movement assessment of the time course in facial expression recognition: neurophysiological implications. Cognitive, Affective and Behavioral Neuroscience, 9, 398–411.

    Article  PubMed  Google Scholar 

  • Calvo, M. G., & Nummenmaa, L. (2011). Time course of discrimination between emotional facial expressions: the role of visual saliency. Vision Research, 51, 1751–1759.

    Article  PubMed  Google Scholar 

  • Calvo, M. G., Nummenmaa, L., & Avero, P. (2010). Recognition advantage of happy faces in extrafoveal vision: featural and affective processing. Visual Cognition, 18, 1274–1297.

    Article  Google Scholar 

  • Calvo, M. G., Nummenmaa, L., & Hyönä, J. (2008). Emotional scenes in peripheral vision: selective orienting and gist processing, but not content identification. Emotion, 8, 68–80.

    Article  PubMed  Google Scholar 

  • De Cesarei, A., Codispoti, M., & Schupp, H. T. (2009). Peripheral vision and preferential emotion processing. NeuroReport, 20, 1439–1443.

    Article  PubMed  Google Scholar 

  • Einhäuser, W., Rutishauser, U., & Koch, C. (2008). Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. Journal of Vision, 8(2), 1–19.

    Article  Google Scholar 

  • Ekman, P., & Friesen, W. V. (1976). Pictures of Facial Affect. Palo Alto: Consulting Psychologists Press.

    Google Scholar 

  • Esteves, F., & Öhman, A. (1993). Masking the face: recognition of emotional facial expressions as a function of the parameters of backward masking. Scandinavian Journal of Psychology, 34, 1–18.

    Article  PubMed  Google Scholar 

  • Goren, D., & Wilson, H. R. (2006). Quantifying facial expression recognition across viewing conditions. Vision Research, 46, 1253–1262.

    Article  PubMed  Google Scholar 

  • Horstmann, G., Lipp, O. V., & Becker, S. I. (2012). Of toothy grins and angry snarls—open mouth displays contribute to efficiency gains in search for emotional faces. Journal of Vision, 12(5), 1–15.

    Article  Google Scholar 

  • Itti, L. (2006). Quantitative modeling of perceptual salience at human eye position. Visual Cognition, 14, 959–984.

    Article  Google Scholar 

  • Itti, L., & Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40, 1489–1506.

    Article  PubMed  Google Scholar 

  • Itti, L., & Koch, C. (2001). Computational modeling of visual attention. Nature Reviews Neuroscience, 2, 1–10.

    Article  Google Scholar 

  • Johnston, L., Miles, L., & Macrae, C. (2010). Why are you smiling at me? Social functions of enjoyment and non-enjoyment smiles. British Journal of Social Psychology, 49, 107–127.

    Article  PubMed  Google Scholar 

  • Juth, P., Lundqvist, D., Karlsson, A., & Öhman, A. (2005). Looking for foes and friends: perceptual and emotional factors when finding a face in the crowd. Emotion, 5, 379–395.

    Article  PubMed  Google Scholar 

  • Kirchner, H., & Thorpe, S. J. (2006). Ultra-rapid object detection with saccadic eye movements: visual processing speed revisited. Vision Research, 46, 1762–1776.

    Article  PubMed  Google Scholar 

  • Kohler, C. G., Turner, T., Stolar, N. M., Bilker, W. B., Brensinger, C. M., Gur, R. E., et al. (2004). Differences in facial expressions of four universal emotions. Psychiatry Research, 128, 235–244.

    Article  PubMed  Google Scholar 

  • Krumhuber, E. G., & Scherer, K. R. (2011). Affect bursts: dynamic patterns of facial expression. Emotion, 11, 825–841.

    Article  PubMed  Google Scholar 

  • Leppänen, J., & Hietanen, J. K. (2004). Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research, 69, 22–29.

    Article  PubMed  Google Scholar 

  • Leppänen, J., & Hietanen, J. K. (2007). Is there more in a happy face than just a big smile? Visual Cognition, 15, 468–490.

    Article  Google Scholar 

  • Liu, L., & Ioannides, A. A. (2010). Emotion separation is completed early and it depends on visual field presentation. PLoSOne, 5, e9790.

    Article  Google Scholar 

  • Loughead, J., Gur, R. C., Elliott, M., & Gur, R. E. (2008). Neural circuitry for accurate identification of facial emotions. Brain Research, 1194, 37–44.

    Article  PubMed  Google Scholar 

  • Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska Directed Emotional Faces—KDEF. Stockholm, Sweden: CD-ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet. ISBN 91-630-7164-9.

    Google Scholar 

  • Maxwell, J. S., & Davidson, R. J. (2004). Unequally masked: indexing differences in the perceptual salience of “unseen” facial expressions. Cognition and Emotion, 18, 1009–1026.

    Article  Google Scholar 

  • McLellan, T., Johnston, L., Dalrymple-Alford, J., & Porter, R. (2010). Sensitivity to genuine vs. posed emotion specified in facial displays. Cognition and Emotion, 24, 1277–1292.

    Article  Google Scholar 

  • Mendolia, M. (2007). Explicit use of categorical and dimensional strategies to decode facial expressions of emotion. Journal of Nonverbal Behavior, 31, 57–75.

    Article  Google Scholar 

  • Milders, M., Sahraie, A., & Logan, S. (2008). Minimum presentation time for masked facial expression discrimination. Cognition and Emotion, 22, 63–82.

    Article  Google Scholar 

  • Nummenmaa, L., Hyönä, J., & Calvo, M. G. (2010). Semantic categorization precedes affective evaluation of visual scenes. Journal of Experimental Psychology: General, 139, 222–246.

    Article  Google Scholar 

  • Nusseck, M., Cunningham, D. W., Wallraven, C., & Bülthoff, H. H. (2008). The contribution of different facial regions to the recognition of conversational expressions. Journal of Vision, 8(8), 1–23.

    Article  PubMed  Google Scholar 

  • Palermo, R., & Coltheart, M. (2004). Photographs of facial expression: accuracy, response times, and ratings of intensity. Behavior Research Methods, 36, 634–638.

    Article  Google Scholar 

  • Parkhurst, D., Law, K., & Niebur, E. (2002). Modelling the role of salience in the allocation of overt visual attention. Vision Research, 42, 107–123.

    Article  PubMed  Google Scholar 

  • Rigoulot, S., D’Hondt, F., Defoort-Dhellemmes, S., Despretz, P., Honoré, J., & Sequeira, H. (2011). Fearful faces impact in peripheral vision: behavioral and neural evidence. Neuropsychologia, 49, 213–221.

    Article  Google Scholar 

  • Rigoulot, S., D’Hondt, F., Honoré, J., & Sequeira, H. (2012). Implicit emotional processing in peripheral vision: behavioral and neural evidence. Neuropsychologia, 50, 2887–2896.

    Article  PubMed  Google Scholar 

  • Rigoulot, S., Delplanque, S., Despretz, P., Defoort-Dhellemmes, S., Honore, J., & Sequeira, H. (2008). Peripherally presented emotional scenes: a spatiotemporal analysis of early ERP responses. Brain Topography, 20, 216–223.

    Article  PubMed  Google Scholar 

  • Scherer, K. R., & Ellgring, H. (2007). Are facial expressions of emotion produced by categorical affect programs or dynamically driven by appraisal? Emotion, 7, 113–130.

    Article  PubMed  Google Scholar 

  • Smith, M. L., Cottrell, G., Gosselin, F., & Schyns, P. G. (2005). Transmitting and decoding facial expressions of emotions. Psychological Science, 16, 184–189.

    Article  PubMed  Google Scholar 

  • Snodgrass, J. G., & Corwin, J. (1988). Pragmatics of measuring recognition memory: applications to dementia and amnesia. Journal of Experimental Psychology: General, 117, 34–50.

    Article  Google Scholar 

  • Stefanics, G., Csukly, G., Komlósi, S., Czobor, P., & Czigler, I. (2012). Processing of unattended facial emotions: a visual mismatch negativity study. NeuroImage, 59, 3042–3049.

    Article  PubMed  Google Scholar 

  • Stone, A., & Valentine, T. (2007). Angry and happy faces perceived without awareness: a comparison with the affective impact of masked famous faces. European Journal of Cognitive Psychology, 19, 161–186.

    Article  Google Scholar 

  • Svärd, J., Wiens, S., & Fischer, H. (2012). Superior recognition performance for happy masked and unmasked faces in both younger and older adults. Frontiers in Psychology, 3, 1–11.

    Article  Google Scholar 

  • Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011). Eye guidance in natural vision: reinterpreting salience. Journal of Vision, 11(5), 1–23.

    Article  Google Scholar 

  • Torralba, A., Oliva, A., Castelhano, M. S., & Henderson, J. (2006). Contextual guidance of eye movements in real-world scenes: the role of global features in object search. Psychological Review, 113, 766–786.

    Article  PubMed  Google Scholar 

  • Tottenham, N., Borscheid, A., Ellertsen, K., Marcus, D., & Nelson, C. A. (2002). The NimStim Face Set. http://www.macbrain.org/faces/index.htm.

  • Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., et al. (2009). The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Research, 168, 242–249.

    Article  PubMed Central  PubMed  Google Scholar 

  • Underwood, G., & Foulsham, T. (2006). Visual saliency and semantic incongruency influence eye movements when inspecting pictures. The Quarterly Journal of Experimental Psychology, 59, 1931–1949.

    Article  PubMed  Google Scholar 

  • Walther, D., & Koch, C. (2006). Modelling attention to salient proto-objects. Neural Networks, 19, 1395–1407.

    Article  PubMed  Google Scholar 

  • Wandell, B. A. (1995). Foundations of vision. Sunderland: Sinauer.

    Google Scholar 

  • Wijers, A. A., & Banis, S. (2012). Foveal and parafoveal spatial attention and its impact on the processing of facial expression: an ERP study. Clinical Neurophysiology, 123, 513–526.

    Article  PubMed  Google Scholar 

  • Yoon, K. L., Hong, S. W., Joorman, J., & Kang, P. (2009). Perception of facial expressions of emotion during binocular rivalry. Emotion, 9, 172–182.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This research was supported by Grant PSI2009-07245 from the Spanish Ministerio de Ciencia e Innovación, and the Agencia Canaria de Investigación, Innovación y Sociedad de la Información (Neurocog Project), and the European Regional Development Funds.

Author information

Authors and Affiliations

  1. Department of Cognitive Psychology, University of La Laguna, 38205, Tenerife, Spain

    Manuel G. Calvo & Andrés Fernández-Martín

  2. School of Science, Aalto University, Espoo, Finland

    Lauri Nummenmaa

  3. Turku PET Centre, Turku, Finland

    Lauri Nummenmaa

Authors
  1. Manuel G. Calvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrés Fernández-Martín
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lauri Nummenmaa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuel G. Calvo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Calvo, M.G., Fernández-Martín, A. & Nummenmaa, L. Facial expression recognition in peripheral versus central vision: role of the eyes and the mouth. Psychological Research 78, 180–195 (2014). https://doi.org/10.1007/s00426-013-0492-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00426-013-0492-x

Keywords

Search

Navigation