Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
CNN-PS: CNN-Based Photometric Stereo for General Non-convex Surfaces Read chapter Read first chapter

2018 | OriginalPaper | Chapter

Visual Psychophysics for Making Face Recognition Algorithms More Explainable

Authors : Brandon RichardWebster, So Yon Kwon, Christopher Clarizio, Samuel E. Anthony, Walter J. Scheirer

Published in: Computer Vision – ECCV 2018

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Scientific fields that are interested in faces have developed their own sets of concepts and procedures for understanding how a target model system (be it a person or algorithm) perceives a face under varying conditions. In computer vision, this has largely been in the form of dataset evaluation for recognition tasks where summary statistics are used to measure progress. While aggregate performance has continued to improve, understanding individual causes of failure has been difficult, as it is not always clear why a particular face fails to be recognized, or why an impostor is recognized by an algorithm. Importantly, other fields studying vision have addressed this via the use of visual psychophysics: the controlled manipulation of stimuli and careful study of the responses they evoke in a model system. In this paper, we suggest that visual psychophysics is a viable methodology for making face recognition algorithms more explainable. A comprehensive set of procedures is developed for assessing face recognition algorithm behavior, which is then deployed over state-of-the-art convolutional neural networks and more basic, yet still widely used, shallow and handcrafted feature-based approaches.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter On Offline Evaluation of Vision-Based Driving Models
next chapter Conditional Prior Networks for Optical Flow
Appendix
Available only for authorised users
Footnotes
Literature
1.
2.
Hern, A.: Apple: Face ID didn’t fail at iPhone X launch, our staff did. The Guardian, 14 Sep 2017
3.
Embretson, S.E., Reise, S.P.: Item Response Theory for Psychologists. Lawrence Erlbaum Associates, Inc., Mahwah (2000)
4.
Lu, Z.L., Dosher, B.: Visual Psychophysics: From Laboratory to Theory. MIT Press (2013)
5.
Kingdom, F., Prins, N.: Psychophysics: A Practical Introduction. Academic Press (2016)
6.
Yager, N., Dunstone, T.: The biometric menagerie. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 220–230 (2010)CrossRef
7.
8.
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you?: explaining the predictions of any classifier. In: ACM KDD (2016)
9.
10.
Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)CrossRef
11.
Phillips, P.J., et al.: Overview of the face recognition grand challenge. In: IEEE CVPR (2005)
12.
13.
Beveridge, J.R., et al.: The challenge of face recognition from digital point-and-shoot cameras. In: IEEE BTAS (2013)
14.
Phillips, P.J., et al.: An introduction to the good, the bad, & the ugly face recognition challenge problem. In: IEEE FG (2011)
15.
Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., Brossard, E.: The megaface benchmark: 1 million faces for recognition at scale. In: IEEE CVPR (2016)
16.
Klare, B.F., et al.: Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A. In: IEEE CVPR (2015)
17.
18.
Ortiz, E.G., Becker, B.C.: Face recognition for web-scale datasets. Comput. Vis. Image Underst. 118, 153–170 (2014)CrossRef
19.
Bhattarai, B., Sharma, G., Jurie, F., Pérez, P.: Some faces are more equal than others: hierarchical organization for accurate and efficient large-scale identity-based face retrieval. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 160–172. Springer, Cham (2015). https://​doi.​org/​10.​1007/​978-3-319-16181-5_​12CrossRef
20.
Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: IEEE CVPR (2015)
21.
Wang, D., Otto, C., Jain, A.K.: Face search at scale. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1122–1136 (2017)CrossRef
22.
Tanaka, J.W., Farah, M.J.: Parts and wholes in face recognition. Q. J. Exp. Psychol. 46(2), 225–245 (1993)CrossRef
23.
Duchaine, B., Nakayama, K.: The Cambridge face memory test: results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants. Neuropsychologia 44(4), 576–585 (2006)CrossRef
24.
Oosterhof, N.N., Todorov, A.: The functional basis of face evaluation. Proc. Natl. Acad. Sci. 105(32), 11087–11092 (2008)CrossRef
25.
Germine, L.T., Duchaine, B., Nakayama, K.: Where cognitive development and aging meet: face learning ability peaks after age 30. Cognition 118(2), 201–210 (2011)CrossRef
26.
Duchaine, B., Germine, L., Nakayama, K.: Family resemblance: ten family members with prosopagnosia and within-class object agnosia. Cogn. Neuropsychol. 24(4), 419–430 (2007)CrossRef
27.
Russell, R., Duchaine, B., Nakayama, K.: Super-recognizers: people with extraordinary face recognition ability. Psychon. Bull. Rev. 16(2), 252–257 (2009)CrossRef
28.
Wilmer, J.B., et al.: Human face recognition ability is specific and highly heritable. Proc. Natl. Acad. Sci. 107(11), 5238–5241 (2010)CrossRef
29.
RichardWebster, B., Anthony, S., Scheirer, W.: Psyphy: a psychophysics driven evaluation framework for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 1–1 (2018). Preprint
30.
Scheirer, W.J., Anthony, S.E., Nakayama, K., Cox, D.D.: Perceptual annotation: Measuring human vision to improve computer vision. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1679–1686 (2014)CrossRef
31.
32.
33.
Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: IEEE CVPR (2015)
34.
Fong, R.C., Vedaldi, A.: Interpretable explanations of black boxes by meaningful perturbation. In: IEEE ICCV (2017)
35.
Turner, R.: A model explanation system. In: 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP) (2016)
36.
Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350(6266), 1332–1338 (2015)MathSciNetCrossRef
37.
Rajalingham, R., Issa, E.B., Bashivan, P., Kar, K., Schmidt, K., DiCarlo, J.J.: Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks, 240614 (2018). bioRxiv
38.
Gerhard, H.E., Wichmann, F.A., Bethge, M.: How sensitive is the human visual system to the local statistics of natural images? PLoS Comput. Biol. 9(1), e1002873 (2013)CrossRef
39.
Eberhardt, S., Cader, J., Serre, T.: How deep is the feature analysis underlying rapid visual categorization? In: NIPS (2016)
40.
Geirhos, R., Janssen, D.H.J., Schütt, H.H., Rauber, J., Bethge, M., Wichmann, F.A.: Comparing deep neural networks against humans: object recognition when the signal gets weaker (2017). arXiv preprint arXiv:​1706.​06969
41.
Heath, M., Sarkar, S., Sanocki, T., Bowyer, K.: Comparison of edge detectors: a methodology and initial study. In: 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Proceedings CVPR 1996, pp. 143–148. IEEE (1996)
42.
McCurie, M., Beletti, F., Parzianello, L., Westendorp, A., Anthony, S.E., Scheirer, W.J.: Predicting first impressions with deep learning. In: IEEE FG (2017)
43.
Germine, L., Nakayama, K., Duchaine, B.C., Chabris, C.F., Chatterjee, G., Wilmer, J.B.: Is the web as good as the lab? Comparable performance from web and lab in cognitive/perceptual experiments. Psychon. Bull. Rev. 19(5), 847–857 (2012)CrossRef
44.
Sinha, P., Balas, B., Ostrovsky, Y., Russell, R.: Face recognition by humans: nineteen results all computer vision researchers should know about. Proc. IEEE 94(11), 1948–1962 (2006)CrossRef
45.
O’Toole, A.J., Phillips, P.J., Jiang, F., Ayyad, J., Penard, N., Abdi, H.: Face recognition algorithms surpass humans matching faces over changes in illumination. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1642–1646 (2007)CrossRef
46.
O’Toole, A.J., Phillips, P.J., Narvekar, A.: Humans versus algorithms: comparisons from the face recognition vendor test 2006. In: IEEE FG (2008)
47.
O’Toole, A.J., An, X., Dunlop, J., Natu, V., Phillips, P.J.: Comparing face recognition algorithms to humans on challenging tasks. ACM Trans. Appl. Percept. (TAP) 9(4), 16 (2012)
48.
Phillips, P.J., O’Toole, A.J.: Comparison of human and computer performance across face recognition experiments. Image Vis. Comput. 32(1), 74–85 (2014)CrossRef
49.
Phillips, P.J., Hill, M.Q., Swindle, J.A., O’Toole, A.J.: Human and algorithm performance on the PaSC face recognition challenge. In: IEEE BTAS (2015)
50.
Karahan, S., Yildirum, M.K., Kirtac, K., Rende, F.S., Butun, G., Ekenel, H.K.: How image degradations affect deep CNN-based face recognition? In: International Conference of the Biometrics Special Interest Group (BIOSIG) (2016)
51.
Grm, K., Struc, V., Artiges, A., Caron, M., Ekenel, H.K.: Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biom. 7(1), 81–89 (2018)CrossRef
52.
Ding, C., Tao, D.: Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans. Pattern Anal. Mach. Intell. (2017, to appear)
53.
Blanz, V., Tarr, M.J., Bülthoff, H.H.: What object attributes determine canonical views? Perception 28(5), 575–599 (1999)CrossRef
54.
Doddington, G., Liggett, W., Martin, A., Przybocki, M., Reynolds, D.: Sheep, goats, lambs and wolves: a statistical analysis of speaker performance in the NIST 1998 speaker recognition evaluation. Technical report, National Institute of Standards and Technology (1998)
55.
Teli, M.N., Beveridge, J.R., Phillips, P.J., Givens, G.H., Bolme, D.S., Draper, B.A.: Biometric zoos: theory and experimental evidence. In: IEEE/IAPR IJCB (2011)
56.
Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: NIPS (2011)
57.
Bergstra, J., Yamins, D., Cox, D.D.: Hyperopt: a python library for optimizing the hyperparameters of machine learning algorithms. In: 12th Python in Science Conference (2013)
58.
Bergstra, J., Komer, B., Eliasmith, C., Yamins, D., Cox, D.D.: Hyperopt: a python library for model selection and hyperparameter optimization. Comput. Sci. Discov. 8(1), 014008 (2015)CrossRef
59.
Parkhi, O.M., Vedaldi, A., Zisserman, A., et al.: Deep face recognition. In: BMVC, vol. 1, p. 6 (2015)
60.
61.
Amos, B., Ludwiczuk, B., Satyanarayanan, M.: Openface: a general-purpose face recognition library with mobile applications. Technical report, CMU-CS-16-118, CMU School of Computer Science (2016)
62.
Cox, D.D., Pinto, N.: Beyond simple features: a large-scale feature search approach to unconstrained face recognition. In: IEEE FG (2011)
63.
Klontz, J.C., Klare, B.F., Klum, S., Jain, A.K., Burge, M.J.: Open source biometric recognition. In: IEEE BTAS (2013)
64.
Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report, 07–49, University of Massachusetts, Amherst (2007)
65.
Yildirim, I., Kulkarni, T.D., Freiwald, W.A., Tenenbaum, J.B.: Efficient and robust analysis-by-synthesis in vision: a computational framework, behavioral tests, and modeling neuronal representations. In: Annual Conference of the Cognitive Science Society (2015)
66.
67.
68.
Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)CrossRef
69.
Lowe, D.G.: Object recognition from local scale-invariant features. In: IEEE ICCV (1999)
70.
Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: IEEE FG (2002)
71.
Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-PIE. Image Vis. Comput. 28(5), 807–813 (2010)CrossRef
72.
Dutta, A., Veldhuis, R., Spreeuwers, L.: A Bayesian model for predicting face recognition performance using image quality. In: IEEE/IAPR IJCB (2014)
73.
Ng, H.W., Winkler, S.: A data-driven approach to cleaning large face datasets. In: IEEE ICIP (2014)
74.
75.
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI (2017)
76.
Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)CrossRef
77.
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
78.
Graves, A.: Practical variational inference for neural networks. In: NIPS (2011)
79.
Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep learning, vol. 1. MIT Press Cambridge (2016)
80.
O’Toole, A.J., Deffenbacher, K.A., Valentin, D., McKee, K., Huff, D., Abdi, H.: The perception of face gender: the role of stimulus structure in recognition and classification. Mem. Cogn. 26(1), 146–160 (1998)CrossRef
81.
Webster, M.A., MacLeod, D.I.: Visual adaptation and face perception. Philos. Trans. R. Soc. Lond. B Biol. Sci. 366(1571), 1702–1725 (2011)CrossRef
82.
Dick, A.O.: Iconic memory and its relation to perceptual processing and other memory mechanisms. Percept. Psychophys. 16(3), 575–596 (1974)CrossRef
Metadata
Title
Visual Psychophysics for Making Face Recognition Algorithms More Explainable
Authors
Brandon RichardWebster
So Yon Kwon
Christopher Clarizio
Samuel E. Anthony
Walter J. Scheirer
Copyright Year
2018
Book
Computer Vision – ECCV 2018

Print ISBN: 978-3-030-01266-3

Electronic ISBN: 978-3-030-01267-0

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-030-01267-0_16

Premium Partner

    Image Credits