Inioluwa Deborah Raji
ABSTRACT
Although essential to revealing biased performance, well intentioned attempts at algorithmic auditing can have effects that may harm the very populations these measures are meant to protect. This concern is even more salient while auditing biometric systems such as facial recognition, where the data is sensitive and the technology is often used in ethically questionable manners. We demonstrate a set of fiveethical concerns in the particular case of auditing commercial facial processing technology, highlighting additional design considerations and ethical tensions the auditor needs to be aware of so as not exacerbate or complement the harms propagated by the audited system. We go further to provide tangible illustrations of these concerns, and conclude by reflecting on what these concerns mean for the role of the algorithmic audit and the fundamental product limitations they reveal.
- Amazon. 2019. Amazon Rekognition FAQs. Retrieved Oct 31, 2019 from https://aws.amazon.com/rekognition/faqs/Google Scholar
- Ruha Benjamin. 2019. Race after technology: Abolitionist tools for the new jim code. John Wiley & Sons.Google Scholar
- Sebastian Benthall and Bruce D. Haynes. 2019. Racial Categories in Machine Learning. In Proc. of the Conference on Fairness, Accountability, and Transparency (FAT). 10.Google Scholar
- Sen. Roy Blunt. 2019. S.847 - Commercial Facial Recognition Privacy Act of 2019. https://www.congress.gov/bill/116th-congress/senate-bill/847/textGoogle Scholar
- Joy Buolamwini. 2018. When the Robot Doesn't See Dark Skin.Google Scholar
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proc. of the Conference on Fairness, Accountability, and Transparency (FAT).Google Scholar
- Matt Cagle and Nicole Ozer. 2018. Amazon Teams Up With Government to Deploy Dangerous New Facial Recognition Technology. (2018).Google Scholar
- Clarifai. 2019. Custom Face Recognition. Retrieved Oct 31, 2019 from https: //www.clarifai.com/custom-face-recognitionGoogle Scholar
- Rep. Yvette D. Clarke. 2019. H.R.2231 - Algorithmic Accountability Act of 2019. https://www.congress.gov/bill/116th-congress/house-bill/2231Google Scholar
- Rep. Yvette D. Clarke. 2019. H.R.4008 - No Biometric Barriers to Housing Act of 2019. https://www.congress.gov/bill/116th-congress/house-bill/4008/text?r= 11&s=1Google Scholar
- Lord Clement-Jones. 2019. Automated Facial Recognition Technology (Moratorium and Review) Bill [HL] 2019--20. https://services.parliament.uk/bills/2019- 20/automatedfacialrecognitiontechnologymoratoriumandreview.htmlGoogle Scholar
- Kimberle Crenshaw. 1989. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. The University of Chicago Legal Forum (1989), 139.Google Scholar
- Kimberle Crenshaw. 2017. Kimberle Crenshaw on Intersectionality, More than Two Decades Later.Google Scholar
- Craig R Ehlen and Robert B Welker. 1996. Procedural fairness in the peer and quality review programs. Auditing 15, 1 (1996), 38.Google Scholar
- Norman L Enger and Paul William Howerton. 1980. Computer Security: A Management Audit Approach. Amacom New York.Google Scholar
- Face++. 2019. Face Attributes. Retrieved Oct 31, 2019 from https://www. faceplusplus.com/attributes/Google Scholar
- Sellywati Mohd Faizal, Mohd Rizal Palil, Ruhanita Maelah, and Rosiati Ramli. 2017. Perception on justice, trust and tax compliance behavior in Malaysia. Kasetsart Journal of Social Sciences 38, 3 (2017), 226--232.Google ScholarCross Ref
- Sheera Frenkel. 2018. Microsoft Employees Question CEO Over Company's Contract With ICE. (2018).Google Scholar
- Timnit Gebru. 2019. Oxford Handbook on AI Ethics Book Chapter on Race and Gender. arXiv preprint arXiv:1908.06165 (2019).Google Scholar
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, JenniferWortman Vaughan, HannaWallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018).Google Scholar
- Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. 2016. Ms-celeb-1m: Challenge of recognizing one million celebrities in the real world. Electronic imaging 2016, 11 (2016), 1--6.Google Scholar
- Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M Branham. 2018. Gender recognition or gender reductionism?: The social implications of embedded gender recognition systems. In Proc. of the ACM Conference on Human Factors in Computing Systems (CHI).Google ScholarDigital Library
- Amy Hawkins. 2018. Beijing's Big Brother Tech Needs African Faces. Retrieved October 31, 2019 from https://foreignpolicy.com/2018/07/24/beijings-big-brothertech- needs-african-faces/Google Scholar
- Michael Hind, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Alexandra Olteanu, and Kush R Varshney. 2018. Increasing Trust in AI Services through Supplier's Declarations of Conformity. arXiv preprint arXiv:1808.07261 (2018).Google Scholar
- Anna Lauren Hoffmann. 2019. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society 22, 7 (2019), 900--915.Google ScholarCross Ref
- IBM. 2019. Release notes. Retrieved Oct 31, 2019 from https://cloud.ibm.com/ docs/services/visual-recognition?topic=visual-recognition-release-notesGoogle Scholar
- Kairos. 2019. Kairos Face Recognition Pricing Guide. Retrieved Oct 31, 2019 from https://www.kairos.com/pricingGoogle Scholar
- Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018. Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. In Proc. of the International Conference on Machine Learning (ICML).Google Scholar
- Os Keyes. 2018. The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. Proc. of the Human Computer Interact action 2, CSCW, Article 88 (Nov. 2018).Google ScholarDigital Library
- Steven Melendez. 2018. Despite a surge of tech activism, Clarifai plans to push further into government work. (2018).Google Scholar
- Steven Melendez. 2018. Uber driver troubles raise concerns about transgender face recognition. Retrieved October 31, 2019 from https://www.fastcompany.com/90216258/uber-face-recognition-tool-haslocked- out-some-transgender-driversGoogle Scholar
- Michele Merler, Nalini Ratha, Rogerio S. Feris, and John R. Smith. 2019. Diversity in Faces. arXiv preprints, Article arXiv:1901.10436 (Jan. 2019), arXiv:1901.10436 pages.Google Scholar
- Microsoft. 2019. What is the Azure Face API? Retrieved Oct 31, 2019 from https://docs.microsoft.com/en-us/azure/cognitive-services/face/overviewGoogle Scholar
- Margaret Mitchell, SimoneWu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2018. Model Cards for Model Reporting. CoRR abs/1810.03993 (2018). arXiv:1810.03993 http://arxiv.org/abs/1810.03993Google Scholar
- Paul Mozur. 2019. One Month, 500,000 Face Scans: How China Is Using A.I. to Profile a Minority. Retrieved October 31, 2019 from https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificialintelligence- racial-profiling.htmlGoogle Scholar
- Kristina Murphy. 2003. Procedural justice and tax compliance. Australian Journal of Social Issues (Australian Council of Social Service) 38, 3 (2003).Google Scholar
- Shruti Nagpal, Maneet Singh, Richa Singh, Mayank Vatsa, and Nalini Ratha. 2019. Deep Learning for Face Recognition: Pride or Prejudiced? arXiv preprint arXiv:1904.01219 (2019).Google Scholar
- Mei Ngan, Mei Ngan, and Patrick Grother. 2015. Face recognition vendor test (FRVT) performance of automated gender classification algorithms. Government Technical Report. US Department of Commerce, National Institute of Standards and Technology.Google Scholar
- Inioluwa Deborah Raji and Joy Buolamwini. 2019. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products. In Prof. of the Conference on Artificial Intelligence, Ethics, and Society.Google ScholarDigital Library
- Rasmus Rothe, Radu Timofte, and Luc Van Gool. 2016. Deep expectation of real and apparent age from a single image without facial landmarks. International Journal of Computer Vision (IJCV) (7 2016).Google Scholar
- Silonie Sachdeva et al. 2009. Fitzpatrick skin typing: Applications in dermatology. Indian Journal of Dermatology, Venereology, and Leprology 75, 1 (2009), 93.Google ScholarCross Ref
- Morgan Klaus Scheuerman, Jacob M Paul, and Jedr Brubaker. 2019. How Computers See Gender: An Evaluation of Gender Classification in Commercial Facial Analysis and Image Labeling Services. (2019).Google Scholar
- Jacob Snow. 2018. Amazon's Face Recognition Falsely Matched 28 Members of Congress With Mugshots. Retrieved August 24, 2017 from https://www.aclu.org/blog/privacy-technology/surveillance-technologies/ amazons-face-recognition-falsely-matched-28Google Scholar
- Olivia Solon. 2019. Facial recognition's 'dirty little secret': Millions of online photos scraped without consent. Retrieved October 31, 2019 from https://www.nbcnews.com/tech/internet/facial-recognition-s-dirty-littlesecret- millions-online-photos-scraped-n981921Google Scholar
- Nisha Srinivas, Karl Ricanek, Dana Michalski, David S Bolme, and Michael King. 2019. Face Recognition Algorithm Bias: Performance Differences on Images of Children and Adults. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 0--0.Google ScholarCross Ref
Index Terms
- Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Recommendations
Machines and the face of ethics
In this article I try to show in what sense Emmanuel Levinas' `ethics as first philosophy' moves our ethical thinking away from what has been called `centrist ethics'. Proceeding via depictions of the structure of Levinasian ethics and including ...
Onward for the freedom of others: marching beyond the AI ethics
FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and TransparencyThe debate on the ethics of Artificial Intelligence brought together different stakeholders including but not limited to academics, policymakers, CEOs, activists, workers' representatives, lobbyists, journalists, and 'moral machines'. Prominent ...
Ethics and emerging technology: ethical concerns from a cognitive, media & technology focused psychology perspective concerning augmented reality, privacy, and sigularity: panel
ETHICS '14: Proceedings of the IEEE 2014 International Symposium on Ethics in Engineering, Science, and TechnologyDeveloping technologies such as augmented reality merges an individual's real life with a digital life that enhances both the visual and the physical experience. Augmented reality creates many ethical concerns in regards to impact on society and human ...
Comments