ABSTRACT
Algorithmic systems increasingly shape information people are exposed to as well as influence decisions about employment, finances, and other opportunities. In some cases, algorithmic systems may be more or less favorable to certain groups or individuals, sparking substantial discussion of algorithmic fairness in public policy circles, academia, and the press. We broaden this discussion by exploring how members of potentially affected communities feel about algorithmic fairness. We conducted workshops and interviews with 44 participants from several populations traditionally marginalized by categories of race or class in the United States. While the concept of algorithmic fairness was largely unfamiliar, learning about algorithmic (un)fairness elicited negative feelings that connect to current national discussions about racial injustice and economic inequality. In addition to their concerns about potential harms to themselves and society, participants also indicated that algorithmic fairness (or lack thereof) could substantially affect their trust in a company or product.
- ACM US Public Policy Council. 2017. Statement on Algorithmic Transparency and Accountability. Retrieved September 16, 2017 from https://www.acm.org/binaries/content/assets/publicpolicy/2017_usacm_statement_algorithms.pdfGoogle Scholar
- Airbnb, Inc. Airbnb's Nondiscrimination Policy. Retrieved September 14, 2017 from https://www.airbnb.com/terms/nondiscrimination_polic yGoogle Scholar
- Mariam Asad, Sarah Fox, and Christopher A. Le Dantec. 2014. Speculative Activist Technologies. Proceedings iConference 2014.Google Scholar
- Paul Baker and Amanda Potts. 2013. 'Why do white people have thin lips?' Google and the perpetuation of stereotypes via auto-complete search forms. Critical Discourse Studies 10, 2: 187--204.Google ScholarCross Ref
- Solon Barocas and Andrew D. Selbst. 2016. Big Data's Disparate Impact. California Law Review 104, 3: 671-- 732.Google Scholar
- David Beer. 2009. Power through the algorithm? Participatory web cultures and the technological unconscious. New Media & Society 11, 6: 985--1002.Google ScholarCross Ref
- Hugh Beyer and Karen Holtzblatt. 1998. Contextual Design: Defining Customer-centered Systems. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. Google ScholarDigital Library
- danah boyd and Kate Crawford. 2012. Critical Questions for Big Data. Information, Communication & Society 15, 5: 662--679.Google ScholarCross Ref
- danah boyd, Karen Levy, and Alice Marwick. 2014. The Networked Nature of Algorithmic Discrimination. In Data and Discrimination: Collected Essays, Seeta Peña Gangadharan, Virginia Eubanks and Solon Barocas (eds.). Open Technology Institute, New America Foundation, Washington, D.C., 53--57.Google Scholar
- Engin Bozdag. 2013. Bias in algorithmic filtering and personalization. Ethics and Information Technology 15, 3: 209--227. Google ScholarDigital Library
- Taina Bucher. 2017. The algorithmic imaginary: exploring the ordinary affects of Facebook algorithms. Information, Communication & Society 20, 1: 30--44.Google ScholarCross Ref
- Jenna Burrell. 2016. How the machine 'thinks': Understanding opacity in machine learning algorithms. Big Data & Society 3, 1: 1--12.Google ScholarCross Ref
- Kathleen Chaykowski. 2016. Facebook To Ban "Ethnic Affinity" Targeting For Housing, Employment, CreditRelated Ads. Forbes.Google Scholar
- Danielle Keats Citron and Frank Pasquale. 2014. The Scored Society: Due Process for Automated Predictions. Washington Law Review 89, 1.Google Scholar
- Ashley Colley, Jacob Thebault-Spieker, Allen Yilun Lin, Donald Degraen, Benjamin Fischman, Jonna Häkkilä, Kate Kuehl, Valentina Nisi, Nuno Jardim Nunes, Nina Wenig, Dirk Wenig, Brent Hecht, and Johannes Schöning. 2017. The Geography of PokéMon GO: Beneficial and Problematic Effects on Places and Movement. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 1179--1192. Google ScholarDigital Library
- Anne Corden and Roy Sainsbury. 2006. Using Verbatim Quotations in Reporting Qualitative Social Research. University of York, York, UK.Google Scholar
- Tressie McMillan Cottom. 2015. Credit Scores, Life Chances, and Algorithms. Retrieved September 15, 2017 from https://tressiemc.com/uncategorized/creditscores-life-chances-and-algorithms/Google Scholar
- Kate Crawford. 2014. The Anxieties of Big Data. The New Inquiry.Google Scholar
- Kate Crawford. 2016. Artificial Intelligence's White Guy Problem. The New York Times.Google Scholar
- Amit Datta, Michael Carl Tschantz, and Anupam Datta. 2015. Automated Experiments on Ad Privacy Settings. In Proceedings on Privacy Enhancing Technologies (PETS 2015), 92--112.Google ScholarCross Ref
- Michael A. DeVito, Jeremy Birnholtz, and Jeffery T. Hancock. Platforms, People, and Perception: Using Affordances to Understand Self-Presentation on Social Media. In Proceedings of the 20th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '17), 740--754. Google ScholarDigital Library
- Nicholas Diakopoulos. 2015. Algorithmic Accountability. Digital Journalism 3, 3: 398--415.Google ScholarCross Ref
- Carl DiSalvo, Thomas Lodato, Laura Fries, Beth Schechter, and Thomas Barnwell. 2011. The collective articulation of issues as design practice. CoDesign 7, 3--4: 185--197.Google ScholarCross Ref
- Carl DiSalvo, Illah Nourbakhsh, David Holstius, Ayça Akin, and Marti Louw. 2008. The Neighborhood Networks Project: A Case Study of Critical Engagement and Creative Expression Through Participatory Design. In Proceedings of the Tenth Anniversary Conference on Participatory Design 2008 (PDC '08), 41--50. Google ScholarDigital Library
- Carl DiSalvo, Phoebe Sengers, and Hrönn Brynjarsdóttir. 2010. Mapping the Landscape of Sustainable HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 1975--1984. Google ScholarDigital Library
- Paul Dourish. 2006. Implications for Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 541--550. Google ScholarDigital Library
- Benjamin Edelman and Michael Luca. 2014. Digital Discrimination: The Case of Airbnb.com. Harvard Business School Working Paper 14-054.Google Scholar
- Benjamin Edelman, Michael Luca, and Daniel Svirsky. 2017. Racial Discrimination in the Sharing Economy: Evidence from a Field Experiment. American Economic Journal: Applied Economics 9, 2: 1--22.Google ScholarCross Ref
- Erin Egan. 2016. Improving Enforcement and Promoting Diversity: Updates to Ethnic Affinity Marketing. Facebook Newsroom Blog. Retrieved September 14, 2017 from https://newsroom.fb.com/news/2016/11/updates-toethnic-affinity-marketing/Google Scholar
- Pelle Ehn. 1990. Work-Oriented Design of Computer Artifacts. L. Erlbaum Associates Inc., Hillsdale, NJ, USA. Google ScholarDigital Library
- Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. "I Always Assumed That I Wasn't Really That Close to {Her}": Reasoning About Invisible Algorithms in News Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 153--162. Google ScholarDigital Library
- Daniel C. Esty and Andrew Winston. 2006. Green to Gold: How Smart Companies Use Environmental Strategy to Innovate, Create Value, and Build Competitive Advantage. Yale University Press.Google Scholar
- Executive Office of the President. 2016. Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights.Google Scholar
- Federal Trade Commission. 2016. Big Data: A Tool for Inclusion or Exclusion? Understanding the Issues.Google Scholar
- Batya Friedman and Helen Nissenbaum. 1996. Bias in Computer Systems. ACM Transactions on Information Systems 14, 3: 330--347. Google ScholarDigital Library
- Shaun L. Gabbidon. 2003. Racial Profiling by Store Clerks and Security Personnel in Retail Establishments: An Exploration of "Shopping While Black." Journal of Contemporary Criminal Justice 19, 3: 345--364.Google ScholarCross Ref
- Marilyn Geewax. 2015. The Tipping Point: Most Americans No Longer Are Middle Class. NPR.Google Scholar
- Tarleton Gillespie. 2014. The relevance of algorithms. In Media Technologies: Essays on Communication, Materiality, and Society - University Press Scholarship, Tarleton Gillespie, Pablo Boczkowski and Kirsten Foot (eds.). MIT Press.Google Scholar
- Bryce Goodman and Seth Flaxman. 2016. European Union regulations on algorithmic decision-making and a "right to explanation." In ICML Workshop on Human Interpretability in Machine Learning (WHI 2016).Google Scholar
- David Graff. 2016. An update to our AdWords policy on lending products. Google Public Policy Blog. Retrieved September 15, 2017 from https://www.blog.google/topics/public-policy/anupdate-to-our-adwords-policy-on/Google Scholar
- Anthony G. Greenwald, Mahzarin R. Banaji, and Brian A. Nosek. 2015. Statistically small effects of the Implicit Association Test can have societally large effects. Journal of Personality and Social Psychology 108, 4: 553--561.Google ScholarCross Ref
- Maya Gupta, Andrew Cotter, Jan Pfeifer, Konstantin Voevodski, Kevin Canini, Alexander Mangylov, Wojciech Moczydlowski, and Alexander van Esbroeck. 2016. Monotonic Calibrated Interpolated Look-Up Tables. Journal of Machine Learning Research 17, 109: 1--47. Google ScholarDigital Library
- Bernard E. Harcourt. 2007. Against prediction: Profiling, policing, and punishing in an actuarial age. University of Chicago Press.Google Scholar
- Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of Opportunity in Supervised Learning. In Advances in Neural Information Processing Systems (NIPS 2016), 3315--3323. Google ScholarDigital Library
- Shin Inouye. 2016. Advocates Applaud Google's Ban on Payday Loan Advertisements. The Leadership Conference on Civil and Human Rights. Retrieved September 15, 2017 from http://civilrights.org/advocates-applaud-googles-banon-payday-loan-advertisements/Google Scholar
- Lucas D. Introna and Helen Nissenbaum. 2000. Shaping the Web: Why the Politics of Search Engines Matters. The Information Society 16, 3: 169--185.Google ScholarCross Ref
- Isaac Johnson, Connor McMahon, Johannes Schöning, and Brent Hecht. 2017. The Effect of Population and "Structural" Biases on Social Media-based Algorithms: A Case Study in Geolocation Inference Across the Urban-Rural Spectrum. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 1167--1178. Google ScholarDigital Library
- Robert Jungk, Norbert Müllert, and Institute for Social Inventions. 1987. Future workshops: how to create desirable futures. Institute for Social Inventions, London.Google Scholar
- Matthew Kay, Cynthia Matuszek, and Sean A. Munson. 2015. Unequal Representation and Gender Stereotypes in Image Search Results for Occupations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 3819--3828. Google ScholarDigital Library
- Jonathan Lazar, Julio Abascal, Simone Barbosa, Jeremy Barksdale, Batya Friedman, Jens Grossklags, Jan Gulliksen, Jeff Johnson, Tom McEwan, Loïc Martínez-Normand, Wibke Michalk, Janice Tsai, Gerrit van der Veer, Hans von Axelson, Ake Walldius, Gill Whitney, Marco Winckler, Volker Wulf, Elizabeth F. Churchill, Lorrie Cranor, Janet Davis, Alan Hedge, Harry Hochheiser, Juan Pablo Hourcade, Clayton Lewis, Lisa Nathan, Fabio Paterno, Blake Reid, Whitney Quesenbery, Ted Selker, and Brian Wentz. 2016. Human-Computer Interaction and International Public Policymaking: A Framework for Understanding and Taking Future Actions. Foundations and Trends in Human-Computer Interaction 9, 2: 69--149. Google ScholarDigital Library
- Min Kyung Lee and Su Baykal. 2017. Algorithmic Mediation in Group Decisions: Fairness Perceptions of Algorithmically Mediated vs. Discussion-Based Social Division. In Proceedings of the 20th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '17), 1035--1048. Google ScholarDigital Library
- Min Kyung Lee, Ji Tae Kim, and Leah Lizarondo. 2017. A Human-Centered Approach to Algorithmic Services: Considerations for Fair and Motivating Smart Community Service Management that Allocates Donations to Non-Profit Organizations. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 3365--3376. Google ScholarDigital Library
- Richard J. Lundman and Robert L. Kaufman. 2003. Driving While Black: Effects of Race, Ethnicity, and Gender on Citizen Self-Reports of Traffic Stops and Police Actions. Criminology 41, 1: 195--220.Google ScholarCross Ref
- Caitlin Lustig and Bonnie Nardi. 2015. Algorithmic Authority: The Case of Bitcoin. In 48th Hawaii International Conference on System Sciences (HICSS 2015), 743--752. Google ScholarDigital Library
- Caitlin Lustig, Katie Pine, Bonnie Nardi, Lilly Irani, Min Kyung Lee, Dawn Nafus, and Christian Sandvig. 2016. Algorithmic Authority: The Ethics, Politics, and Economics of Algorithms That Interpret, Decide, and Manage. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 1057--1062. Google ScholarDigital Library
- Peggy McIntosh. 1990. White Privilege: Unpacking the Invisible Knapsack. Independent School 49, 4: 31--5.Google Scholar
- Amanda Menking and Ingrid Erickson. 2015. The Heart Work of Wikipedia: Gendered, Emotional Labor in the World's Largest Online Encyclopedia. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 207-- 210. Google ScholarDigital Library
- Claire Cain Miller. 2015. When Algorithms Discriminate. The New York Times.Google Scholar
- Robert M. Morgan and Shelby D. Hunt. 1994. The Commitment-Trust Theory of Relationship Marketing. Journal of Marketing 58, 3: 20--38.Google ScholarCross Ref
- Laura W. Murphy. 2016. Airbnb's Work to Fight Discrimination and Build Inclusion: A Report Submitted to Airbnb. Retrieved September 15, 2017 from http://blog.atairbnb.com/wpcontent/uploads/2016/09/REPORT_Airbnbs-Work-toFight-Discrimination-and-Build-Inclusion.pdf?3c10beGoogle Scholar
- Carey Nadeau and Amy K. Glasmeier. 2016. Minimum Wage: Can an Individual or a Family Live on It? Retrieved September 15, 2017 from http://livingwage.mit.edu/articles/15-minimum-wagecan-an-individual-or-a-family-live-on-itGoogle Scholar
- Safiya Umoja Noble. 2014. Teaching Trayvon: Race, Media, and the Politics of Spectacle. The Black Scholar 44, 1: 12--29.Google ScholarCross Ref
- Cathy O'Neil. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown, New York. Google Scholar
- Jahna Otterbacher, Jo Bates, and Paul Clough. 2017. Competent Men and Warm Women: Gender Stereotypes and Backlash in Image Search Results. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 6620--6631. Google ScholarDigital Library
- Sarah Perez. 2016. Microsoft silences its new A.I. bot Tay, after Twitter users teach it racism. TechCrunch.Google Scholar
- Sarah Pink. 2014. Doing Visual Ethnography. Sage Publications.Google Scholar
- Angelisa Plane, Elissa Redmiles, Michelle Mazurek, and Michael Tschantz. 2017. Exploring User Perceptions of Discrimination in Online Targeted Advertising. In Proceedings of the 2017 USENIX Security Symposium.Google Scholar
- Bernadette D. Proctor, Jessica L. Semega, and Melissa A. Kollar. 2016. Income and Poverty in the United States: 2015. The United States Census Bureau. Retrieved September 15, 2017 from https://www.census.gov/library/publications/2016/dem o/p60--256.htmlGoogle Scholar
- Emilee Rader and Rebecca Gray. 2015. Understanding User Beliefs About Algorithmic Curation in the Facebook News Feed. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 173--182. Google ScholarDigital Library
- Matt Ratto. 2011. Critical Making: Conceptual and Material Studies in Technology and Social Life. The Information Society 27, 4: 252--260. Google ScholarDigital Library
- Noopur Raval and Paul Dourish. 2016. Standing Out from the Crowd: Emotional Labor, Body Labor, and Temporal Labor in Ridesharing. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16), 97--107. Google ScholarDigital Library
- Horst W. J. Rittel and Melvin M. Webber. 1973. Dilemmas in a general theory of planning. Policy Sciences 4, 2: 155--169.Google ScholarCross Ref
- Rosemary Rodriguez. 2015. Discovery. The Good Wife.Google Scholar
- Daniela K. Rosner, Saba Kawas, Wenqi Li, Nicole Tilly, and Yi-Chen Sung. 2016. Out of Time, Out of Place: Reflections on Design Workshops as a Research Method. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16), 1131--1141. Google ScholarDigital Library
- Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. CoDesign 4, 1: 5--18.Google ScholarCross Ref
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2015. Can an Algorithm be Unethical? In 65th Annual Meeting of the International Communication Association.Google Scholar
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and Discrimination: Converting Critical Concerns into Productive Inquiry: May 2014.Google Scholar
- Sassafras Tech Collective. 2016. Icebreaker. In Exploring Social Justice, Design, and HCI Workshop at CHI 2016.Google Scholar
- Clay Shirky. 2011. A Speculative Post on the Idea of Algorithmic Authority. Retrieved September 15, 2017 from http://www.shirky.com/weblog/2009/11/aspeculative-post-on-the-idea-of-algorithmic-authority/Google Scholar
- Kiley Sobel, Katie O'Leary, and Julie A. Kientz. 2015. Maximizing Children's Opportunities with Inclusive Play: Considerations for Interactive Technology Design. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15), 39--48. Google ScholarDigital Library
- Derald Wing Sue. 2010. Microaggressions in Everyday Life: Race, Gender, and Sexual Orientation. Wiley.Google Scholar
- Astra Taylor and Jathan Sadowski. 2015. How Companies Turn Your Facebook Activity Into a Credit Score. The Nation.Google Scholar
- David R. Thomas. 2006. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation 27, 2: 237--246.Google ScholarCross Ref
- Vanessa Thomas, Christian Remy, Mike Hazas, and Oliver Bates. 2017. HCI and Environmental Public Policy: Opportunities for Engagement. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 6986--6992. Google ScholarDigital Library
- Blase Ur, Pedro Giovanni Leon, Lorrie Faith Cranor, Richard Shay, and Yang Wang. 2012. Smart, Useful, Scary, Creepy: Perceptions of Online Behavioral Advertising. In Proceedings of the Eighth Symposium on Usable Privacy and Security (SOUPS '12), 4:1-- 4:15. Google ScholarDigital Library
- Jeff Warshaw, Nina Taft, and Allison Woodruff. 2016. Intuitions, analytics, and killing ants: Inference literacy of high school-educated adults in the US. In Proceedings of the Twelfth Symposium on Usable Privacy and Security (SOUPS '16).Google Scholar
- Brian Wynne. 1991. Knowledges in Context. Science, Technology, & Human Values 16, 1: 111--121.Google ScholarCross Ref
Index Terms
- A Qualitative Exploration of Perceptions of Algorithmic Fairness
Recommendations
Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction
WWW '18: Proceedings of the 2018 World Wide Web ConferenceAs algorithms are increasingly used to make important decisions that affect human lives, ranging from social benefit assignment to predicting risk of criminal recidivism, concerns have been raised about the fairness of algorithmic decision making. Most ...
Dimensions of Diversity in Human Perceptions of Algorithmic Fairness
EAAMO '22: Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and OptimizationA growing number of oversight boards and regulatory bodies seek to monitor and govern algorithms that make decisions about people’s lives. Prior work has explored how people believe algorithmic decisions should be made, but there is little ...
The methodology of studying fairness perceptions in Artificial Intelligence: Contrasting CHI and FAccT
AbstractThe topic of algorithmic fairness is of increasing importance to the Human–Computer Interaction research community following accumulating concerns regarding the use and deployment of Artificial Intelligence-based systems. How we ...
Highlights- Mapping of CHI and FAccT papers on algorithmic fairness published 1993–2022.
- ...
Comments