Michael Katell
Editorial Notes
The authors have requested minor, non-substantive changes to the VoR and, in accordance with ACM policies, a Corrected VoR was published on April 21, 2020. For reference purposes the VoR may still be accessed via the Supplemental Material section on this page.
ABSTRACT
Research to date aimed at the fairness, accountability, and transparency of algorithmic systems has largely focused on topics such as identifying failures of current systems and on technical interventions intended to reduce bias in computational processes. Researchers have given less attention to methods that account for the social and political contexts of specific, situated technical systems at their points of use. Co-developing algorithmic accountability interventions in communities supports outcomes that are more likely to address problems in their situated context and re-center power with those most disparately affected by the harms of algorithmic systems. In this paper we report on our experiences using participatory and co-design methods for algorithmic accountability in a project called the Algorithmic Equity Toolkit. The main insights we gleaned from our experiences were: (i) many meaningful interventions toward equitable algorithmic systems are non-technical; (ii) community organizations derive the most value from localized materials as opposed to what is "scalable" beyond a particular policy context; (iii) framing harms around algorithmic bias suggests that more accurate data is the solution, at the risk of missing deeper questions about whether some technologies should be used at all. More broadly, we found that community-based methods are important inroads to addressing algorithmic harms in their situated contexts.
Supplemental Material
Available for Download
pdf
3372874-vor.pdf (1.1 MB)
Version of Record for "Toward situated interventions for algorithmic equity: lessons from the field" by Katell et al., Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* '20).
- Philip E Agre. 1997. Lessons learned in trying to reform AI. Social Science, Technical Systems, and Cooperative Work: Beyond the Great Divide (1997), 131.Google Scholar
- Mariam Asad, Christopher A Le Dantec, Becky Nielsen, and Kate Diedrick. 2017. Creating a sociotechnical API: Designing city-scale community engagement. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2295--2306.Google ScholarDigital Library
- Stephanie Ballard, Karen M Chappell, and Kristen Kennedy. 2019. Judgment call the game: Using value sensitive design and design fiction to surface ethical concerns related to technology. In Proceedings of the 2019 on Designing Interactive Systems Conference. ACM, 421--433.Google ScholarDigital Library
- Shaowen Bardzell. 2010. Feminist HCI: Taking stock and outlining an agenda for design. In Proceedings of the SIGCHI Conference on Human Factor in Computing Systems. ACM, 1301--1310.Google ScholarDigital Library
- Solon Barocas and Danah Boyd. 2017. Engaging the ethics of data science in practice. Commun. ACM 60, 11 (2017), 23--25.Google ScholarDigital Library
- Ruha Benjamin. 2019. Captivating Technology: Race, Carceral Technoscience, and Liberatory Imagination in Everyday Life. Duke University Press.Google ScholarCross Ref
- Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley & Sons.Google Scholar
- Davide Beraldo and Stefania Milan. 2019. From data politics to the contentious politics of data. Big Data & Society 6, 2 (2019), 2053951719885967.Google ScholarCross Ref
- O Fals Borda. 2006. Participatory (action) research in social theory: Origins and challenges. Handbook of Action Research (2006), 27--37.Google Scholar
- Dimitrios Bountouridis, Jaron Harambam, Mykola Makhortykh, Mónica Marrero, Nava Tintarev, and Claudia Hauff. 2019. SIREN: A Simulation framework for understanding the effects of recommender systems in online news environments. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 150--159.Google ScholarDigital Library
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency. 77--91.Google Scholar
- Sasha Costanza-Chock. 2018. Design Justice: Towards an intersectional feminist framework for design theory and practice. Proceedings of the Design Research Society (2018).Google ScholarCross Ref
- Sasha Costanza-Chock. 2020. Design Justice: Community-led Practices to Build the World We Need. MIT Press.Google ScholarCross Ref
- Lina Dencik, Arne Hintz, Joanna Redden, and Emiliano Treré. 2019. Exploring Data Justice: Conceptions, Applications and Directions.Google Scholar
- Carl DiSalvo, Jonathan Lukens, Thomas Lodato, Tom Jenkins, and Tanyoung Kim. 2014. Making public things: How HCI design can express matters of concern. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2397--2406.Google ScholarDigital Library
- Paul Dourish. 2004. What we talk about when we talk about context. Personal and Ubiquitous Computing 8, 1 (2004), 19--30.Google ScholarDigital Library
- Jenny Durkan. 2012. EXHIBIT A (Settlement Agreement and Stipulated [Proposed] Order of Resolution, Office of the Mayor of Seattle) - 1 12-CV-1282., 76 pages.Google Scholar
- Catherine D'Ignazio and Lauren Klein. 2020. Data Feminism.Google Scholar
- Severin Engelmann, Mo Chen, Felix Fischer, Ching-yu Kao, and Jens Grossklags. 2019. Clear sanctions, vague rewards: How China's social credit system currently defines good and bad behavior. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 69--78.Google ScholarDigital Library
- Christine Floyd, Wolf-Michael Mehl, Fanny-Michaela Resin, Gerhard Schmidt, and Gregor Wolf. 1989. Out of Scandinavia: Alternative approaches to software design and system development. Human-Computer Interaction 4, 4 (1989), 253--350.Google ScholarDigital Library
- Sarah Fox, Mariam Asad, Katherine Lo, Jill P Dimond, Lynn S Dombrowski, and Shaowen Bardzell. 2016. Exploring social justice, design, and HCI. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 3293--3300.Google ScholarDigital Library
- Paulo Freire. 2018. Pedagogy of the Oppressed. Bloomsbury Publishing USA.Google Scholar
- Batya Friedman and David Hendry. 2012. The envisioning cards: A toolkit for catalyzing humanistic and technical imaginations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1145--1148.Google ScholarDigital Library
- Bill Gaver, Tony Dunne, and Elena Pacenti. 1999. Design: Cultural probes. interactions 6, 1 (1999), 21--29.Google ScholarDigital Library
- Ben Green and Yiling Chen. 2019. Disparate interactions: An algorithm-in-the-loop analysis of fairness in risk assessments. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 90--99.Google ScholarDigital Library
- Egon G Guba and Yvonna S Lincoln. 2005. Paradigmatic controversies, contradictions, and emerging confluences. (2005).Google Scholar
- Donna Haraway. 1988. Situated knowledges: The science question in feminism and the privilege of partial perspective. Feminist Studies 14, 3 (1988), 575--599.Google ScholarCross Ref
- Sandra G Harding. 2004. The Feminist Standpoint Theory Reader: Intellectual and Political Controversies. Psychology Press.Google Scholar
- Nancy CM Hartsock. 2017. The feminist standpoint: Developing the ground for a specifically feminist historical materialism. In Karl Marx. Routledge, 565--592.Google Scholar
- Woodrow Hartzog and Evan Selinger. 2018. Facial recognition is the perfect tool for oppression. Medium (2018).Google Scholar
- Gillian R. Hayes. 2011. The relationship of action research to human-computer interaction. ACM Transactions on Computer-Human Interaction 18, 3 (2011), 1--20.Google ScholarDigital Library
- Anna Lauren Hoffmann. 2019. Where fairness fails: Data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society 22, 7 (2019), 900--915.Google ScholarCross Ref
- Elizabeth E Joh. 2014. Policing by numbers: Big data and the Fourth Amendment. Wash. L. Rev. 89 (2014), 35.Google Scholar
- Holtzblatt Karen and Jones Sandra. 2017. Contextual inquiry: A participatory technique for system design. In Participatory Design. CRC Press, 177--210.Google Scholar
- Michael Katell. 2018. Adverse detection: The promise and peril of body-worn cameras. In SURVEILLANCE, PRIVACY AND PUBLIC SPACE., Bryce Clayton Newell, Tjerk Timan, and Bert-Jaap Koops (Eds.). Taylor & Francis, Abingdon, 99--118. OCLC: 1009072661.Google Scholar
- Michael Katell, Meg Young, Bernease Herman, Dharma Dailey, Aaron Tam, Vivian Guetler, Corinne Binz, Daniella Raz, and PM Krafft. 2019. An Algorithmic Equity Toolkit for Technology Audits by Community Advocates and Activists. arXiv preprint arXiv:1912.02943 (2019).Google Scholar
- Finn Kensing and Jeanette Blomberg. 1998. Participatory design: Issues and concerns. Computer Supported Cooperative Work (CSCW) 7, 3-4 (1998), 167--185.Google ScholarDigital Library
- Finn Kensing and Kim Halskov Madsen. 1992. Generating visions: Future workshops and metaphorical design. L. Erlbaum Associates Inc.Google Scholar
- Os Keyes, Josephine Hoy, and Margaret Drouhard. 2019. Human-computer insurrection: Notes on an anarchist HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 339.Google ScholarDigital Library
- Sean A Kidd and Michael J Kral. 2005. Practicing participatory action research. Journal of Counseling Psychology 52, 2 (2005), 187.Google ScholarCross Ref
- PM Krafft, Meg Young, Michael Katell, Karen Huang, and Ghislain Bugingo. 2020. Defining Artificial Intelligence in Policy versus Practice. Proceedings of the 2020 AAAI/ACM Conference on AI, Ethics, and Society (AIES). (2020).Google ScholarDigital Library
- Nick Logler, Daisy Yoo, and Batya Friedman. 2018. Metaphor cards: A how-to-guide for making and using a generative metaphorical design toolkit. In Proceedings of the 2018 Designing Interactive Systems Conference. ACM, 1373--1386.Google ScholarDigital Library
- Lassana Magassa, Meg Young, and Batya Friedman. 2017. Diverse Voices: A how-to guide for creating more inclusive tech policy documents. Tech Policy Lab (2017).Google Scholar
- Momin M Malik, Katja Mayer, Hemank Lamba, and Claudia Müller-Birn. [n. d.]. Workshop on Critical Data Science. ([n. d.]).Google Scholar
- Robin McTaggart. 1991. Principles for participatory action research. Adult Education Quarterly 41, 3 (1991), 168--187.Google ScholarCross Ref
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 220--229.Google ScholarDigital Library
- Laura Moy. 2019. How police technology aggravates racial inequity: A taxonomy of problems and a path forward. Available at SSRN 3340898 (2019).Google Scholar
- Jakob Nielsen. 1994. Heuristic evaluation. In Usability Inspection Methods. John Wiley & Sons, Inc., 25--62.Google ScholarDigital Library
- Safiya Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. NYU Press.Google Scholar
- Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447--453.Google Scholar
- Frank Pasquale. 2015. The Black Box Society. Harvard University Press.Google Scholar
- Samir Passi and Solon Barocas. 2019. Problem formulation and fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT* '19. ACM Press, Atlanta, GA, USA, 39--48.Google ScholarDigital Library
- Seeta Peña Gangadharan and Jędrzej Niklas. 2019. Decentering technology in discourse on discrimination. Information, Communication & Society 22, 7 (2019), 882--899.Google ScholarCross Ref
- Md Anisur Rahman. 2008. Some trends in the praxis of participatory action research. The Sage Handbook of Action Research: Participative Inquiry and Practice (2008), 49--62.Google Scholar
- Marc Rettig. 1994. Prototyping for tiny fingers. Commun. ACM 37, 4 (1994), 21--27.Google ScholarDigital Library
- Elizabeth B-N Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. Co-design 4, 1 (2008), 5--18.Google Scholar
- Douglas Schuler and Aki Namioka. 1993. Participatory design: Principles and practices. CRC Press.Google Scholar
- Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 59--68.Google ScholarDigital Library
- Clay Spinuzzi. 2005. The methodology of participatory design. Technical Communication 52, 2 (2005), 163--174.Google Scholar
- Luke Stark. 2019. Facial recognition is the plutonium of AI. XRDS: Crossroads, The ACM Magazine for Students 25, 3 (2019), 50--55.Google ScholarDigital Library
- Lucy A Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge University Press.Google Scholar
- Meredith Whittaker, Kate Crawford, Roel Dobbe, Genevieve Fried, Elizabeth Kaziunas, Varoon Mathur, Sarah Mysers West, Rashida Richardson, Jason Schultz, and Oscar Schwartz. 2018. AI Now Report 2018. AI Now Institute at New York University.Google Scholar
- Ke Yang, Julia Stoyanovich, Abolfazl Asudeh, Bill Howe, HV Jagadish, and Gerome Miklau. 2018. A nutritional label for rankings. In Proceedings of the 2018 International Conference on Management of Data. ACM, 1773--1776.Google ScholarDigital Library
- Daisy Yoo, Alina Huldtgren, Jill Palzkill Woelfer, David G Hendry, and Batya Friedman. 2013. A value sensitive action-reflection model: Evolving a co-design space with stakeholder and designer prompts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 419--428.Google ScholarDigital Library
- Meg Young, Michael Katell, and P. M. Krafft. 2019. Municipal surveillance regulation and algorithmic accountability. Big Data & Society 6, 2 (2019).Google Scholar
- Meg Young, Lassana Magassa, and Batya Friedman. 2019. Toward inclusive tech policy design: A method for underrepresented voices to strengthen tech policy documents. Ethics and Information Technology 21, 2 (2019), 89--103.Google ScholarDigital Library
- Meg Young, Luke Rodriguez, Emily Keller, Feiyang Sun, Boyang Sa, Jan Whittington, and Bill Howe. 2019. Beyond open vs. closed: Balancing individual privacy and public accountability in data sharing. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 191--200.Google ScholarDigital Library
Recommendations
Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsAlgorithmic decision-making systems are increasingly being adopted by government public service agencies. Researchers, policy experts, and civil rights groups have all voiced concerns that such systems are being deployed without adequate consideration ...
Conceptualizing Algorithmic Stigmatization
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsAlgorithmic systems have infiltrated many aspects of our society, mundane to high-stakes, and can lead to algorithmic harms known as representational and allocative. In this paper, we consider what stigma theory illuminates about mechanisms leading to ...
Equity & Inclusivity at IDC
IDC '17: Proceedings of the 2017 Conference on Interaction Design and ChildrenIn this one-day workshop, we aim to bring together a community of researchers at the Interaction Design and Children (IDC) conference who will share how they already make or plan to make equity-promoting fairness by allocating more resources and ...
Comments