Top

TechTrends

Published in:

19-09-2022 | Original Paper

Interrogating Algorithmic Bias: From Speculative Fiction to Liberatory Design

Author: Nettrice Gaskins

Published in: TechTrends | Issue 3/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This paper reviews algorithmic or artificial intelligence (AI) bias in education technology, especially through the lenses of speculative fiction, speculative and liberatory design. It discusses the causes of the bias and reviews literature on various ways that algorithmic/AI bias manifests in education and in communities that are underrepresented in EdTech software development. While other recent work has responded to mainstream or private sector technology development, this review looks elsewhere where practitioners, artists, and activists engage underrepresented communities in brainstorming processes to identify and solve tough challenges. Their creative work includes films, toolkits, applications, prototypes and other physical artifacts, and other future-facing ideas that can provide guideposts for private sector development. Acknowledging the gaps in what has been studied, this paper proposes a different approach that includes speculative and liberatory design thinking, which can help developers better understand the educational and personal contexts of underrepresented groups. Early efforts to advocate for fairness and equity in AI and EdTech by groups such as the Algorithmic Justice League, the EdTech Equity Project, and EdSAFE AI Alliance is also explored.

previous article Expanding Theories of Sociopolitical Development: Centering the Intersectional Experiences of Girls of Color in an Informal STEM Program

next article Resisting Techno-Orientalism and Mimicry Stereotypes in and Through Data Science Education

Ali, S., Williams, R., Payne B., Park H., Breazeal C. (2019). Constructionism, Ethics, and Creativity: Developing Primary and Middle School Artificial Intelligence Education. Presented in the International Workshop on Education in Artificial Intelligence K-12 (EDUAI ‘19). Palo Alto, CA, USA. Proceedings of IJCAI 2019.

Angwin, J., J. Larson, S. Mattu, & L. Kirchner. (2016). Machine bias. Retrieved September 3, 2022, from https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Atske, S., & Perrin, A. (2022). Home broadband adoption, computer ownership vary by race, ethnicity in the U.S. Retrieved April 5, 2022, from https://www.pewresearch.org/fact-tank/2021/07/16/home-broadband-adoption-computer-ownership-vary-by-race-ethnicity-in-the-u-s

Attewell, P., Kasinitz, P., & Dunn, K. (2010). Black Canadians and black Americans: Racial income inequality in comparative perspective. Ethnic and Racial Studies, 33(3), 473–495. https://doi.org/10.1080/01419870903085883CrossRef

Atwood, M. (2014). In other worlds: SF and the human imagination. Virago.

Baraka, I. A. (1971). Technology & Ethos. Raise, race, rays, raze: Essays since 1965. Amistad, 2, 155–158.

Baym, N. K. (2015). Social media and the struggle for society. Social Media + Society, 1(1), 205630511558047. https://doi.org/10.1177/2056305115580477CrossRef

Benjamin, R. (2020). Race after technology: Abolitionist tools for the new jim code. Polity.

Bradley, B. (2021). Tool aims to track disparities in Ed-tech usage by race, poverty level. Retrieved April 5, 2022, from https://marketbrief.edweek.org/marketplace-k-12/tool-aims-track-disparities-ed-tech-usage-race-poverty-level

Brigham, S. E. (1993). TQM: Lessons we can learn from industry. Change, 42–48.

Broussard, M. (2019). Artificial unintelligence: How computers misunderstand the world. The MIT Press.

Buehring, J. H., & Liedtka, J. (2018). Embracing systematic futures thinking at the intersection of strategic planning, foresight and design. Journal of Innovation Management, 6(3), 134. https://doi.org/10.24840/2183-0606_006.003_0006CrossRef

Bulut, E. (2018). Interview with Safiya Umoja Noble: Algorithms of oppression, gender and race. Journal of Cultural Studies, Faculty of Communication, Hacettepe University, 5(2): 294-301.

Buolamwini, J. & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, in Proceedings of Machine Learning Research (Vol. 81, pp. 77–91).

Cheney, G., McMillan, J. J., & Schwartzman, R. (1997). Should we buy the ‘student-as-consumer’ metaphor? The Montana Professor, 7(3), 8–11.

Civin, M. (2022). Stephanie dinkins: On love and data. Retrieved September 3, 2022, from https://brooklynrail.org/2022/04/artseen/Stephanie-Dinkins-On-Love-and-Data

Crawford, K. (2017). The trouble with bias - NIPS 2017 keynote . YouTube. Retrieved September 14, 2022, from https://youtu.be/fMym_BKWQzk

Dery, M. (1997). Flame wars: The discourse of cyberculture. Duke Univ. Press.

Dimaggio, P., Harittai, E., Celeste, C., & Shafer, S. (2020). Excerpts from “digital inequality: From unequal access to differentiated use”. Inequality in the United States: A Reader, 98-113. https://doi.org/10.4324/9781003071709-13.

Dobson, J. E. (2019). Critical digital humanities: The search for a methodology. University of Illinois Press.CrossRef

Dorans, N. (2010). Misrepresentations in unfair treatment by Santelices and Wilson. Harvard Educational Review, 80(3), 404–413. https://doi.org/10.17763/haer.80.3.l253473353686748CrossRef

Dunne, A., & Raby, F. (2013). Speculative everything: Design, fiction, and social dreaming. The MIT Press.

Dunne, A., & Raby, F. (2021). Design noir: The secret life of electronic objects. Bloomsbury Visual Arts.CrossRef

EdSAFE AI (2022). Retrieved September 3, 2022, from https://www.edsafeai.org

Fergus, J. (2020). A facial recognition company lied to a school about how racist its algorithm is. Retrieved March 29, 2022, from https://www.inputmag.com/tech/a-facial-recognition-company-lied-to-a-school-about-how-racist-its-algorithm-is

Fortunato, S. (2020). Straight talk: Ayodamola Okunseinde. Retrieved April 1, 2022, from https://www.sciartmagazine.com/straight-talk-ayodamola-okunseinde.html

Fouché, R. (2004). Not made for black history month: Lewis Latimer and technological assimilation. Appropriating technology: Vernacular science and social power (pp. 315–330). University of Minnesota Press.

Fuchs, C., & Horak, E. (2008). Africa and the digital divide. Telematics and Informatics, 25(2), 99–116.CrossRef

Gaskins, N. R. (2021). Techno-vernacular creativity and innovation: Culturally relevant making inside and outside of the classroom. The MIT Press.CrossRef

Gee, J. P. (2017). Teaching, learning, literacy in our high-risk high-tech world: A framework for becoming human. Teachers College Press.

Grother, P, Ngan, M., & Hanaoka, K. (2019). Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects. National Institute of Standards and Technology.

Harrison, L. M., & Risler, L. (2015). The role consumerism plays in student learning. Act Learn High Educ, 16(1), 67–76. https://doi.org/10.1177/1469787415573356CrossRef

Hebbar, N., & M. Jacobs. (2022) What do Edtech and AI have to do with racial bias? Retrieved September 3, 2022, from https://digitalpromise.org/2021/10/28/what-do-edtech-and-ai-have-to-do-with-race-bias/

Hebbar, N., Jacobs, M., Volk, M., & Allendorf, E. (2020). AI in Education Toolkit for Racial Equity. The EdTech Equity Project.

Itō, M., Martin, C., Pfister, R. C., Rafalow, M. H., Salen, K., & Wortman, A. (2019). Affinity online: How connection and shared interest fuel learning. New York University Press.

Karumbaiah, S., & J. Brooks. (2021). How Colonial Continuities Underlie Algorithmic Injustices in Education. 2021 Conference on Research in Equitable and Sustained Participation in Engineering, Computing, and Technology (RESPECT). https://doi.org/10.1109/respect51740.2021.9620605.

Koenig, R. (2019). The new Jim code? race and discriminatory design - edsurge news. Retrieved April 5, 2022, from https://www.edsurge.com/news/2019-08-20-the-new-jim-code-race-and-discriminatory-design

Lee, J. (2020). When bias is coded into our technology. NPR. Retrieved September 14, 2022, from https://www.npr.org/sections/codeswitch/2020/02/08/770174171/when-bias-is-coded-into-our-technology

Lenhart, A. (2019). Mobile access shifts social media use and other online activities. Retrieved April 5, 2022, from https://www.pewresearch.org/internet/2015/04/09/mobile-access-shifts-social-media-use-and-other-online-activities

Love, B. (2016). Get to the future. Retrieved September 3, 2022, from http://getfreehiphopcivics.com/get-to-the-future/

Lynch, M. (2018). Analyzing Edtech's diversity problem. The Tech Edvocate. Retrieved September 14, 2022, from https://www.thetechedvocate.org/analyzing-EdTechs-diversity-problem

Mawasi, A., Ahmed, I., Walker, E., Wang, S., Marasli, Z., Whitehurst, A., Wylie, R. (2020). Using Design-Based Research to Improve Help Giving in Middle School Math Classroom. International Conference of the Learning Sciences.

Naidoo, R., & Whitty, G. (2013). Students as consumers: Commodifying or democratising learning? International Journal of Chinese Education, 2(2), 212–240. https://doi.org/10.1163/22125868-12340022CrossRef

Nguyen, M. H., Hargittai, E., & Marler, W. (2021). Digital inequality in communication during a time of physical distancing: The case of covid-19. Computers in Human Behavior, 120, 106717. https://doi.org/10.1016/j.chb.2021.106717CrossRef

Nielsen, M. (2011). Reinventing discovery: The new era of networked science. Princeton University Press.CrossRef

Njee, N. (2016). Share cropping blackness: White supremacy and the hyper-consumption of black popular culture. Sociology.

Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.CrossRef

Okunseinde, A. T. (2020). Iyapo repository: Constructing and archiving alternate futures. In V. Bradbury & S. O’Hara (Authors), Art hack practice: Critical intersections of art, innovation and the maker movement (pp. 87-98). : Routledge.

Pabst, A. A. (2021). Afro algorithms: Imagining new possibilities for race, technology, and the future through animated storytelling. Patterns, 2(8). https://doi.org/10.1016/j.patter.2021.100327.

Perry, A., & Turner-Lee, N. (2020). AI can disrupt racial inequity in schools, or make it much worse. Retrieved April 1, 2022, from https://hechingerreport.org/ai-can-disrupt-racial-inequity-in-schools-or-make-it-much-worse/

Pinkard, N., Erete, S., Martin, C. K., & McKinney de Royston, M. (2017). Digital youth divas: Exploring narrative-driven curriculum to spark middle school girls’ interest in computational activities. Journal of the Learning Sciences, 26(3), 477–516. https://doi.org/10.1080/10508406.2017.1307199CrossRef

Raji, I. D., Gebru, T., Mitchell, M., Buolamwini, J., Lee, J., & Denton, E. (2020). Saving face. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3375627.3375820

Reich, J, & Itō, M. (2017). From good intentions to real outcomes: Equity by design in learning technologies. Digital Media and Learning Research Hub.

Riddle, T., & Sinclair, S. (2019). Racial disparities in school-based disciplinary actions are associated with county-level rates of racial bias. Proceedings of the National Academy of Sciences, 116(17), 8255–8260. https://doi.org/10.1073/pnas.1808307116CrossRef

Robinson, L., Schulz, J., Khilnani, A., Ono, H., Cotten, S. R., McClain, N., ... Tolentino, N. (2020). Digital inequalities in time of pandemic: Covid-19 exposure risk profiles and new forms of vulnerability. First Monday. https://doi.org/10.5210/fm.v25i7.10845.

Santelices, M. V., & Wilson, M. (2010). Unfair treatment? The case of Freedle, the SAT, and the standardization approach to differential item functioning. Harvard Educational Review, 80(1), 106–134. https://doi.org/10.17763/haer.80.1.j94675w001329270CrossRef

Sinclair, K. (2017). We have the power to code a new operating system for humanity. Retrieved September 3, 2022, from https://www.weforum.org/agenda/2017/06/artificial-intelligence-code-better-world/

Srinivasan, R. (2018). Whose global village? Rethinking how technology shapes our world. NYU Press.

Sweeney, L. (2013). Discrimination in online ad delivery. Communications of the ACM, 56(5), 44–54. https://doi.org/10.1145/2447976.2447990CrossRef

Toyama, K. (2015). Geek heresy: Rescuing social change from the cult of technology. PublicAffairs.

Vincent, J. (2022). Afrobrainiac. Retrieved September 3, 2022, from https://www.afrobrainiac.com/.

Watkins, S. C., & Cho, A. (2018). The digital edge: How black and Latino youth navigate digital inequality. NYU Press.

Winchester III, W. W. (2019). Engaging the black ethos: Afrofuturism as a design Lens for inclusive technological innovation. Journal of Futures Studies, 24(2), 55–62.

Winerip, M. (2012). Facing a robo-grader? just keep obfuscating mellifluously. Retrieved March 29, 2022, from https://www.nytimes.com/2012/04/23/education/robo-readers-used-to-grade-test-essays.html

Young, J. K. (2006). Black writers, white publishers: Marketplace politics in twentieth-century African American literature. University of Mississippi.

Title: Interrogating Algorithmic Bias: From Speculative Fiction to Liberatory Design
Author: Nettrice Gaskins
Publication date: 19-09-2022
Publisher: Springer US
Published in: TechTrends / Issue 3/2023
Print ISSN: 8756-3894
Electronic ISSN: 1559-7075
DOI: https://doi.org/10.1007/s11528-022-00783-0

Introducing the May 2023 Issue

Column: Editor’s Notes

Discovering Unproductive Learning Patterns of Wheel-spinning Students in Intelligent Tutors Using Cluster Analysis

Original Paper

Digital Technology for a Borderless World: Innovative Educators in Practice

Guest Editorial

Race-Positive Career and Technical Education: Techno-Social Agency Beyond the Vocational-Liberal Divide

Original Paper

Informal Learning in United States Libraries: A Systematic Review

Original Paper

Microlearning through the Lens of Gagne’s Nine Events of Instruction: A Qualitative Study

Original Paper

Springer Professional

Interrogating Algorithmic Bias: From Speculative Fiction to Liberatory Design

Abstract

Other articles of this Issue 3/2023

Introducing the May 2023 Issue

Discovering Unproductive Learning Patterns of Wheel-spinning Students in Intelligent Tutors Using Cluster Analysis

Digital Technology for a Borderless World: Innovative Educators in Practice

Race-Positive Career and Technical Education: Techno-Social Agency Beyond the Vocational-Liberal Divide

Informal Learning in United States Libraries: A Systematic Review

Microlearning through the Lens of Gagne’s Nine Events of Instruction: A Qualitative Study

Premium Partner

Springer Professional

Abstract

Please log in to get access to your license.

Introducing the May 2023 Issue

Discovering Unproductive Learning Patterns of Wheel-spinning Students in Intelligent Tutors Using Cluster Analysis

Digital Technology for a Borderless World: Innovative Educators in Practice

Race-Positive Career and Technical Education: Techno-Social Agency Beyond the Vocational-Liberal Divide

Informal Learning in United States Libraries: A Systematic Review

Microlearning through the Lens of Gagne’s Nine Events of Instruction: A Qualitative Study