Top

AI & SOCIETY

Published in:

27-09-2022 | Original Paper

The cyclical ethical effects of using artificial intelligence in education

Authors: Edward Dieterle, Chris Dede, Michael Walker

Published in: AI & SOCIETY | Issue 2/2024

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Our synthetic review of the relevant and related literatures on the ethics and effects of using AI in education reveals five qualitatively distinct and interrelated divides associated with access, representation, algorithms, interpretations, and citizenship. We open our analysis by probing the ethical effects of algorithms and how teams of humans can plan for and mitigate bias when using AI tools and techniques to model and inform instructional decisions and predict learning outcomes. We then analyze the upstream divides that feed into and fuel the algorithmic divide, first investigating access (who does and does not have access to the hardware, software, and connectivity necessary to engage with AI-enhanced digital learning tools and platforms) and then representation (the factors making data either representative of the total population or over-representative of a subpopulation’s preferences, thereby preventing objectivity and biasing understandings and outcomes). After that, we analyze the divides that are downstream of the algorithmic divide associated with interpretation (how learners, educators, and others understand the outputs of algorithms and use them to make decisions) and citizenship (how the other divides accumulate to impact interpretations of data by learners, educators, and others, in turn influencing behaviors and, over time, skills, culture, economic, health, and civic outcomes). At present, lacking ongoing reflection and action by learners, educators, educational leaders, designers, scholars, and policymakers, the five divides collectively create a vicious cycle and perpetuate structural biases in teaching and learning. However, increasing human responsibility and control over these divides can create a virtuous cycle that improves diversity, equity, and inclusion in education. We conclude the article by looking forward and discussing ways to increase educational opportunity and effectiveness for all by mitigating bias through a cycle of progressive improvement.

previous article Are AI systems biased against the poor? A machine learning analysis using Word2Vec and GloVe embeddings

next article Reading vs. Scanning: Notes on Re:Print

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Adjei S, Ostrow K, Erickson, E, Heffernan N (2017) Clustering students in ASSISTments: exploring system and school-level traits to advance personalization. In: Proceedings of the 10th international conference on Educational Data Mining, Hubei Province, China

Baker RS (2019) Challenges for the future of educational data mining: the Baker learning analytics prizes. J Educ Data Mining 11(1):1–17. https://doi.org/10.5281/zenodo.3554745CrossRef

Bakhshi H, Downing J, Osborne M, Schneider P (2017). The future of skills: employment in 2030. Pearson and Nesta. https://futureskills.pearson.com/research/assets/pdfs/technical-report.pdf

Buolamwini J, Gebru T (2018) Gender shades: Intersectional accuracy disparities in commercial gender classification. In: Proceedings of the 1st conference on fairness, accountability and transparency, New York, NY. http://proceedings.mlr.press

CAST (2018) Universal design for learning guidelines. Version 2.2. https://udlguidelines.cast.org/

Caliskan A, Bryson JJ, Narayanan A (2017) Semantics derived automatically from language corpora contain human-like biases. Science 356(6334):183–186. https://doi.org/10.1126/science.aal4230CrossRef

Chandra S, Chang A, Day L, Fazlullah A, Liu J, McBride L, Mudalige T, Weiss D (2020) Closing the K-12 digital divide in the age of distance learning. https://www.commonsensemedia.org/sites/default/files/uploads/pdfs/common_sense_media_report_final_7_1_3pm_web.pdf

Chardin M, Novak K (2020) Equity by design: Delivering on the power and promise of UDL. Corwin Press, California

Clark DB, Tanner-Smith EE, Killingsworth SS (2016) Digital games, design, and learning: a systematic review and meta-analysis. Rev Educ Res 86(1):79–122. https://doi.org/10.3102/0034654315582065CrossRef

Council of Chief State School Officers (2018) States leading for equity: promising practices advancing the equity commitments. https://ccsso.org/sites/default/files/2018-03/States%20Leading%20for%20Equity%20Online_031418.pdf

Culp KM, Honey M, Mandinach E (2005) A retrospective on twenty years of education technology policy. J Educ Comput Res 32(3):279–307. https://doi.org/10.2190/7w71-qvt2-pap2-udx7CrossRef

D’Angelo C, Rutstein D, Harris C, Bernard R, Borokhovski E, Haertel G (2014) Simulations for STEM learning: systematic review and meta-analysis. SRI International. https://www.sri.com/wp-content/uploads/pdf/simulations-for-stem-learning-full-report.pdf

Daniel BK (2019) Big data and data science: a critical review of issues for educational research. Br J Edu Technol 50(1):101–113. https://doi.org/10.1111/bjet.12595CrossRef

Decuypere M, Grimaldi E, Landri P (2021) Introduction: critical studies of digital education platforms. Crit Stud Educ 62(1):1–16. https://doi.org/10.1080/17508487.2020.1866050CrossRef

Dede C (2015) Data-intensive research in education: current work and next steps. Computing Research Association. https://cra.org/wp-content/uploads/2015/10/CRAEducationReport2015.pdf

Dede C (2019) Improving efficiency and effectiveness through learning engineering. In: Dede C, Richards J, Saxberg B (eds) Learning engineering for online education: theoretical contexts and design-based examples. Routledge, UK, pp 1–14

Dede C (2020) Introduction: Reconceptualizing higher education and lifelong learning in the era of the synergistic digital economy. In: Dede C, Richards J (eds) The 60-year curriculum: new models for lifelong learning in the digital economy. Routledge, UK, pp 1–24CrossRef

Dede C, Richards J, Saxberg B (eds) (2019) Learning engineering for online education: theoretical contexts and design-based examples. Routledge, UK. https://doi.org/10.4324/9781351186193CrossRef

Díaz P, Ioannou A, Bhagat KK, Spector JM (eds) (2019) Learning in a digital world: perspective on interactive technologies for formal and informal education. Springer, Berlin. https://doi.org/10.1007/978-981-13-8265-9CrossRef

Digital Promise (n.d.) Product certifications. Retrieved December 17, 2021. https://productcertifications.digitalpromise.org/

D’Mello S, Dieterle E, Duckworth A (2017) Advanced, analytic, automated (AAA) measurement of engagement during learning. Educ Psychol 52(2):104–123. https://doi.org/10.1080/00461520.2017.1281747CrossRef

Educational Testing Service. (2014) ETS standards for quality and fairness. https://www.ets.org/s/about/pdf/standards.pdf

Ercikan K, Pellegrino JW (eds) (2017) Validation of score meaning for the next generation of assessments: the use of response processes. Routledge, UK

Eubanks V (2017) Automating in equality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press

European Commission. (n.d.). AI watch: monitor the development, uptake and impact of Artificial Intelligence for Europe. Retrieved July 25, 2021. https://knowledge4policy.ec.europa.eu/ai-watch_en

European Economic and Social Committee (2017) The ethics of big data: balancing economic benefits and ethical questions of big data in the EU policy context. European Union. https://doi.org/10.2864/18937CrossRef

Ferrero F, Barujel AG (2019) Algorithmic driven decision-making systems in education: analyzing bias from the sociocultural perspective. In: 2019 XIV Latin American Conference on Learning Technologies (LACLO), https://doi.org/10.1109/LACLO49268.2019.00038

Finlay L (2016) Negotiating the swamp: the opportunity and challenge of reflexivity in research practice. Qual Res 2(2):209–230. https://doi.org/10.1177/146879410200200205CrossRef

Fischer C, Pardos ZA, Baker RS, Williams JJ, Smyth P, Yu R, Slater S, Baker R, Warschauer M (2020) Mining big data in education: affordances and challenges. Rev Res Educ 44(1):130–160. https://doi.org/10.3102/0091732x20903304CrossRef

Fishman B, Dede C (2016) Teaching and technology: new tools for new times. In: Gitomer DH, Bell CA (eds) Handbook of research on teaching, 5th edn. American Educational Research Association, Washington, pp 1269–1334CrossRef

Giannakos MN, Sharma K, Pappas IO, Kostakos V, Velloso E (2019) Multimodal data as a means to understand the learning experience. Int J Inf Manage 48:108–119. https://doi.org/10.1016/j.ijinfomgt.2019.02.003CrossRef

Gilman M (2020) Poverty lawgorithms: a poverty lawyer’s guide to fighting automated decision-making harms on low-income communities. https://ssrn.com/abstract=3699650

Guo H, Rios JA, Haberman S, Liu OL, Wang J, Paek I (2016) A new procedure for detection of students’ rapid guessing responses using response time. Appl Meas Educ 29(3):173–183. https://doi.org/10.1080/08957347.2016.1171766

Hao K (2020) The coming war on the hidden algorithms that trap people in poverty. https://www.technologyreview.com/2020/12/04/1013068/algorithms-create-a-poverty-trap-lawyers-fight-back/

Heffernan NT, Heffernan CL (2014) The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. Int J Artif Intell Educ 24(4):470–497. https://doi.org/10.1007/s40593-014-0024-xMathSciNetCrossRef

IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019) Ethically aligned design: a vision for prioritizing human well-being with autonomous and intelligent systems, 1st edn. IEEE. https://standards.ieee.org/content/dam/ieee-standards/standards/web/documents/other/ead1e.pdf

Jiang W, Pardos ZA, Wei Q (2019) Goal-based course recommendation. In: Proceedings of the 9th international conference on learning analytics & knowledge, Tempe, AZ. https://doi.org/10.1145/3303772.3303814

Kirkpatrick K (2016) Battling algorithmic bias. Commun ACM 59(10):16–17. https://doi.org/10.1145/2983270CrossRef

Kirsch I, Sands A, Robbins S, Goodman M, Tannenbaum R (2021) Buttressing the middle: a case for reskilling and upskilling America’s middle-skill workers in the 21st century. https://www.ets.org/s/research/pdf/buttressing-policy-report.pdf

Kostyuk V, Almeda MV, Baker RS (2018) Correlating affect and behavior in reasoning mind with state test achievement. In: Proceedings of the 8th international conference on learning analytics and knowledge, Sydney, Australia

LearnPlatform (2021) Edtech engagement & digital learning equity gaps. Retrieved July 14, 2021. https://learnplatform.com/insights/infographic/2020-digital-equity-in-review

Lee Y-H, Haberman SJ (2016) Investigating test-taking behaviors using timing and process data. Int J Test 16(3):240–267. https://doi.org/10.1080/15305058.2015.1085385CrossRef

Levy R (2019) Dynamic Bayesian network modeling of game-based diagnostic assessments. Multivar Behav Res 54(6):771–794. https://doi.org/10.1080/00273171.2019.1590794CrossRef

Luckin R, Holmes W, Griffiths M, Forcier LB (2016) Intelligence unleashed: an argument for AI in education. Pearson. https://www.pearson.com/content/dam/one-dot-com/one-dot-com/global/Files/about-pearson/innovation/open-ideas/Intelligence-Unleashed-v15-Web.pdf

MacLean CL, Dror IE (2016) A primer on the psychology of cognitive bias. In: Robertson CT, Kesselheim AS (eds) Blinding as a solution to bias: strengthening biomedical science, forensic science, and law. Academic Press, Cambridge, pp 13–24

Mandinach EB, Miskell RC (2018) Blended learning and data use in three technology-infused charter schools. Learn Landsc 11(1):183–198. https://doi.org/10.36510/learnland.v11i1.931CrossRef

Mandinach EB, Schildkamp K (2021) Misconceptions about data-based decision making in education: an exploration of the literature. Stud Educ Eval 69:1–10. https://doi.org/10.1016/j.stueduc.2020.100842CrossRef

Marco CA, Larkin GL (2000) Research ethics: ethical issues of data reporting and the quest for authenticity. Acad Emerg Med 7(6):691–694. https://doi.org/10.1111/j.1553-2712.2000.tb02049.xCrossRef

Mitchell M, Baker D, Moorosi N, Denton E, Hutchinson B, Hanna A, Gebru T, Morgenstern J (2020) Diversity and inclusion metrics in subset selection. In: 2020 AAAI/ACM conference on AI, ethics, and society (AIES ’20), New York, NY

Mozur P (2018) A genocide incited on Facebook, with posts from Myanmar's military. The New York Times. https://www.nytimes.com/2018/10/15/technology/myanmar-facebook-genocide.html

Nance JP (2019) Implicit racial bias and students' fourth amendment rights. University of Florida Levin College of Law Research Paper No. 19-26. https://ssrn.com/abstract=3140638

National Academies of Sciences, Engineering, and Medicine (2000). How people learn: brain, mind, experience, and school (expanded ed.). National Academy Press. https://doi.org/10.17226/9853

National Academies of Sciences, Engineering, and Medicine (2018) How people learn II: learners, contexts, and cultures. The National Academy Press, Washington. https://doi.org/10.17226/24783CrossRef

National Academies of Sciences, Engineering, and Medicine (2020) Climate change: evidence and causes Update 2020. The National Academies Press, Washington. https://doi.org/10.17226/25733CrossRef

Nielsen MW, Bloch CW, Schiebinger L (2018) Making gender diversity work for scientific discovery and innovation. Nat Hum Behav 2(10):726–734. https://doi.org/10.1038/s41562-018-0433-1CrossRef

Niemi DM, Pea RD, Saxburg B, Clark RE (eds) (2018) Learning analytics in education. Information Age Publishing Inc., USA

Olteanu A, Castillo C, Diaz F, Kıcıman E (2019) Social data: biases, methodological pitfalls, and ethical boundaries. Front Big Data. https://doi.org/10.3389/fdata.2019.00013CrossRef

Organisation for Economic Co-operation and Development (2018) The future of education and skills: education 2030. http://www.oecd.org/education/2030/oecd-education-2030-position-paper.pdf

Osei Bonsu K, Song J (2020) Turbulence on the global economy influenced by artificial intelligence and foreign policy inefficiencies. J Liberty Int Affairs 6(2):113–122. https://doi.org/10.47305/JLIA2020113obCrossRef

Pardos ZA, Baker RS, San Pedro M, Gowda SM, Gowda SM (2014) Affective states and state tests: Investigating how affect and engagement during the school year predict end-of-year learning outcomes. J Learn Anal 1(1):107–128. https://doi.org/10.18608/jla.2014.11.6CrossRef

Perkins D (1992) Smart schools: better thinking and learning for every child. Free Press, New York

Powell A (2021) Two early vaccination surveys worse than worthless thanks to ‘big data paradox,’ analysts say. https://news.harvard.edu/gazette/story/2021/12/vaccination-surveys-fell-victim-to-big-data-paradox-harvard-researchers-say/

Raghavan M, Barocas S, Kleinberg J, Levy K (2020) Mitigating bias in algorithmic hiring: Evaluating claims and practices. Proceedings of the 2020 conference on fairness, accountability, and transparency

Ramsetty A, Adams C (2020) Impact of the digital divide in the age of COVID-19. J Am Med Inform Assoc 27(7):1147–1148. https://doi.org/10.1093/jamia/ocaa078CrossRef

RELX (2021) RELX emerging tech executive report 2021: accelerated adoption of AI brings new obstacles to light for US business leaders. Retrieved December 7, 2021. https://stories.relx.com/relx-emerging-tech-2021/index.html

Ritzhaupt AD, Cheng L, Luo W, Hohlfeld TN (2020) The digital divide in formal educational settings: The past, present, and future relevance. In: Bishop MJ, Boling E, Elen J, Svihla V (eds) Handbook of research in educational communications and technology, 5th edn. Springer, New York, pp 483–504. https://doi.org/10.1007/978-3-030-36119-8_23CrossRef

Sao Pedro MA, Baker RS, Gobert JD, Montalvo O, Nakama A (2013) Leveraging machine-learned detectors of systematic inquiry behavior to estimate and predict transfer of inquiry skill. User Model User-Adap Inter 23(1):1–39. https://doi.org/10.1007/s11257-011-9101-0CrossRef

Schiff D (2021) Out of the laboratory and into the classroom: the future of artificial intelligence in education. AI Soc 36(1):331–348. https://doi.org/10.1007/s00146-020-01033-8CrossRef

Schildkamp K, Lai MK, Earl LM (eds) (2013) Data-based decision making in education: challenges and opportunities. Springer, New York

Schildkamp K, Poortman CL, Ebbeler J, Pieters JM (2019) How school leaders can build effective data teams: five building blocks for a new wave of data-informed decision making. J Educ Change 20(3):283–325. https://doi.org/10.1007/s10833-019-09345-3CrossRef

Shah DS, Schwartz HA, Hovy D (2020) Predictive biases in natural language processing models: a conceptual framework and overview. In: Proceedings of the 58th annual meeting of the association for computational linguistics

Shao E, Guo S, Pardos ZA (2021) Degree planning with PLAN-BERT: multi-semester recommendation using future courses of interest. Proc AAAI Conf Artif Intell 35(17):14920–14929

Shepard LA (2019) Classroom assessment to support teaching and learning. Ann Am Acad Pol Soc Sci 683(1):183–200. https://doi.org/10.1177/0002716219843818CrossRef

Shohamy E (2016) Democratic assessment as an alternative. Lang Test 18(4):373–391. https://doi.org/10.1177/026553220101800404CrossRef

Shute VJ, Ventura M (2013) Stealth assessment: measuring and supporting learning in video games. The MIT Press, CambridgeCrossRef

Shute V, Wang L (2017) Assessing and supporting hard-to-measure constructs in video games. In: Rupp AA, Leighton JP (eds) The handbook of cognition and assessment: frameworks, methodologies, and applications. John Wiley & Sons, New Jersey, pp 535–562

Shute V, Rahimi S, Smith G, Ke F, Almond R, Dai CP, Kuba R, Liu Z, Yang X, Sun C (2020) Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity and learning supports in educational games. J Comput Assist Learn. https://doi.org/10.1111/jcal.12473CrossRef

Soedirgo J, Glas A (2020) Toward active reflexivity: positionality and practice in the production of knowledge. PS Polit Sci Polit 53(3):527–531. https://doi.org/10.1017/s1049096519002233CrossRef

Stadler M, Fischer F, Greiff S (2019) Taking a closer look: an exploratory analysis of successful and unsuccessful strategy use in complex problems. Front Psychol. https://doi.org/10.3389/fpsyg.2019.00777CrossRef

UNESCO (2019) Artificial intelligence in education. Retrieved July 26, 2021. https://en.unesco.org/artificial-intelligence/education

United Nations. (2015). Transforming our world: the 2030 agenda for sustainable development. https://sdgs.un.org/sites/default/files/publications/21252030%20Agenda%20for%20Sustainable%20Development%20web.pdf

Valadez JR, Duran R (2007) Redefining the digital divide: beyond access to computers and the internet. High School J 90(3):31–44CrossRef

van Rijn PW, Ali US (2017) A comparison of item response models for accuracy and speed of item responses with applications to adaptive testing. Br J Math Stat Psychol 70(2):317–345. https://doi.org/10.1111/bmsp.12101CrossRef

Wang Y, Kai S, Baker RS (2020) Early detection of wheel-spinning in ASSISTments. In: Bittencourt II, Cukurova M, Muldner K, Luckin R, Millán E (eds) Artificial intelligence in education. AIED 2020. Lecture notes in computer science, vol 12163. Springer International Publishing, New York, pp 574–585

Yapo A, Weiss J (2018) Ethical implications of bias in machine learning. In: Hawaii international conference on system sciences 2018, Waikoloa Village, HI

Završnik A (2019) Algorithmic justice: Algorithms and big data in criminal justice settings. Eur J Criminol. https://doi.org/10.1177/1477370819876762

Zwitter A (2014) Big data ethics. Big Data Soc 1(2):1–6. https://doi.org/10.1177/2053951714559253CrossRef

Title: The cyclical ethical effects of using artificial intelligence in education
Authors: Edward Dieterle
Chris Dede
Michael Walker
Publication date: 27-09-2022
Publisher: Springer London
Published in: AI & SOCIETY / Issue 2/2024
Print ISSN: 0951-5666
Electronic ISSN: 1435-5655
DOI: https://doi.org/10.1007/s00146-022-01497-w

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 2/2024

Anthropomorphization and beyond: conceptualizing humanwashing of AI-enabled machines

Expert responsibility in AI development

What do we really know about the drivers of undeclared work? An evaluation of the current state of affairs using machine learning

Rules for privately owned robots in public spaces

Is LaMDA sentient?

Gender preferences for robots and gender equality orientation in communication situations

Premium Partner