Top

TechTrends

Published in:

03-11-2023 | Original Paper

Can ChatGPT Pass Graduate-Level Instructional Design assignments? Potential Implications of Artificial Intelligence in Education and a Call to Action

Authors: Bruce Parsons, John H. Curry

Published in: TechTrends | Issue 1/2024

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This article investigates an artificial intelligence language model, ChatGPT, and its ability to complete graduate-level instructional design assignments. The approach subjected ChatGPT to a needs, task, and learner analysis for a 12th-grade media literacy module and benchmarked its performance by expert evaluation and measurements via grading rubrics. Initial results show ChatGPT’s ability to write graduate-level instructional design assignments but with some challenges and frustration in the bot's ability to customize for location and school-specifics. Key benefits of the findings include new ways to challenge instructional design students, faculty customization of future assignments, and the potential of AI as a collaboration tool in the instructional design and educational process.

previous article Generative Artificial Intelligence in Education and Its Implications for Assessment

next article Design and Development of an Advising Chatbot as a Student Support Intervention in a University System

Albirini, A. (2007). The crisis of educational technology, and the prospect of reinventing education. Educational Technology & Society, 10(1), 227–236.

Banathy, B. H. (1992). A systems view of education: Concepts and principles for effective practice. Educational Technology.

Bienkowski, M., Feng, M., & Means, B. (2014). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief (pp. 1–60).

Ceres, P. (n.d.). ChatGPT Is Coming for Classrooms. Don’t Panic. Wired. Retrieved October 2, 2023, from https://www.wired.com/story/chatgpt-is-coming-for-classrooms-dont-panic/

ChatGPT Passed an MBA Exam. What’s Next? (n.d.). Knowledge at Wharton. Retrieved February 23, 2023, from https://knowledge.wharton.upenn.edu/podcast/wharton-business-daily-podcast/chatgpt-passed-an-mba-exam-whats-next/. Accessed 10/2/23

Chiu, T. K. F. (2021). A holistic approach to the design of artificial intelligence (AI) Education for K-12 Schools. TechTrends, 65(5), 796–807. https://doi.org/10.1007/s11528-021-00637-1CrossRef

Chocarro, R., Cortinas, Monica, & Marcos-Matas, G. (2021). Teachers’ attitudes towards chatbots in education: A technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. Educational Studies, 1–19. https://doi.org/10.1080/03055698.2020.1850426

Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of denerative artificial intelligence. Journal of Science Education and Technology, 32(3), 444–452.CrossRef

Design Thinking. (n.d.). Retrieved October 2, 2023, from https://hbr.org/2008/06/design-thinking

Dimitriadis, G. (2020). Evolution in education: Chatbots. Homo Virtualis, 3(1), 47. https://doi.org/10.12681/homvir.23456CrossRef

Eisenstein, E. L. (1980). The printing press as an agent of change. Cambridge University Press.CrossRef

Elias, T. (2011). Learning analytics: Definitions, processes and potential.

Faye, C. J. F., Cathy. (2018). A history of panic over entertainment technology. Behavioral Scientist. https://behavioralscientist.org/history-panic-entertainment-technology/. Accessed 10/2/23

Grafton, A. (1999). The footnote: A curious history. Harvard University Press.

Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S., Santos, O., Rodrigo, M., Cukurova, M., Bittencourt, I., & Koedinger, K. (2021). Ethics of AI in education: Towards a community-wide framework. International Journal of Artificial Intelligence in Education, 32. https://doi.org/10.1007/s40593-021-00239-1

Huang, W. H.-Y., & Soman, D. (2013). A practitioner’s guide to gamification of education. https://inside.rotman.utoronto.ca/behaviouraleconomicsinaction/files/2013/09/GuideGamificationEducationDec2013.pdf

Huang, K. (2023). Alarmed by A.I. chatbots, universities start revamping how they teach. The New York Times. https://www.nytimes.com/2023/01/16/technology/chatgpt-artificial-intelligence-universities.html. Accessed 10/2/23

Kerly, A., Hall, P., & Bull, S. (2007). Bringing Chatbots into education: Towards natural language negotiation of open learner models. Knowledge-Based Systems, 20, 177–185. https://doi.org/10.1016/j.knosys.2006.11.014CrossRef

Luckin, R. (2017). Towards artificial intelligence-based assessment systems. Nature Human Behaviour, 1(3), 0028.CrossRef

Markauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Buckingham Shum, S., Gašević, D., & Siemens, G. (2022). Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? Computers and Education: Artificial Intelligence, 3, 100056. https://doi.org/10.1016/j.caeai.2022.100056CrossRef

Marsicano, C. [@ChrisMarsicano]. (2023, January 16). I expect I’m going to get blasted for this by some, but here is my new ChatGPT policy for my [Tweet]. Twitter. https://twitter.com/ChrisMarsicano/status/1615015435069448192. Accessed 10/2/23

Mckie, I. A. S., & Narayan, B. (2019). Enhancing the academic library experience with chatbots: An exploration of research and implications for practice. Journal of the Australian Library and Information Association, 68(3), 268–277. https://doi.org/10.1080/24750158.2019.1611694CrossRef

Meadows, D. H. (2008). Thinking in systems: A primer. Chelsea Green Publishing.

Moraes Neto, A. J., & Fernandes, M. A. (2019). Chatbot and conversational analysis to promote collaborative learning in distance education. 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), 2161–377X, 324–326. https://doi.org/10.1109/ICALT.2019.00102

Norman, D. A., & Stappers, P. J. (2016). DesignX: Complex sociotechnical systems. She Ji: The Journal of Design, Economics, and Innovation, 1(2), 83–106.

Norris, P. (2001). Digital divide: Civic engagement, information poverty and the internet World-Wide. In Info-The journal of policy, regulation and strategy for telecommunications, 5(1), 77–78. https://doi.org/10.1108/146366903322008287CrossRef

Reigeluth, C. M., & Carr-Chellman, A. A. (2009). Instructional-Design theories and models, Volume III: Building a common knowledge base. Routledge. https://www.routledge.com/Instructional-Design-Theories-and-Models-Volume-III-Building-a-Common/Reigeluth-Carr-Chellman/p/book/9780805864564. Accessed 10/2/23

Reiser, R. A. (2001). A history of instructional design and technology: Part II: A history of instructional design. Educational Technology Research and Development, 49(2), 57–6. https://doi.org/10.1007/BF02504928CrossRef

Roose, K. (2023). Don’t Ban ChatGPT in Schools. Teach With It. The New York Times. https://www.nytimes.com/2023/01/12/technology/chatgpt-schools-teachers.html. Accessed 10/2/23

Russell, S., & Norvig, P. (2003). Artificial intelligence: A modern approach. In Artificial Intelligence—AI (2nd ed., Vol. 82), Prentice Hall.

Saettler, L. P. (2004). The evolution of American educational technology. Information Age Publishing.

Selwyn, N. (2019). Should robots replace teachers? (1st ed.). Polity Press. Retrieved from https://www.perlego.com/book/1536481/should-robots-replace-teachers-ai-and-the-future-of-education-pdf. Accessed 10/2/23

Senge, P. M. (1990). The fifth discipline: The art and practice of the learning organization (1st ed). Doubleday/Currency. http://catdir.loc.gov/catdir/enhancements/fy0703/90002991-s.html. Accessed 10/2/23

Shum, S. B., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-Centred learning analytics. Journal of Learning Analytics, 6(2), 2. https://doi.org/10.18608/jla.2019.62.1CrossRef

Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862CrossRef

Srinivasan, V. (2022). AI & learning: A preferred future | Elsevier Enhanced Reader. Computers and Education: Artificial Intelligence, 3. https://doi.org/10.1016/j.caeai.2022.100062

Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning and Teaching, 6(1), 1. https://doi.org/10.37074/jalt.2023.6.1.17CrossRef

Tomlinson, C. A. (2001). How to Differentiate Instruction in Mixed-Ability Classrooms. Upper Saddle River, NJ: Pearson Education.

Turkle, S. (1995). Life on the screen: Identity in the age of the internet. Simon & Schuster.

Whitford, E. (2023). ChatGPT And AI Will Fuel New EdTech Boom. Forbes. Retrieved February 23, 2023, from https://www.forbes.com/sites/emmawhitford/2023/01/18/chatgpt-and-ai-will-fuel-new-edtech-boom/. Accessed 10/2/23

Winkler, R., & Soellner, M. (2018). Unleashing the potential of chatbots in education: A state-of-the-art analysis. Academy of Management Proceedings, 2018(1), 15903. https://doi.org/10.5465/ambpp.2018.15903abstractCrossRef

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are we there yet? - A systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4. https://doi.org/10.3389/frai.2021.654924

Yang, S., & Evans, C. (2019, November 5). Opportunities and challenges in using AI chatbots in higher education. Proceedings of the 2019 3rd International Conference on Education and E-Learning. https://doi.org/10.1145/3371647.3371659

Zahour, O. (2020). Towards a chatbot for educational and vocational guidance in Morocco: Chatbot E-Orientation. International Journal of Advanced Trends in Computer Science and Engineering, 9(2), 2479–2487. https://doi.org/10.30534/ijatcse/2020/237922020CrossRef

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0CrossRef

Title: Can ChatGPT Pass Graduate-Level Instructional Design assignments? Potential Implications of Artificial Intelligence in Education and a Call to Action
Authors: Bruce Parsons
John H. Curry
Publication date: 03-11-2023
Publisher: Springer US
Published in: TechTrends / Issue 1/2024
Print ISSN: 8756-3894
Electronic ISSN: 1559-7075
DOI: https://doi.org/10.1007/s11528-023-00912-3

Springer Professional

Can ChatGPT Pass Graduate-Level Instructional Design assignments? Potential Implications of Artificial Intelligence in Education and a Call to Action

Abstract

Premium Partner

Springer Professional

Abstract

Please log in to get access to your license.

Other articles of this Issue 1/2024

Game-Based Learning and Systems Thinking: an Innovative Instructional Approach for the 21st Century

The Influence of Systems Thinking on Immersive Learning Design Decisions

The Change We Work: Professional Agency and Ethics for Emerging AI Technologies

International Students' Unique Challenges and Reflections

Innovation of Instructional Design and Assessment in the Age of Generative Artificial Intelligence

In Consideration of Context: The Potential of Geographic Information Systems (GIS)