Top

Published in:

2023 | OriginalPaper | Chapter

85. Artificial Intelligence for Competency Assessment in Design Education: A Review of Literature

Authors : Sumbul Khan, Lucienne Blessing, Yakhoub Ndiaye

Published in: Design in the Era of Industry 4.0, Volume 3

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

One of the biggest challenges for instructors involved in design education (DE) is fair and thorough assessment of student design competencies. Over the past few years, new technological innovations have emerged to support teaching and learning in DE, especially the complex process of assessment. One of these technological innovations is artificial intelligence (AI). AI is a promising research topic at the edge of technology, innovation, and education that is gaining great attention in contemporary design society. It is often defined as the effort to automate intellectual tasks normally performed by humans. AI-based assessment of competencies in DE has the potential to make the assessment more efficient and effective. Although AI itself is not recent, it is a relatively new and innovative way to assess students’ design competencies. We conducted a review of the literature to identify and analyze research in the assessment of competencies in DE. In this paper, we (1) review the issues faced by students and instructors in the assessment of DE, (2) review AI-based assessment of competencies in engineering design, and (3) propose recommendations for AI-based assessment of competencies for design education.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Micro-credentials-oriented Education—Current State of the New Norm in India

next chapter Urban Design: Yesterday, Today and Tomorrow—A Brief Chronological Review

Wagenaar, R.: Competences and learning outcomes: a panacea for understanding the (new) role of higher education? Tuning J. High. Educ. 1, 279 (2014). https://doi.org/10.18543/tjhe-1(2)-2014pp279-302CrossRef

Oda, S., Inoue, M., Yamazaki, A.K.: Assessment of global competency for engineering students in a multicultural and multidisciplinary project based learning course. In: 7th World Engineering Education Forum, pp. 439–443 (2017). https://doi.org/10.1109/WEEF.2017.8467071

Rahman, M., Ismail, H.N., Daud, K.A.M., Jaafar, M.F.: Competency based assessment (CBA) of engineering students’ product development via project-based learning (PjBL) process. Int. J. Bus. Soc. Sci. 2, 221–229 (2011)

Shah, J.J.: Identification, measurement & development of design skills in engineering education. Int. Conf. Eng. Des. 49, 1–15 (2005). https://doi.org/10.1016/j.oraloncology.2013.05.007CrossRef

Lawson, B., Dorst, K.: Acquiring design expertise. Design Expertise, pp. 2–19 (2015)

Diefes-Dux, H.A., Zawojewski, J.S., Hjalmarson, M.A.: Using educational research in the design of evaluation tools for open-ended problems. Int. J. Eng. Educ. 26, 807–819 (2010)

Davis, D.C., Gentili, K.L., Trevisan, M.S., Calkins, D.E.: Engineering design assessment processes and scoring scales for program improvement and accountability. J. Eng. Educ. 91, 211–221 (2002). https://doi.org/10.1002/j.2168-9830.2002.tb00694.xCrossRef

Beyerlein, S., Davis, D., Trevisan, M., Thompson, L., Harrison, K.: Assessment framework for capstone design courses. In: 2006 ACE, Chicago, Illinois, (2006). https://doi.org/10.18260/1-2--842

Demirkan, H., Afacan, Y.: Assessing creativity in design education: analysis of creativity factors in the first-year design studio. Des. Stud. 33, 262–278 (2012). https://doi.org/10.1016/j.destud.2011.11.005CrossRef

10.

Aflatoony, L., Wakkary, R., Neustaedter, C.: Becoming a design thinker: assessing the learning process of students in a secondary level design thinking course. Int. J. Art Des. Educ. 37, 438–453 (2018). https://doi.org/10.1111/jade.12139 CrossRef

11.

Blizzard, J., Klotz, L., Potvin, G., Hazari, Z., Cribbs, J., Godwin, A.: Using survey questions to identify and learn more about those who exhibit design thinking traits. Des. Stud. 38, 92–110 (2015). https://doi.org/10.1016/j.destud.2015.02.002CrossRef

12.

Christensen, K.S., Hjorth, M., Iversen, O.S., Blikstein, P.: Towards a formal assessment of design literacy: analyzing K-12 students’ stance towards inquiry. Des. Stud. 46, 125–151 (2016). https://doi.org/10.1016/j.destud.2016.05.002CrossRef

13.

Kimbell, R.: Evolving project e-scape for national assessment. Int. J. Technol. Des. Educ. 22, 135–155 (2012). https://doi.org/10.1007/s10798-011-9190-4CrossRef

14.

Bartholomew, S.R.: Assessing open-ended design problems. Technol. Eng. Teacher 76(6), 13–17 (2017)

15.

Doppelt, Y.: Assessing creative thinking in design-based learning. Int. J. Technol. Des. Educ. 19, 55–65 (2009). https://doi.org/10.1007/s10798-006-9008-yCrossRef

16.

Baker, S.K., Simmons, D.C., Kameenui, E.J.: Vocabulary acquisition: research bases. In: What Reading Research Tells Us About Children with Diverse Learning Needs: Bases and Basics (ed), p. 183, Routledge, New York (1998)

17.

Avery, J.: Leveraging crowdsourced peer-to-peer assessments to enhance the case method of learning. J. Adv. Mark. Educ. 22, 1–15 (2014)

18.

Atman, C., Kilgore, D., Mckenna, M.: Characterizing design learning: a mixed-methods study of engineering designers’ use of language. J. Eng. Educ. 97(3), 309–326 (2008). https://doi.org/10.1002/j.2168-9830.2008.tb00981.xCrossRef

19.

Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., et al.: The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, 71 (2021). https://doi.org/10.1136/bmj.n71CrossRef

20.

Pellegrino, J.W., Chudowsky, N., Glaser, O.: Knowing what students know: the science and design of educational assessment. National Academy Press (2001). https://doi.org/10.17226/10019

21.

Flores, M.A., Veiga Simão, A.M., Barros, A., Pereira, D.: Perceptions of effectiveness, fairness and feedback of assessment methods: a study in higher education. Stud. High. Educ. 49(9), (2015). https://doi.org/10.1080/03075079.2014.881348

22.

Custer, R.L., Valesey, B.G., Burke, B.N.: An assessment model for a design approach to technological problem solving. J. T. Educ. 12, 5–20 (2016). https://doi.org/10.21061/jte.v12i2.a.1CrossRef

23.

Davis, D.C., Gentili, K.L., Trevisan, M.S., Christianson, R.K., McCauley, J.F.: Measuring learning outcomes for engineering design education. In: ASEE Annual Conference Proceedings, pp. 4193–4199 (2000)

24.

Falchikovab, N.: Improving assessment through student involvement: practical solutions for aiding learning in higher and further education. Routledge (2013). https://doi.org/10.4324/9780203220993

25.

Patel, A., Elena, M.-V., Summers, J.: A systematic approach to evaluating design prompts in supporting experimental design research. In: Proceedings of the Design Society: International Conference on Engineering Design, vol. 1, pp. 2755–2764 (2019). https://doi.org/10.1017/dsi.2019.282

26.

Jonassen, D., Strobel, J., Lee, C.B.: Everyday problem solving in engineering: lessons for engineering educators. J. Eng. Educ. 95, 139–151 (2006). https://doi.org/10.1002/j.2168-9830.2006.tb00885.xCrossRef

27.

Eshun, E.F., Osei-Poku, P.: Design students perspectives on assessment rubric in studio-based learning design students perspectives on assessment rubric in studio-based. J. Univ. Teach. Learn. Pract. 10, 15 (2013). https://doi.org/10.53761/1.10.1.8CrossRef

28.

Balfour, S.P.: Assessing writing in MOOCs: automated essay scoring and calibrated peer review^TM. Res. Pract. Assess. 8, 40–48 (2013)

29.

Francis Williams, R., Dreher, H.: Automatically grading essays with Markit©. Issues Informing Sci. Inf. Technol. 1, 693–700 (2004). https://doi.org/10.28945/769CrossRef

30.

Weld, D.S. et al.: Personalized online education: a crowdsourcing challenge. In: Proceedings of the 4th Human Computation Workshop (HCOMP-12), pp. 159–163 (2012)

31.

Siddiqi, R., Harrison, C.J., Siddiqi, R.: Improving teaching and learning through automated short-answer marking. IEEE Trans. Learn. Technol. 3, 237–249 (2010)CrossRef

32.

Blumenstein, M., Green, S., Fogelman, S., Nguyen, A., Muthukkumarasamy, V.: Performance analysis of GAME: a generic automated marking environment. Comp. Educ. 50, 1203–1216 (2008)CrossRef

33.

Naudé, K.A., Greyling, J.H., Vogts, D.: Marking student programs using graph similarity. Comp. Educ. 54, 545–561 (2010)CrossRef

34.

Insa, D., Silva, J.: Automatic assessment of Java code. Comp. L. S. Struct. 53, 59–72 (2018)

35.

Tekin, C., Braun, J., van der Schaar, M.: etutor: online learning for personalized education. In: IEEE ICASSP, pp. 5545–5549. IEEE, Australia (2015)

36.

Tejeda-Lorente, Á., Bernabé-Moreno, J., Porcel, C., Galindo-Moreno, P., Herrera-Viedma, E.: A dynamic recommender system as reinforcement for personalized education by a fuzzly linguistic web system. Procedia Comp. Sci. 55, 1143–1150 (2015)CrossRef

37.

Hemachandran, K., Verma, P., Pareek, P., Arora, N., Rajesh Kumar, K.V., Ahanger, T.A., Pise, A.A., Ratna, R.: Artificial intelligence: a universal virtual tool to augment tutoring in higher education. Comp. Intel. Neurosci. (2022). https://doi.org/10.1155/2022/1410448

38.

Zhu, X., Tang, S.: Design of an artificial intelligence algorithm teaching system for universities based on probabilistic neuronal network model. Sci. Prog. (2022). https://doi.org/10.1155/2022/4131058CrossRef

39.

Hooda, M., Rana, C., Dahiya, O., Rizwan, A. and Hossain, M.S.: Artificial intelligence for assessment and feedback to enhance student success in higher education. Math. Prob. Eng. (2022). https://doi.org/10.1155/2022/5215722

40.

Chaudhuri, N.B., Dhar, D., Yammiyavar, P.G.: A computational model for subjective evaluation of novelty in descriptive aptitude. Int. J. Technol. Des. Educ. 32, 1121–1158 (2022). https://doi.org/10.1007/s10798-020-09638-2CrossRef

41.

Dworschak, F., Dietze, S., Wittmann, M., Schleich, B., Wartzack, S.: Reinforcement learning for engineering design automation. Adv. Eng. Info. 52, 101612 (2022). https://doi.org/10.1016/j.aei.2022.101612CrossRef

42.

Fan, X., Zhong, X.: Artificial intelligence-based creative thinking skill analysis model using human–computer interaction in art design teaching. Comp. Elect. Eng. 100, 107957 (2022). https://doi.org/10.1016/j.compeleceng.2022.107957CrossRef

43.

Xinhan, N.: Intelligent analysis of classroom student state based on neural network algorithm and emotional feature recognition. J. Intel. Fuzzy Syst. 40(4), 7171–7182 (2021). https://doi.org/10.3233/JIFS-189545

44.

Liu, C., Feng, Y., & Wang, Y.: An innovative evaluation method for undergraduate education: an approach based on BP neural network and stress testing. Stud. High. Educ., 47(1), 212–228 (2022). https://doi.org/10.1080/03075079.2020.1739013

45.

Dehbozorgi, N., Norkham, A.: An architecture model of recommender system for pedagogical design patterns. In: 2021 IEEE Frontiers in Education Conference, pp. 1–4. IEEE, Lincoln, USA (2021). https://doi.org/10.1109/FIE49875.2021.9637342

46.

Lu, O.H., Huang, A.Y., Tsai, D.C., Yang, S.J.: Expert-authored and machine-generated short-answer questions for assessing students learning performance. Educ. Technol. Soc. 24(3), 159–173 (2021)

47.

Huang, H.L., Hwang, G.J., Chen, P.Y.: An integrated concept mapping and image recognition approach to improving students’ scientific inquiry course performance. Brit. J. Educ. Technol. 53(3), 706–727 (2022). https://doi.org/10.1111/bjet.13177CrossRef

48.

Yang, J., Chen, Z., Qiu, G., Li, X., Li, C., Yang, K., Chen, Z., Gao, L., Lu, S.: Exploring the relationship between children’s facial emotion processing characteristics and speech communication ability using deep learning on eye tracking and speech performance measures. Comp. Speech L. 76, 101389 (2022). https://doi.org/10.1016/j.csl.2022.101389CrossRef

49.

Cajander, Å., Daniels, M., Von Konsky, B.R.: Development of professional competencies in engineering education. IEEE, RD, USA (2011). https://doi.org/10.1109/FIE.2011.6143003

50.

Cajander, Å., Daniels, M., McDermott, R., Von Konsky, B.R.. Assessing professional skills in engineering education In: Hamer, J., de Raadt, M. (ed.), 13th ACE2011, Australian Computer Society, Perth, Western Australia (2011)

51.

Anderson, M.: Crowdsourcing higher education: a design proposal for distributed learning. J. Online Learn. Teach. 7(4), 1–8 (2011)

52.

De Alfaro, L., Shavlovsky, M.: CrowdGrader: a tool for crowdsourcing the evaluation of homework assignments (2014). https://doi.org/10.1145/2538862.2538900

53.

Heimerl, K., Gawalt, B., Chen, K., Parikh, T., Hartmann, B.: Community sourcing: engaging local crowds to perform expert work via physical kiosks. In: Proceedings of the 2012 ACM Conference on Human Factors in Computing Systems, p. 1539 (2012). https://doi.org/10.1145/2207676.2208619.

54.

Czaplewski, A.J.: Computer-assisted grading rubrics: automating the process of providing comments and student feedback. Mark. Educ. Rev. 19(1), (2015). https://doi.org/10.1080/10528008.2009.11489057

55.

Klein, S., Benjamin, R., Shavelson, R., Bolus, R.: The collegiate learning assessment: facts and fantasies. Eval. Rev. 31, 415–439 (2007). https://doi.org/10.1177/0193841X07303318CrossRef

56.

Bennani, S., Idrissi, M.K., Fadouli, N., Benjelloun Touimi, Y., Ouguengay, Y.A.: Online project based learning driven by competencies: a systematic strategy proposal for assessment. In: International Conference on Interactive Mobile and Computer Aided Learning, IMCL 2012, pp. 92–99 (2012). https://doi.org/10.1109/IMCL.2012.6396457

57.

Idrissi, M.K., Hnida, M., Bennani, S.: Competency-based assessment: from conceptual model to operational tool. In: Innovative Practices for Higher Education Assessment and Measurement, pp. 57–78.IGI Global (2016). https://doi.org/10.4018/978-1-5225-0531-0.ch004

58.

Villamañe, M., Álvarez, A., Larrañaga, M., Caballero, J., Hernández-Rivas, O.: Using visual learning analytics to support competence-based learning. In: ACM International Conference Proceeding Series, pp. 333–338. ACM (2018). https://doi.org/10.1145/3284179.3284233

59.

Yago, H., Clemente, J., Rodriguez, D.: Competence-based recommender systems: a systematic literature review. Behav. Info. Technol. 37, 958–977 (2018). https://doi.org/10.1080/0144929X.2018.1496276CrossRef

60.

Khan, S., Maheshwary, K., Arlitt, R., Blessing, L.: Assessment of conceptual design problems comprising design rationale and sketches. In: Proceedings of the ASME 2020 IDETC/CIE2020. ASME, St. Louis, USA (2020). https://doi.org/10.1115/DETC2020-22376

61.

McGown, A., Green, G., Rodgers, P.A.: Visible ideas: information patterns of conceptual sketch activity. Des. Stud. 19, 431–453 (1998). https://doi.org/10.1016/S0142-694X(98)00013-1CrossRef

62.

Funke, I., Mees, S.T., Weitz, J., Speidel, S.: Video-based surgical skill assessment using 3D convolutional neural networks. Int. J. Comp. Ass. Radio Surg. 14, 1217–1225 (2019)CrossRef

63.

Brumm, T.J., Hanneman, L.F., Mickelson, S.K.: Assessing and developing program outcomes through workplace competencies. Int. J. Eng. Educ. 22(1), 123–129 (2006)

64.

Vargas, H., Heradio, R., Chacon, J., De La Torre, L., Farias, G., Galan, D., Dormido, S.: Automated assessment and monitoring support for competency-based courses. IEEE, pp. 41043–41051 (2019). https://doi.org/10.1109/ACCESS.2019.2908160

Title: Artificial Intelligence for Competency Assessment in Design Education: A Review of Literature
Authors: Sumbul Khan
Lucienne Blessing
Yakhoub Ndiaye
Publisher: Springer Nature Singapore
Book: Design in the Era of Industry 4.0, Volume 3
Print ISBN: 978-981-9904-27-3

Electronic ISBN: 978-981-9904-28-0

Copyright Year: 2023
DOI: https://doi.org/10.1007/978-981-99-0428-0_85