Top

Published in:

2024 | OriginalPaper | Chapter

Evaluation of Generative AI-Assisted Software Design and Engineering: A User-Centered Approach

Authors : Mahsa Fischer, Carsten Lanquillon

Published in: Artificial Intelligence in HCI

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This paper evaluates the impact of generative artificial intelligence (AI) on software design and engineering through a user-centered approach. The integration of generative AI tools in software development processes is scrutinized across various phases, from ideation to deployment. By conducting a literature review and a preliminary evaluation with 18 students, this study identifies critical tasks within the software development life cycle where generative AI tools can enhance productivity and creativity. The paper outlines the potential of generative AI to expedite tasks like code completion, prototype design, requirements validation, and documentation, thereby potentially transforming software engineering practices. It emphasizes a user-centered perspective, assessing tools based on criteria such as usability, effectiveness, and integration within existing workflows. Furthermore, the study highlights the importance of human-AI collaboration, suggesting that while generative AI can significantly support software development tasks, human oversight and critical evaluation of AI-generated outputs remain essential. This research contributes to understanding how generative AI tools can be effectively integrated into software development processes, offering insights into the benefits and challenges of these emerging technologies from a user-centric viewpoint.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Examining User Perceptions to Vocal Interaction with AI Bots in Virtual Reality and Mobile Environments: A Focus on Foreign Language Learning and Communication Dynamics

next chapter A Three-Year Analysis of Human Preferences in Delegating Tasks to AI

Bilgram, V., Laarmann, F.: Accelerating innovation with generative AI: AI-augmented digital prototyping and innovation methods. IEEE Eng. Manage. Rev. 51(2), 18–25 (2023). https://doi.org/10.1109/EMR.2023.3272799CrossRef

Buglione, L., Abran, A.: Improving the user story agile technique using the INVEST criteria. In: 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement, pp. 49–53 (2013). https://doi.org/10.1109/IWSM-Mensura.2013.18

Camburn, B.: A systematic method for design prototyping. J. Mech. Des. 137(8), 081102 (2015). https://doi.org/10.1115/1.4030331

Camburn, B.: Design prototyping methods: State of the art in strategies, techniques, and guidelines. Des. Sci. 3, e13 (2017). https://doi.org/10.1017/dsj.2017.10

Carvallo, J.P., Erazo-Garzón, L.: On the use of ChatGPT to support requirements engineering teaching and learning process. In: Berrezueta, S. (ed.) Proceedings of the 18th Latin American Conference on Learning Technologies (LACLO 2023), pp. 328–342. Springer Nature Singapore, Singapore (2023)

Cash, P., Štorga, M.: Multifaceted assessment of ideation: Using networks to link ideation and design activity. J. Eng. Des. 26(10–12), 391–415 (2015). https://doi.org/10.1080/09544828.2015.1070813

Cocci, A.: Quality of information and appropriateness of ChatGPT outputs for urology patients. Prostate Cancer Prostatic Dis. 27(1), 103–108 (2023).https://doi.org/10.1038/s41391-023-00705-y, https://doi.org/10.1038/s41391-023-00705-y

Deniz, B.K., Gnanasambandam, C., Harrysson, M., Hussin, A., Srivastava, S.: Unleashing developer productivity with generative AI. https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/unleashing-developer-productivity-with-generative-ai (2013) Accessed 30 Jan 2024

Doran, G.: There’s a S.M.A.R.T. way to write management’s goals and objectives. Manag. Rev. 70(11), 35–36 (1981)

10.

Dunbar, K.N., Klahr, D.: Scientific thinking and reasoning. In: The Oxford Handbook of Thinking and Reasoning. Oxford University Press (2012).https://doi.org/10.1093/oxfordhb/9780199734689.013.0035

11.

Ebert, C., Louridas, P.: Generative AI for software practitioners. IEEE Softw. 40(4), 30–38 (2023). https://doi.org/10.1109/MS.2023.3265877CrossRef

12.

Epstein, Z., et al.: The investigators of human creativity, art and the science of generative AI. Science 380(6650), 1110–1111 (2023).https://doi.org/10.1126/science.adh4451

13.

Felfernig, A.: AI Techniques for software requirements prioritization. World Sci. Publishing Company, pp. 29–47 (2021).https://doi.org/10.1142/9789811239922_0002

14.

Goodfellow, I., Bengio, Y., Courville, A.: Deep learning. MIT Press (2016). http://www.deeplearningbook.org

15.

Hauser, A.: Unleash your innovation power: Combining design thinking, agile and lean. https://medium.com/@HauserAndreas/unleash-your-innovation-power-combining-design-thinking-agile-and-lean-part-1-cb7a3360c2af (2019) Accessed 30 Jan 2024

16.

Hind, M., et al.: Experiences with improving the transparency of AI models and services (2019)

17.

Hinderberger, F.: Vom Proof-of-Concept zum Full-Scale-Product. Springer Fachmedien Wiesbaden, Wiesbaden, pp. 125–147 (2021)

18.

ISO12207-2008: ISO/IEC/IEEE International Standard - Systems and software engineering – Software life cycle processes. IEEE STD 12207-2008, pp. 1–138 (2008). https://doi.org/10.1109/IEEESTD.2008.4475826

19.

Khan, J., Rehman, I., Khan, Y., Khan, I., Rashid, S.: Comparison of requirement prioritization techniques to find best prioritization technique. Int. J. Mod. Edu. Comput. Sci. 7(11), 53–59 (2015). https://doi.org/10.5815/ijmecs.2015.11.06

20.

Khan, J.Y., Uddin, G.: Automatic code documentation generation using GPT-3. In: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. ASE ’22, Association for Computing Machinery, New York, NY, USA (2023). https://doi.org/10.1145/3551349.3559548

21.

Königstorfer, F., Thalmann, S.: Software documentation is not enough! Requirements for the documentation of AI. Digit. Policy Regul. Gov. 23(5), 475–488 (2021). https://doi.org/10.1108/DPRG-03-2021-0047

22.

Kunicina, N., Zabasta, A., Patlins, A., Bilic, I., Peksa, J.: Prototyping process in education and science. In: 2020 IEEE 61th International Scientific Conference on Power and Electrical Engineering of Riga Technical University (RTUCON), pp. 1–6 (2020). https://doi.org/10.1109/RTUCON51174.2020.9316550

23.

Li, J., Mynatt, E., Mishra, V., Bell, J.: Always Nice and Confident. Developer’s Experiences Engaging Generative AI Chatbots Versus Human-Powered Q &A Platforms, Sometimes Wrong (2023)

24.

Ma, W., et al.: ChatGPT: Understanding code syntax and semantics (2023)

25.

Ness, R.B.: Promoting innovative thinking. Am. J. Pub. Health 105(S1), S114–S118 (2015). https://doi.org/10.2105/AJPH.2014.302365, PMID: 25706005

26.

Noy, S., Zhang, W.: Experimental evidence on the productivity effects of generative artificial intelligence. Science 381(6654), 187–192 (2023). https://doi.org/10.1126/science.adh2586CrossRef

27.

Nuzula, M.I.F., Rochimah, S.: Evaluation of service quality in human resource information systems using the ISO/IEC 25010. In: 2023 International Seminar on Application for Technology of Information and Communication: Smart Technology Based on Industry 4.0: A New Way of Recovery from Global Pandemic and Global Economic Crisis, iSemantic 2023. Institute of Electrical and Electronics Engineers Inc, pp. 215–220 (2023).https://doi.org/10.1109/iSemantic59612.2023.10295365

28.

Ozkaya, I.: Application of large language models to software engineering tasks: Opportunities, risks, and implications. IEEE Softw. 40(3), 4–8 (2023). https://doi.org/10.1109/MS.2023.3248401CrossRef

29.

Peng, S., Kalliamvakou, E., Cihon, P., Demirer, M.: The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (2023)

30.

Pennefather, P.P.: Creative prototyping with generative AI: Augmenting creative workflows with generative AI. Apress (2023). https://doi.org/10.1007/978-1-4842-9579-3CrossRef

31.

Pothukuchi, A.S., Kota, L.V., Mallikarjunaradhya, V.: Impact of generative AI on the software development lifecycle (sdlc). Int. J. Creative Res. Thoughts 11(8) (2023), available at SSRN: https://ssrn.com/abstract=4536700

32.

Ricca, F., Marchetto, A., Stocco, A.: AI-based test automation: A grey literature analysis. In: 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 263–270 (2021). https://doi.org/10.1109/ICSTW52544.2021.00051

33.

Ries, E.: The lean startup: How today’s entrepreneurs use continuous innovation to create radically successful businesses. Crown (2011)

34.

Rodriguez, A.D., Dearstyne, K.R., Cleland-Huang, J.: Prompts matter: Insights and strategies for prompt engineering in automated software traceability (2023)

35.

Romero, J., Medina-Bulo, I., Chicano, F.: Optimising the software development process with artificial intelligence. Springer Nature (2023). https://books.google.de/books?id=KAU50AEACAAJ

36.

Singh, V., Asari, V., Kumar, S., Patel, R.: Computational methods and data engineering: Proceedings of ICMDE 2020, Volume 2. Springer Singapore, Imprint: Springer (2021). https://books.google.de/books?id=Gfd9zwEACAAJ

37.

Sommerville, I.: Engineering software products: An introduction to modern software engineering. Pearson Education, Incorporated (2020). https://books.google.de/books?id=yFAkzgEACAAJ

38.

Tholander, J., Jonsson, M.: Design ideation with AI - sketching, thinking and talking with generative machine learning models. In: Proceedings of the 2023 ACM Designing Interactive Systems Conference. DIS’23, Association for Computing Machinery, New York, NY, USA, pp. 1930–1940 (2023).https://doi.org/10.1145/3563657.3596014

39.

Tian, H., et al.: Is ChatGPT the ultimate programming assistant – how far is it? (2023)

40.

Toubia, O.: Idea generation, creativity, and incentives. Mark. Sci. 25(5), 411–425 (2006). https://doi.org/10.1287/mksc.1050.0166CrossRef

41.

Waseem, M., Das, T., Ahmad, A., Fehmideh, M., Liang, P., Mikkonen, T.: Using ChatGPT throughout the software development life cycle by novice developers (2023)

42.

White, J., et al.: A prompt pattern catalog to enhance prompt engineering with ChatGPT (2023)

43.

Winston, P.H.: Artificial Intelligence. Addison-Wesley, 3 edn (1992)

44.

Wohlin, C., Aurum, A.: Criteria for selecting software requirements to create product value: An industrial empirical study. In: Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., Grünbacher, P. (eds.) Value-Based Software Engineering, pp. 179–200. Springer, Heidelberg (2006). https://doi.org/10.1007/3-540-29263-2_9CrossRef

45.

Wu, L.: Agile design and AI integration: revolutionizing MVP development for superior product design. Int. J. Edu. Humanit. 9(1), 226–230 (2023)https://doi.org/10.54097/ijeh.v9i1.9417

46.

York, E.: Evaluating ChatGPT: Generative AI in UX design and web development pedagogy. In: Proceedings of the 41st ACM International Conference on Design of Communication. SIGDOC’23, Association for Computing Machinery, New York, NY, USA, pp. 197–201 (2023). https://doi.org/10.1145/3615335.3623035

Title: Evaluation of Generative AI-Assisted Software Design and Engineering: A User-Centered Approach
Authors: Mahsa Fischer
Carsten Lanquillon
Publisher: Springer Nature Switzerland
Book: Artificial Intelligence in HCI
Print ISBN: 978-3-031-60605-2

Electronic ISBN: 978-3-031-60606-9

Copyright Year: 2024
DOI: https://doi.org/10.1007/978-3-031-60606-9_3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"