Top

Published in:

2025 | OriginalPaper | Chapter

AI-Based Automotive Test Case Generation: An Action Research Study on Integration of Generative AI into Test Automation Frameworks

Authors : Albin Karlsson, Erik Lindmaa, Simin Sun, Miroslaw Staron

Published in: Product-Focused Software Process Improvement. Industry-, Workshop-, and Doctoral Symposium Papers

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Generative AI is transforming software development, particularly in unit and regression testing. However, it’s rarely used in Hardware-in-the-Loop (HIL) testing due to hardware-specific environments. This paper examines integrating GitHub Copilot into automotive test automation frameworks, focusing on Volvo’s Test Automation Framework (TAF). It explores how Copilot can automate test case generation and compares AI-generated test cases with manually written ones in terms of reliability and robustness. Using an iterative action research methodology, the study evaluates the functional suitability of AI-generated test cases and the challenges of integration. Results show that in the first iteration, 23% of AI-generated test cases passed in Jenkins and received high functionality scores. In the second iteration, this increased to 36%. These findings highlight the potential of Generative AI to enhance HIL testing.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter A Multi-model Approach for Video Data Retrieval in Autonomous Vehicle Development

next chapter AI Act High-Risk Requirements Readiness: Industrial Perspectives and Case Company Insights

Dakhel, A.M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M.C., Jiang, Z.M.J.: GitHub copilot AI pair programmer: asset or liability? J. Syst. Softw. 203, 111734 (2023)CrossRef

Ebert, C., Louridas, P.: Generative AI for software practitioners. IEEE Softw. 40(4), 30–38 (2023)CrossRef

Garousi, V., Felderer, M., Kuhrmann, M., Herkiloğlu, K., Eldh, S.: Exploring the industry’s challenges in software testing: an empirical study. J. Softw. Evol. Process 32(8), e2251 (2020)CrossRef

GitHub: GitHub copilot your AI pair programmer (2024). https://github.com/features/copilot. 10 June 2024

Haghighatkhah, A., Banijamali, A., Pakanen, O.P., Oivo, M., Kuvaja, P.: Automotive software engineering: a systematic mapping study. J. Syst. Softw. 128, 25–55 (2017)CrossRef

Majdinasab, V., Bishop, M.J., Rasheed, S., Moradidakhel, A., Tahir, A., Khomh, F.: Assessing the security of GitHub copilot generated code–a targeted replication study. arXiv preprint arXiv:2311.11177 (2023)

Ochodek, M., Hebig, R., Meding, W., Frost, G., Staron, M.: Chapter 8 recognizing lines of code violating company-specific coding guidelines using machine learning. In: Bosch, J., Carlson, J., Holmstrom Olsson, H., Sandahl, K., Staron, M. (eds.) Accelerating Digital Transformation: 10 Years of Software Center, pp. 211–251. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-10873-0_11

Panichella, A., Kifetew, F.M., Tonella, P.: Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans. Software Eng. 44(2), 122–158 (2017)CrossRef

Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., Karri, R.: Asleep at the keyboard? Assessing the security of GitHub copilot’s code contributions. In: 2022 IEEE Symposium on Security and Privacy (SP), pp. 754–768. IEEE (2022)

10.

Reynolds, L., McDonell, K.: Prompt programming for large language models: beyond the few-shot paradigm. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–7 (2021)

11.

Schäfer, M., Nadi, S., Eghbali, A., Tip, F.: An empirical evaluation of using large language models for automated unit test generation. IEEE Trans. Software Eng. 50, 85–105 (2023)CrossRef

12.

Shin, K.W., Kim, S.S., Lim, D.J.: Automatic test-case generation for hardware-in-the-loop testing of automotive body control modules. Technical report, SAE Technical Paper (2013)

13.

Siddiq, M.L., Majumder, S.H., Mim, M.R., Jajodia, S., Santos, J.C.: An empirical study of code smells in transformer-based code generation techniques. In: 2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM), pp. 71–82. IEEE (2022)

14.

Staron, M.: Action Research in Software Engineering. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-32610-4

15.

Staron, M.: Automotive Software Architectures. Springer, Cham (2021)

16.

Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

17.

Wong, D., Kothig, A., Lam, P.: Exploring the verifiability of code generated by GitHub copilot. arXiv preprint arXiv:2209.01766 (2022)

18.

Zhang, B., Liang, P., Zhou, X., Ahmad, A., Waseem, M.: Practices and challenges of using GitHub copilot: an empirical study. arXiv preprint arXiv:2303.08733 (2023)

Title: AI-Based Automotive Test Case Generation: An Action Research Study on Integration of Generative AI into Test Automation Frameworks
Authors: Albin Karlsson
Erik Lindmaa
Simin Sun
Miroslaw Staron
Publisher: Springer Nature Switzerland
Book: Product-Focused Software Process Improvement. Industry-, Workshop-, and Doctoral Symposium Papers
Print ISBN: 978-3-031-78391-3

Electronic ISBN: 978-3-031-78392-0

Copyright Year: 2025
DOI: https://doi.org/10.1007/978-3-031-78392-0_4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"