Skip to main content

2025 | OriginalPaper | Buchkapitel

AI-Based Automotive Test Case Generation: An Action Research Study on Integration of Generative AI into Test Automation Frameworks

verfasst von : Albin Karlsson, Erik Lindmaa, Simin Sun, Miroslaw Staron

Erschienen in: Product-Focused Software Process Improvement. Industry-, Workshop-, and Doctoral Symposium Papers

Verlag: Springer Nature Switzerland

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Generative AI is transforming software development, particularly in unit and regression testing. However, it’s rarely used in Hardware-in-the-Loop (HIL) testing due to hardware-specific environments. This paper examines integrating GitHub Copilot into automotive test automation frameworks, focusing on Volvo’s Test Automation Framework (TAF). It explores how Copilot can automate test case generation and compares AI-generated test cases with manually written ones in terms of reliability and robustness. Using an iterative action research methodology, the study evaluates the functional suitability of AI-generated test cases and the challenges of integration. Results show that in the first iteration, 23% of AI-generated test cases passed in Jenkins and received high functionality scores. In the second iteration, this increased to 36%. These findings highlight the potential of Generative AI to enhance HIL testing.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Dakhel, A.M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M.C., Jiang, Z.M.J.: GitHub copilot AI pair programmer: asset or liability? J. Syst. Softw. 203, 111734 (2023)CrossRef Dakhel, A.M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M.C., Jiang, Z.M.J.: GitHub copilot AI pair programmer: asset or liability? J. Syst. Softw. 203, 111734 (2023)CrossRef
2.
Zurück zum Zitat Ebert, C., Louridas, P.: Generative AI for software practitioners. IEEE Softw. 40(4), 30–38 (2023)CrossRef Ebert, C., Louridas, P.: Generative AI for software practitioners. IEEE Softw. 40(4), 30–38 (2023)CrossRef
3.
Zurück zum Zitat Garousi, V., Felderer, M., Kuhrmann, M., Herkiloğlu, K., Eldh, S.: Exploring the industry’s challenges in software testing: an empirical study. J. Softw. Evol. Process 32(8), e2251 (2020)CrossRef Garousi, V., Felderer, M., Kuhrmann, M., Herkiloğlu, K., Eldh, S.: Exploring the industry’s challenges in software testing: an empirical study. J. Softw. Evol. Process 32(8), e2251 (2020)CrossRef
5.
Zurück zum Zitat Haghighatkhah, A., Banijamali, A., Pakanen, O.P., Oivo, M., Kuvaja, P.: Automotive software engineering: a systematic mapping study. J. Syst. Softw. 128, 25–55 (2017)CrossRef Haghighatkhah, A., Banijamali, A., Pakanen, O.P., Oivo, M., Kuvaja, P.: Automotive software engineering: a systematic mapping study. J. Syst. Softw. 128, 25–55 (2017)CrossRef
6.
Zurück zum Zitat Majdinasab, V., Bishop, M.J., Rasheed, S., Moradidakhel, A., Tahir, A., Khomh, F.: Assessing the security of GitHub copilot generated code–a targeted replication study. arXiv preprint arXiv:2311.11177 (2023) Majdinasab, V., Bishop, M.J., Rasheed, S., Moradidakhel, A., Tahir, A., Khomh, F.: Assessing the security of GitHub copilot generated code–a targeted replication study. arXiv preprint arXiv:​2311.​11177 (2023)
7.
Zurück zum Zitat Ochodek, M., Hebig, R., Meding, W., Frost, G., Staron, M.: Chapter 8 recognizing lines of code violating company-specific coding guidelines using machine learning. In: Bosch, J., Carlson, J., Holmstrom Olsson, H., Sandahl, K., Staron, M. (eds.) Accelerating Digital Transformation: 10 Years of Software Center, pp. 211–251. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-10873-0_11 Ochodek, M., Hebig, R., Meding, W., Frost, G., Staron, M.: Chapter 8 recognizing lines of code violating company-specific coding guidelines using machine learning. In: Bosch, J., Carlson, J., Holmstrom Olsson, H., Sandahl, K., Staron, M. (eds.) Accelerating Digital Transformation: 10 Years of Software Center, pp. 211–251. Springer, Cham (2022). https://​doi.​org/​10.​1007/​978-3-031-10873-0_​11
8.
Zurück zum Zitat Panichella, A., Kifetew, F.M., Tonella, P.: Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans. Software Eng. 44(2), 122–158 (2017)CrossRef Panichella, A., Kifetew, F.M., Tonella, P.: Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. IEEE Trans. Software Eng. 44(2), 122–158 (2017)CrossRef
9.
Zurück zum Zitat Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., Karri, R.: Asleep at the keyboard? Assessing the security of GitHub copilot’s code contributions. In: 2022 IEEE Symposium on Security and Privacy (SP), pp. 754–768. IEEE (2022) Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., Karri, R.: Asleep at the keyboard? Assessing the security of GitHub copilot’s code contributions. In: 2022 IEEE Symposium on Security and Privacy (SP), pp. 754–768. IEEE (2022)
10.
Zurück zum Zitat Reynolds, L., McDonell, K.: Prompt programming for large language models: beyond the few-shot paradigm. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–7 (2021) Reynolds, L., McDonell, K.: Prompt programming for large language models: beyond the few-shot paradigm. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–7 (2021)
11.
Zurück zum Zitat Schäfer, M., Nadi, S., Eghbali, A., Tip, F.: An empirical evaluation of using large language models for automated unit test generation. IEEE Trans. Software Eng. 50, 85–105 (2023)CrossRef Schäfer, M., Nadi, S., Eghbali, A., Tip, F.: An empirical evaluation of using large language models for automated unit test generation. IEEE Trans. Software Eng. 50, 85–105 (2023)CrossRef
12.
Zurück zum Zitat Shin, K.W., Kim, S.S., Lim, D.J.: Automatic test-case generation for hardware-in-the-loop testing of automotive body control modules. Technical report, SAE Technical Paper (2013) Shin, K.W., Kim, S.S., Lim, D.J.: Automatic test-case generation for hardware-in-the-loop testing of automotive body control modules. Technical report, SAE Technical Paper (2013)
13.
Zurück zum Zitat Siddiq, M.L., Majumder, S.H., Mim, M.R., Jajodia, S., Santos, J.C.: An empirical study of code smells in transformer-based code generation techniques. In: 2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM), pp. 71–82. IEEE (2022) Siddiq, M.L., Majumder, S.H., Mim, M.R., Jajodia, S., Santos, J.C.: An empirical study of code smells in transformer-based code generation techniques. In: 2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM), pp. 71–82. IEEE (2022)
15.
Zurück zum Zitat Staron, M.: Automotive Software Architectures. Springer, Cham (2021) Staron, M.: Automotive Software Architectures. Springer, Cham (2021)
16.
Zurück zum Zitat Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017) Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
17.
Zurück zum Zitat Wong, D., Kothig, A., Lam, P.: Exploring the verifiability of code generated by GitHub copilot. arXiv preprint arXiv:2209.01766 (2022) Wong, D., Kothig, A., Lam, P.: Exploring the verifiability of code generated by GitHub copilot. arXiv preprint arXiv:​2209.​01766 (2022)
18.
Zurück zum Zitat Zhang, B., Liang, P., Zhou, X., Ahmad, A., Waseem, M.: Practices and challenges of using GitHub copilot: an empirical study. arXiv preprint arXiv:2303.08733 (2023) Zhang, B., Liang, P., Zhou, X., Ahmad, A., Waseem, M.: Practices and challenges of using GitHub copilot: an empirical study. arXiv preprint arXiv:​2303.​08733 (2023)
Metadaten
Titel
AI-Based Automotive Test Case Generation: An Action Research Study on Integration of Generative AI into Test Automation Frameworks
verfasst von
Albin Karlsson
Erik Lindmaa
Simin Sun
Miroslaw Staron
Copyright-Jahr
2025
DOI
https://doi.org/10.1007/978-3-031-78392-0_4