nach oben

Automated Software Engineering

Erschienen in:

01.05.2024

Large language models for qualitative research in software engineering: exploring opportunities and challenges

verfasst von: Muneera Bano, Rashina Hoda, Didar Zowghi, Christoph Treude

Erschienen in: Automated Software Engineering | Ausgabe 1/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The recent surge in the integration of Large Language Models (LLMs) like ChatGPT into qualitative research in software engineering, much like in other professional domains, demands a closer inspection. This vision paper seeks to explore the opportunities of using LLMs in qualitative research to address many of its legacy challenges as well as potential new concerns and pitfalls arising from the use of LLMs. We share our vision for the evolving role of the qualitative researcher in the age of LLMs and contemplate how they may utilize LLMs at various stages of their research experience.

Vorheriger Artikel A class integration test order generation approach based on Sarsa algorithm

Nächster Artikel Regression test selection in test-driven development

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

https://www.csiro.au/en/news/all/articles/2023/june/humans-and-ai-hallucinate.

https://www.cyberdaily.au/digital-transformation/9779-researchers-apologies-to-big-4-consultancy-firms-for-false-ai-based-accusations.

https://c3.ai/glossary/data-science/model-drift/.

https://www.sciencedirect.com/journal/information-and-software-technology/publish/guide-for-authors.

Alkaissi, H., McFarlane, S.I.: Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus 15, 192 (2023)

Arora, C., John, G., Mohamed, A.: Advancing requirements engineering through generative AI: assessing the role of LLMs. (2023) arXiv preprint arXiv:2310.13976.

Balel, Y.: The role of artificial intelligence in academic paper writing and its potential as a co-author’, Euro. J. Therap.. (2023)

Bano, M., Didar Z., Jon W.: Exploring qualitative research using LLMs. (2023) arXiv preprint arXiv:2306.13298.

Bender, E.M., Timnit G., Angelina M.-M., Shmargaret S.: On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp 610–23 (2021)

Byun, C., Piper, V., Kevin, S.: Dispensing with Humans in Human-Computer Interaction Research. In: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–26 (2023)

Easterbrook, S., Singer, J., Storey, M.A., Damian, D.: Selecting empirical methods for software engineering research. Guide to Adv. Emp. Softw. Eng. 8, 285–311 (2008)CrossRef

Ebert, C., Louridas, P.: Generative AI for software practitioners. IEEE Softw. 40, 30–38 (2023)CrossRef

Emmert-Streib, F.: Importance of critical thinking to understand ChatGPT. Europ. J. Human Genet. 15, 1–2 (2023)

Gentles, S.J., Cathy, C., Jenny, P., Ann Mckibbon, K.: Sampling in qualitative research: insights from an overview of the methods literature. Qual. Rep. 20, 1772–1789 (2015)

Hoda, R.: Socio-technical grounded theory for software engineering. IEEE Transaction Software Engineering. 48, 3808–3832 (2021)CrossRef

Hou, X., Yanjie, Z., Yue, L., Zhou, Y., Kailong, W., Li, L., Xiapu, L., David, L., John, G., Haoyu, W.: Large language models for software engineering: a systematic literature review. arXiv preprint arXiv:2308.10620

Jalil, S., Suzzana, R., Thomas, D.L., Kevin, M., Wing, L.: Chatgpt and software testing education: Promises & perils. In: 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 4130–37. IEEE (2023)

Jiang, D., Xiang R., Bill Y-L.: LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. (2023) arXiv preprint arXiv:2306.02561.

Kitchenham, B.: Procedures for performing systematic reviews. Keele UK Keele Univ. 33(2004), 1–26 (2004)

Kuhail, M.A., Sujith, S.M., Ashraf, K., Jose, B., Syed J.S.: Will I be replaced? Assessing chatgpt’s effect on software development and programmer perceptions of Ai tools. Assessing Chatgpt’s Effect on Software Development and Programmer Perceptions of Ai Tools.

Navigli, R., Simone, C., and Björn, R.: Biases in large language models: origins, inventory and discussion. ACM J. Data Inform. Qual. (2023)

Nguyen-Duc, A., Beatriz C.-D., Adam, P., Chetan, A., Dron, K., Tomas, H., Usman, R., Jorge, M., Eduardo, G., Kai-Kristian K.: Generative artificial intelligence for software engineering–a research Agenda, (2023) arXiv preprint arXiv:2310.18648.

Ozkaya, I.: Application of large language models to software engineering tasks: opportunities. Risks Implicat. IEEE Software. 40, 4–8 (2023)

Polonsky, M.J., Jeffrey D.R.: Should artificial intelligent agents be your co-author? Arguments in favour, informed by ChatGPT. In: 91–96. SAGE Publications Sage UK: London, England (2023)

Rudolph, J., Tan, S., Tan, S.: ChatGPT: bullshit spewer or the end of traditional assessments in higher education? J. Appl. Learn. Teach. 24, 6 (2023)

Scoccia, G.L.: Exploring Early Adopters’ Perceptions of ChatGPT as a Code Generation Tool. In: 2023 38th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW), pp 88–93 (2023)

Treude, C., Hideaki H.: She Elicits Requirements and he tests: software engineering gender bias in large language models. (2023) arXiv preprint arXiv:2303.10131.

Watkins, R.: Guidance for researchers and peer-reviewers on the ethical use of large language models (LLMs) in scientific research workflows. AI Ethics 16, 1–6 (2023)

Watson, C.: Unreliable narrators?’Inconsistency’(and some inconstancy) in interviews. Qual. Res. 6, 367–384 (2006)CrossRef

Titel: Large language models for qualitative research in software engineering: exploring opportunities and challenges
verfasst von: Muneera Bano
Rashina Hoda
Didar Zowghi
Christoph Treude
Publikationsdatum: 01.05.2024
Verlag: Springer US
Erschienen in: Automated Software Engineering / Ausgabe 1/2024
Print ISSN: 0928-8910
Elektronische ISSN: 1573-7535
DOI: https://doi.org/10.1007/s10515-023-00407-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 1/2024

Using data mining techniques to generate test cases from graph transformation systems specifications

DL4SC: a novel deep learning-based vulnerability detection framework for smart contracts

Correction to: A class integration test order generation approach based on Sarsa algorithm

Sound analysis and migration of data from Ethereum smart contracts

An extensive study of the effects of different deep learning models on code vulnerability detection in Python code

Software defect prediction: future directions and challenges

Premium Partner