Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Empirical Software Engineering
Published in: Empirical Software Engineering 5/2023

01-09-2023

Tag that issue: applying API-domain labels in issue tracking systems

Authors: Fabio Santos, Joseph Vargovich, Bianca Trinkenreich, Italo Santos, Jacob Penney, Ricardo Britto, João Felipe Pimentel, Igor Wiese, Igor Steinmacher, Anita Sarma, Marco A. Gerosa

Published in: Empirical Software Engineering | Issue 5/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Labeling issues with the skills required to complete them can help contributors to choose tasks in Open Source Software projects. However, manually labeling issues is time-consuming and error-prone, and current automated approaches are mostly limited to classifying issues as bugs/non-bugs. We investigate the feasibility and relevance of automatically labeling issues with what we call “API-domains,” which are high-level categories of APIs. Therefore, we posit that the APIs used in the source code affected by an issue can be a proxy for the type of skills (e.g., DB, security, UI) needed to work on the issue. We ran a user study (n=74) to assess API-domain labels’ relevancy to potential contributors, leveraged the issues’ descriptions and the project history to build prediction models, and validated the predictions with contributors (n=20) of the projects. Our results show that (i) newcomers to the project consider API-domain labels useful in choosing tasks, (ii) labels can be predicted with a precision of 84% and a recall of 78.6% on average, (iii) the results of the predictions reached up to 71.3% in precision and 52.5% in recall when training with a project and testing in another (transfer learning), and (iv) project contributors consider most of the predictions helpful in identifying needed skills. These findings suggest our approach can be applied in practice to automatically label issues, assisting developers in finding tasks that better match their skills.
previous article An empirically based model of software prototyping: a mapping study and a multi-case study
next article What constitutes debugging? An exploratory study of debugging episodes

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now
Appendix
Available only for authorised users
Footnotes
2
In this study, the words “tasks” and “issues” are used interchangeably.
 
Literature
Agrawal R, Imieliński T, Swami A (1993) Mining association rules between sets of items in large databases. in Proceedings of the 1993 ACM SIGMOD international conference on management of data, pp 207–216
Antoniol G, Ayari K, Di Penta M, Khomh F, Guéhéneuc Y-G (2008) Is it a bug or an enhancement? a text-based approach to classify change requests. in Proceedings of the 2008 conference of the center for advanced studies on collaborative research: meeting of minds, pp 304–318
Balali S, Steinmacher I, Annamalai U, Sarma A, Gerosa MA (2018) Newcomer’ barriers. . . is that all? an analysis of mentors’ and newcomers’ barriers in OSS projects. Comput Supported Coop Work 27(3-6):679–714
Barcomb A, Stol K, Fitzgerald B, Riehle D (2020) Managing episodic volunteers in free/libre/open source software communities. IEEE Trans Softw Eng:1–1
Behl D, Handa S, Arora A (2014) A bug mining tool to identify and analyze security bugs using naive bayes and tf-idf. in 2014 International Conference on Reliability Optimization and Information Technology (ICROIT). IEEE, pp 294–299
Bettenburg N, Just S, Schröter A, WeißC, Premraj R, Zimmermann T (2007) Quality of bug reports in eclipse. in Proceedings of the 2007 OOPSLA workshop on eclipse technology exchange, ser. eclipse ’07. New York, NY, USA: ACM, pp 21–25
Blanco A, Casillas A, Pérez A, de Ilarraza AD (2019) Multi-label clinical document classification: impact of label-density. Expert Syst Appl 138:112835CrossRef
Charte F, Rivera AJ, del Jesus MJ, Herrera F (2015) Mlsmote: approaching imbalanced multilabel learning through synthetic instance generation. Knowl-Based Syst 89:385–397CrossRef
Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: pre-training of deep bidirectional transformers for language understanding. in NAACL,
Ducasse S, Pollet D (2009) Software architecture reconstruction: a process-oriented taxonomy. IEEE Trans Softw Eng 35(4):573–591CrossRef
El Zanaty F, Rezk C, Lijbrink S, van Bergen W, Côté M, McIntosh S (2020) Automatic recovery of missing issue type labels. IEEE Softw
Feng Y, Jones J, Chen Z, Fang C (2018) An empirical study on software failure classification with multi-label and problem-transformation techniques. in 2018 IEEE 11th International Conference on Software Testing, verification and validation (ICST). IEEE, pp 320–330
Ferreira Moreno M, Sousa Dos Santos WH, Costa Mesquita Santos R, Fontoura De Gusmao Cerqueira R (2018) Supporting knowledge creation through has: the hyperknowledge annotation system. in 2018 IEEE International Symposium on Multimedia (ISM), 239–246
Flach PA, Kull M (2015) Precision-recall-gain curves: Pr analysis done right. in NIPS 15
Goel E, Abhilasha E, Goel E, Abhilasha E (2017) Random forest: a review. Int J Adv Res Comput Sc Softw Eng 7(1)
Guggulothu T, Moiz SA (2020) Code smell detection using multi-label classification approach. Softw Quality J 28(3):1063–1086CrossRef
Herrera F, Charte F, Rivera AJ, del Jesus MJ (2016) Multilabel classification: problem analysis, metrics and techniques, 1st ed. Springer publishing company, incorporated
Huang Y, Wang J, Wang S, Liu Z, Wang D, Wang Q (2021) Characterizing and predicting good first issues. in Proceedings of the 15th ACM/IEEE international symposium on Empirical Software Engineering and Measurement (ESEM), pp 1–12
Izadi M, Ganji S, Heydarnoori (2021) Topic recommendation for software repositories using multi-label classification algorithms. Empir Softw Eng 26:93
Izadi M, Heydarnoori A, Gousios G (2021) Topic recommendation for software repositories using multi-label classification algorithms. Empirical Softw Eng 26:09CrossRef
Izadi M, Akbari K, Heydarnoori A (2022) Predicting the objective and priority of issue reports in software repositories. Empirical Softw Eng 27(2):1–37CrossRef
Kallis R, Di Sorbo A, Canfora G, Panichella S (2019) Ticket tagger: machine learning driven issue classification. in 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, pp 406–409
Klock ACT, Gasparini I, Pimenta MS (2016) 5W2H framework: a guide to design, develop and evaluate the user-centered gamification. in Proceedings of the 15th Brazilian symposium on human factors in computing systems, pp 1–10
Lin B, Zampetti F, Bavota G, Di Penta M, Lanza M (2019) Pattern-based mining of opinions in q &a websites. in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, pp 548–559
Lu Y, Li G, Zhao Z, Wen L, Jin Z (2017) Learning to infer API mappings from API documents. in International conference on knowledge science, engineering and management. Springer, pp 237–248
Nam J, Pan SJ, Kim S (2013) Transfer defect learning. in 2013 35th international conference on software engineering (ICSE). IEEE, pp 382–391
Ohno T (1982) How the toyota production system was created. Japanese Econ Studies 10(4):83–101CrossRef
Pacaiova H (2015) Analysis and identification of nonconforming products by 5W2H method. Center for Quality
Park Y, Jensen C (2009) Beyond pretty pictures: examining the benefits of code visualization for open source newcomers. in Proceedings of the 5th IEEE international workshop on visualizing software for understanding and analysis, ser. VISSOFT ’09. IEEE, pp 3–10
Pereira RB, Plastino A, Zadrozny B, Merschmann LH (2018) Correlation analysis of performance measures for multi-label classification. Inf Process Manag 54(3):359–369CrossRef
Perez Q, Jean P-A, Urtado C, Vauttier S (2021) Bug or not bug? that is the question. in 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). IEEE, pp 47–58
Petkovic D, Sosnick-Pérez M, Okada K, Todtenhoefer R, Huang S, Miglani N, Vigil A (2016) Using the random forest classifier to assess and predict student learning of software engineering teamwork. in 2016 IEEE Frontiers in Education Conference (FIE). IEEE, pp 1–7
Pingclasai N, Hata H, Matsumoto K-I (2013) Classifying bug reports to bugs and other requests using topic modeling. in 2013 20Th asia-pacific software engineering conference (APSEC), vol 2. IEEE, pp 13–18
Pinto GHL, Figueira Filho F, Steinmacher I, Gerosa MA (2017) Training software engineers using open-source software: the professors’ perspective. in 2017 IEEE 30th Conference on Software Engineering Education and Training (CSEE &T). IEEE, pp 117–121
Pinto G, Steinmacher I, Gerosa MA (2016) More common than you think: an in-depth study of casual contributors. in IEEE 23rd international conference on software analysis, evolution, and reengineering, SANER 2016, Suita, Osaka, Japan, 14-18 March 2016 - vol 1, pp 112–123
Pushphavathi T, Suma V, Ramaswamy V (2014) A novel method for software defect prediction: hybrid of fcm and random forest. in 2014 International Conference on Electronics and Communication Systems (ICECS). IEEE, pp 1–5
Qiu D, Li B, Leung H (2016) Understanding the API usage in Java. Inf Softw Technol 73:81–100CrossRef
Ramos J, et al (2003) Using TF-IDF to determine word relevance in document queries. in Proceedings of the first instructional conference on machine learning, vol 242. Piscataway, NJ, pp 133–142
Ravichandiran S (2021) Getting started with google BERT: build and train state-of-the-art natural language processing models using BERT. Packt Publishing Ltd
Romano J, Kromrey J, Coraggio J, Skowronek J (2006) Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’sd for evaluating group differences on the NSSE and other surveys? in Annual meeting of the florida association of institutional research, pp 1–3
Saito T, Rehmsmeier M (2015) The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PloS One 10(3):e0118432CrossRef
Santos F, Trinkenreich B, Nicolati Pimentel JF, Wiese I, Steinmacher I, Sarma A, Gerosa M (2022) How to choose a task? mismatches in perspectives of newcomers and existing contributors. Empirical Softw Eng Meas
Santos I, Wiese I, Steinmacher I, Sarma A, Gerosa MA (2022) Hits and misses: newcomers’ ability to identify skills needed for OSS tasks. in 2022 IEEE international conference on software analysis, evolution and reengineering (SANER), pp 174–183
Santos F, Wiese I, Trinkenreich B, Steinmacher I, Sarma A, Gerosa MA (2021) Can i solve it? identifying apis required to complete OSS tasks. in 2021 IEEE/ACM 18th international conference on Mining Software Repositories (MSR). IEEE, pp 346–257
Sarma A, Gerosa MA, Steinmacher I, Leano R (2016) Training the future workforce through task curation in an OSS ecosystem. in Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, pp 932–935
Satapathy SM, Acharya BP, Rath SK (2016) Early stage software effort estimation using random forest technique based on use case points. IET Softw 10(1):10–17CrossRef
Savidis A, Savaki C (2021) Software architecture mining from source code with dependency graph clustering and visualization. in IVAPP, 12
Seah C-W, Tsang IW, Ong Y-S (2013) Transfer ordinal label learning. IEEE Trans Neural Netw Learn Syst 24(11):1863–1876CrossRef
Sheskin D (2020) Handbook of parametric and nonparametric statistical procedures, 5th edn. Chapman & HallCrossRefMATH
Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manag 45(4):427–437CrossRef
Stanik C, Montgomery L, Martens D, Fucci D, Maalej W (2018) A simple nlp-based approach to support onboarding and retention in open source communities. in 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, pp 172–182
Steinmacher I, Silva MAG, Gerosa MA, Redmiles DF (2015) A systematic literature review on the barriers faced by newcomers to open source software projects. Inf Softw Technol 59:67–85CrossRef
Steinmacher I, Conte TU, Gerosa MA (2015) Understanding and supporting the choice of an appropriate task to start with in open source software communities. in 2015 48th Hawaii international conference on system sciences. IEEE, pp 5299–5308
Steinmacher I, Conte T, Gerosa MA, Redmiles D (2015) Social barriers faced by newcomers placing their first contribution in open source software projects. in Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, ser. CSCW’15. New York, NY, USA: Association for computing machinery, pp 1379–1392
Steinmacher I, Treude C, Gerosa MA (2018) Let me in: guidelines for the successful onboarding of newcomers to open source projects. IEEE Softw, vol 36(4):41–49
Strauss A, Corbin J (1998) Basics of qualitative research techniques. Sage publications, Thousand oaks, CA
Szumilas M (2010) Explaining odds ratios. J Canadian Acad Child Adolescent Psych 19(3):227
Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2019) The impact of automated parameter optimization on defect prediction models. IEEE Trans Softw Eng 45(7):683–711CrossRef
Tsoumakas G, Katakis I, Vlahavas I (2009) Mining multi-label data. Data Mining Knowl Discover Handbook:667–685
Uddin G, Khomh F (2019) Automatic mining of opinions expressed about apis in stack overflow. IEEE Trans Softw Eng:1–1
Vadlamani SL, Baysal O (2020) Studying software developer expertise and contributions in stack overflow and GitHub. in 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, pp 312–323
Van Gompel M, Van Den Bosch A (2016) Efficient n-gram, skipgram and flexgram modelling with colibri core. J Open Res Softw 4(1)
Vargas-Baldrich S, Linares-Vásquez M, Poshyvanyk D (2015) Automated tagging of software projects using bytecode and dependencies. in 2015 30th IEEE/ACM international conference on Automated Software Engineering (ASE). IEEE, pp 289–294
Vaz L, Steinmacher I, Marczak S (2019) An empirical study on task documentation in software crowdsourcing on topcoder. in 2019 ACM/IEEE 14th International Conference on Global Software Engineering (ICGSE). IEEE, pp 48–57
Wang J, Sarma A (2011) Which bug should i fix: helping new developers onboard a new project. In Proceedings of the 4th international workshop on cooperative and human aspects of software engineering, ACM, pp 76–79
Wiese IS, Ré R, Steinmacher I, Kuroda RT, Oliva GA, Treude C, Gerosa MA (2017) Using contextual information to predict co-changes. J Syst Softw 128:220–235CrossRef
Xia X, Lo D, Wang X, Zhou B (2013) Tag recommendation in software information sites. in 2013 10th Working conference on mining software repositories (MSR). IEEE, pp 287–296
You Y, Li J, Reddi S, Hseu J, Kumar S, Bhojanapalli S, Song X, Demmel J, Keutzer K, Hsieh C-J (2020) Large batch optimization for deep learning: training bert in 76 minutes. in International conference on learning representations. Available: https://​openreview.​net/​forum?​id=​Syx4wnEtvH
Zhang M-L, Zhou Z-H (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recogni 40(7):2038–2048CrossRefMATH
Zhou Y, Tong Y, Gu R, Gall H (2016) Combining text mining and data mining for bug report classification. J Softw Evol Process 28(3):150–176CrossRef
Zimmermann T, Premraj R, Bettenburg N, Just S, Schroter A, Weiss C (2010) What makes a good bug report IEEE Trans Softw Eng, vol 36(5), pp 618–643
Metadata
Title
Tag that issue: applying API-domain labels in issue tracking systems
Authors
Fabio Santos
Joseph Vargovich
Bianca Trinkenreich
Italo Santos
Jacob Penney
Ricardo Britto
João Felipe Pimentel
Igor Wiese
Igor Steinmacher
Anita Sarma
Marco A. Gerosa
Publication date
01-09-2023
Publisher
Springer US
Published in
Empirical Software Engineering / Issue 5/2023
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI
https://doi.org/10.1007/s10664-023-10329-4

Other articles of this Issue 5/2023

Empirical Software Engineering 5/2023 Go to the issue

OriginalPaper

A comparison of reinforcement learning frameworks for software testing tasks

OriginalPaper

On the validity of retrospective predictive performance evaluation procedures in just-in-time software defect prediction

OriginalPaper

A user study for evaluation of formal verification results and their explanation at Bosch

OriginalPaper

More than React: Investigating the Role of Emoji Reaction in GitHub Pull Requests

OriginalPaper

Can the configuration of static analyses make resolving security vulnerabilities more effective? - A user study

OriginalPaper

Adversarial domain adaptation for cross-project defect prediction

Premium Partner

    Image Credits