Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Empirical Software Engineering
Erschienen in: Empirical Software Engineering 1/2024

01.02.2024

Using knowledge units of programming languages to recommend reviewers for pull requests: an empirical study

verfasst von: Md Ahasanuzzaman, Gustavo A. Oliva, Ahmed E. Hassan

Erschienen in: Empirical Software Engineering | Ausgabe 1/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Determining the right code reviewer for a given code change requires understanding the characteristics of the changed code, identifying the skills of each potential reviewer (expertise profile), and finding a good match between the two. To facilitate this task, we design a code reviewer recommender that operates on the knowledge units (KUs) of a programming language. We define a KU as a cohesive set of key capabilities that are offered by one or more building blocks of a given programming language. We operationalize our KUs using certification exams for the Java programming language. We detect KUs from 10 actively maintained Java projects from GitHub, spanning 290K commits and 65K pull requests (PRs). We generate developer expertise profiles based on the detected KUs. We use these KU-based expertise profiles to build a code reviewer recommender (KUREC). We compare KUREC’s performance to that of seven baseline recommenders. KUREC ranked first along with the top-performing baseline recommender (RF) in a Scott-Knott ESD analysis of recommendation accuracy (the top-5 accuracy of KUREC is 0.84 (median) and the MAP@5 is 0.51 (median)). From a practical standpoint, we highlight that KUREC’s performance is more stable (lower interquartile range) than that of RF, thus making it more consistent and potentially more trustworthy. We also design three new recommenders by combining KUREC with our baseline recommenders. These new combined recommenders outperform both KUREC and the individual baselines. Finally, we evaluate how reasonable the recommendations from KUREC and the combined recommenders are when those deviate from the ground truth. We observe that KUREC is the recommender with the highest percentage of reasonable recommendations (63.4%). Overall we conclude that KUREC and one of the combined recommenders (e.g., AD_HYBRID) are overall superior to the baseline recommenders that we studied. Future work in the area should thus (i) consider KU-based recommenders as baselines and (ii) experiment with combined recommenders.
Vorheriger Artikel The Impact of Personality on Requirements Engineering Activities: A Mixed-Methods Study
Nächster Artikel An empirical study of task infections in Ansible scripts

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
http://​www.​bit.​ly/​3bhSFux. The contents will be made available on a public GitHub repository once the paper is accepted.
 
Literatur
Adomavicius G, Zhang J (2014) Improving stability of recommender systems: a meta-algorithmic approach. IEEE Trans Knowl Data Eng 27(6):1573–1587CrossRef
Al-Subaihin AA, Sarro F, Black S, Capra L, Harman M, Jia Y, Zhang Y (2016) Clustering Mobile Apps Based on Mined Textual Features. In: Proceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement, ESEM’16
Al-Zubaidi WHA, Thongtanunam P, Dam HK, Tantithamthavorn C, Ghose A (2020) Workload-aware reviewer recommendation using a multi-objective search-based approach. In: Proceedings of the 16th ACM international conference on predictive models and data analytics in software engineering, pp 21–30
Anagnostopoulos A, Kumar R, Mahdian M (2008) Influence and correlation in social networks. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 7–15
Anvik J, Murphy GC (2007) Determining Implementation Expertise from Bug Reports. In: Proceedings of the 4th international workshop on mining software repositories, pp 1–8
Asthana S, Kumar R, Bhagwan R, Bird C, Bansal C, Maddila C, Mehta S, Ashok B (2019) Whodo: automating reviewer suggestions at scale. In: Proceedings of the 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 937–945
Avelino G, Passos L, Hora A, Valente MT (2016) A novel approach for estimating truck factors. In: Proceedings of the 24th international conference on program comprehension (ICPC), IEEE, pp 1–1
Bacchelli A, Bird C (2013) Expectations, outcomes, and challenges of modern code review. In: Proceedings of the 2013 international conference on software engineering, IEEE Press, ICSE’13, p 712–721
Balachandran V (2013) Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In: Proceedings of the 35th international conference on software engineering (ICSE), pp 931–940
Bishnu PS, Bhattacherjee V (2012) Software Fault Prediction Using Quad Tree-Based K-Means Clustering Algorithm. IEEE Trans Knowl Data Eng 24(6):1146–1150CrossRef
Bishop M, Burley D, Buck S, Ekstrom JJ, Futcher L, Gibson D, Hawthorne EK, Kaza S, Levy Y, Mattord H et al. (2017) Cybersecurity curricular guidelines. In: IFIP world conference on information security education, pp 3–13
Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Learn Res 3(Jan):993–1022
Campos PG, Díez F, Sánchez-Montañés M (2011) Towards a more realistic evaluation: testing the ability to predict future tastes of matrix factorization-based recommenders. In: Proceedings of the 5th ACM conference on recommender systems, pp 309–312
Chouchen M, Ouni A, Mkaouer MW, Kula RG, Inoue K (2021) Whoreview: A multi-objective search-based approach for code reviewers recommendation in modern code review. Appl Soft Comput 100:106908CrossRef
Cogo FR, Xia X, Hassan AE (2022) Assessing the alignment between the information needs of developers and the documentation of programming languages: A case study on rust. arXiv:​2202.​04431
de Lima Júnior ML, Soares DM, Plastino A, Murta L (2015) Developers assignment for analyzing pull requests. In: Proceedings of the 30th annual ACM symposium on applied computing, pp 1567–1572
Dey T, Karnauch A, Mockus A (2021) Representation of Developer Expertise in Open Source Software. In: Proceedings of the 43rd international conference on software engineering, p 995–1007
Ding C, He X (2004) K-Means Clustering via Principal Component Analysis. In: Proceedings of the 21st international conference on machine learning, pp 29–3
Fejzer M, Przymus P, Stencel K (2018) Profile based recommendation of code reviewers. J Intell Inf Syst 50:597–619CrossRef
Ferreira M, Mombach T, Valente MT, Ferreira K (2019) Algorithms for estimating truck factors: a comparative study. Softw Qual J 27(4):1583–1617CrossRef
Fritz T, Murphy GC, Hill E (2007) Does a Programmer’s Activity Indicate Knowledge of Code? In: Proceedings of the 6th joint meeting of the european software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering, p 341–350
Fritz T, Murphy GC, Murphy-Hill E, Ou J, Hill E (2014) Degree-of-Knowledge: Modeling a Developer’s Knowledge of Code. ACM Trans Softw Eng Methodol 23(2)
Fritz T, Ou J, Murphy GC, Murphy-Hill E (2010) A Degree-of-Knowledge Model to Capture Source Code Familiarity. In: Proceedings of the 32nd ACM/IEEE international conference on software engineering, p 385–394
Gauthier IX, Lamothe M, Mussbacher G, McIntosh S (2021) Is historical data an appropriate benchmark for reviewer recommendation systems?: A case study of the gerrit community. In: Proceedings of the 36th IEEE/ACM international conference on automated software engineering (ASE), IEEE, pp 30–41
Ghotra B, McIntosh S, Hassan AE (2015) Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models. In: Proceedings of the 37th IEEE international conference on software engineering, pp 789–800
Ghotra B, Mcintosh S, Hassan AE (2017) A large-scale study of the impact of feature selection techniques on defect classification models. In: Porceedings of the 14th international conference on mining software repositories, p 146–157
Girba T, Kuhn A, Seeberger M, Ducasse S (2005) How developers drive software evolution. In: Proceedings of the 8th international workshop on principles of software evolution, pp 113–122
Goebl S, He X, Plant C, Böhm C (2014) Finding the optimal subspace for clustering. In: Proceedings of the 2014 IEEE international conference on data mining, pp 130–139
Greene GJ, Fischer B (2016) Cvexplorer: Identifying candidate developers by mining and exploring their open source contributions. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering, pp 804–809
Hannebauer C, Patalas M, Stünkel S, Gruhn V (2016) Automatically recommending code reviewers based on their expertise: an empirical comparison. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering, p 99–110
Hauff C, Gousios G (2015) Matching GitHub developer profiles to job advertisements. In: 2015 IEEE/ACM 12th working conference on mining software repositories, pp 362–366
He Q, Li B, Chen F, Grundy J, Xia X, Yang Y (2020) Diversified third-party library prediction for mobile app development. IEEE Trans Software Eng 48(1):150–165CrossRef
Jiang J, He JH, Chen XY (2015) Coredevrec: automatic core member recommendation for contribution evaluation. J Comput Sci Technol 30:998–1016CrossRef
Jiang J, Yang Y, He J, Blanc X, Zhang L (2017) Who should comment on this pull request? analyzing attributes for more accurate commenter recommendation in pull based development. Inf Softw Technol 84:48–62CrossRef
Jiang J, Lo D, Zheng J, Xia X, Yang Y, Zhang L (2019) Who should make decision on this pull request? analyzing time-decaying relationships and file similarities for integrator prediction. J Syst Softw 154:196–210CrossRef
Jureczko M, Madeyski L (2010) Towards identifying software project clusters with regard to defect prediction. In: Proceedings of the 6th international conference on predictive models in software engineering, PROMISE’10
Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2014) The promises and Perils of mining GitHub. In: Proceedings of the 11th working conference on mining software repositories, pp 92–101
Kassambara A, Mundt F (2017) Package ‘factoextra’. Extract and visualize the results of multivariate data analyses 76
Kleinberg JM (1999) Authoritative sources in a hyperlinked environment. J ACM (JACM) 46(5):604–632
Kondo M, Bezemer CP, Kamei Y, Hassan AE, Mizuno O (2019) The impact of feature reduction techniques on defect prediction models. Empirical Softw Eng 24(4):1925–1963
Lewis C, Lin Z, Sadowski C, Zhu X, Ou R, Whitehead Jr EJ (2013) Does bug prediction support human developers? Findings from a Google Case Study. In: Proceedings of the 35th international conference on software engineering, p 372–381
Li T, Zhu S, Ogihara M (2003) Algorithms for clustering high dimensional and distributed data. Intell Data Anal 7(4):305–326CrossRef
Liang JT, Zimmermann T, Ford D (2022) Towards mining oss skills from github activity. In: Proceedings of the 44th international conference on software engineering - new ideas and emerging results (NIER track)
Liao Z, Wu Z, Wu J, Zhang Y, Liu J, Long J (2019) Tirr: a code reviewer recommendation algorithm with topic model and reviewer influence. In: Proceedings of the 2019 IEEE global communications conference, pp 1–6
Li X, Peng S, Du J (2021) Towards medical knowmetrics: representing and computing medical knowledge using semantic predications as the knowledge unit and the uncertainty as the knowledge context. Scientometrics pp 1–27
Malik H, Hassan AE (2008) Supporting software evolution using adaptive change propagation heuristics. In: Proceedings of the 24th IEEE international conference on software maintenance, pp 177–18
Ma D, Schuler D, Zimmermann T, Sillito J (2009) Expert recommendation with usage expertise. In: Proceedings of the 25th IEEE international conference on software maintenance, pp 535–538
McDonald DW, Ackerman MS (2000) Expertise recommender: a flexible recommendation system and Architecture. In: Proceedings of the 2000 ACM conference on computer supported cooperative work, pp 231–240
McDonald DW (2001) Evaluating expertise recommendations. In: Proceedings of the 2001 International ACM SIGGROUP conference on supporting group work, p 214–223
Mirsaeedi E, Rigby PC (2020) Mitigating turnover with code review recommendation: balancing expertise, workload, and knowledge distribution. In: Proceedings of the 42nd international conference on software engineering, pp 1183–1195
Mittas N, Angelis L (2013) Ranking and clustering software cost estimation models through a multiple comparisons algorithm. IEEE Trans Software Eng 39(4):537–551CrossRef
Mockus A, Herbsleb JD (2002) Expertise browser: a quantitative approach to identifying expertise. In: Proceedings of the 24th international conference on software engineering, pp 503–512
Montandon JE, Silva LL, Valente MT (2019) Identifying experts in software libraries and frameworks among github users. In: Porceedings of the 2019 16th international conference on mining software repositories (MSR), pp 276–287
Moradi Dakhel A, C Desmarais M, Khomh F (2021) Assessing developer expertise from the statistical distribution of programming syntax patterns. In: Proceedings of the 25th international conference on evaluation and assessment in software engineering, pp 90–99
Munaiah N, Kroh S, Cabrey C, Nagappan M (2017) Curating GitHub for engineered software projects. Empir Softw Eng 22(6):3219–3253
Nidheesh N, Nazeer K, Ameer P (2020) A Hierarchical Clustering algorithm based on Silhouette Index for cancer subtype discovery from genomic data. Neural Comput Appl 32(15):11459–11476CrossRef
Ouni A, Kula RG, Inoue K (2016) Search-based peer reviewers recommendation in modern code review. In: Proceedings of the 32nd IEEE international conference on software maintenance and evolution, pp 367–377
Pandya P, Tiwari S (2022) Corms: a github and gerrit based hybrid code reviewer recommendation approach for modern code review. In: Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering, pp 546–557
Panichella A, Dit B, Oliveto R, Di Penta M, Poshynanyk D, De Lucia A (2013) How to effectively use topic models for software engineering tasks? an approach based on genetic algorithms. In: Proceedings of the 35th international conference on software engineering, pp 522–531
Parsons L, Haque E, Liu H (2004) Subspace Clustering for High Dimensional Data: A Review. SIGKDD Explorations Newsletter 6(1):90–105CrossRef
Patel A, Jain S, Shandilya SK (2018) Data of semantic web as unit of knowledge. J Web Eng
Peták M, Brožová H, Houška M (2020) Modelling of knowledge via fuzzy knowledge unit in a case of the ERP systems upgrade. Autom Control Comput Sci 54(6):529–540
Peták M, Houška M (2018) Fuzzy knowledge unit. In: Proceedings of 12th international scientific conference on distance learning in applied informatics, pp 491–502
Rahman MM, Roy CK, Redl J, Collins JA (2016) Correct: code reviewer recommendation at github for vendasta technologies. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering, pp 792–797
Rigby PC, Bird C (2013) Convergent contemporary software peer review practices. In: Proceedings of the 2013 9th joint meeting on foundations of software engineering, Association for Computing Machinery, New York, NY, USA, ESEC/FSE 2013, pp 202–212. https://​doi.​org/​10.​1145/​2491411.​2491444
Robbes R, Röthlisberger D (2013) Using developer interaction data to compare expertise metrics. In: Proceedings of the 10th working conference on mining software repositories, p 297–300
Rong G, Zhang Y, Yang L, Zhang F, Kuang H, Zhang H (2022) Modeling review history for reviewer recommendation: a hypergraph approach. In: Proceedings of the 44th international conference on software engineering, pp 1381–1392
Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65CrossRef
Spadini D, Aniche M, Bacchelli A (2018) PyDriller: Python framework for mining software repositories. In: Proceedings of the 2018 26th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, p 908–911
Strand A, Gunnarson M, Britto R, Usman M (2020) Using a context-aware approach to recommend code reviewers: findings from an industrial case study. In: Proceedings of the 42nd international conference on software engineering: software engineering in practice, pp 1–10
Sülün E, Tüzün E, Doǧrusöz U (2019) Reviewer recommendation using software artifact traceability graphs. In: Proceedings of the 15th international conference on predictive models and data analytics in software engineering, pp 66–75
Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2017) An empirical comparison of model validation techniques for defect prediction models. IEEE Trans Software Eng 43(1):1–18CrossRef
Tecimer KA, Tüzün E, Dibeklioglu H, Erdogmus H (2021) Detection and elimination of systematic labeling bias in code reviewer recommendation systems. In: Evaluation and assessment in software engineering, pp 181–190
Thongtanunam P, Tantithamthavorn C, Kula RG, Yoshida N, Iida H, Matsumoto Ki (2015) Who should review my code? a file location-based code-reviewer recommendation approach for modern code review. In: Proceedings of the 22nd international conference on software analysis, evolution, and reengineering (SANER), pp 141–150
Tsantalis N, Chatzigeorgiou A, Stephanides G, Halkidis ST (2006) Design pattern detection using similarity scoring. IEEE Trans Software Eng 32(11):896–909CrossRef
Vivacqua A, Lieberman H (2000) Agents to assist in finding help. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 65–72
Von Solms S, Futcher L (2018) Identifying the cybersecurity body of knowledge for a postgraduate module in systems engineering. In: IFIP world conference on information security education, pp 121–132
Wan Y, Chen L, Xu G, Zhao Z, Tang J, Wu J (2018) SCSMiner: mining social coding sites for software developer recommendation with relevance propagation. World Wide Web 21(6):1523–1543CrossRef
Xia X, Lo D, Wang X, Yang X (2015) Who should review this change?: Putting text and file location analyses together for more accurate recommendations. In: Proceedings of the 31st IEEE international conference on software maintenance and evolution, pp 261–270
Xia Z, Sun H, Jiang J, Wang X, Liu X (2017) A hybrid approach to code reviewer recommendation with collaborative filtering. In: Proceedings of the 6th international workshop on software mining, pp 24-31
Xia Z, Sun H, Jiang J, Wang X, Liu X (2017) A hybrid approach to code reviewer recommendation with collaborative filtering. In: Proceedings of the 6th on international workshop software mining, pp 24–31
Xibilia MG, Latino M, Marinković Z, Atanasković A, Donato N (2020) Soft sensors based on deep neural networks for applications in security and safety. IEEE Trans Instrum Meas 69(10):7869–7876CrossRef
Xie X, Yang X, Wang B, He Q (2021) DevRec: multi-relationship embedded software developer recommendation. IEEE Trans Softw Eng 1–1
Xiong R, Li B (2019) Accurate design pattern detection based on idiomatic implementation matching in java language context. In: 2019 IEEE 26th international conference on software analysis, evolution and reengineering (SANER), pp 163–174
Ying H, Chen L, Liang T, Wu J (2016) Earec: leveraging expertise and authority for pull-request reviewer recommendation in github. In: Proceedings of the 3rd international workshop on crowdsourcing in software engineering, pp 29–35
Yitzhaki S (1979) Relative Deprivation and the Gini Coefficient. Q J Econ 93(2):321–324CrossRef
Yoon KA, Kwon OS, Bae DH (2007) An approach to outlier detection of software measurement data using the k-means clustering method. In: First international symposium on empirical software engineering and measurement, pp 443–445
Yu Y, Wang H, Yin G, Wang T (2016) Reviewer recommendation for pull-requests in GitHub: What can we learn from code review and bug assignment? Inf Softw Technol 74:204–218CrossRef
Yu Y, Wang H, Yin G, Wang T (2016) Reviewer recommendation for pull-requests in GitHub: What can we learn from code review and bug assignment? Inf Softw Technol 74:204–2CrossRef
Yu Y, Wang H, Yin G, Ling CX (2014) Who should review this pull-request: Reviewer recommendation to expedite crowd collaboration. In: Proceedings of the 21st Asia-Pacific software engineering conference, pp 335–342
Zanjani MB, Kagdi H, Bird C (2016) Automatically recommending peer reviewers in modern code review. IEEE Trans Software Eng 42(6):530–543CrossRef
Zhang F, Zheng Q, Zou Y, Hassan AE (2016) Cross-project defect prediction using a connectivity-based unsupervised classifier. In: Proceedings of the 38th international conference on software engineering, p 309–320
Metadaten
Titel
Using knowledge units of programming languages to recommend reviewers for pull requests: an empirical study
verfasst von
Md Ahasanuzzaman
Gustavo A. Oliva
Ahmed E. Hassan
Publikationsdatum
01.02.2024
Verlag
Springer US
Erschienen in
Empirical Software Engineering / Ausgabe 1/2024
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI
https://doi.org/10.1007/s10664-023-10421-9

Weitere Artikel der Ausgabe 1/2024

Empirical Software Engineering 1/2024 Zur Ausgabe

EditorialNotes

Guest editorial: special issue on empirical software engineering and measurement

OriginalPaper

SAFe transformation in a large financial corporation

OriginalPaper

Silent bugs in deep learning frameworks: an empirical study of Keras and TensorFlow

OriginalPaper

Search-based Automatic Repair for Fairness and Accuracy in Decision-making Software

OriginalPaper

Assessing the utility of text-to-SQL approaches for satisfying software developer information needs

OriginalPaper

On the intuitive comprehensibility of contribution links in goal models: an experimental study

Premium Partner

    Bildnachweise