nach oben

Empirical Software Engineering

Erschienen in:

01.03.2024

Studying the impact of risk assessment analytics on risk awareness and code review performance

verfasst von: Xueyao Yu, Filipe R. Cogo, Shane McIntosh, Michael W. Godfrey

Erschienen in: Empirical Software Engineering | Ausgabe 2/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

While code review is a critical component of modern software quality assurance, defects can still slip through the review process undetected. Previous research suggests that the main reason for this is a lack of reviewer awareness about the likelihood of defects in proposed changes; even experienced developers may struggle to evaluate the potential risks. If a change’s riskiness is underestimated, it may not receive adequate attention during review, potentially leading to defects being introduced into the codebase. In this paper, we investigate how risk assessment analytics can influence the level of awareness among developers regarding the potential risks associated with code changes; we also study how effective and efficient reviewers are at detecting defects during code review with the use of such analytics. We conduct a controlled experiment using Gherald, a risk assessment prototype tool that analyzes the riskiness of change sets based on historical data. Following a between-subjects experimental design, we assign participants to the treatment (i.e., with access to Gherald) or control group. All participants are asked to perform risk assessment and code review tasks. Through our experiment with 48 participants, we find that the use of Gherald is associated with statistically significant improvements (one-tailed, unpaired Mann-Whitney U test, \(\alpha \) = 0.05) in developer awareness of riskiness of code changes and code review effectiveness. Moreover, participants in the treatment group tend to identify the known defects more quickly than those in the control group; however, the difference between the two groups is not statistically significant. Our results lead us to conclude that the adoption of a risk assessment tool has a positive impact on code review practices, which provides valuable insights for practitioners seeking to enhance their code review process and highlights the importance for further research to explore more effective and practical risk assessment approaches.

Vorheriger Artikel Demystifying API misuses in deep learning applications

Nächster Artikel Evaluating the impact of flaky simulators on testing autonomous driving systems

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

This experiment was reviewed by and received ethics clearance from the University of Waterloo Research Ethics Committee (ORE #44022).

https://commons.apache.org/proper/commons-lang/

https://www.apache.org

https://issues.apache.org/jira/projects/LANG/issues/

https://github.com/apache/commons-lang/

Ackerman AF, Fowler PJ, Ebenau RG (1984) In: Proc. of a symposium on software validation: inspection-testing-verification-alternatives (Elsevier North-Holland, Inc., USA), p. 13–40

Ackerman A, Buchwald L, Lewski F (1989) IEEE Software 6(3):31. https://doi.org/10.1109/52.28121CrossRef

Alami A, Leavitt Cohn M, Wasowski A (2019) 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . https://doi.org/10.1109/icse.2019.00111

Ayewah N, Pugh W (2010) In: Proceedings of the 19th international symposium on software testing and analysis (Association for Computing Machinery, New York, USA, 2010), ISSTA ’10, pp 241–252. https://doi.org/10.1145/1831708.1831738

Bacchelli A, Bird C (2013) 35th International Conference on Software Engineering (ICSE) (2013). https://doi.org/10.1109/icse.2013.6606617

Ball T, Bounimova E, Cook B, Levin V, Lichtenberg J, McGarvey C, Ondrusek B, Rajamani SK, Ustuner A In: Proceedings of the 1st ACM SIGOPS/EuroSys European conference on computer systems 2006 (Association for Computing Machinery, New York, USA, 2006), EuroSys ’06, pp 73-85. https://doi.org/10.1145/1217935.1217943

Barnett M, Bird C, Brunet J, Lahiri SK (2015) In: 2015 IEEE/ACM 37th IEEE International conference on software engineering, vol 1, pp 134–144. https://doi.org/10.1109/ICSE.2015.35

Baum T, Schneider K, Bacchelli A (2019) Empirical Software Engineering 24(4):1762–1798. https://doi.org/10.1007/s10664-018-9676-8CrossRef

Baum T, Liskin O, Niklas K, Schneider K (2016) In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering (Association for Computing Machinery, New York, USA, 2016), FSE 2016, pp 85–96

Baum T, Schneider K, Bacchelli A (2017) In: 2017 IEEE international conference on software maintenance and evolution (ICSME), pp 329–340. https://doi.org/10.1109/ICSME.2017.28

Baysal O, Kononenko O, Holmes R, Godfrey MW (2016) Empir Softw Eng 21(3):932. https://doi.org/10.1007/s10664-015-9366-8CrossRef

Beller M, Bholanath R, McIntosh S, Zaidman A (2016) In: Proc. of the international conference on software analysis, evolution, and reengineering (SANER), pp 470–481

Biffl S (2000) In: Proceedings seventh Asia-Pacific software engeering conference. APSEC 2000, pp 136–145. https://doi.org/10.1109/APSEC.2000.896692

Bosu A, Greiler M, Bird C (2015) In: Proceedings of the 12th working conference on mining software repositories (IEEE Press), MSR ’15, pp 146–156

Cohen J (2013) Current directions in psychological science 1(3):98CrossRef

Fagan ME (1976) IBM Syst J 15(3):182. https://doi.org/10.1147/sj.153.0182CrossRef

Feitelson DG, Frachtenberg E, Beck KL (2013) IEEE Internet Comput 17(4):8. https://doi.org/10.1109/MIC.2013.25CrossRef

Fregnan E (2023) Assessing review outcomes and cognitive factors to improve code review. Ph.D. thesis

Fregnan E, Braz L, D’Ambros M, Çalıklı G, Bacchelli A (2022) In: Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering (Association for Computing Machinery, New York, USA), ESEC/FSE 2022, pp 483-494. https://doi.org/10.1145/3540250.3549177

Fukushima T, Kamei Y, Mcintosh S, Yamashita K, Ubayashi N (2014) Proceedings of the 11th working conference on mining software repositories - MSR 2014 . https://doi.org/10.1145/2597073.2597075

Gasparini L, Fregnan E, Braz L, Baum T, Bacchelli A (2021) In: 2021 Working conference on software visualization (VISSOFT), pp 115–119. https://doi.org/10.1109/VISSOFT52517.2021.00022

Gonçalves PW, Fregnan E, Baum T, Schneider K, Bacchelli A (2022) Empirical Softw. Engg. 27(4). https://doi.org/10.1007/s10664-022-10123-8

Gousios G, Pinzger M, Deursen Av (2014) In: Proceedings of the 36th international conference on software engineering (Association for Computing Machinery, New York, USA), ICSE 2014, pp 345-355. https://doi.org/10.1145/2568225.2568260

Graves T, Karr A, Marron J, Siy H (2000) IEEE Transactions on Software Engineering 26(7):653. https://doi.org/10.1109/32.859533CrossRef

Hampton J (2018) In: Laboratory Psychology (Psychology Press), pp 15–37

Heckman S, Williams L (2008) In: Proceedings of the second ACM-IEEE international symposium on empirical software engineering and measurement (Association for Computing Machinery, New York, USA), ESEM ’08, pp 41–50. https://doi.org/10.1145/1414004.1414013

Heckman S, Williams L (2009) In: 2009 International conference on software testing verification and validation, pp 161–170. https://doi.org/10.1109/ICST.2009.45

Herzig K, Zeller A (2013) In: 2013 10th working conference on mining software repositories (MSR), pp 121–130. https://doi.org/10.1109/MSR.2013.6624018

Hoang T, Dam HK, Kamei Y, Lo D, Ubayashi N (2019) 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). https://doi.org/10.1109/msr.2019.00016

Hoang T, Kang HJ, Lo D, Lawall J (2020) Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering . https://doi.org/10.1145/3377811.3380361

Hovemeyer D, Pugh W (2004) SIGPLAN Not 39(12):92–106. https://doi.org/10.1145/1052883.1052895CrossRef

Janes A, Mairegger M, Russo B (2018) In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering (Association for Computing Machinery, New York, USA, 2018), ASE 2018, pp 876-879. https://doi.org/10.1145/3238147.3240488

Johnson B, Song Y, Murphy-Hill E, Bowdidge R (2013) In: 2013 35th international conference on software engineering (ICSE), pp 672–681. https://doi.org/10.1109/ICSE.2013.6606613

Kamei Y, Shihab E, Adams B, Hassan AE, Mockus A, Sinha A, Ubayashi N (2013) IEEE Trans Softw Eng 39(6):757–773. https://doi.org/10.1109/tse.2012.70CrossRef

Kamei Y, Matsumoto S, Monden A, Matsumoto Ki, Adams B, Hassan AE (2010) In: 2010 IEEE International conference on software maintenance, pp 1–10. https://doi.org/10.1109/ICSM.2010.5609530

Kendall MG (1948) Rank correlation methods (Griffin)

Khanan C, Luewichana W, Pruktharathikoon K, Jiarpakdee J, Tantithamthavorn C, Choetkiertikul M, Ragkhitwetsagul C, Sunetnanta T (2020) In: 2020 35th IEEE/ACM International conference on automated software engineering (ASE), pp 1336–1339

Kim S, Whitehead EJ, Zhang Y (2008) IEEE Transactions on Software Engineering 34(2):181–196. https://doi.org/10.1109/tse.2007.70773CrossRef

Kim S, Ernst MD (2007) In: Proceedings of the the 6th joint meeting of the european software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering (Association for Computing Machinery, New York, USA), ESEC-FSE ’07, pp 45-54. https://doi.org/10.1145/1287624.1287633

Kim S, Zimmermann T, Pan K, Jr. Whitehead EJ (2006) In: 21st IEEE/ACM international conference on automated software engineering (ASE’06), pp 81–90. https://doi.org/10.1109/ASE.2006.23

Kollanus S, Koskinen J (2009) Open Softw Eng J 3(1):15–34. https://doi.org/10.2174/1874107x00903010015CrossRef

Kononenko O, Baysal O, Godfrey MW (2016) In: Proceedings of the 38th international conference on software engineering (Association for Computing Machinery, New York, USA), ICSE ’16, pp 1028-1038. https://doi.org/10.1145/2884781.2884840

Kononenko O, Baysal O, Guerrouj L, Cao Y, Godfrey MW (2015) In: 2015 IEEE international conference on software maintenance and evolution (ICSME), pp 111–120. https://doi.org/10.1109/ICSM.2015.7332457

Lewis C, Lin Z, Sadowski C, Zhu X, Ou R, Whitehead EJ (2013) In: 2013 35th international conference on software engineering (ICSE), pp 372–381. https://doi.org/10.1109/ICSE.2013.6606583

Li PL, Herbsleb J, Shaw M, Robinson B (2006) In: Proceedings of the 28th international conference on software engineering (Association for Computing Machinery, New York, USA), ICSE ’06, pp 413-422. https://doi.org/10.1145/1134285.1134343

Matsumoto S, Kamei Y, Monden A, Matsumoto Ki, Nakamura M (2010) In: Proceedings of the 6th international conference on predictive models in software engineering (Association for Computing Machinery, New York, USA), PROMISE ’10. https://doi.org/10.1145/1868328.1868356

Mcintosh S, Kamei Y (2018) IEEE Transactions on Software Engineering 44(5):412–428. https://doi.org/10.1109/tse.2017.2693980CrossRef

McIntosh S, Kamei Y, Adams B, Hassan AE (2014) In: Proc. of the working conference on mining software repositories (MSR), pp 192–201

Mockus A, Weiss DM (2000) Bell Labs Technical Journal 5(2):169. https://doi.org/10.1002/bltj.2229CrossRef

Oosterwaal S, Deursen Av, Coelho R, Sawant AA, Bacchelli A (2016) In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering (Association for Computing Machinery, New York, USA), FSE 2016, pp 1038-1041. https://doi.org/10.1145/2950290.2983929

Paas F, Tuovinen J, Tabbers H, Van Gerven P (2003) Educational Psychologist - EDUC PSYCHOL 38:63. https://doi.org/10.1207/S15326985EP3801_8CrossRef

Paixao M, Krinke J, Han D, Ragkhitwetsagul C, Harman M (2017) In: 2017 32nd IEEE/ACM international conference on automated software engineering (ASE) , pp 95–105. https://doi.org/10.1109/ASE.2017.8115622

Pascarella L, Spadini D, Palomba F, Bruntink M, Bacchelli A (2018) Proc ACM Hum-Comput Interact 2(CSCW). https://doi.org/10.1145/3274404

Pornprasit C, Tantithamthavorn CK (2021) 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). https://doi.org/10.1109/msr52588.2021.00049

Rezk C, Kamei Y, McIntosh S (2022) IEEE Transactions on Software Engineering 48(9):3297–3309CrossRef

Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) In: Annual meeting of the Southern association for institutional research, pp 1–51

Rutar N, Almazan C, Foster J (2004) In: 15th international symposium on software reliability engineering, pp 245–256. https://doi.org/10.1109/ISSRE.2004.1

Sadowski C, Aftandilian E, Eagle A, Miller-Cushon L, Jaspan C (2018) Commun ACM 61(4):58–66. https://doi.org/10.1145/3188720CrossRef

Sadowski C, Söderberg E, Church L, Sipko M, Bacchelli A (2018) In: 2018 IEEE/ACM 40th international conference on software engineering: software engineering in practice track (ICSE-SEIP), pp 181–190

Sadowski C, Van Gogh J, Jaspan C, Soderberg E, Winter C (2015) In: 2015 IEEE/ACM 37th IEEE international conference on software engineering, vol 1, pp 598–608. https://doi.org/10.1109/ICSE.2015.76

Shihab E, Hassan AE, Adams B, Jiang ZM (2012) In: Proceedings of the ACM SIGSOFT 20th international symposium on the foundations of software engineering (Association for Computing Machinery, New York, USA), FSE ’12. https://doi.org/10.1145/2393596.2393670

Shull F, Seaman C (2008) IEEE Softw 25(1):88. https://doi.org/10.1109/MS.2008.7CrossRef

Spadini D, Bacchelli A (2020) In: 2020 IEEE/ACM 17th international conference on mining software repositories (MSR), pp 528–532. https://doi.org/10.1145/3379597.3387455

Tang H, Lan T, Hao D, Zhang L (2015) In: Proceedings of the 7th Asia-Pacific symposium on internetware (Association for Computing Machinery, New York, USA), Internetware ’15, pp 43-51. https://doi.org/10.1145/2875913.2875922

Tan M, Tan L, Dara S, Mayeux C (2015) In: 2015 IEEE/ACM 37th IEEE international conference on software engineering vol 2, pp 99–108. https://doi.org/10.1109/ICSE.2015.139

Tao Y, Dang Y, Xie T, Zhang D, Kim S (2012) In: Proceedings of the ACM SIGSOFT 20th international symposium on the foundations of software engineering (Association for Computing Machinery, New York, USA), FSE ’12. https://doi.org/10.1145/2393596.2393656

Tao Y, Kim S (2015) In: 2015 IEEE/ACM 12th working conference on mining software repositories, pp 180–190. https://doi.org/10.1109/MSR.2015.24

Thongtanunam P, McIntosh S, Hassan AE, Iida H (2015) In: 2015 IEEE/ACM 12th working conference on mining software repositories pp 168–179. https://doi.org/10.1109/MSR.2015.23

Tomassi DA, Rubio-González C (2021) In: 2021 36th IEEE/ACM international conference on automated software engineering (ASE), pp 292–303. https://doi.org/10.1109/ASE51524.2021.9678535

Tymchuk Y, Mocci A, Lanza M (2015) In: 2015 IEEE 22nd International conference on software analysis, evolution, and reengineering (SANER) , pp 151–160. https://doi.org/10.1109/SANER.2015.7081825

Undefinedliwerski J, Zimmermann T, Zeller A (2005) SIGSOFT Softw. Eng. Notes 30(4):1–5. https://doi.org/10.1145/1082983.1083147CrossRef

Undefinedliwerski J, Zimmermann T, Zeller A (2005) (Association for Computing Machinery, New York, USA), MSR ’05, p 1-5. https://doi.org/10.1145/1083142.1083147

Uquillas Gómez V, Ducasse S, D’Hondt T (2015) Sci Comput Program 98(P3):376–393. https://doi.org/10.1016/j.scico.2013.08.002CrossRef

Wan Z, Xia X, Hassan AE, Lo D, Yin J, Yang X (2020) IEEE Transactions on Software Engineering 46(11):1241. https://doi.org/10.1109/TSE.2018.2877678CrossRef

Zhang T, Song M, Pinedo J, Kim M (2015) In: 2015 IEEE/ACM 37th IEEE international conference on software engineering, vol 1, pp 111–122. https://doi.org/10.1109/ICSE.2015.33

Zimmermann T, Premraj R, Zeller A (2007) In: Third International workshop on predictor models in software engineering (PROMISE’07: ICSE Workshops 2007) , p 9 https://doi.org/10.1109/PROMISE.2007.10

Titel: Studying the impact of risk assessment analytics on risk awareness and code review performance
verfasst von: Xueyao Yu
Filipe R. Cogo
Shane McIntosh
Michael W. Godfrey
Publikationsdatum: 01.03.2024
Verlag: Springer US
Erschienen in: Empirical Software Engineering / Ausgabe 2/2024
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-024-10443-x

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 2/2024

Quantifying and characterizing clones of self-admitted technical debt in build systems

When less is more: on the value of “co-training” for semi-supervised software defect predictors

An empirical study of untangling patterns of two-class dependency cycles

An empirical study on the usage of mocking frameworks in Apache software foundation

A study of common bug fix patterns in Rust

Demystifying API misuses in deep learning applications

Premium Partner