Luca Pascarella
Alberto Bacchelli
Skip Abstract Section
Abstract
Contemporary code review is a widespread practice used by software engineers to maintain high software quality and share project knowledge. However, conducting proper code review takes time and developers often have limited time for review. In this paper, we aim at investigating the information that reviewers need to conduct a proper code review, to better understand this process and how research and tool support can make developers become more effective and efficient reviewers. Previous work has provided evidence that a successful code review process is one in which reviewers and authors actively participate and collaborate. In these cases, the threads of discussions that are saved by code review tools are a precious source of information that can be later exploited for research and practice. In this paper, we focus on this source of information as a way to gather reliable data on the aforementioned reviewers' needs. We manually analyze 900 code review comments from three large open-source projects and organize them in categories by means of a card sort. Our results highlight the presence of seven high-level information needs, such as knowing the uses of methods and variables declared/modified in the code under review. Based on these results we suggest ways in which future code review tools can better support collaboration and the reviewing task.
- Ulrike Abelein and Barbara Paech. 2015. Understanding the Influence of User Participation and Involvement on System Success: a Systematic Mapping Study . Empirical Software Engineering , Vol. 20, 1 (2015), 28--81. Google Scholar
Digital Library
- Alberto Bacchelli and Christian Bird. 2013. Expectations, Outcomes, and Challenges of Modern Code Review. In 35th International Conference on Software Engineering (ICSE 2013a). 710--719. Google ScholarDigital Library
- Richard A Baker Jr. 1997. Code reviews enhance software quality. In Proceedings of the 19th International Conference on Software Engineering. ACM, 570--571. Google ScholarDigital Library
- Vipin Balachandran. 2013. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In Proceedings of the 2013 International Conference on Software Engineering. IEEE Press, 931--940. Google ScholarDigital Library
- Mike Barnett, Christian Bird, Joao Brunet, and Shuvendu K Lahiri. 2015. Helping developers help themselves: Automatic decomposition of code review changesets. In Proceedings of the 2015 International Conference on Software Engineering. IEEE Press . Google ScholarDigital Library
- Tobias Baum, Olga Liskin, Kai Niklas, and Kurt Schneider. 2016. A Faceted Classification Scheme for Change-Based Industrial Code Review Processes. In Software Quality, Reliability and Security (QRS), 2016 IEEE International Conference on. IEEE, Vienna, Austria.Google ScholarCross Ref
- Tobias Baum, Kurt Schneider, and Alberto Bacchelli. 2017. On the Optimal Order of Reading Source Code Changes for Review. In 33rd IEEE International Conference on Software Maintenance and Evolution (ICSME), Proceedings .Google Scholar
- Gabriele Bavota and Barbara Russo. 2015. Four eyes are better than two: On the impact of code reviews on software quality. In 2015 IEEE 31st International Conference on Software Maintenance and Evolution, ICSME 2015 - Proceedings. 81--90. Google ScholarDigital Library
- Olga Baysal, Oleksii Kononenko, Reid Holmes, and Michael W Godfrey. 2013. The influence of non-technical factors on code review. In Reverse Engineering (WCRE), 2013 20th Working Conference on. IEEE, 122--131.Google ScholarCross Ref
- Moritz Beller, Alberto Bacchelli, Andy Zaidman, and Elmar Juergens. 2014. Modern code reviews in open-source projects: which problems do they fix?. In Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 202--211. Google ScholarDigital Library
- Fevzi Belli and Radu Crisan. 1996. Towards automation of checklist-based code-reviews. In Software Reliability Engineering, 1996. Proceedings., Seventh International Symposium on. IEEE, 24--33. Google ScholarDigital Library
- Silvia Breu, Rahul Premraj, Jonathan Sillito, and Thomas Zimmermann. 2010. Information Needs in Bug Reports : Improving Cooperation Between Developers and Users . Proceedings of the 2010 Computer Supported Cooperative Work Conference (2010), 301--310. Google ScholarDigital Library
- Raymond PL Buse and Westley R Weimer. 2010. Automatically documenting program changes. In Proceedings of the IEEE/ACM international conference on Automated software engineering. ACM, 33--42. Google ScholarDigital Library
- Raymond P.L. Buse and Thomas Zimmermann. 2012. Information needs for software development analytics. In Proceedings - International Conference on Software Engineering. 987--996. Google ScholarDigital Library
- Gerardo Canfora, Luigi Cerulo, and Massimiliano Di Penta. 2009. Ldiff: An enhanced line differencing tool. In Proceedings of the 31st International Conference on Software Engineering. IEEE Computer Society, 595--598. Google ScholarDigital Library
- Robert Chatley and Lawrence Jones. 2018. Diggit: Automated code review via software repository mining. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 567--571.Google ScholarCross Ref
- Jason Cohen. 2010. Modern Code Review. In Making Software , , Andy Oram and Greg Wilson (Eds.). O'Reilly, Chapter 18, 329--338.Google Scholar
- W. J. Conover. 1999. Practical Nonparametric Statistics 3rd ed.). John Wiley & Sons, Inc.Google Scholar
- Luis Fernando Cortés-Coy, Mario Linares-Vásquez, Jairo Aponte, and Denys Poshyvanyk. 2014. On automatically generating commit messages via summarization of source code changes. In Source Code Analysis and Manipulation (SCAM), 2014 IEEE 14th International Working Conference on. IEEE, 275--284. Google ScholarDigital Library
- Jacek Czerwonka, Michaela Greiler, and Jack Tilford. 2015. Code Reviews Do Not Find Bugs. How the Current Code Review Best Practice Slows Us Down. In Proceedings of the 2015 International Conference on Software Engineering . IEEE -- Institute of Electrical and Electronics Engineers. http://research.microsoft.com/apps/pubs/default.aspx?id=242201 Google ScholarDigital Library
- Marco di Biase, Magiel Bruntink, Arie van Deursen, and Alberto Bacchelli. 2018. The effects of change-decomposition on code review-A Controlled Experiment. arXiv preprint arXiv:1805.10978 (2018).Google Scholar
- Mart'in Dias, Alberto Bacchelli, Georgios Gousios, Damien Cassou, and Stéphane Ducasse. 2015. Untangling fine-grained code changes. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 341--350.Google Scholar
- Felipe Ebert, Fernando Castor, Nicole Novielli, and Alexander Serebrenik. 2017. Confusion Detection in Code Reviews. In 33rd International Conference on Software Maintenance and Evolution (ICSME), Proceedings. ICSME.Google Scholar
- Deborah Finfgeld-Connett. 2014. Use of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews. Qualitative Research , Vol. 14, 3 (2014), 341--352.Google ScholarCross Ref
- Georgios Gousios, Margaret-Anne Storey, and Alberto Bacchelli. 2016. Work Practices and Challenges in Pull-Based Development: The Contributor's Perspective. In Proceedings of the 38th International Conference on Software Engineering (ICSE '16). ACM, 285--296. Google ScholarDigital Library
- Georgios Gousios, Andy Zaidman, Margaret-Anne Storey, and Arie Van Deursen. 2015. Work practices and challenges in pull-based development: the integrator's perspective. In Proceedings of the 37th International Conference on Software Engineering-Volume 1. IEEE Press, 358--368. Google ScholarCross Ref
- Robert J Grissom and John J Kim. 2005. Effect sizes for research: A broad practical approach. Lawrence Erlbaum Associates Publishers.Google Scholar
- Kazuki Hamasaki, Raula Gaikovina Kula, Norihiro Yoshida, AE Cruz, Kenji Fujiwara, and Hajimu Iida. 2013. Who does what during a code review? datasets of oss peer review repositories. In Proceedings of the 10th Working Conference on Mining Software Repositories. IEEE Press, 49--52. Google ScholarDigital Library
- JD Herbsleb and E Kuwana. 1993. Preserving knowledge in design projects: What designers need to know . Chi '93 & Interact '93 (1993), 7--14. Google ScholarDigital Library
- Kim Herzig and Andreas Zeller. 2013. The impact of tangled code changes. In Proceedings of the 10th Working Conference on Mining Software Repositories. IEEE Press, 121--130. Google ScholarDigital Library
- Yujuan Jiang, Bram Adams, and Daniel M. German. 2013. Will my patch make it? And how fast?: Case study on the linux kernel. In IEEE International Working Conference on Mining Software Repositories . 101--110. Google ScholarDigital Library
- Norihito Kitagawa, Hideaki Hata, Akinori Ihara, Kiminao Kogiso, and Kenichi Matsumoto. 2016. Code review participation: game theoretical modeling of reviewers in gerrit datasets. In Proceedings of the 9th International Workshop on Cooperative and Human Aspects of Software Engineering. ACM, 64--67. Google ScholarDigital Library
- Andrew J. Ko, Robert DeLine, and Gina Venolia. 2007. Information Needs in Collocated Software Development Teams. In 29th International Conference on Software Engineering (ICSE'07). 344--353. Google ScholarCross Ref
- Oleksii Kononenko, Olga Baysal, Latifa Guerrouj, Yaxin Cao, and Michael W. Godfrey. 2015. Investigating code review quality: Do people and participation matter?. In 2015 IEEE 31st International Conference on Software Maintenance and Evolution, ICSME 2015 - Proceedings. 111--120. Google ScholarDigital Library
- Klaus Krippendorff. 2011. Agreement and information in the reliability of coding. Communication Methods and Measures , Vol. 5, 2 (2011), 93--112.Google ScholarCross Ref
- Mike Kuniavsky. 2003. Observing the user experience: a practitioner's guide to user research .Elsevier. Google ScholarDigital Library
- Mika V Mantyla and Casper Lassenius. 2009. What types of defects are really discovered in code reviews? Software Engineering, IEEE Transactions on , Vol. 35, 3 (2009), 430--448. Google ScholarDigital Library
- Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E Hassan. 2014. The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects. In Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 192--201. Google ScholarDigital Library
- Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. 2016. An empirical study of the impact of modern code review practices on software quality . , Vol. 21, 5 (2016), 2146--2189. Google ScholarDigital Library
- Robert K Merton and Patricia L Kendall. 1946. The focused interview. American journal of Sociology , Vol. 51, 6 (1946), 541--557.Google ScholarCross Ref
- Rodrigo Morales, Shane McIntosh, and Foutse Khomh. 2015. Do code review practices impact design quality? a case study of the qt, vtk, and itk projects. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 171--180.Google Scholar
- Hazel E Nelson. 1976. A modified card sorting test sensitive to frontal lobe defects. Cortex , Vol. 12, 4 (1976), 313--324.Google ScholarCross Ref
- Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, Denys Poshyvanyk, and Andrea De Lucia. 2015. Mining version histories for detecting code smells. IEEE Transactions on Software Engineering , Vol. 41, 5 (2015), 462--489.Google ScholarDigital Library
- Fabio Palomba, Annibale Panichella, Andy Zaidman, Rocco Oliveto, and Andrea De Lucia. 2017. The scent of a smell: An extensive comparison between textual and structural smells. IEEE Transactions on Software Engineering (2017). Google ScholarDigital Library
- Chris Parnin and Carsten G”org. 2008. Improving change descriptions with change contexts. In Proceedings of the 2008 international working conference on Mining software repositories. ACM, 51--60. Google ScholarDigital Library
- Luca Ponzanelli, Simone Scalabrino, Gabriele Bavota, Andrea Mocci, Massimiliano Di Penta, Rocco Oliveto, and Michele Lanza. 2017. Supporting Software Developers with a Holistic Recommender System. In Proceedings of ICSE 2017 (39th ACM/IEEE International Conference on Software Engineering) . to be published. Google ScholarDigital Library
- Adam Porter, Harvey Siy, Audris Mockus, and Lawrence Votta. 1998. Understanding the sources of variation in software inspections. ACM Transactions on Software Engineering and Methodology (TOSEM) , Vol. 7, 1 (1998), 41--79. Google ScholarDigital Library
- Achyudh Ram, Anand Ashok Sawant, Marco Castelluccio, and Alberto Bacchelli. 2018. What Makes A Code Change Easier To Review? An Empirical Investigation On Code Change Reviewability. In 26th Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018). forthcoming. Google ScholarDigital Library
- Peter C Rigby and Christian Bird. 2013. Convergent contemporary software peer review practices. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, Saint Petersburg, Russia, 202--212. Google ScholarDigital Library
- Peter C Rigby, Daniel M German, Laura Cowen, and Margaret-Anne Storey. 2014. Peer Review on Open Source Software Projects: Parameters, Statistical Models, and Theory. ACM Transactions on Software Engineering and Methodology (2014), 34. Google ScholarDigital Library
- Caitlin Sadowski, Emma Söderberg, Luke Church, Michal Sipko, and Alberto Bacchelli. 2018. Modern code review: a case study at google. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice. ACM, 181--190. Google ScholarDigital Library
- Chris Sauer, D Ross Jeffery, Lesley Land, and Philip Yetton. 2000. The effectiveness of software development technical reviews: A behaviorally motivated program of research. Software Engineering, IEEE Transactions on , Vol. 26, 1 (2000), 1--14. Google ScholarDigital Library
- Jonathan Sillito, Gail C Murphy, and Kris De Volder. 2006. Questions programmers ask during software evolution tasks. In Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering. ACM, 23--34. Google ScholarDigital Library
- Davide Spadini, Maur'icio Aniche, Margaret-Anne Storey, Magiel Bruntink, and Alberto Bacchelli. 2018. When Testing Meets Code Review: Why and How Developers Review Tests. In Software Engineering (ICSE), 2018 IEEE/ACM 40th International Conference on. to appear. Google ScholarDigital Library
- Andrew Sutherland and Gina Venolia. 2009a. Can peer code reviews be exploited for later information needs?. In Software Engineering-Companion Volume, 2009. ICSE-Companion 2009. 31st International Conference on. IEEE, 259--262.Google ScholarCross Ref
- Andrew Sutherland and Gina Venolia. 2009b. Can peer code reviews be exploited for later information needs?. In Software Engineering-Companion Volume, 2009. ICSE-Companion 2009. 31st International Conference on. IEEE, 259--262.Google ScholarCross Ref
- Yida Tao, Yingnong Dang, Tao Xie, Dongmei Zhang, and Sunghun Kim. 2012. How do software engineers understand code changes?: an exploratory study in industry. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM. Google ScholarDigital Library
- Yida Tao and Sunghun Kim. 2015. Partitioning composite code changes to facilitate code review. In Proceedings of the 12th Working Conference on Mining Software Repositories. IEEE Press, 180--190. Google ScholarDigital Library
- Patanamon Thongtanunam, Raula Gaikovina Kula, Ana Erika Camargo Cruz, Norihiro Yoshida, and Hajimu Iida. 2014. Improving code review effectiveness through reviewer recommendations. In Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering. ACM, 119--122. Google ScholarDigital Library
- Patanamon Thongtanunam, Shane McIntosh, Ahmed E Hassan, and Hajimu Iida. 2015a. Investigating Code Review Practices in Defective Files: An Empirical Study of the Qt System. In MSR '15 Proceedings of the 12th Working Conference on Mining Software Repositories . Google ScholarDigital Library
- Patanamon Thongtanunam, Shane Mcintosh, Ahmed E. Hassan, and Hajimu Iida. 2016a. Review Participation in Modern Code Review . Empirical Software Engineering (EMSE) (2016), to appear. Google ScholarDigital Library
- Patanamon Thongtanunam, Shane McIntosh, Ahmed E Hassan, and Hajimu Iida. 2016b. Revisiting code ownership and its relationship with software quality in the scope of modern code review. In Proceedings of the 38th international conference on software engineering. ACM, 1039--1050. Google ScholarDigital Library
- Patanamon Thongtanunam, Chakkrit Tantithamthavorn, Raula Gaikovina Kula, Norihiro Yoshida, Hajimu Iida, and Ken-ichi Matsumoto. 2015b. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 141--150.Google Scholar
- Jason Tsay, Laura Dabbish, and James Herbsleb. 2014. Let's talk about it: evaluating contributions through discussion in GitHub. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 144--154. Google ScholarDigital Library
- Yuriy Tymchuk, Andrea Mocci, and Michele Lanza. 2015. Code Review: Veni, ViDI, Vici. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 151--160.Google Scholar
- Bogdan Vasilescu, Daryl Posnett, Baishakhi Ray, Mark GJ van den Brand, Alexander Serebrenik, Premkumar Devanbu, and Vladimir Filkov. 2015. Gender and tenure diversity in GitHub teams. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3789--3798. Google ScholarDigital Library
- Martin Vechev, Eran Yahav, et almbox. 2016. Programming with “Big Code”. Foundations and Trends® in Programming Languages , Vol. 3, 4 (2016), 231--284. Google ScholarDigital Library
- Robert S Weiss. 1995. Learning from strangers: The art and method of qualitative interview studies .Simon and Schuster.Google Scholar
- Kenji Yamauchi, Jiachen Yang, Keisuke Hotta, Yoshiki Higo, and Shinji Kusumoto. 2014. Clustering commits for understanding the intents of implementation. In Software Maintenance and Evolution (ICSME), 2014 IEEE International Conference on. IEEE, 406--410. Google ScholarDigital Library
Index Terms
- Information Needs in Contemporary Code Review
Recommendations
Code review quality: how developers see it
ICSE '16: Proceedings of the 38th International Conference on Software EngineeringIn a large, long-lived project, an effective code review process is key to ensuring the long-term quality of the code base. In this work, we study code review practices of a large, open source project, and we investigate how the developers themselves ...
Code smells detection via modern code review: a study of the OpenStack and Qt communities
AbstractCode review plays an important role in software quality control. A typical review process involves a careful check of a piece of code in an attempt to detect and locate defects and other quality issues/violations. One type of issue that may impact ...
Understanding code snippets in code reviews: a preliminary study of the OpenStack community
ICPC '22: Proceedings of the 30th IEEE/ACM International Conference on Program ComprehensionCode review is a mature practice for software quality assurance in software development with which reviewers check the code that has been committed by developers, and verify the quality of code. During the code review discussions, reviewers and ...
Comments