Peter C. Rigby
Christian Bird
ABSTRACT
Software peer review is practiced on a diverse set of software projects that have drastically different settings, cultures, incentive systems, and time pressures. In an effort to characterize and understand these differences we examine two Google-led projects, Android and Chromium OS, three Microsoft projects, Bing, Office, and MS SQL, and projects internal to AMD. We contrast our findings with data taken from traditional software inspection conducted on a Lucent project and from open source software peer review on six projects, including Apache, Linux, and KDE. Our measures of interest include the review interval, the number of developers involved in review, and proxy measures for the number of defects found during review. We find that despite differences among projects, many of the characteristics of the review process have independently converged to similar values which we think indicate general principles of code review practice. We also introduce a measure of the degree to which knowledge is shared during review. This is an aspect of review practice that has traditionally only had experiential support. Our knowledge sharing measure shows that conducting peer review increases the number of distinct files a developer knows about by 66% to 150% depending on the project. This paper is one of the first studies of contemporary review in software firms and the most diverse study of peer review to date.
- J. Asundi and R. Jayant. Patch review processes in open source software development communities: A comparative case study. In HICSS: Proceedings of the 40th Annual Hawaii International Conference on System Sciences, page 10, 2007. Google Scholar
Digital Library
- A. Bacchelli and C. Bird. Expectations, outcomes, and challenges of modern code review. In Proceedings of the International Conference on Software Engineering. IEEE, 2013. Google ScholarDigital Library
- C. Bird, A. Gourley, and P. Devanbu. Detecting patch submission and acceptance in oss projects. In MSR: Proceedings of the Fourth International Workshop on Mining Software Repositories, page 4. IEEE Computer Society, 2007. Google ScholarDigital Library
- D. Bisant and J. Lyle. A two-person inspection method to improve programming productivity. IEEE Transactions on Software Engineering, 15(10):1294–1304, 1989. Google ScholarDigital Library
- F. Buck. Indicators of quality inspections. IBM Syst. Commun. Division, Tech. Rep. TR, 21, 1981.Google Scholar
- J. Cohen. Best Kept Secrets of Peer Code Review. Smart Bear Inc., 2006.Google Scholar
- T. DeMarco. Contolling Software Projects: Management, Measurement, and Estimation. Prentice Hall, 1986. Google ScholarDigital Library
- P. M. Duvall, S. Matyas, and A. Glover. Continuous integration: improving software quality and reducing risk. Addison-Wesley Professional, 2007. Google ScholarDigital Library
- M. Fagan. Design and Code Inspections to Reduce Errors in Program Development. IBM Systems Journal, 15(3):182–211, 1976. Google ScholarDigital Library
- M. Fagan. A history of software inspections. Software pioneers: contributions to software engineering, Springer-Verlag, Inc., pages 562–573, 2002. Google ScholarDigital Library
- Gerrit. Web based code review and project management for git based projects. http://code.google.com/p/gerrit/.Google Scholar
- P. M. Johnson. Reengineering inspection. ACM Communications, 41(2):49–52, 1998. Google ScholarDigital Library
- P. Kampstra. Beanplot: A boxplot alternative for visual comparison of distributions. Journal of Statistical Software, Code Snippets 1, 28:1–9, 2008.Google Scholar
- C. Larman and V. Basili. Iterative and incremental developments: a brief history. Computer, 36(6):47 – 56, June 2003. Google ScholarDigital Library
- A. Mockus, R. T. Fielding, and J. Herbsleb. Two case studies of open source software development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology, 11(3):1–38, 2002. Google ScholarDigital Library
- A. Mockus and J. D. Herbsleb. Expertise browser: a quantitative approach to identifying expertise. In ICSE: Proceedings of the 24th International Conference on Software Engineering, pages 503–512. ACM Press, 2002. Google ScholarDigital Library
- M. Mukadam, C. Bird, and P. C. Rigby. Gerrit software code review data from android. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR ’13, pages 45–48, Piscataway, NJ, USA, 2013. IEEE Press. Google ScholarDigital Library
- M. Nurolahzade, S. M. Nasehi, S. H. Khandkar, and S. Rawal. The role of patch review in software evolution: an analysis of the mozilla firefox. In International Workshop on Principles of Software Evolution, pages 9–18, 2009. Google ScholarDigital Library
- A. Porter, H. Siy, A. Mockus, and L. Votta. Understanding the sources of variation in software inspections. ACM Transactions Software Engineering Methodology, 7(1):41–79, 1998. Google ScholarDigital Library
- J. Ratcliffe. Moving software quality upstream: The positive impact of lightweight peer code review. In Pacific NW Software Quality Conference, 2009.Google Scholar
- E. S. Raymond. The Cathedral and the Bazaar. O’Reilly and Associates, 1999. Google ScholarDigital Library
- P. Rigby, B. Cleary, F. Painchaud, M. Storey, and D. German. Contemporary peer review in action: Lessons from open source development. Software, IEEE, 29(6):56 –61, nov.-dec. 2012. Google ScholarDigital Library
- P. C. Rigby. Understanding Open Source Software Peer Review: Review Processes, Parameters and Statistical Models, and Underlying Behaviours and Mechanisms. http://hdl.handle.net/1828/3258 University of Victoria, Canada, Dissertation, 2011. Google ScholarDigital Library
- P. C. Rigby, D. M. German, and M.-A. Storey. Open source software peer review practices: A case study of the apache server. In ICSE: Proceedings of the 30th international conference on Software Engineering, pages 541–550, 2008. Google ScholarDigital Library
- P. C. Rigby and M.-A. Storey. Understanding broadcast based peer review on open source software projects. In Proceeding of the 33rd international conference on Software engineering, ICSE ’11, pages 541–550, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- G. V. Rossum. An open source app: Rietveld code review tool. https://developers.google.com/ appengine/articles/rietveld.Google Scholar
- C. Sauer, D. R. Jeffery, L. Land, and P. Yetton. The Effectiveness of Software Development Technical Reviews: A Behaviorally Motivated Program of Research. IEEE Transactions Software Engineering, 26(1):1–14, 2000. Google ScholarDigital Library
- R. Schwartz. Interview with Shawn Pearce, Google Engineer, on FLOSS Weekly. http://www.youtube.com/watch?v=C3MvAQMhC_M.Google Scholar
- L. G. Votta. Does every inspection need a meeting? SIGSOFT Softw. Eng. Notes, 18(5):107–114, 1993. Google ScholarDigital Library
- E. Weller. Lessons from three years of inspection data. IEEE Software, 10(5):38–45, 1993. Google ScholarDigital Library
- K. E. Wiegers. Peer Reviews in Software: A Practical Guide. Addison-Wesley Information Technology Series. Addison-Wesley, 2001.Google Scholar
- R. K. Yin. Case Study Research: Design and Methods, volume 5 of Applied Social Research Methods Series. Sage Publications Inc., 3 edition, 2003.Google Scholar
Index Terms
- Convergent contemporary software peer review practices
Recommendations
Open source software peer review practices: a case study of the apache server
ICSE '08: Proceedings of the 30th international conference on Software engineeringPeer review is seen as an important quality assurance mechanism in both industrial development and the open source software (OSS) community. The techniques for performing inspections have been well studied in industry; in OSS development, peer reviews ...
Comparative case studies of open source software peer review practices
ContextThe power of open source software peer review lies in the involvement of virtual communities, especially users who typically do not have a formal role in the development process. As communities grow to a certain extent, how to organize and ...
Exploring the evolution of software practices
ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringWhen software products and services are developed and maintained over longer time, software engineering practices tend to drift away from both structured and agile methods. Nonetheless, in many cases the evolving practices are far from ad hoc or ...
Comments