Dieter Rombach
Abstract
Software inspections, reviews, and walkthroughs have become a standard process component in many software development domains. Maturity level 3 of the CMM-I requires establishment of peer reviews [12] and substantial sustained improvements in quality and productivity have been reported as a result of using reviews ([16], [21], [22], [27]). The NSF Impact project identifies the degree to which these industrial success cases have been instigated and improved by research in software engineering.
This research identifies that there is widespread adoption of inspections, reviews or walkthroughs but that companies do not generally exploit their full potential. However there exist sustained industrial success cases with respect to the wide-spread and measurably successful application of them. It also identifies research in software engineering that can be credibly documented as having influenced the industrial success cases. Credible documentation may exist in the form of publications or documented reports by witnesses. Due to the semi-formal nature of inspections, reviews, and walkthroughs, a specific focus is given to empirical research results as motivators for adoption. Through the examination of one detailed case study, it is shown that software engineering research has had a significant impact on practice and that the impact can be traced in this case from research to that practice. The case study chosen provides evidence of both success and failure regarding sustained application in practice.
Thus the analysis of historic impact chains of research reveals a clear impact of software engineering research on sustained industrial success for inspections, reviews and walkthroughs. More importantly, in impact chains where the empirical results have not been established, we conclude that success has not been achieved or has not been sustained.
The paper closes with (1) lessons learned for creating the sustained use and impact of semi-formal software engineering processes, (2) a request for researchers and practitioners to further consider how their work can improve the effectiveness of research and practice, and (3) a request to contribute additional success cases and impact factors to the authors database for future enhancements of this paper.
- Aurum, A. and Petersson, H. and Wohlin, C., State-of-the-art: software inspections after 25 years, Journal of Software Testing, Verification and Reliability, 12(3), Sept. 2002, pp. 133--54.Google Scholar
Cross Ref
- Beck: Extreme Programming Explained: Embrace Change. Addison Wesley 1999. Google ScholarDigital Library
- Basili, V.R.: Evolving and Packaging Reading Technologies, J. Systems and Software, vol. 38, no. 1, July 1997, pp. 3--12. Google ScholarDigital Library
- Basili, V.R., Caldiera, G., and Rombach, H.D., "Goal Question Metric Paradigm," in Encyclopedia of Software Engineering, vol. 1, J.J. Marciniak, Ed., 1994, pp. 528--532.Google Scholar
- Basili, V., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sorumgard, S., and Zelkowitz, M., 1996. The Empirical Investigation of Perspective-based Reading. Journal of Empirical Software Engineering, 2(1):133164.Google Scholar
- Basili, V.R., McGarry, F.E., Pajerski, R., and Zelkowitz, M.V., Lessons learned from 25 years of process improvement, In proceedings of International Conference on Software Engineering, Orlando, Florida, May 2002, pp. 69--79. Google ScholarDigital Library
- Basili, V.R. and Selby, R.W. Comparing the effectiveness of software testing techniques. IEEE Transactions on Software Engineering, 13(12):12781296, 1987. Google ScholarDigital Library
- Boehm, B., and V.R. Basili, Software Defect Reduction Top 10 List, Computer, vol. 34, no. 1, Jan. 2001, pp. 135--137. Google ScholarDigital Library
- L. Briand et al., Quality Assurance Technologies for the EURO Conversion--Industrial Experience at Allianz Life Assurance, Proc. 2nd Int'l Software Quality Week Europe, Software Research Inc., 1998, pp. 1--23.Google Scholar
- Ciolkowski, M., Shull, F., and Biffl, S.: Practical Experiences in the Design and Conduct of Surveys in Empirical Software Engineering, ESERNET Method and Experience, R. Conradi, ed., LNCS, Springer-Verlag, 2003, Volume 2765 / 2003, August 2003, pp. 104--128.Google Scholar
- Ciolkowski, Laitenberger, Biffl. Software Reviews: The state of the practice, IEEE Software Special Issue on the State of the Practice in Software Engineering, December 2003, pp. 46--51. Google ScholarDigital Library
- CMMI Product Team, Software Engineering Institute, Capability Maturity Model® Integration (CMMISM), Version 1.1, CMU/SEI-2002-TR-012.Google Scholar
- Deming, W.: Out of the crisis, SPC Press, 1982.Google Scholar
- Endres, A., Rombach, H.D., A Handbook of Software and Systems Engineering - Empirical Observations, Laws and Theories, Edison-Wesley, 2003.Google Scholar
- Estublier, J., Leblang, D., Hoek, A., Conradi, R., Clemm, G., Tichy, W., and Wiborg-Weber, D. 2005. Impact of software engineering research on the practice of software configuration management. ACM Trans. Softw. Eng. Methodol. 14, 4 (Oct. 2005), 383--430. Google ScholarDigital Library
- Fagan, M.E.: Design and Code Inspections to Reduce Errors in Program Development, IBM Systems J., vol. 15, no. 3, 1976, pp. 182--211.Google ScholarDigital Library
- Gilb, T. and Graham, D., Software Inspection, Addison-Wesley Publishing Company, 1993. Google ScholarDigital Library
- Hetzel, W.C., 1976. An Experimental Analysis of Program Verification Methods. PhD thesis, University of North Carolina at Chapel Hill, Department of Computer Science. Google ScholarDigital Library
- IEEE Std. 1028-1997 (1998): Standard for Software Reviews, The Institute of Electrical and Electronics Engineering, Inc. ISBN 1-55937-987-1Google Scholar
- Kohli, Robert, O., and Radice, R.A., Low Level Design Inspection Specification, TR 21.629, IBM System Communications Division, Kingston NY 12401.Google Scholar
- Laitenberger, O. and DeBaud, J.: An Encompassing Life-Cycle Centric Survey of Software Review, In Journal of Systems and Software, vol. 50, no. 1, Jan. 2000, pp. 5--31. Google ScholarDigital Library
- Laitenberger, O., Leszak, M., Stoll, D., El Emam, K., Quantitative Modeling of Software Reviews in an Industrial Setting, in Proceedings of the 6th International Software Metrics Symposium. Google ScholarDigital Library
- O. Laitenberger, S. Vegas, and M. Ciolkowski, The State of the Practice of Review and Review Technologies in Germany, VISEK technical report 011.02, 2002.Google Scholar
- Linger, R.C., Mills, H.D., and Witt, B.I., 1979. Structured Programming: Theory and Practice. Addison-Wesley Publishing Company. Google ScholarDigital Library
- Lott, C.M. and Rombach, H.D., "Repeatable Software Engineering Experiments for Comparing Defect-Detection Techniques," 1996.Google Scholar
- Marciniak, J.J., 1994. Reviews and Audits. In: Marciniak, J.J., editor, Encyclopedia of Software Engineering, volume 2, pages 10841090. John Wiley and Sons.Google Scholar
- McGarry, F., Decker, B., Attaining Level 5 in CMM process maturity, IEEE Computer 19(6), Nov/Dec 2002, 27-29 Nov. 2001 Page(s):83--90. Google ScholarDigital Library
- H.D. Mills, "Stepwise Refinement and Verification in Boxstructured systems," IEEE Computer, 1988. Google ScholarDigital Library
- Mills, H: "Cleanroom Engineering", American Programmer, Pages 31--37, May 1991.Google Scholar
- Myers, G.J., 1978. A controlled experiment in program testing and code walkthroughs/inspections. Communications of the ACM, 21(9):760768. Google ScholarDigital Library
- National Institute of Standards and Technology, The Economic Impact of Inadequate Infrastructure for Software Testing, http://www.nist.gov/public_affairs/releases/n02-10.htm, May 2002Google Scholar
- Osterweil et al., The Impact Project: Determining the Impact of Software Engineering Research upon Practice, Software Eng. Notes, vol. 25, no. 6, Nov. 2000, pp. 108--109. Google ScholarDigital Library
- Ryder, B.G., Soffa, M.L., and Burnett, M. 2005. The impact of software engineering research on modern progamming languages. ACM Trans. Softw. Eng. Methodol. 14, 4 (Oct. 2005), 431--477. Google ScholarDigital Library
- Selby, R.W., Basili, V.R., and Baker, F.T., "Cleanroom Software Development: An Empirical Evaluation," IEEE Transactions on Software Engineering, vol. SE-13, pp. 1027--1037, 1987. Google ScholarDigital Library
- Solingen, R. van, Berghout, E., The Goal/Question/Metric Method. A Practical Guide for Quality Improvement of Software Development, London: McGraw-Hill, 1999, ISBN: 0-07-709553-7.Google Scholar
- Strauss, S.H. and Ebenau, R.G.: Software Inspection Process. McGraw Hill Systems Design & Implementation Series. Google ScholarDigital Library
- Votta, L.G.: Does Every Inspection Need a Meeting? ACM Software Eng Notes, vol. 18, no. 5, December 1993, pp. 107--114. Google ScholarDigital Library
- Weinberg, G.M. and Freedman, D.P., 1984. Reviews, Walkthroughs, and Inspections. IEEE Transactions on Software Engineering, 12(1):6872.Google Scholar
- Wiegers, K.: Peer Reviews in Software--A Practical Guide, Addison-Wesley, 2002. Google ScholarDigital Library
- Yourdon, E.: Structured Walkthroughs, N.Y.: Prentice Hall, 4th edition, 1989. Google ScholarDigital Library
Index Terms
- Impact of research on practice in the field of inspections, reviews and walkthroughs: learning from successful industrial uses
Recommendations
Bridging Software Engineering Research and Industrial Practice
There is a long-standing challenge to narrow the gap between software engineering research and industrial practice. This gap is reinforced by a number of challenges, including differing timelines, metrics, expectations, and perceptions of these two ...
Coding for inspections and reviews
XP '18: Proceedings of the 19th International Conference on Agile Software Development: CompanionInspections/reviews are a part of agile software development and have been shown to be a cost-effective means of eliminating faults. However, inspections are not a standard practice in many organizations. It is important to identify characteristics of ...
Good Practices in Aligning Software Engineering Research and Industry Practice
There is a long-standing challenge to narrow the gap between software engineering research and industry practice, to align their interests and realize true synergies between the two communities. Some difficulties to this challenge include mismatched ...
Comments