Thirumalesh Bhat
Nachiappan Nagappan
ABSTRACT
This paper discusses software development using the Test Driven Development (TDD) methodology in two different environments (Windows and MSN divisions) at Microsoft. In both these case studies we measure the various context, product and outcome measures to compare and evaluate the efficacy of TDD. We observed a significant increase in quality of the code (greater than two times) for projects developed using TDD compared to similar projects developed in the same organization in a non-TDD fashion. The projects also took at least 15% extra upfront time for writing the tests. Additionally, the unit tests have served as auto documentation for the code when libraries/APIs had to be used as well as for code maintenance.
- "IEEE Std 982.2-1988 IEEE guide for the use of IEEE standard dictionary of measures to produce reliable software," 1988.Google Scholar
- V. Basili, Shull, F.,Lanubile, F., "Building Knowledge through Families of Experiments", IEEE Transactions on Software Engineering, 25(4), pp. 456--472, 1999 Google ScholarDigital Library
- K. Beck, Extreme Programming Explained, Embrace Change: Addison Wesley, 2000. Google ScholarDigital Library
- K. Beck, Test Driven Development- by Example. Boston: Addison-Wesley, 2003. Google ScholarDigital Library
- J. Carver, Jaccheri, L., Morasca, S., and Shull, F., "Issues Using Students in Empirical Studies in Software Engineering Education ", Proceedings of IEEE Metrics, pp. 239--249, 2003. Google ScholarDigital Library
- B. Curtis, "Three Problems Overcome with Behavioral Models of the Software Development Process (Panel)", Proceedings of International Conference on Software Engineering, Pittsburgh, PA, pp. 398--399, 1989. Google ScholarDigital Library
- H. Erdogmus, Morisio, M., Torchiano, M., "On the effectiveness of the Test-First Approach to Programming", IEEE Transactions in Software Engineering, 31(3), pp. 226--237, 2005. Google ScholarDigital Library
- D. Gelperin and W. Hetzel, "Software Quality Engineering", Proceedings of Fourth International Conference on Software Testing, Washington D.C., June 1987.Google Scholar
- B. George and L. Williams, "An Initial Investigation of Test- Driven Development in Industry", Proceedings of ACM Symposium on Applied Computing, Melbourne, FL, pp. 1135--1139, 2003. Google ScholarDigital Library
- B. George and L. Williams, "A Structured Experiment of Test-Driven Development", Information and Software Technology (IST), 46(5), pp. 337--342, 2003.Google ScholarCross Ref
- W. S. Humphrey, Managing the Software Process. Reading, Massachusetts: Addison-Wesley, 1989. Google ScholarDigital Library
- C. Larman and V. Basili, "A History of Iterative and Incremental Development", IEEE Computer, 36(6), pp. 47--56, June 2003. Google ScholarDigital Library
- M. M. Müller and O. Hagner, "Experiment about Test-first Programming", Proceedings of Conference on Empirical Assessment in Software Engineering (EASE), 2002.Google ScholarCross Ref
- M. M. Müller and W. F. Tichy, "Case Study: Extreme Programming in a University Environment", Proceedings of 23rd International Conference on Software Engineering, pp. 537--544, May 2001. Google ScholarDigital Library
- L. Williams, Krebs, W., Layman, L., "Extreme Programming Evaluation Framework for Object-Oriented Languages -- Version 1.1," Technical Report, North Carolina State University, NCSU CSC TR-2003-20, 2003.Google Scholar
- L. Williams, E. M. Maximilien, and M. Vouk, "Test-Driven Development as a Defect-Reduction Practice", Proceedings of IEEE International Symposium on Software Reliability Engineering, Denver, CO, pp. 34--45, 2003. Google ScholarDigital Library
Index Terms
- Evaluating the efficacy of test-driven development: industrial case studies
Recommendations
Test driven development: the state of the practice
ACM-SE '12: Proceedings of the 50th Annual Southeast Regional ConferenceTest-Driven Development has been a practice used primarily in agile software development circles for a little more than a decade now. In software development circles, this is a relatively young and immature practice. How much acceptance has it gained it ...
Test-Driven Development: Concepts, Taxonomy, and Future Direction
Test-driven development creates software in very short iterations with minimal upfront design. Poised for widespread adoption TDD has become the focus of an increasing number of researchers and developers.
Does Test-Driven Development Improve the Program Code? Alarming Results from a Comparative Case Study
Balancing Agility and Formalism in Software EngineeringIt is suggested that test-driven development (TDD) is one of the most fundamental practices in agile software development, which produces loosely coupled and highly cohesive code. However, how the TDD impacts on the structure of the program code have ...
Reviews
Andrew Brooks
Test-driven development (TDD) reduces defect density by at least a factor of two at the expense of increasing coding time by 15 to 35 percent. Finally, we have measures on which to base project management decisions. Or do we__?__ The measures derive from two industrial case studies at Microsoft called project A and project B. Defect densities from these two projects were compared with defect densities from comparable projects that employed a non-TDD methodology. Managers supplied the estimates of how much employing TDD had increased coding time. However, we do not know about the operational profiles of the software. Would using the comparable projects more extensively have resulted in the discovery of more defects__?__ Also, we do not know about other software quality assurance activities. What did the test teams do__?__ Developer ability can vary considerably. Yet, project A had six developers, while its comparable project had two. Complexity typically correlates with lines of code (LOC). Yet, project B was one-fifth the size of its comparable project in terms of source LOC. Finally, we do not know about defect severity. Do measured improvements rely on the inclusion of minor defects__?__ The notion of comparable project used by the authors is fatally flawed, and too much is unknown about the data. The reader cannot simply accept that TDD reduces defect density by at least a factor of two. This paper will only interest those researching test-driven development. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments