Continuous Test-Driven Development: A Preliminary Empirical Evaluation Using Agile Experimentation in Industrial Settings

Test-Driven Development (TDD) is an agile software development and design practice popularized by the eXtreme Programming methodology. Continuous Test-Driven Development (CTDD), proposed by the authors, is the recent enhancement of the TDD practice and combines TDD with the continuous testing (CT) practice that recommends background testing. Thus CTDD eliminates the need to manually execute the tests by a developer. This paper uses CTDD research to test out the idea of Agile Experimentation. It is a refined approach performing disciplined scientific research in an industrial setting. The objective of this paper is to evaluate the new CTDD practice versus the well-established TDD practice via a Single Case empirical study involving a professional developer in a real, industrial software development project employing Microsoft .NET. We found that there was a slight (4 min) drop in the mean red-to-green time (i.e., time from the moment when any of the tests fails or the project does not build to the time when the project compiles and all the tests are passing), while the size of the CTDD versus TDD effect was non-zero but small (\(d-index=0.22\)). The recorded results are not conclusive but are in accordance with the intuition. By eliminating the mundane need to execute the tests we have made the developer slightly faster. If the developers that use TDD embrace CTDD it can lead to small improvements in their coding performance that, taking into account a number of developers using TDD, could lead to serious savings in the entire company or industry itself.