Top

Published in:

2025 | OriginalPaper | Chapter

Benchmarking Ongoing Development Output in Real-Life Software Projects

Authors : Jonathan Streit, Lukas Feye

Published in: Product-Focused Software Process Improvement. Industry-, Workshop-, and Doctoral Symposium Papers

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this case study we compare six different metrics and their suitability for productivity benchmarking on development output level. We use detailed data from four software industry projects performed by one company with overall 264 months of development and 1.1 million source lines of code. Code change, absolute growth and number of commits as well as invested effort are measured in consecutive 3-month periods. This allows us to observe alterations in productivity throughout the course of a project as well as inter-project comparisons. We find correlations between effort and the chosen output metrics as well as significant and explainable productivity differences between projects and project phases.

We also analyze whether the use of a clone detection algorithm can improve measurement by adjusting for copy & paste additions and renamed or moved code, and find that a small benefit exists. The redundancy-adjusted amount of code tokens added or modified seems to be the best metric among the selected, in particular in ongoing development where an already existing codebase is changed. Number of commits and absolute growth may complement the picture.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Strategies and Challenges in Cloud-to-Cloud Migration Using Infrastructure as Code

next chapter A Multi-model Approach for Video Data Retrieval in Autonomous Vehicle Development

With outcome describing the value for the customer and impact the value that flows back to the IT organization.

Projects A, B and D do not start at zero due to sample and prototype code that was created before the actual project start.

The extreme case would be to analyze every commit separately, as in the Git “Lines of code changed” statistics.

From a business point of view, changing the code multiple times does not represent economic value unless it was for a temporarily needed feature.

International Software Benchmarking Standards Group. http://isbsg.org

Armour, P.G.: Beware of counting LOC. Commun. ACM 47 (2004)

Barb, A., Neill, C., Sangwan, R., Piovoso, M.: A statistical study of the relevance of lines of code measures in software projects. Innov. Syst. Software Eng. 10(4) (2014). https://doi.org/10.1007/s11334-014-0231-5

Beck, K.: Outcome over output: also impact and effort (2000). https://medium.com/@kentbeck_7670/outcome-over-output-also-impact-and-effort-8f9eb0ce0dbb

Beck, K., Fowler, M.: Planning Extreme Programming. Addison-Wesley, Boston (2000)

Boehm, B.: Software Cost estimation with COCOMO II (2002)

Deissenboeck, F., Juergens, E., Hummel, B., Wagner, S., Parareda, B.M.Y., Pizka, M.: Tool support for continuous quality control. IEEE Software 25(5) (2008). https://doi.org/10.1109/MS.2008.129

Drouin, N., Badri, M., Touré, F.: Analyzing software quality evolution using metrics: an empirical study on open source software. J. Softw. 8(10) (2013)

Fernandez-Ramil, J., Lehman, M.: Metrics of software evolution as effort predictors - a case study (2000). https://doi.org/10.1109/ICSM.2000.883036

10.

Forsgren, N., Storey, M.A., Maddila, C., Zimmermann, T., Houck, B., Butler, J.: The SPACE of developer productivity. ACM Queue 19 (2021)

11.

Hill, P. (ed.): Practical Software Project Estimation: A Toolkit for Estimating Software Development Effort & Duration. McGraw Hill, New York (2010)

12.

Jones, C.: Estimating Software Costs: Bringing Realism to Estimating, 2nd edn. McGraw-Hill, New York (2007)

13.

Juergens, E., Deissenboeck, F., Hummel, B., Wagner, S.: Do code clones matter? In: 2009 IEEE 31st International Conference on Software Engineering (2009). https://doi.org/10.1109/ICSE.2009.5070547

14.

Kocaguneli, E., Misirli, A.T., Caglayan, B., Bener, A.: Experiences on developer participation and effort estimation. In: 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications (2011). https://doi.org/10.1109/SEAA.2011.71

15.

Myers, E.: An o(nd) difference algorithm and its variations (1985)

16.

Nagappan, N., Ball, T.: Use of relative code churn measures to predict system defect density. In: Proceedings of 27th International Conference on Software Engineering, ICSE 2005 (2005). https://doi.org/10.1109/ICSE.2005.1553571

17.

Oliveira, E., Viana, D., Cristo, M., Conte, T.: How have software engineering researchers been measuring software productivity? - a systematic mapping study. In: Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS. INSTICC, SciTePress (2017). https://doi.org/10.5220/0006314400760087

18.

Mas y Parareda, B., Pizka, M.: Measuring productivity using the infamous lines of code metric. In: The First International Workshop on Software Productivity Analysis and Cost Estimation, Nagoya, Japan, Proceedings (2007)

19.

Pizka, M., Panas, T.: Establishing economic effectiveness through software health-management. In: 1st International Workshop on Software Health Management (SHM 2009), Pasadena, CA (2009)

20.

Richardson, L.F.: Fractals, vol. 1. Cambridge University Press, Cambridge (1993)

21.

Shin, Y., Meneely, A., Williams, L., Osborne, J.A.: Evaluating complexity, code churn, and developer activity metrics as indicators of software vulnerabilities. IEEE Trans. Software Eng. 37(6) (2011). https://doi.org/10.1109/TSE.2010.81

22.

Tan, T., Li, Q., Boehm, B., Yang, Y., He, M., Moazeni, R.: Productivity trends in incremental and iterative software development (2009). https://doi.org/10.1145/1671248.1671250

Title: Benchmarking Ongoing Development Output in Real-Life Software Projects
Authors: Jonathan Streit
Lukas Feye
Publisher: Springer Nature Switzerland
Book: Product-Focused Software Process Improvement. Industry-, Workshop-, and Doctoral Symposium Papers
Print ISBN: 978-3-031-78391-3

Electronic ISBN: 978-3-031-78392-0

Copyright Year: 2025
DOI: https://doi.org/10.1007/978-3-031-78392-0_2

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"