Skip to main content
Top
Published in: Empirical Software Engineering 4/2020

13-04-2020

Code and commit metrics of developer productivity: a study on team leaders perceptions

Authors: Edson Oliveira, Eduardo Fernandes, Igor Steinmacher, Marco Cristo, Tayana Conte, Alessandro Garcia

Published in: Empirical Software Engineering | Issue 4/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Context

Developer productivity is essential to the success of software development organizations. Team leaders use developer productivity information for managing tasks in a software project. Developer productivity metrics can be computed from software repositories data to support leaders’ decisions. We can classify these metrics in code-based metrics, which rely on the amount of produced code, and commit-based metrics, which rely on commit activity. Although metrics can assist a leader, organizations usually neglect their usage and end up sticking to the leaders’ subjective perceptions only.

Objective

We aim to understand whether productivity metrics can complement the leaders’ perceptions. We also aim to capture leaders’ impressions about relevance and adoption of productivity metrics in practice.

Method

This paper presents a multi-case empirical study performed in two organizations active for more than 18 years. Eight leaders of nine projects have ranked the developers of their teams by productivity. We quantitatively assessed the correlation of leaders’ rankings versus metric-based rankings. As a complement, we interviewed leaders for qualitatively understanding the leaders’ impressions about relevance and adoption of productivity metrics given the computed correlations.

Results

Our quantitative data suggest a greater correlation of the leaders’ perceptions with code-based metrics when compared to commit-based metrics. Our qualitative data reveal that leaders have positive impressions of code-based metrics and potentially would adopt them.

Conclusions

Data triangulation of productivity metrics and leaders’ perceptions can strengthen the organization conviction about productive developers and can reveal productive developers not yet perceived by team leaders and probably underestimated in the organization.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Footnotes
1
Plots of the joint distributions between calculated metric values and rankings (metric-based and leader-informed) are available online at https://​doi.​org/​10.​5281/​zenodo.​3534258
 
Literature
go back to reference Amrit C, Daneva M, Damian D (2014) Human factors in software development: On its underlying theories and the value of learning from related disciplines. a guest editorial introduction to the special issue. Inf Softw Technol (IST) 56(12):1537–1542CrossRef Amrit C, Daneva M, Damian D (2014) Human factors in software development: On its underlying theories and the value of learning from related disciplines. a guest editorial introduction to the special issue. Inf Softw Technol (IST) 56(12):1537–1542CrossRef
go back to reference de Aquino Junior G, Meira S (2009) Towards effective productivity measurement in software projects. In: Proceedings of the 4th International Conference on Software Engineering Advances (ICSEA), pp 241–249 de Aquino Junior G, Meira S (2009) Towards effective productivity measurement in software projects. In: Proceedings of the 4th International Conference on Software Engineering Advances (ICSEA), pp 241–249
go back to reference Basili V, Rombach HD (1988) The tame project: Towards improvement-oriented software environments. IEEE Trans Softw Eng (TSE) 14(6):758–773CrossRef Basili V, Rombach HD (1988) The tame project: Towards improvement-oriented software environments. IEEE Trans Softw Eng (TSE) 14(6):758–773CrossRef
go back to reference Bird C, Nagappan N, Murphy B, Gall H, Devanbu P (2011) Don’t touch my code! examining the effects of ownership on software quality. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE), pp 4–14 Bird C, Nagappan N, Murphy B, Gall H, Devanbu P (2011) Don’t touch my code! examining the effects of ownership on software quality. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE), pp 4–14
go back to reference Boehm BW (1981) Software engineering economics. Prentice Hall, Englewood CliffsMATH Boehm BW (1981) Software engineering economics. Prentice Hall, Englewood CliffsMATH
go back to reference Chand D, Gowda R (1993) An exploration of the impact of individual and group factors on programmer productivity. In: Proceedings of the Conference On Computer Science (CSC), pp 338–345 Chand D, Gowda R (1993) An exploration of the impact of individual and group factors on programmer productivity. In: Proceedings of the Conference On Computer Science (CSC), pp 338–345
go back to reference Chávez A, Ferreira I, Fernandes E, Cedrim D, Garcia A (2017) How does refactoring affect internal quality attributes? a multi-project study. In: Proceedings of the 31st Brazilian Symposium On Software Engineering (SBES), pp 74–83 Chávez A, Ferreira I, Fernandes E, Cedrim D, Garcia A (2017) How does refactoring affect internal quality attributes? a multi-project study. In: Proceedings of the 31st Brazilian Symposium On Software Engineering (SBES), pp 74–83
go back to reference Chrissis MB, Konrad M, Shrum S (2011) CMMI For development: guidelines for process integration and product improvement. Pearson Education Chrissis MB, Konrad M, Shrum S (2011) CMMI For development: guidelines for process integration and product improvement. Pearson Education
go back to reference Cohen J (1988) Statistical power analysis for the behavioural sciences, 2nd edn. Routledge, EvanstonMATH Cohen J (1988) Statistical power analysis for the behavioural sciences, 2nd edn. Routledge, EvanstonMATH
go back to reference Croux C, Dehon C (2010) Influence functions of the spearman and kendall correlation measures. Stat Methods Appl 19(4):497–515MathSciNetCrossRef Croux C, Dehon C (2010) Influence functions of the spearman and kendall correlation measures. Stat Methods Appl 19(4):497–515MathSciNetCrossRef
go back to reference Cruzes D, Dyba T (2011) Recommended steps for thematic synthesis in software engineering. In: Proceedings of the 5th international symposium on Empirical Software Engineering and Measurement (ESEM), pp 275–284 Cruzes D, Dyba T (2011) Recommended steps for thematic synthesis in software engineering. In: Proceedings of the 5th international symposium on Empirical Software Engineering and Measurement (ESEM), pp 275–284
go back to reference De Silva L, Balasubramaniam D (2012) Controlling software architecture erosion: a survey. J Syst Softw (JSS) 85(1):132–151CrossRef De Silva L, Balasubramaniam D (2012) Controlling software architecture erosion: a survey. J Syst Softw (JSS) 85(1):132–151CrossRef
go back to reference Dyer R, Nguyen HA, Rajan H, Nguyen T (2015) Boa: Ultra-large-scale software repository and source-code mining. ACM Trans Softw Eng Methodol (TOSEM) 25(1):7CrossRef Dyer R, Nguyen HA, Rajan H, Nguyen T (2015) Boa: Ultra-large-scale software repository and source-code mining. ACM Trans Softw Eng Methodol (TOSEM) 25(1):7CrossRef
go back to reference Fernandes E, Ferreira LP, Figueiredo E, Valente MT (2017) How clear is your code? an empirical study with programming challenges. In: Proceedings of the Ibero-American conference on software engineering: experimental software engineering track (CIbSE-ESELAW), pp 1–14 Fernandes E, Ferreira LP, Figueiredo E, Valente MT (2017) How clear is your code? an empirical study with programming challenges. In: Proceedings of the Ibero-American conference on software engineering: experimental software engineering track (CIbSE-ESELAW), pp 1–14
go back to reference Fernandes E, Oliveira J, Vale G, Paiva T, Figueiredo E (2016) A review-based comparative study of bad smell detection tools. In: Proceedings of the 20th international conference on Evaluation And Assessment In Software Engineering (EASE), pp 18:1–18:12 Fernandes E, Oliveira J, Vale G, Paiva T, Figueiredo E (2016) A review-based comparative study of bad smell detection tools. In: Proceedings of the 20th international conference on Evaluation And Assessment In Software Engineering (EASE), pp 18:1–18:12
go back to reference Ferreira AIF, Santos G, Cerqueira R, Montoni M, Barreto A, Barreto AOS, Rocha AR (2007) Applying iso 9001: 2000, mps. br and cmmi to achieve software process maturity: Bl informatica’s pathway. In: 29Th International Conference On Software Engineering (ICSE’07). IEEE, pp 642–651 Ferreira AIF, Santos G, Cerqueira R, Montoni M, Barreto A, Barreto AOS, Rocha AR (2007) Applying iso 9001: 2000, mps. br and cmmi to achieve software process maturity: Bl informatica’s pathway. In: 29Th International Conference On Software Engineering (ICSE’07). IEEE, pp 642–651
go back to reference Ferreira M, Valente MT, Ferreira K (2017) A comparison of three algorithms for computing truck factors. In: Proceedings of the 25th International Conference On Program Comprehension (ICPC), pp 207–217 Ferreira M, Valente MT, Ferreira K (2017) A comparison of three algorithms for computing truck factors. In: Proceedings of the 25th International Conference On Program Comprehension (ICPC), pp 207–217
go back to reference Field A (2009) Discovering Statistics using IBM SPSS Statistics, 2nd edn. Sage Publications Ltd, Thousand Oaks, California 91320 Field A (2009) Discovering Statistics using IBM SPSS Statistics, 2nd edn. Sage Publications Ltd, Thousand Oaks, California 91320
go back to reference Gousios G, Kalliamvakou E, Spinellis D (2008) Measuring developer contribution from software repository data. In: Proceedings of the 5th international working conference on Mining Software Repositories (MSR), pp 19–132 Gousios G, Kalliamvakou E, Spinellis D (2008) Measuring developer contribution from software repository data. In: Proceedings of the 5th international working conference on Mining Software Repositories (MSR), pp 19–132
go back to reference Greiler M, Herzig K, Czerwonka J (2015) Code ownership and software quality: a replication study. In: Proceedings of the 12th working conference on Mining Software Repositories (MSR), pp 2–12 Greiler M, Herzig K, Czerwonka J (2015) Code ownership and software quality: a replication study. In: Proceedings of the 12th working conference on Mining Software Repositories (MSR), pp 2–12
go back to reference Halstead M (1977) Elements of software science, 1st edn. Elsevier, AmsterdamMATH Halstead M (1977) Elements of software science, 1st edn. Elsevier, AmsterdamMATH
go back to reference Huselid M (1995) The impact of human resource management practices on turnover, productivity, and corporate financial performance. Acad Manag J (AMJ) 38(3):635–672 Huselid M (1995) The impact of human resource management practices on turnover, productivity, and corporate financial performance. Acad Manag J (AMJ) 38(3):635–672
go back to reference Kendall MG (1955) Rank correlation methods, 2nd edn. Charles Griffin & Co. Ltd., LondonMATH Kendall MG (1955) Rank correlation methods, 2nd edn. Charles Griffin & Co. Ltd., LondonMATH
go back to reference Lawrence M (1981) Programming methodology, organizational environment, and programming productivity. J Syst Softw (JSS) 2(3):257–269CrossRef Lawrence M (1981) Programming methodology, organizational environment, and programming productivity. J Syst Softw (JSS) 2(3):257–269CrossRef
go back to reference Levenshtein V (1966) Binary codes capable of correcting deletions, insertions, and reversals. Dokl Phys 10(8):707–710MathSciNet Levenshtein V (1966) Binary codes capable of correcting deletions, insertions, and reversals. Dokl Phys 10(8):707–710MathSciNet
go back to reference Lokan C (1999) An empirical study of the correlations between function point elements. In: Proceedings of the 6th international software metrics symposium (METRICS), pp 200–206 Lokan C (1999) An empirical study of the correlations between function point elements. In: Proceedings of the 6th international software metrics symposium (METRICS), pp 200–206
go back to reference Lorenz M, Kidd J (1994) Object-oriented software metrics: A practical guide, 1st edn., Prentice Hall, Englewood Cliffs Lorenz M, Kidd J (1994) Object-oriented software metrics: A practical guide, 1st edn., Prentice Hall, Englewood Cliffs
go back to reference Melo C, Cruzes D, Kon F, Conradi R (2011) Agile team perceptions of productivity factors. In: Proceedings of the Agile Conference (Agile2011), pp 57–66 Melo C, Cruzes D, Kon F, Conradi R (2011) Agile team perceptions of productivity factors. In: Proceedings of the Agile Conference (Agile2011), pp 57–66
go back to reference Meyer A, Fritz T, Murphy G, Zimmermann T (2014) Software developers’ perceptions of productivity. In: Proceedings of the 22nd international symposium on Foundations of Software Engineering (FSE), pp 19–29 Meyer A, Fritz T, Murphy G, Zimmermann T (2014) Software developers’ perceptions of productivity. In: Proceedings of the 22nd international symposium on Foundations of Software Engineering (FSE), pp 19–29
go back to reference Mockus A, Fielding R, Herbsleb J (2002) Two case studies of open source software development: Apache and Mozilla. ACM Trans Softw Eng Methodol (TOSEM) 11(3):309–346CrossRef Mockus A, Fielding R, Herbsleb J (2002) Two case studies of open source software development: Apache and Mozilla. ACM Trans Softw Eng Methodol (TOSEM) 11(3):309–346CrossRef
go back to reference Mordal K, Anquetil N, Laval J, Serebrenik A, Vasilescu B, Ducasse S (2013) Software quality metrics aggregation in industry. Soft Evol Process (S:E&P) 25 (10):1117–1135CrossRef Mordal K, Anquetil N, Laval J, Serebrenik A, Vasilescu B, Ducasse S (2013) Software quality metrics aggregation in industry. Soft Evol Process (S:E&P) 25 (10):1117–1135CrossRef
go back to reference Munson J, Elbaum S (1998) Code churn: a measure for estimating the impact of code change. In: Proceedings of the 6th International Conference on Software Maintenance (ICSM), pp 24–31 Munson J, Elbaum S (1998) Code churn: a measure for estimating the impact of code change. In: Proceedings of the 6th International Conference on Software Maintenance (ICSM), pp 24–31
go back to reference Oliveira E, Conte T, Cristo M, Mendes E (2016) Software project managers’ perceptions of productivity factors: Findings from a qualitative study. In: Proceedings of the 10th international symposium on Empirical Software Engineering and Measurement (ESEM), pp 15:1–15:6 Oliveira E, Conte T, Cristo M, Mendes E (2016) Software project managers’ perceptions of productivity factors: Findings from a qualitative study. In: Proceedings of the 10th international symposium on Empirical Software Engineering and Measurement (ESEM), pp 15:1–15:6
go back to reference Oliveira E, Viana D, Cristo M, Conte T (2017) How have software engineering researchers been measuring software productivity? a systematic mapping study. In: Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS), pp 76–87 Oliveira E, Viana D, Cristo M, Conte T (2017) How have software engineering researchers been measuring software productivity? a systematic mapping study. In: Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS), pp 76–87
go back to reference Oliveira R, de Mello R, Fernandes E, Garcia A, Lucena C (2020) Collaborative or individual identification of code smells? on the effectiveness of novice and professional developers. Inf Softw Technol (IST) 106242:120 Oliveira R, de Mello R, Fernandes E, Garcia A, Lucena C (2020) Collaborative or individual identification of code smells? on the effectiveness of novice and professional developers. Inf Softw Technol (IST) 106242:120
go back to reference Ordonez M, Haddad H (2008) The state of metrics in software industry. In: Proceedings of the 5th international conference on Information Technology: New Generations (ITNG), pp 453–458 Ordonez M, Haddad H (2008) The state of metrics in software industry. In: Proceedings of the 5th international conference on Information Technology: New Generations (ITNG), pp 453–458
go back to reference Petersen K (2011) Measuring and predicting software productivity: a systematic map and review. Inf Softw Technol (IST) 53(4):317–343CrossRef Petersen K (2011) Measuring and predicting software productivity: a systematic map and review. Inf Softw Technol (IST) 53(4):317–343CrossRef
go back to reference Rahman F, Devanbu P (2013) How, and why, process metrics are better. In: Proceedings of the 35th International Conference on Software Engineering (ICSE), pp 432–441 Rahman F, Devanbu P (2013) How, and why, process metrics are better. In: Proceedings of the 35th International Conference on Software Engineering (ICSE), pp 432–441
go back to reference Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empiric Soft Eng (EMSE) 14(2):131CrossRef Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empiric Soft Eng (EMSE) 14(2):131CrossRef
go back to reference Runeson P, Host M, Rainer A, Regnell B (2012) Case study research in software engineering: Guidelines and examples, 1st edn. Wiley, New YorkCrossRef Runeson P, Host M, Rainer A, Regnell B (2012) Case study research in software engineering: Guidelines and examples, 1st edn. Wiley, New YorkCrossRef
go back to reference Scholtes I, Mavrodiev P, Schweitzer F (2016) From Aristotle to ringelmann: A large-scale analysis of team productivity and coordination in Open Source Software projects. Empiric Soft Eng (EMSE) 21(2):642–683CrossRef Scholtes I, Mavrodiev P, Schweitzer F (2016) From Aristotle to ringelmann: A large-scale analysis of team productivity and coordination in Open Source Software projects. Empiric Soft Eng (EMSE) 21(2):642–683CrossRef
go back to reference Sheetz S, Henderson D, Wallace L (2009) Understanding developer and manager perceptions of function points and source lines of code. J Syst Softw (JSS) 82 (9):1540–1549CrossRef Sheetz S, Henderson D, Wallace L (2009) Understanding developer and manager perceptions of function points and source lines of code. J Syst Softw (JSS) 82 (9):1540–1549CrossRef
go back to reference Thongtanunam P, McIntosh S, Hassan A, Iida H (2016) Revisiting code ownership and its relationship with software quality in the scope of modern code review. In: Proceedings of the 38th International Conference on Software Engineering (ICSE), pp 1039–1050 Thongtanunam P, McIntosh S, Hassan A, Iida H (2016) Revisiting code ownership and its relationship with software quality in the scope of modern code review. In: Proceedings of the 38th International Conference on Software Engineering (ICSE), pp 1039–1050
go back to reference Trendowicz A, Münch J (2009) Factors influencing software development productivity: State-of-the-art and industrial experiences. Adv Comput 77:185–241CrossRef Trendowicz A, Münch J (2009) Factors influencing software development productivity: State-of-the-art and industrial experiences. Adv Comput 77:185–241CrossRef
go back to reference Verner J, Babar M, Cerpa N, Hall T, Beecham S (2014) Factors that motivate software engineering teams: a four country empirical study. J Syst Softw (JSS) 92(1):115–127CrossRef Verner J, Babar M, Cerpa N, Hall T, Beecham S (2014) Factors that motivate software engineering teams: a four country empirical study. J Syst Softw (JSS) 92(1):115–127CrossRef
go back to reference Wagner S, Ruhe M (2008) A systematic review of productivity factors in software development. In: Proceedings of the 2nd international workshop on Software Productivity Analysis And Cost Estimation (SPACE), pp 1–6 Wagner S, Ruhe M (2008) A systematic review of productivity factors in software development. In: Proceedings of the 2nd international workshop on Software Productivity Analysis And Cost Estimation (SPACE), pp 1–6
go back to reference Weber KC, Araújo EE, da Rocha ARC, Machado CA, Scalet D, Salviano CF (2005) Brazilian software process reference model and assessment method. In: International symposium on computer and information sciences. Springer, pp 402–411 Weber KC, Araújo EE, da Rocha ARC, Machado CA, Scalet D, Salviano CF (2005) Brazilian software process reference model and assessment method. In: International symposium on computer and information sciences. Springer, pp 402–411
go back to reference Wen M, Wu R, Cheung SC (2016) Locus: Locating bugs from software changes. In: Proceedings of the 31st international conference on Automated Software Engineering (ASE), pp 262–273 Wen M, Wu R, Cheung SC (2016) Locus: Locating bugs from software changes. In: Proceedings of the 31st international conference on Automated Software Engineering (ASE), pp 262–273
go back to reference Wloka J, Ryder B, Tip F, Ren X (2009) Safe-commit analysis to facilitate team software development. In: Proceedings of the 31st International Conference on Software Engineering (ICSE), pp 507–517 Wloka J, Ryder B, Tip F, Ren X (2009) Safe-commit analysis to facilitate team software development. In: Proceedings of the 31st International Conference on Software Engineering (ICSE), pp 507–517
go back to reference Wohlin C, Runeson P, Höst M, Ohlsson M, Regnell B, Wesslén A (2012) Experimentation in software engineering, 1st edn. Springer Science & Business Media, BerlinCrossRef Wohlin C, Runeson P, Höst M, Ohlsson M, Regnell B, Wesslén A (2012) Experimentation in software engineering, 1st edn. Springer Science & Business Media, BerlinCrossRef
Metadata
Title
Code and commit metrics of developer productivity: a study on team leaders perceptions
Authors
Edson Oliveira
Eduardo Fernandes
Igor Steinmacher
Marco Cristo
Tayana Conte
Alessandro Garcia
Publication date
13-04-2020
Publisher
Springer US
Published in
Empirical Software Engineering / Issue 4/2020
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI
https://doi.org/10.1007/s10664-020-09820-z

Other articles of this Issue 4/2020

Empirical Software Engineering 4/2020 Go to the issue

Premium Partner