nach oben

Empirical Software Engineering

Erschienen in:

01.05.2022

The effects of continuous integration on software development: a systematic literature review

verfasst von: Eliezio Soares, Gustavo Sizilio, Jadson Santos, Daniel Alencar da Costa, Uirá Kulesza

Erschienen in: Empirical Software Engineering | Ausgabe 3/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Context

Continuous integration (CI) is a software engineering technique that proclaims frequent activities to assure the software product health. Researchers and practitioners mention several benefits related to CI. However, no systematic study surveys state of the art regarding such benefits or cons.

Objective

This study aims to identify and interpret empirical evidence regarding how CI impacts software development.

Method

Through a Systematic Literature Review, we search for studies in six digital libraries. Starting from 479 studies, we select 101 empirical studies that evaluate CI in the context of software development. We thoroughly read and extract information regarding (i) the CI environment, (ii) findings related to the effects of CI, and (iii) the employed research methods. We apply a thematic synthesis to group and summarize the findings.

Results

Existing research has explored the positive effects of CI, such as better cooperation, or negative effects, such as adding technical and process challenges. From our thematic synthesis, we identify six themes: development activities, software process, quality assurance, integration patterns, issues & defects, and build patterns.

Conclusions

Empirical research in CI has been increasing over recent years. We found that much of the existing research reveals that CI brings positive effects to software development. However, CI may also bring technical challenges to software development teams. Despite the overall positive outlook regarding CI, we still find room for improvements in the existing empirical research that evaluates the effects of CI.

Vorheriger Artikel Analyzing developer contributions using artifact traceability graphs

Nächster Artikel Search-based fairness testing for regression-based machine learning systems

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Available at https://www.mendeley.com/

https://travis-ci.com/

https://icsme2020.github.io/cfp/ArtifactROSETrackCFP.html

Beck K, Andres C (2005) Extreme programming explained: embrace change. Addison-Wesley, Boston

Bernardo JH, Costa DAD, Kulesza U (2018) Studying the impact of adopting continuous integration on the delivery time of pull requests. In: Proceedings of the 15th international conference on mining software repositories—MSR 18. https://doi.org/10.1145/3196398.3196421

Beller M, Gousios G, Zaidman A (2017) TravisTorrent: synthesizing travis CI and GitHub for full-stack research on continuous integration. In: Proceedings of the 14th working conference on mining software repositories. http://www.st.ewi.tudelft.nl/~mbeller/publications/2017_beller_gousios_zaidman_travistorrent_synthesizing_travis_ci_and_github_for_full-stack_research_on_continuous_integration.pdf Accessed 22 July 2020

Chen L, Babar MA, Zhang H (2010) Towards an evidence-based understanding of electronic data sources. https://doi.org/10.14236/ewic/ease2010.17

CI Theatre (2017) In: ThoughtWorks. https://www.thoughtworks.com/radar/techniques/ci-theatre. Accessed 25 Aug 2020

Cohen J (1968) Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 70:213–220. https://doi.org/10.1037/h0026256CrossRef

Cruzes DS, Dybå T (2011) Recommended steps for thematic synthesis in software engineering. In: 2011 International symposium on empirical software engineering and measurement. https://doi.org/10.1109/esem.2011.36

Debbiche A, Dienér M, Svensson RB (2014) Challenges when adopting continuous integration: a case study. In: Product-focused software process improvement lecture notes in computer science. https://doi.org/10.1007/978-3-319-13835-0, pp 17–32

Dikert K, Paasivaara M, Lassenius C (2016) Challenges and success factors for large-scale agile transformations: a systematic literature review. J Syst Softw 119:87–108. https://doi.org/10.1016/j.jss.2016.06.013CrossRef

Duvall PM, Matyas S, Glover A (2013) Continuous integration improving software quality and reducing risk. Addison-Wesley, Upper Saddle River

Duvall PM (2018) Continuous delivery—patterns and anti-patterns in the software lifecycle. In: dzone.com. https://dzone.com/refcardz/continuous-delivery-patterns. Accessed 7 Feb 2021

Dybå T, Dingsøyr T (2008) Strength of evidence in systematic reviews in software engineering. In: Proceedings of the second ACM-IEEE international symposium on empirical software engineering and measurement—ESEM 08. https://doi.org/10.1145/1414004.1414034

Dybå T, Dingsoyr T, Hanssen GK (2007) Applying systematic reviews to diverse study types: an experience report. In: First international symposium on empirical software engineering and measurement (ESEM 2007). https://doi.org/10.1109/esem.2007.59

Easterbrook S, Singer J, Storey MA, Damian D (2008) Selecting empirical methods for software engineering research. In: Guide to advanced empirical software engineering. Springer, London, pp 285–311

Embury SM, Page C (2019) Effect of continuous integration on build health in undergraduate team projects. In: Software engineering aspects of continuous development and new paradigms of software production and deployment lecture notes in computer science. https://doi.org/10.1007/978-3-030-06019-0

Felidre W, Furtado L, Costa DAD et al (2019) Continuous integration theater. In: 2019 ACM/IEEE international symposium on empirical software engineering and measurement (ESEM). https://doi.org/10.1109/esem.2019.8870152

Fitzgerald B, Stol K-J (2017) Continuous software engineering: a roadmap and agenda. J Syst Softw 123:176–189. https://doi.org/10.1016/j.jss.2015.06.063CrossRef

Fowler M (2006) Continuous integration. In: Martinfowler.com. https://martinfowler.com/articles/continuousIntegration.html. Accessed 29 Jun 2020

Fowler M (2013) Extremeprogramming. In: Martinfowler.com. https://martinfowler.com/bliki/ExtremeProgramming.html

Fowler M (2017) Continuousintegrationcertification. In: Martinfowler.com. https://martinfowler.com/bliki/ContinuousIntegrationCertification.html. Accessed: 26 Jun 2020

Ghaleb TA, Costa DAD, Zou Y (2019) An empirical study of the long duration of continuous integration builds. Empir Softw Eng 24:2102–2139. https://doi.org/10.1007/s10664-019-09695-9CrossRef

Hilton M, Tunnell T, Huang K, et al. (2016) Usage, costs, and benefits of continuous integration in open-source projects. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering—ASE 2016. https://doi.org/10.1145/2970276.2970358

Holmstrom H, Conchuir E, Agerfalk P, Fitzgerald B (2006) Global software development challenges: a case study on temporal, geographical and socio-cultural distance. In: 2006 IEEE international conference on global software engineering (ICGSE06). https://doi.org/10.1109/icgse.2006.261210

Johanssen JO, Kleebaum A, Paech B, Bruegge B (2018) Practitioners eye on continuous software engineering. In: Proceedings of the 2018 international conference on software and system process—ICSSP 18. https://doi.org/10.1145/3202710.3203150

Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.471. Accessed 26 June 2020

Kitchenham BA, Tore D, Magne J (2004) Evidence-based software engineering. https://doi.org/10.1109/ICSE.2004.1317449

Laukkanen E, Itkonen J, Lassenius C (2017) Problems, causes and solutions when adopting continuous delivery—a systematic literature review. Inf Softw Technol 82:55–79. https://doi.org/10.1016/j.infsof.2016.10.001CrossRef

Leppanen M, Makinen S, Pagels M, et al. (2015) The highways and country roads to continuous deployment. IEEE Softw 32:64–72. https://doi.org/10.1109/ms.2015.50CrossRef

Meedeniya DA, D I, Perera I (2019) Software artefacts consistency management towards continuous integration: a roadmap. Int J Adva Comput Sci Appl. https://doi.org/10.14569/ijacsa.2019.0100411

Pinto G, Castor F, Bonifacio R, Rebouças M (2018) Work practices and challenges in continuous integration: a survey with Travis CI users. Softw: Pract Exp 48:2223–2236. https://doi.org/10.1002/spe.2637

Rausch T, Hummer W, Leitner P, Schulte S (2017) An empirical analysis of build failures in the continuous integration workflows of Java-based open-source software. In: 2017 IEEE/ACM 14th international conference on mining software repositories (MSR). https://doi.org/10.1109/msr.2017.54

Robles G (2010) Replicating MSR: a study of the potential replicability of papers published in the mining software repositories proceedings. In: 2010 7th IEEE working conference on mining software repositories (MSR 2010). https://doi.org/10.1109/msr.2010.5463348

Rodríguez-Pérez G, Robles G, González-Barahona JM (2018) Reproducibility and credibility in empirical software engineering: a case study based on a systematic literature review of the use of the SZZ algorithm. Inf Softw Technol 99:164–176. https://doi.org/10.1016/j.infsof.2018.03.009CrossRef

Rogers RO (2004) Scaling continuous integration. In: Extreme programming and agile processes in software engineering lecture notes in computer science, pp 68–76

Shahin M, Babar MA, Zhu L (2017) Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE Access 5:3909–3943. https://doi.org/10.1109/access.2017.2685629CrossRef

Soares E, Sizilio G, Santos J, Alencar D, Kulesza U (2021) SLR artifacts—continuous integration quality impacts (v.1.0.2) [Data set]. In: Zenodo. https://doi.org/10.5281/zenodo.4545623. Accessed: 10 Aug 2021

Ståhl D, Bosch J (2013) Experienced benefits of continuous integration in industry software product development: a case study. Artificial intelligence and applications/794: modelling, identification and control/795: parallel and distributed computing and networks/796: software engineering/792: web-based education. https://doi.org/10.2316/p.2013.796-012

Ståhl D, Bosch J (2014a) Modeling continuous integration practice differences in industry software development. J Syst Softw 87:48–59. https://doi.org/10.1016/j.jss.2013.08.032CrossRef

Ståhl D, Bosch J (2014b) Automated software integration flows in industry: a multiple-case study. In: Companion proceedings of the 36th international conference on software engineering—ICSE Companion 2014. https://doi.org/10.1145/2591062.2591186

Vasilescu B, Yu Y, Wang H, et al. (2015) Quality and productivity outcomes relating to continuous integration in GitHub. In: Proceedings of the 2015 10th joint meeting on foundations of software engineering—ESEC/FSE 2015. https://doi.org/10.1145/2786805.2786850

Vassallo C, Palomba F, Gall HC (2018) Continuous refactoring in CI: a preliminary study on the perceived advantages and barriers. In: 2018 IEEE international conference on software maintenance and evolution (ICSME). https://doi.org/10.1109/icsme.2018.00068

Vassallo C, Proksch S, Gall HC, Penta MD (2019) Automated reporting of anti-patterns and decay in continuous integration. In: 2019 IEEE/ACM 41st international conference on software engineering (ICSE). https://doi.org/10.1109/icse.2019.00028

Viggiato M, Oliveira J, Figueiredo E, et al. (2019) Understanding similarities and differences in software development practices across domains. In: 2019 ACM/IEEE 14th international conference on global software engineering (ICGSE). https://doi.org/10.1109/icgse.2019.00013

Volf Z, Shmueli E (2017) Screening heuristics for project gating systems. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering—ESEC/FSE 2017. https://doi.org/10.1145/3106237.3117766

Wheelan SA, Hochberger JM (1996) Validation studies of the group development questionnaire. Small Group Res 27(1):143–170CrossRef

Yu Y, Yin G, Wang T et al (2016) Determinants of pull-based development in the context of continuous integration. Sci China Inf Sci. https://doi.org/10.1007/s11432-016-5595-8

Zampetti F, Scalabrino S, Oliveto R et al (2017) How open source projects use static code analysis tools in continuous integration pipelines. In: 2017 IEEE/ACM 14th international conference on mining software repositories (MSR). https://doi.org/10.1109/msr.2017.2

Zhao Y, Serebrenik A, Zhou Y, et al. (2017) The impact of continuous integration on other software development practices: a large-scale empirical study. In: 2017 32nd IEEE/ACM international conference on automated software engineering (ASE). https://doi.org/10.1109/ase.2017.8115619

Titel: The effects of continuous integration on software development: a systematic literature review
verfasst von: Eliezio Soares
Gustavo Sizilio
Jadson Santos
Daniel Alencar da Costa
Uirá Kulesza
Publikationsdatum: 01.05.2022
Verlag: Springer US
Erschienen in: Empirical Software Engineering / Ausgabe 3/2022
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-021-10114-1

Springer Professional

Abstract

Context

Objective

Method

Results

Conclusions

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 3/2022

Can pre-trained code embeddings improve model performance? Revisiting the use of code embeddings in software engineering tasks

Search-based fairness testing for regression-based machine learning systems

The reproducibility of programming-related issues in Stack Overflow questions

Breaking bad? Semantic versioning and impact of breaking changes in Maven Central

Systematic literature review on software quality for AI-based software

Software system comparison with semantic source code embeddings

Premium Partner