Top

Empirical Software Engineering

Published in:

01-11-2021

Fixing vulnerabilities potentially hinders maintainability

Authors: Sofia Reis, Rui Abreu, Luis Cruz

Published in: Empirical Software Engineering | Issue 6/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Security is a requirement of utmost importance to produce high-quality software. However, there is still a considerable amount of vulnerabilities being discovered and fixed almost weekly. We hypothesize that developers affect the maintainability of their codebases when patching vulnerabilities. This paper evaluates the impact of patches to improve security on the maintainability of open-source software. Maintainability is measured based on the Better Code Hub’s model of 10 guidelines on a dataset, including 1300 security-related commits. Results show evidence of a trade-off between security and maintainability for 41.90% of the cases, i.e., developers may hinder software maintainability. Our analysis shows that 38.29% of patches increased software complexity and 37.87% of patches increased the percentage of LOCs per unit. The implications of our study are that changes to codebases while patching vulnerabilities need to be performed with extra care; tools for patch risk assessment should be integrate into the CI/CD pipeline; computer science curricula needs to be updated; and, more secure programming languages are necessary.

previous article A pragmatic approach for hyper-parameter tuning in search-based test case generation

next article Using a balanced scorecard to identify opportunities to improve code review effectiveness: an industrial experience report

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Zero Day Initiative website available at https://www.zerodayinitiative.com/advisories/published/(Accessed on September 20, 2021)

SIG’s website: https://www.sig.eu/ (Accessed on September 20, 2021)

BCH’s website: https://bettercodehub.com/ (Accessed on September 20, 2021)

OpenSSL is a toolkit that contains open-source implementations of the SSL and TLS cryptographic protocols. Repository available at https://github.com/openssl/openssl (Accessed on September 20, 2021)

CVE-2016-6304 details available at http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6304(Accessed on September 20, 2021)

CVE-2016-6304 fix available at https://github.com/openssl/openssl/commit/e408c09bbf7c3057bda4b8d20bec1b3a7771c15b(Accessed on September 20, 2021)

CVE-2014-1608 details available at https://github.com/mantisbt/mantisbt/commit/00b4c17088fa56594d85fe46b6c6057bb3421102(Accessed on September 20, 2021)

CWE-89 details available at https://cwe.mitre.org/data/definitions/89.html (Accessed on September 20, 2021)

Research Concepts list available at https://cwe.mitre.org/data/definitions/1000.html

Information available here: https://www.softwareimprovementgroup.com/methodologies/iso-iec-25010-2011-standard/

Check the answer to How can I adjust the threshold for passing/not passing a guideline? at https://bettercodehub.com/docs/faq (Accessed on September 20, 2021)

Research Concepts is a tree-view provided by the Common Weakness Enumeration (CWE) website that intends to facilitate research into weaknesses. It is organized according to abstractions of behaviors instead of how they can be detected, their usual location in code, and when they are introduced in the development life cycle. The list is available here: https://cwe.mitre.org/data/definitions/1000.html

CVE-2016-0799 patch details available at https://github.com/openssl/openssl/commit/9cb177301fdab492e4cfef376b28339afe3ef663 (Accessed on September 20, 2021)

Acar Y, Stransky C, Wermke D, Weir C, Mazurek ML, Fahl S (2017) Developers need support, too: A survey of security advice for software developers. In: 2017 IEEE cybersecurity development (SecDev), pp 22–26. https://doi.org/10.1109/SecDev.2017.17

Alves TL, Correia JP, Visser J (2011) Benchmark-based aggregation of metrics to ratings. In: 2011 Joint conference of the 21st international workshop on software measurement and the 6th international conference on software process and product measurement, pp 20–29. https://doi.org/10.1109/IWSM-MENSURA.2011.15

Alves TL, Ypma C, Visser J (2010) Deriving metric thresholds from benchmark data. In: 2010 IEEE international conference on software maintenance, pp 1–10. https://doi.org/10.1109/ICSM.2010.5609747

Baggen R, Correia JP, Schill K, Visser J (2012) Standardized code quality benchmarking for improving software maintainability. Softw Qual J 20 (2):287–307. https://doi.org/10.1007/s11219-011-9144-9CrossRef

Berger ED, Hollenbeck C, Maj P, Vitek O, Vitek J (2019) On the impact of programming languages on code quality. arXiv:1901.10220

Bijlsma D, Ferreira MA, Luijten B, Visser J (2012) Faster issue resolution with higher technical quality of software. Softw Qual J. 20(2):265–285. https://doi.org/10.1007/s11219-011-9140-0CrossRef

Chowdhury I, Zulkernine M (2010) Can complexity, coupling, and cohesion metrics be used as early indicators of vulnerabilities?. In: Proceedings of the 2010 ACM symposium on applied computing, SAC ’10. pp 1963–1969, Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1774088.1774504

Common Criteria Working Group (2009) Common methodology for information technology security evaluation. Tech. rep., Technical report, Common Criteria Interpretation Management Board

Cruz L, Abreu R, Grundy J, Li L, Xia X (2019) Do energy-oriented changes hinder maintainability?. In: 2019 IEEE International conference on software maintenance and evolution (ICSME), pp 29–40

di Biase M, Rastogi A, Bruntink M, van Deursen A (2019) The delta maintainability model: Measuring maintainability of fine-grained code changes. In: 2019 IEEE/ACM international conference on technical debt (TechDebt), pp 113–122

Elkhail AA, Cerny T (2019) On relating code smells to security vulnerabilities. In: 2019 IEEE 5th intl conference on big data security on cloud (BigDataSecurity), IEEE Intl Conference on High Performance and Smart Computing, (HPSC) and IEEE intl conference on intelligent data and security (IDS), pp 7–12

Foundation TO (2017) Owasp top 10 - 2017, The ten most critical web application security risks. Tech. rep., The OWASP Foundation. Release Candidate

Hegedűs P, Kádár I, Ferenc R, Gyimóthy T (2018) Empirical evaluation of software maintainability based on a manually validated refactoring dataset. Inf Softw Technol 95:313–327. https://doi.org/10.1016/j.infsof.2017.11.012CrossRef

Hegedűs P, Bán D, Ferenc R, Gyimóthy T (2012) Myth or reality? analyzing the effect of design patterns on software maintainability. In: Computer applications for software engineering, disaster recovery, and business continuity. Springer, Berlin, pp 138–145

Heitlager I, Kuipers T, Visser J (2007) A practical model for measuring maintainability. In: 6th International conference on the quality of information and communications technology (QUATIC 2007), pp 30–39. https://doi.org/10.1109/QUATIC.2007.8

International Organization for Standardization (2011) International standard ISO/IEC 25010 systems and software engineering - systems and software quality requirements and evaluation (SQuaRE) - system and software quality models

Islam MR, Zibran MF (2016) A comparative study on vulnerabilities in categories of clones and non-cloned code. In: 2016 IEEE 23rd international conference on software analysis, evolution, and reengineering (SANER), vol 3, pp 8–14

Just R, Jalali D, Inozemtseva L, Ernst MD, Holmes R, Fraser G (2014) Are mutants a valid substitute for real faults in software testing?. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering. ACM, pp 654–665

Kataoka Y, Imai T, Andou H, Fukaya T (2002) A quantitative evaluation of maintainability enhancement by refactoring. In: International conference on software maintenance, 2002. Proceedings., pp 576–585. https://doi.org/10.1109/ICSM.2002.1167822

Khomh F, Gueheneuce Y (2008) Do design patterns impact software quality positively?. In: 2008 12th European conference on software maintenance and reengineering, pp 274–278. https://doi.org/10.1109/CSMR.2008.4493325

Kurilova D, Potanin A, Aldrich J (2014) Wyvern: Impacting software security via programming language design. In: Proceedings of the 5th workshop on evaluation and usability of programming languages and tools, pp 57–58

Li F, Paxson V (2017) A large-scale empirical study of security patches. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, CCS ’17, pp 2201–2215, Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3133956.3134072

Malavolta I, Verdecchia R, Filipovic B, Bruntink M, Lago P (2018) How maintainability issues of android apps evolve. In: 2018 IEEE international conference on software maintenance and evolution (ICSME), pp 334–344. https://doi.org/10.1109/ICSME.2018.00042

Maruyama K, Tokoda K (2008) Security-aware refactoring alerting its impact on code vulnerabilities. In: 2008 15th Asia-pacific software engineering conference, pp 445–452. https://doi.org/10.1109/APSEC.2008.57

McCabe TJ (1976) A complexity measure. IEEE Trans Softw Eng SE-2(4):308–320. https://doi.org/10.1109/TSE.1976.233837MathSciNetCrossRef

McGraw G (2004) Software security. IEEE Secur Priv 2(2):80–83CrossRef

McGraw KO, Wong SP (1992) A common language effect size statistic psychological bulletin. https://doi.org/10.1037/0033-2909.111.2.361

Nistor L, Kurilova D, Balzer S, Chung B, Potanin A, Aldrich J (2013) Wyvern: A simple, typed, and pure object-oriented language. In: Proceedings of the 5th Workshop on MechAnisms for SPEcialization, Generalization and InHerItance, MASPEGHI ’13, pp 9–16, Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2489828.2489830

Olivari M (2018) Maintainable production: A model of developer productivity based on source code contributions. Master’s thesis University of Amsterdam

Palomba F, Bavota G, Penta MD, Fasano F, Oliveto R, Lucia AD (2018) On the diffuseness and the impact on maintainability of code smells: A large scale empirical investigation. Empirical Softw Engg 23(3):1188–1221. https://doi.org/10.1007/s10664-017-9535-zCrossRef

Ponta SE, Plate H, Sabetta A, Bezzi M, Dangremont C (2019) A manually-curated dataset of fixes to vulnerabilities of open-source software. In: Proceedings of the 16th international conference on mining software repositories, MSR ’19. IEEE Press, p 383–387. https://doi.org/10.1109/MSR.2019.00064

Pothamsetty V (2005) Where security education is lacking. In: Proceedings of the 2Nd annual conference on information security curriculum development, InfoSecCD ’05, pp 54–58, ACM, New York, NY, USA. https://doi.org/10.1145/1107622.1107635

Pratt JW (1959) Remarks on zeros and ties in the wilcoxon signed rank procedures. J Am Stat Assoc 54(287):655–667MathSciNetCrossRef

Ray B, Posnett D, Devanbu P, Filkov V (2017) A large-scale study of programming languages and code quality in github. Commun ACM 60 (10):91–100. https://doi.org/10.1145/3126905CrossRef

Ray B, Posnett D, Filkov V, Devanbu P (2014) A large scale study of programming languages and code quality in Github. In: Proceedings of the 22Nd ACM SIGSOFT international symposium on foundations of software engineering, FSE 2014, 155–165, ACM, New York, NY, USA. https://doi.org/10.1145/2635868.2635922

Reis S, Abreu R (2017) A database of existing vulnerabilities to enable controlled testing studies. Int J Secur Softw Eng (IJSSE) 8(3). https://doi.org/10.4018/IJSSE.2017070101

Reis S, Abreu R (2017) Secbench: A database of real security vulnerabilities. In: Proceedings of the international workshop on secure software engineering in devops and agile development (SecSE 2017)

Schneier B (2006) Beyond fear: Thinking sensibly about security in an uncertain world. Berlin, Springer Science & Business Media

Shin Y, Meneely A, Williams L, Osborne JA (2010) Evaluating complexity, code churn, and developer activity metrics as indicators of software vulnerabilities. IEEE Trans Softw Eng 37(6):772–787CrossRef

Slaughter SA, Harter DE, Krishnan MS (1998) Evaluating the cost of software quality. Commun ACM 41(8):67–73CrossRef

Telang R, Wattal S (2007) An empirical analysis of the impact of software vulnerability announcements on firm stock price. IEEE Trans Softw Eng 33(8):544–557CrossRef

The OWASP Foundation (2009) OWASP application security verification standard 2009 - web application standard. Tech rep

Visser J (2016) Building maintainable software, java edition: Ten guidelines for future-proof code. O’Reilly Media, Inc

Visser J (2020) Sig/tUvit evaluation criteria trusted product maintainability: Guidance for producers. Available: https://bit.ly/3hnY0Am

Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics Bulletin 1(6):80–83CrossRef

Xu H, Heijmans J, Visser J (2013) A practical model for rating software security. In: 2013 IEEE seventh international conference on software security and reliability companion, pp 231–232. https://doi.org/10.1109/SERE-C.2013.11

Zazworka N, Shaw MA, Shull F, Seaman C (2011) Investigating the impact of design debt on software quality. In: Proceedings of the 2nd workshop on managing technical debt, MTD ’11, pp 17–23, Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1985362.1985366

Zibran MF, Saha RK, Asaduzzaman M, Roy CK (2011) Analyzing and forecasting near-miss clones in evolving software: An empirical study. In: 2011 16th IEEE international conference on engineering of complex computer systems, pp 295–304

Title: Fixing vulnerabilities potentially hinders maintainability
Authors: Sofia Reis
Rui Abreu
Luis Cruz
Publication date: 01-11-2021
Publisher: Springer US
Published in: Empirical Software Engineering / Issue 6/2021
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-021-10019-z

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 6/2021

A study of how Docker Compose is used to compose multi-component systems

FACER: An API usage-based code-example recommender for opportunistic reuse

Why and what happened? Aiding bug comprehension with automated category and causal link identification

Where were the repair ingredients for Defects4j bugs?

Characterizing refactoring graphs in Java and JavaScript projects

The forgotten role of search queries in IR-based bug localization: an empirical study

Premium Partner