The cost of errors in software development: evidence from industry
Section snippets
Software life cycle costs
This research develops a model for costs generated in software development, focusing on costs associated with software error correction and detection. It draws on empirical data extracted for one phase of the software development life cycle by a vendor of packaged microcomputer software.
In the past, empirical investigations of software costs have been rare, most likely due to the difficulty in obtaining sufficient data for analysis. Financial accounting systems do not require separate account
Data collection
Data was collected from a packaged software vendor with global operations and a number of competitive products. The vendor developed, researched, specified, and tested each software release at corporate headquarters before allowing global release. Each completed, tested, and stable product was then released in the vendors home market simultaneously with its being passed on for localisation (i.e., translation to the local language, culture and practice) to 31 strategically important global
Error cost
The vendor's internal studies concluded that the largest single contributor to localisation cost was the detection and correction of errors. Consequently, this research first investigated evidence on error costs.
Prior literature has suggested a growth model for software errors and reliability. Evolutionary growth models assume that the number of errors discovered depends on time and that the underlying error distribution is exponential. The most important of these was proposed by Jelinski and
Specification searches
This part of the research reports the findings from a general multiple variable analysis of software costs and cost drivers for development tasks during the `localisation' phase. Because part of the data used is accounting cost data, there is the possibility of results being confounded by double-entry bookkeeping. Double-entry bookkeeping, by definition, records information about the same transactions in multiple places around the corporate accounts simultaneously. This can weaken our
Conclusions and implications for the software industry
The dataset acquired for this research provides strong support for an evolutionary model of error occurrence with an exponential distribution of error count overtime. It also provided some additional specifications for enhanced models for software cost estimation. Specifically:
(1) The regression shows that the serious errors have less influence on cost than other errors, assuming that they are ultimately resolved. This likely reflects greater discretion available in how and when to resolve the
J. Christopher Westland is Professor and Head of the Department of Information Systems and Management at the Hong Kong University of Science and Technology. He received a BA in mathematics, and an MBA in accounting from Indiana University. He received his Ph.D. in information systems from the University of Michigan. He has professional experience in the US as a certified public accountant and as a consultant in information systems in the US, Europe, Latin America and Asia. He is a US CPA, an
References (21)
- et al.
An integrated environment for reuse reengineering C code
The Journal of Systems and Software
(1998) Costs and productivity estimation in computer engineering economics
The Engineering Economist
(1996)Software quality standards: do software quality initiatives such as ISO 9000 reduce the introduction of dangerous programmatic errors?
DBMS
(1993)Software and its impact: a quantitative assessment
Datamation
(1973)8th World computer congress developing small-scale application software products: some experimental results
- et al.
An Analysis of Transformations
Journal of the Royal Statistical Society, Series B
(1964) Coupling and control flow measures in practice
The Journal of Systems and Software
(2000)Formal methods can cut your error rate
Computing Canada
(1993)To err is human, to forgive uncommon in IS
Computing Canada
(1994)
Cited by (70)
IFC+: Towards the integration of IoT into early stages of building design
2022, Automation in ConstructionCitation Excerpt :The possibility of working with models is a very desirable characteristic as far as the support of IoT features in BIM processes is concerned, since each IoT infrastructure is managed by an underlying software. The use of models to represent IoT devices can helps to detect errors and inconsistencies in early stages of the software development, which in turn, has been repeatedly proved to reduce software lifecycle costs exponentially [23–25]. Similarly, there are starting to appear studies reflecting the positive economic impact of using BIM models in construction processes, although the lack of records and available data still difficulties to carry out these kinds of research [26,27].
The information content of Sarbanes-Oxley in predicting security breaches
2020, Computers and SecurityCitation Excerpt :This is somewhat more frequent than the 5.6 years expected between breaches by the hazard structural model, but with more extensive data collection, a more robust estimate could be obtained, and we consider this estimate to be realistic for the occupancy structural model. Such failures where they do occur in the software field are apocryphally called 5000 errors and have been studied in Boehm et al. (1995), Westland (2000), Westland 2002, Westland (2004) . The probability of a breach at any given time is small, with a large enough population of firms one can expect many failures to show up every year across the full population of firms and industry wide risk of breach is thus substantial.
Supporting agent oriented requirement analysis with ontologies
2016, International Journal of Human Computer StudiesCitation Excerpt :Throughout these activities, cognitive limitations of modellers, communication gaps with the users, together with the interdependencies of models, can lead to errors. These errors can snowball in the course of the development process and become more costly to fix [Westland, 2002]. As such, capturing errors in the requirement models during the verification and validation activities before they propagate into the design and later phases is often pursued.
User Involvement in Software Development Processes
2016, Procedia Computer ScienceAn Assessment of Rules of Thumb for Software Phase Management, and the Relationship Between Phase Effort and Schedule Success
2024, IEEE Transactions on Software Engineering
J. Christopher Westland is Professor and Head of the Department of Information Systems and Management at the Hong Kong University of Science and Technology. He received a BA in mathematics, and an MBA in accounting from Indiana University. He received his Ph.D. in information systems from the University of Michigan. He has professional experience in the US as a certified public accountant and as a consultant in information systems in the US, Europe, Latin America and Asia. He is a US CPA, an active member of Mensa, and sits on the editorial boards of several of the leading academic journals in information technology and has written two books, The New Science of Wealth (forthcoming October 2001, Wiley) and Global Electronic Commerce (MIT Press, 2000). He has been a frequent commentator on Hong Kong's technology policy. A full biography and selected papers may be found at http://www.ismt.ust.hk/faculty/westland/.