A Bayesian network approach to assess and predict software quality using activity-based quality models
Introduction
Despite the importance of software quality, its management is still an immature discipline in software engineering research and practice. Research work has gone in many directions and produced a variety of useful results. However, there is still no commonly agreed way for quality management. The practice varies strongly from a concentration on testing to a large-scale quality management process.
One main problem is that many of the tools and methods in quality assurance and management work isolated. For example, development teams usually perform peer reviews whose results are often not set into relation to results of integration or system tests performed by the quality assurance team. Hence, quality is tackled on many levels without a combined strategy [1]. What is missing is a clear integration of these single efforts. One prerequisite for such an integration is a quality management sub-process in the overall development process. The process defines roles, activities, and artefacts and how they work together. Hence, the maturity level of the organisation’s processes plays an important role. A second prerequisite is a clear quality model that specifies the quality of the software to be developed.
Current quality models such as the ISO 9126 [2] have widely acknowledged problems [3], [4], [5]. Especially as a basis for assessment and prediction, the defined “-ilities” are too abstract. A clear transition to measurements is therefore difficult in practice. Hence, quantitative quality assessment and prediction is usually done without direct use of such a quality model. This, in turn, leads to isolated solutions in quality management.
We use the previously proposed activity-based quality models (ABQM) [4] as a basis for quality assessment and prediction. They provide a clear structure of quality and detailed information about quality-influences. Activity-based quality models have proven useful in practice to structure quality and to generate corresponding guidelines and checklists. In this paper, we add a systematic transition from ABQMs to Bayesian networks in order to enhance their assessment and prediction capabilities. A four-step approach is defined that generates a Bayesian network using an activity-based quality model and an assessment or prediction goal. The approach is demonstrated in several examples.
We first motivate and introduce activity-based quality models in Section 2. In Section 3 the four-step approach for systematically constructing a Bayesian network from an activity-based quality model is proposed. The approach is then demonstrated in an initial evaluation in Section 4 using publicly available data from NASA projects and measured data from the open source system Tomcat. Related work is discussed in Section 5 and final conclusions are given in Section 6.
Section snippets
Activity-based quality models
Quality models describe in a structured way the meaning of a software’s quality. We introduce the use of general quality models and how the modelling of activities and facts helps to define quality more precisely.
Assessment and prediction approach
Although activity-based quality models have proven to be useful in practice, there is no systematic measurement approach for them. Hence, there are no quantitative assessments and predictions possible so far. We propose an approach that can be used for systematically deriving assessment and prediction models from an activity-based quality model.
Initial evaluation
The approach presented above can be used in various contexts to answer assessment and prediction questions. In this section, we provide an initial evaluation by applying the approach to several publicly available data sets as well as automatically collectable measures for an open source system.
Related work
The basic idea to use Bayesian networks for assessing and predicting software quality has been developed foremost by Fenton, Neil, and Littlewood. They introduced Bayesian networks as a tool in this area and applied it in various contexts related to software quality. In [21] they formulate a critique on current defect prediction models and suggest to use Bayesian networks. Other researchers also used Bayesian networks for software quality prediction similarly [22], [23].
The work closest to the
Conclusions
A high goal in software quality management is the reliable quantitative assessment and prediction of software quality. Many efforts in building assessment and prediction models have given insights in the usefulness but also the constraints of such models. However, these models have not been tightly integrated into other quality management activities. Activity-based quality models have proven in practice to be a solid foundation for defining quality on a detailed level. However, quantitative
Acknowledgments
This work has partially been supported by the German Federal Ministry of Education and Research (BMBF) in the project QuaMoCo (01 IS 08023B). I would like to thank the anonymous reviewers at PROMISE for feedback on an earlier version and Vic Basili, Sebastian Winter, and the anonymous IST reviewers for helpful suggestions for this version.
References (28)
- S. Wagner, F. Deissenboeck, An integrated approach to quality modelling, in: Proc. 5th International Workshop on...
- ISO 9126-1, Software Engineering – Product Quality – Part 1: Quality Model,...
- et al.
Software quality: the elusive target
IEEE Software
(1996) - F. Deissenboeck, S. Wagner, M. Pizka, S. Teuchert, J.-F. Girard, An activity-based quality model for maintainability,...
- S. Wagner, K. Lochmann, S. Winter, A. Goeb, M. Klaes, Quality models in practice: a preliminary analysis, in: Proc. 3rd...
- et al.
A comprehensive model of usability
- S. Wagner, D. Méndez Fernández, S. Islam, K. Lochmann, A security requirements approach for web systems, in: Proc....
- ISO/IEC 14764, IEEE Std. 14764-2006, Software Engineering – Software Life Cycle Processes – Maintenance,...
- et al.
Goal question metric paradigm
- M. Lindvall, P. Donzelli, S. Asgari, V. Basili. Towards reusable measurement patterns, in: Proc. 11th IEEE...
Using ranked nodes to model qualitative judgments in Bayesian networks
IEEE Transactions on Knowledge and Data Engineering
Cited by (41)
Comprehensive assessment of open source software ecosystem health
2023, Internet of Things (Netherlands)An ERP Data Quality Assessment Framework for the Implementation of an APS system using Bayesian Networks
2022, Procedia Computer ScienceA Method to Estimate Software Strategic Indicators in Software Development: An Industrial Application
2021, Information and Software TechnologyCitation Excerpt :There is plenty of literature on indicators and metrics aimed to specify meaningful aspects of software development products and processes. One specific type of instrument that has been widely used in the literature to specify and create assessment models for software-related indicators refers to Quality Models (QMs) [39–42]. Although useful for specification, most of these works are academic and provide too abstract procedures to be operational [49].
Quantitative software reliability assessment methodology based on Bayesian belief networks and statistical testing for safety-critical software
2020, Annals of Nuclear EnergyCitation Excerpt :Therefore, through the analysis of relevant standards (such as IEEE 1012 (IEEE, 2017)) and guidelines (e.g. BTP 7–14 (U.S. NRC, 2007)), a series of indirect metrics are gained as indicator nodes (Fenton et al., 2007), which we can think of as providing observations for the parent node (Chu et al., 2018). In other words, indicator nodes represent metrics for the quality of the actual activity (Wagner, 2010). In this study, according to the degree of compliance with relevant standards and guidelines, the measurement results of indicator nodes can be divided into three levels, which are cited Chu et al. (2018):
Development of a cyber security risk model using Bayesian networks
2015, Reliability Engineering and System SafetyOperationalised product quality models and assessment: The Quamoco approach
2015, Information and Software TechnologyCitation Excerpt :Moreover, we can go the other way round. We can pick quality aspects, for example, ISO/IEC 25010 quality characteristics, which we consider important and costly for a specific software system, and trace what product factors affect them and what measures quantify them (cf. [73]). This allows us to put emphasis on the product factors with the largest impact on these quality aspects.