Quality of Software Architectures and Software Quality

The architecture of an IT system is of crucial importance for its success. In order to assess architecture’s fitness, a number of standardized architecture evaluation methods have been proposed. Most of them are intended for the evaluation of a single architecture at a certain point in time. Furthermore, the results are often highly dependent on the person performing the evaluation. Thus, such methods cannot be used to compare and rate different architectures. The DoSAM method instead provides an evaluation framework for comparing different software architectures in a certain domain. After adapting this framework to the application domain at hand once, it can then be used repeatedly for all future evaluations in a methodical and reproducible way.

Software architecture has been advocated as an effective means to produce quality systems. In this paper, we argue that integration between analysis conducted at different stages of development is still lacking. Hence we propose an architecture-centric approach which, by combining different technologies and tools for analysis and testing, supports a seamless tool-supported approach to validate required properties. The idea is to start from the requirements, produce a validated model of the SA, and then use the SA to derive a set of conformance test cases. In this paper, we outline the process, and discuss how some existing tools, namely QuARS,ModTest, CowTest and UIT, could be exploited to support the approach. The integrated framework is under development.

It is now recognized that nonfunctional properties are important to practical software development and maintenance. Many of these properties involve involving time and probabilities – for example, reliability and availability. One approach to ensuring conformance to nonfunctional requirements is the use of runtime monitoring. Currently, such monitoring is done in one of two ways: 1) monitoring through use of a generic tool or 2) by adding instrumentation code within system software and writing a tool to manage resulting datasets. The first approach is often not flexible while the second approach can lead to a higher development cost. In this paper, we present a flexible framework for runtime verification of timed and probabilistic nonfunctional properties of component-based architectures. We describe a Microsoft .NET-based implementation of our framework built upon the Windows Management Instrumentation (WMI) infrastructure and the Distributed Management Task Force’s Common Information Model standard. We use a language for contracts based on Probabilistic Computational Tree Logic (PCTL). We provide a formal semantics for this language based on possible application execution traces. The semantics is generic over the aspects of an application that are represented by states and transitions of traces. This enables us to use the language to define a wide range of nonfunctional properties.

One of the aims of component-based software engineering is the reuse of existing software components in different deployment contexts. With the redeployment of a component, its performance changes, since it depends on the performance of external services, the underlying hardware and software, and the operational profile. Therefore, performance prediction models are required that are able to handle these dependencies and use the properties of component-based software systems. Parametric contracts model the relationship of provided and required services of a component. In this paper, we analyse the influence of external services on the service execution time applying parametric contracts and a performance prediction algorithm based on Markov chains. We verbalise the assumptions of this approach and evaluate their validity with an experiment. We will see that most of the assumptions hold only under certain constraints.

In recent years, the focus of software development has progressively shifted upward, in the direction of the abstract level of architecture specification. However, while the functional properties of the systems have been extensively dealt with in the literature, relatively less attention has been given until recently to the specification and analysis at the architectural level of quality attributes such as performance and reliability. The contribution of this paper is twofold: first we discuss the type of information that should be provided at the architectural level in order to successfully address the problem of performance and reliability modeling and analysis of software systems; based on this discussion, we define an extension of the xADL architectural language that enables the support for stochastic modeling and analysis of performance and reliability in software architectures.

Objective: In this paper, we present a formalization of the definition of metrics to assess quality attributes of CORBA components and assemblies. The focus is on the formalization technique, rather than on the evaluation of the metrics themselves. Method: We represent a component assembly as an instantiation of the CORBA Component Model metamodel. The resulting meta-object diagram can then be traversed using Object Constraint Language clauses. With these clauses, we construct a formal and executable definition of the metrics. Results: We demonstrate the expressiveness of our technique by formally de fining metrics proposed informally by several authors on different aspects of components and assemblies’ quality attributes. Conclusion: Providing a formal and executable definition of metrics for CORBA components and assemblies is an enabling precondition to allow for independent scrutiny of such metrics, which is, in turn, essential to increase practitioners’ confidence on predictable quality attributes.

Effective standard architectures promise to improve the efficiency of software projects and the quality of resulting systems. However, hardly any standard architecture has become established in practise to date. They are either too general or too specific to be effective – the architect’s dilemma. Reference architectures have a less binding character than standard architectures and still are of high value. This paper presents exemplary parts of the Quasar reference architecture for business information systems, the result of more than seven years of architectural research at sd&m.

When organizations cooperate closely, for example after a company merger, there is typically a need to integrate their in-house developed software into one coherent system, preferably by reusing from all of the existing systems. The parts that can be reused may be arbitrarily small or large, ranging from code snippets to large self-containing components. Not only implementations can be reused however; sometimes it may be more appropriate to only reuse experiences in the form of architectural solutions and requirements. In order to investigate the circumstances under which different types of reuse are appropriate, we have performed a multiple case study, consisting of nine cases. Our conclusions are, summarized: reuse of components from one system requires reuse of architectural solutions from the same system; merge of architectural solutions cannot occur unless the solutions already are similar, or if some solutions from one are incorporated into the other. In addition, by hierarchically decomposing the systems we make the same observations. Finally, among the cases we find more architectural similarities than might had been expected, due to common domain standards and common solutions within a domain. Although these observations, when presented, should not be surprising, our experiences from the cases show that in practice organizations have failed to recognize when the necessary prerequisites for reuse have not been present.

Security is an important quality attribute required in many software intensive systems. However, software development methodologies do not provide sufficient support to address security related issues. Furthermore, the majority of the software designers do not have adequate expertise in the security domain. Thus, security is often treated as an add-on to the designed architecture. Such ad-hoc practices to deal with security issues can result in a system that is vulnerable to different types of attacks. The security community has discovered several security sensitive design patterns, which can be used to compose a security sensitive architecture. However, there is little awareness about the relationship between security and software architecture. Our research has identified several security patterns along with the properties that can be achieved through those patterns. This paper presents those patterns and properties in a framework that can provide appropriate support to address security related issues during architecture processes.

A central tenet of software architecture design is to base this on a formulation of desired quality attributes, such as buildability, performance, and availability of the target system. Thus there is a need for architectural evaluation—ensuring the architecture’s support for desired quality attributes—and a variety of evaluation techniques have been developed, described, and used. Architectural prototyping is an experimental approach that creates executable ‘skeleton’ systems to investigate architectural qualities of a future system. Architectural prototyping is a learning vehicle for exploring an architectural design space as well as an evaluation technique. The contribution of this paper is to explore the evaluation aspect of architectural prototypes from an analytical standpoint. We present an analysis and discussion of architectural prototyping in the context of two well-established quality frameworks. Our analysis concludes that architectural prototyping is a viable evaluation technique that may evaluate architectural quality attributes and especially valuable in cases where the balance between opposing qualities must be assessed.

Component based development, which had been successful in enterprise computing, shows promises to be a good development model for automotive systems. This is possible if several dependability concerns of the embedded systems can be properly addressed by the models, frameworks and integration platforms. SaveCCM is a component model for automotive systems developed by employing component based system design. Our ongoing research is related to estimation of software reliability for this model. In this paper, we provide a survey of the state of the art on research techniques for the estimation of system reliability based on component reliabilities and architectures. We further discuss their pros and cons with reference to our architectural model and discuss some practical considerations. Based on this we also present the basics of our approach to reliability modeling of SaveCCM architectures.

Predicting the performance of software architectures during early design stages is an active field of research in software engineering. It is expected that accurate predictions minimize the risk of performance problems in software systems by a great extent. This would improve quality and save development time and costs of subsequent code fixings. Although a lot of different methods have been proposed, none of them have gained widespread application in practice. In this paper we describe the evaluation and comparison of three approaches for early performance predictions (Software Performance Engineering (SPE), Capacity Planning (CP) and umlPSI). We conducted an experiment with 31 computer science students. Our results show that SPE and CP are suited for supporting performance design decisions in our scenario. CP is also able to validate performance goals as stated in requirement documents under certain conditions. We found that SPE and CP are matured, yet lack the proper tool support that would ease their application in practice.

n-way combinatorial testing is a specification-based testing criterion, which requires that for a system consisting of a few parameters, every combination of valid values of arbitrary n (n ≥ 2) parameters be covered by at least one test. In this paper, we propose two different tests generation algorithms based on the combinatorial design for the n-way combination testing. We show that the produced tests can cover all the combinations of parameters to the greatest degree with the small quantity. We implemented the automatic test generators based on the algorithms and obtained some valuable empirical results.

In this research paper, an approach to fully automating the generation of test data for object-oriented programs fulfilling dataflow-based testing criteria and the subsequent evaluation of its fault-detection capability are presented. The underlying aim of the generation is twofold: to achieve a given dataflow coverage measure and to minimize the effort to reach this goal in terms of the number of test cases required. In order to solve the inherent conflict of this task, hybrid self-adaptive and multiobjective evolutionary algorithms are adopted. Our approach comprises the following steps: a preliminary activity provides support for the automatic instrumentation of source code in order to record the relevant dataflow information. Based on the insight gained hereby, test data sets are continuously enhanced towards the goals mentioned above. Afterwards, the generated test set is evaluated by means of mutation testing. Progress achieved so far in our ongoing project will be described in this paper.

This paper presents Jartege, a tool which allows random generation of unit tests for Java classes specified in JML. JML (Java Modeling Language) is a specification language for Java which allows one to write invariants for classes, and pre- and postconditions for operations. As in the JML-JUnit tool, we use JML specifications on the one hand to eliminate irrelevant test cases, and on the other hand as a test oracle. Jartege randomly generates test cases, which consist of a sequence of constructor and method calls for the classes under test. The random aspect of the tool can be parameterized by associating weights to classes and operations, and by controlling the number of instances which are created. The practical use of Jartege is illustrated by a small case study.

This paper examines whether test problems that occur speci- fically during unit testing of object-oriented programs can be solved using the aspect-oriented programming paradigm.

Performance is an important aspect of software quality. To prevent performance degradation during software development, performance can be monitored and software modifications that damage performance can be reverted or optimized. Regression benchmarking provides means for an automated monitoring of performance, yielding a list of software modifications potentially associated with performance changes. We focus on locating individual modifications as causes of individual performance changes and present three methods that help narrow down the list of modifications potentially associated with a performance change. We illustrate the entire process on a real world project.