Software Quality: Quality Intelligence in Software and Systems Engineering

For a highly relevant and applied research area like software testing industry-academia collaboration is of uttermost importance. In this paper we reflect on how industry-academia collaboration can be improved based on evidence from four empirical studies. We therefore first present four studies providing evidence on the (1) perceived level of challenges in testing activities, (2) focus areas in industrial and academic software testing conferences, (3) synergies between industrial and academic software testing conferences, as well as (4) the need for consideration of grey literature. Then, we reflect on issues, which we think can improve the link and synergies between industry and academia in software testing, i.e., research topics, guidelines and evidence, value and risk, context and scalability, action research and education as well as grey literature and open science.

An increasing number of companies incorporate Big Data in order to increase business value and gain competitive advantages. With this new paradigm, also testing methodologies need to be revised to suit the specific requirements of Big Data. This paper summarizes the outcome of an exploratory survey conducted in 2018 with seven participants from different industries (Healthcare, Technology). The issues can be divided into four categories: (1) limited resources and performance problems, (2) verifying test results, (3) finding an optimal test coverage, and (4) availability of test data.

Regression Testing is an important quality assurance activity for combating unwanted side-effects, which might have been introduced in a new software release. Selecting and prioritizing regression test cases is a challenge in practice – especially in a world of ever increasing complexity, distribution, and size of the software solutions. Current approaches try to minimize the number of regression test cases by analyzing the change and the coverage of the tests with regards to this change. Our approach utilizes usage frequencies from the previous, productive software version in order to select or prioritize test cases by calculating the Regression Risk of a change. This takes into account that not all features of a software are used the same. We successfully validate our approach in a case study of an industry project which develops a complex process integration platform.

As a software project evolves over time, the associated test suite usually grows with it. If test suites are not carefully maintained, this can easily result in massive test execution duration, reducing the benefits of regression testing because faults are found later in development or even after release. Test suite minimization aims to combat long running test suites by removing redundant test cases. Previous work mainly evaluates test suite minimization techniques based on comparably small projects, which are less practically relevant. In this paper, we compare four test suite minimization techniques by applying them to several open source software projects and evaluate the results. We find that the size and execution time of all the test suites can be reduced by over 70% on average. However, there is a substantial loss in fault detection capability of, on average, around 12.5%, restricting the applicability of this form of test suite minimization.

The importance of software engineers’ competency has long been established as a key pillar for the development of robust software in order to achieve quality software. Software engineering competency research is not necessarily lacking. Nevertheless, the satisfaction derived from using software competency needs more investigation. The aim of this study is to identify soft competencies from empirical data and create satisfaction levels for software engineers’ soft competencies. The result shows 63 soft competencies with three different satisfaction levels consisting of basic, performance and delighters. The paper contributes to the SEC research by highlighting the satisfaction levels of soft competency for the benefit of the educators (academia), software engineers (possessor) and users of software competency (practitioner).

Semantic similarity information supports requirements tracing and helps to reveal important requirements quality defects such as redundancies and inconsistencies.

Previous work has applied semantic similarity algorithms to requirements, however, we do not know enough about the performance of machine learning and deep learning models in that context.

Therefore, in this work we create the largest dataset for analyzing the similarity of requirements so far through the use of Amazon Mechanical Turk, a crowd-sourcing marketplace for micro-tasks. Based on this dataset, we investigate and compare different types of algorithms for estimating semantic similarities of requirements, covering both relatively simple bag-of-words and machine learning models.

In our experiments, a model which relies on averaging trained word and character embeddings as well as an approach based on character sequence occurrences and overlaps achieve the best performances on our requirements dataset.

Software products evolve increasingly fast as markets continuously demand new features and agility to customer’s need. This evolution of products triggers an evolution of software development practices in a different way. Compared to classical methods, where products were developed in projects, contemporary methods for continuous integration, delivery, and deployment develop products as part of continuous programs. In this context, software architects, designers, and quality engineers need to understand how the processes evolve over time since there is no natural start and stop of projects. For example, they need to know how similar two iterations of the same program or how similar two development programs are. In this paper, we compare three methods for calculating the degree of similarity between projects by comparing their Git commit series. We test three approaches—the DNA-motifs-inspired SimSAX measure and clustering of subsequences (k-Means and Hierarchical clustering). Our results show that the clustering algorithms are much more sensitive to parameters and often find similarities that are not correct. SimSAX, on the other hand, can be calibrated to find fewer similarities between the projects; the similarities are also more consistent for SimSAX than they are for the clustering. We conclude that it is better to use DNA-inspired motifs as they provide more accurate results.

Code reviews have been used to improve code quality since the 1970s. Most practitioners in the field of software have some experience with respect to the technique. In this mapping study we illustrate what kinds of research questions are addressed in code review literature. The following themes emerged from analysis of 75 original articles: (1) description or comparison of different code review practices, (2) behavior of reviewers (e.g., eye tracking studies), (3) communication and teamwork, (4) outcomes of code reviews (e.g., what kinds of problems are identified), (5) how properties of code to be reviewed affect reviewing, and (6) reasons for conducting code reviews. About half of the studies have been conducted with students and novices. The numbers of industry papers has significantly increased when compared to the previous reviews in the field.

While optimising quality assurance has been an important research area for many years, we still see interesting new ideas in this area such as incorporating psychological factors, detecting pseudo-tested code and detecting code with low fault risk.