Software Quality: The Next Big Thing in Software Engineering and Quality

Software is becoming a critical component of most products and organizational functions. The ability to continuously improve software determines how well the organization can respond to market opportunities. Continuous software engineering promises numerous advantages over sprint-based or plan-driven development. However, implementing a continuous software engineering pipeline in an existing organization is challenging.

In this invited position paper, we discuss the adoption challenges and argue for a more systematic methodology to drive the adoption of continuous engineering. Our discussion is based on ongoing work with several industrial partners as well as experience reported in both state-of-practice and state-of-the-art.

We conclude that the adoption of continuous software engineering primarily requires analysis of the organization, its goals, and constraints. One size does not fit all purposes, meaning that many of the principles behind continuous engineering are relevant for most organizations, but the level of realization and the benefits may still vary. The main hindrances to continuous flow of software arise from sub-optimal organizational structures and the lack of alignment. Once those are removed, the organization can implement automation to further improve the software delivery.

Test Maturity Model integration (TMMi) is a popular model for maturity assessment and capability improvement of software testing practices in industry. Originally inspired by the Capability Maturity Model Integration (CMMI), and managed by the TMMi Foundation, the TMMi specification provides guidelines for assessing and improving testing capabilities of teams and organizations. In this invited paper, we discuss motivations for and benefits of adopting the TMMi. The discussion is based on an international user survey, which received data from 74 companies that have received TMMi assessments and certifications.

Modern Code Reviews (MCRs) are a widely-used quality assurance mechanism in continuous integration and deployment. Unfortunately, in medium and large projects, the number of changes that need to be integrated, and consequently the number of comments triggered during MCRs could be overwhelming. Therefore, there is a need for quickly recognizing which comments are concerning issues that need prompt attention to guide the focus of the code authors, reviewers, and quality managers. The goal of this study is to design a method for automated classification of review comments to identify the needed change faster and with higher accuracy. We conduct a Design Science Research study on three open-source systems. We designed a method (CommentBERT) for automated classification of the code-review comments based on the BERT (Bidirectional Encoder Representations from Transformers) language model and a new taxonomy of comments. When applied to 2,672 comments from Wireshark, The Mono Framework, and Open Network Automation Platform (ONAP) projects, the method achieved accuracy, measured using Matthews Correlation Coefficient, of 0.46–0.82 (Wireshark), 0.12–0.8 (ONAP), and 0.48–0.85 (Mono). Based on the results, we conclude that the proposed method seems promising and could be potentially used to build machine-learning-based tools to support MCRs as long as there is a sufficient number of historical code-review comments to train the model.

Machine learning has emerged as a useful tool to aid software quality control. It can support identifying problematic code snippets or predicting maintenance efforts. The majority of these frameworks rely on code metrics as input.

However, evidence suggests great potential for text- and image-based approaches to predict code quality as well. Using a manually labeled dataset, this preliminary study examines the use of five text- and two image-based algorithms to predict the readability, understandability, and complexity of source code.

While the overall performance can still be improved, we find Support Vector Machines (SVM) outperform sophisticated text transformer models and image-based neural networks. Furthermore, text-based SVMs tend to perform well on predicting readability and understandability of code, while image-based SVMs can predict code complexity more accurately.

Our study both shows the potential of text- and image-based algorithms for software quality prediction and outlines their weaknesses as a starting point for further research.

Testing is the key activity in ensuring the quality of automotive systems. The corresponding test case specifications often contain test cases expressed in natural language. However, there is a lack of review approaches that are easy to apply for practitioners to ensure appropriate quality of those test case specifications. We therefore present an analytical quality assurance method based on a quality model and review checklists derived from it. Especially, we focus on quality criteria that are relevant for natural language test cases and in the context of the automotive domain. To ensure applicability in industrial practice, we stringently involve practitioners in the development of the quality model via expert workshops. The systematic derivation of quality characteristics results in a quality model for automotive test case specifications. Furthermore, we show how review checklists for a multidimensional review were derived from it. A first evaluation indicates that these review checklists support practitioners in conducting reviews and also foster the understanding of qualitative test case specifications.