Tests and Proofs

Active automata learning comprises techniques for learning automata models of black-box systems by testing such systems. While this form of learning enables model-based analysis and verification, it may also require a substantial amount of interactions with considered systems to learn adequate models, which capture the systems’ behaviour.

The test cases executed during learning can be divided into two categories: (1) test cases to gain knowledge about a system and (2) test cases to falsify a learned hypothesis automaton. The former are selected by learning algorithms, whereas the latter are selected by conformance-testing algorithms. There exist various options for both types of algorithms and there are dependencies between them. In this paper, we investigate the performance of combinations of four different learning algorithms and seven different testing algorithms. For this purpose, we perform learning experiments using 39 benchmark models. Based on experimental results, we discuss insights regarding the performance of different configurations for various types of systems. These insights may serve as guidance for future users of active automata learning.

It is crucial that smart contracts are tested thoroughly due to their immutable nature. Even small bugs in smart contracts can lead to huge monetary losses. However, testing is not enough; it is also important to ensure the quality and completeness of the tests. There are already several approaches that tackle this challenge with mutation testing, but their effectiveness is questionable since they only considered small contract samples. Hence, we evaluate the quality of smart contract mutation testing at scale. We choose the most promising of the existing (smart contract specific) mutation operators, analyse their effectiveness in terms of killability and highlight severe vulnerabilities that can be injected with the mutations. Moreover, we improve the existing mutation methods by introducing a novel killing condition that is able to detect a deviation in the gas consumption, i.e., in the monetary value that is required to perform transactions.

There is a high demand in practical methods and tools to ensure total correctness of critical software components. A usual assumption is that the machine code (or binary code) generated by a compiler follows the semantics of the programming language. Unfortunately, modern compilers such as GCC and LLVM are too complex to be thoroughly verified, and bugs in the generated code are not uncommon. As an alternative approach, we suggest proving that the generated machine code still satisfies the functional properties expressed at the source code level. The suggested approach takes an ACSL-annotated C function along with its binary code and checks that the binary code meets the ACSL annotations. The main steps are as follows: (1) disassembling the machine code and extracting its semantics; (2) adapting the annotations to the machine level and generating the verification conditions. The implementation utilizes MicroTESK, Frama-C, Why3, and other tools. To illustrate the solution, we use the RISC-V microprocessor architecture; however, the approach is relatively independent of the target platform as it is based on formal specifications of the instruction set. It is worth noting that the presented method can be exploited as a test oracle for compiler testing.

We explore the application of graph database technology to spatio-temporal model checking of cooperating cyber-physical systems-of- systems such as vehicle platoons. We present a translation of spatio-temporal automata (STA) and the spatio-temporal logic STAL to semantically equivalent property graphs and graph queries respectively. We prove a sound reduction of the spatio-temporal verification problem to graph database query solving. The practicability and efficiency of this approach is evaluated by introducing NeoMC, a prototype implementation of our explicit model checking approach based on Neo4j. To evaluate NeoMC we consider case studies of verifying vehicle platooning models. Our evaluation demonstrates the effectiveness of our approach in terms of execution time and counterexample detection.

Differential equations are of immense importance for modeling physical phenomena, often in combination with discrete modeling formalisms. In current industrial practice, properties of the resulting models are checked by testing, using simulation tools. Research on SAT solvers that are able to handle differential equations has aimed at replacing tests by correctness proofs. However, there are fundamental limitations to such approaches in the form of undecidability, and moreover, the resulting solvers do not scale to problems of the size commonly handled by simulation tools. Also, in many applications, classical mathematical semantics of differential equations often does not correspond well to the actual intended semantics, and hence a correctness proof wrt. mathematical semantics does not ensure correctness of the intended system.

In this paper, we head at overcoming those limitations by an alternative approach to handling differential equations within SAT solvers. This approach is usually based on the semantics used by tests in simulation tools, but still may result in mathematically precise correctness proofs wrt. that semantics. Experiments with a prototype implementation confirm the promise of such an approach.

Runtime Assertion Checking (RAC) for expressive specification languages is a non-trivial verification task, that becomes even more complex for memory-related properties of imperative languages with dynamic memory allocation. It is important to ensure the soundness of RAC verdicts, in particular when RAC reports the absence of failures for execution traces. This paper presents a formalization of a program transformation technique for RAC of memory properties for a representative language with memory operations. It includes an observation memory model that is essential to record and monitor memory-related properties. We prove the soundness of RAC verdicts with regard to the semantics of this language.

Data races, a condition where two memory accesses to the same memory location occur concurrently, have been shown to be a major source of concurrency bugs in distributed systems. Unfortunately, data races are often triggered by non-deterministic event orderings that are hard to detect when testing complex distributed systems.

In this paper, we propose Spider, an automated tool for identifying data races in distributed system traces. Spider encodes the causal relations between the events in the trace as a symbolic constraint model, which is then fed into an SMT solver to check for the presence of conflicting concurrent accesses. To reduce the constraint solving time, Spider employs a pruning technique aimed at removing redundant portions of the trace.

In this paper, we present sasa, an open-source SimulAtor of Self-stabilizing Algorithms. Self-stabilization defines the ability of a distributed algorithm to recover after transient failures. sasa is implemented as a faithful representation of the atomic-state model. This model is the most commonly used in the self-stabilizing area to prove both the correct operation and complexity bounds of self-stabilizing algorithms.

The design of sasa relies as much as possible on existing tools: ocaml, dot, and tools developed in the Synchrone Group of the VERIMAG laboratory.

Detect and Avoid (DAA) systems are safety enhancement software applications that provide situational awareness and maneuvering guidance to aid aircraft pilots in avoiding and remaining well clear from other aircraft in the airspace. This paper presents a graphical toolkit, called DAA-Displays, designed to facilitate the assessment of compliance of DAA software implementations to formally specified functional and operational requirements. The toolkit integrates simulation and prototyping technologies allowing designers, domain experts, and pilots to compare the behavior of a DAA implementation against its formal specification. The toolkit has been used to validate an actual software implementation of DAA for unmanned aircraft systems against a standard reference algorithm that has been formally verified.

We present ScAmPER, a tool that generates test suites that maximise coverage for a class of interactive fiction computer games from the early 1980s. These games customise a base game engine with scripts written in a simple language. The tool uses a heuristic-guided search to evaluate whether these lines of code can in fact be executed during gameplay and, if so, outputs a sequence of game inputs that achieves this. Equivalently, the tool can be seen as attempting to generate a set of test cases that maximises coverage of the scripted code. The tool also generates a visualisation of the search process.