Computer Safety, Reliability, and Security

This paper considers the design-phase safety analysis of vehicle guidance systems. The proposed approach constructs dynamic fault trees (DFTs) to model a variety of safety concepts and E/E architectures for drive automation. The fault trees can be used to evaluate various quantitative measures by means of model checking. The approach is accompanied by a large-scale evaluation: The resulting DFTs with up to 300 elements constitute larger-than-before DFTs, yet the concepts and architectures can be evaluated in a matter of minutes.

This paper presents a novel method for rare event simulation of dynamic fault trees with complex repairs that requires only a modest number of simulations, while retaining statistically justified confidence intervals. Our method exploits the importance sampling technique for rare event simulation, together with a compositional state space generation method for dynamic fault trees.

We demonstrate our approach using three parameterized sets of case studies, showing that our method can handle fault trees that could not be evaluated with either existing analytical techniques, nor with standard simulation techniques.

In this paper, we investigate the pondered selection of innovative software verification technology in the safety-critical domain and its implications. Verification tools perform analyses, testing or simulation activities. The compliance of the techniques implemented by these tools to fulfill standard-mandated objectives (i.e., to be means of compliance in the context of DO-178C and related supplements) should be explained to the certification body. It is thereby difficult for practitioners to use novel techniques, without a systematic method for arguing their appropriateness. Thus, we offer a method for arguing the appropriate application of a certain verification technique (potentially in combination with other techniques) to produce the evidence needed to satisfy certification objectives regarding fault detection and mitigation in a realistic avionics application via safety cases. We use this method for the choice of an appropriate compiler to support the development of a drone.

Confidence in safety critical systems is often justified by safety arguments. The excessive complexity of systems nowadays introduces more uncertainties for the arguments reviewing. This paper proposes a framework to support the argumentation assessment based on experts’ decision and confidence in the decision for the lowest level claims of the arguments. Expert opinion is extracted and converted in a quantitative model based on Dempster-Shafer theory. Several types of argument and associated formulas are proposed. A preliminary validation of this framework is realized through a survey for safety experts.

Like most systems, automotive software systems evolve due to many reasons including adding, removing or modifying features, fixing bugs, or improving system quality. In this context, safety cases, used to demonstrate that a system satisfies predefined safety requirements, often dictated by a standard such as ISO 26262, need to co-evolve. A necessary step is performing an impact assessment to identify how changes in the system affect the safety case. In previous work, we introduced a generic model-based impact assessment approach, that, while sound, was not particularly precise. In this work, we show how exploiting knowledge about system changes, the particular safety case language, and the standard can increase the precision of the impact assessment, reducing any unnecessary revision work required by a safety engineer. We present six precision improvement techniques illustrated on a GSN safety case used with ISO 26262.

Human-Robot Collaboration is increasingly prominent in people’s lives and in the industrial domain, for example in manufacturing applications. The close proximity and frequent physical contacts between humans and robots in such applications make guaranteeing suitable levels of safety for human operators of the utmost importance. Formal verification techniques can help in this regard through the exhaustive exploration of system models, which can identify unwanted situations early in the development process. This work extends our SAFER-HRC methodology with a rich non-deterministic formal model of operator behaviors, which captures the hazardous situations resulting from human errors. The model allows safety engineers to refine their designs until all plausible erroneous behaviors are considered and mitigated.

SPARK 2014 is a modern programming language and a new state-of-the-art tool set for development and verification of high-integrity software. In this paper, we explore the capabilities and limitations of its latest version in the context of building a flight stack for a high-altitude unmanned glider. Towards that, we deliberately applied static analysis early and continuously during implementation, to give verification the possibility to steer the software design. In this process we have identified several limitations and pitfalls of software design and verification in SPARK, for which we give workarounds and protective actions to avoid them. Finally, we give design recommendations that have proven effective for verification, and summarize our experiences with this new language.

Our approach of using BaCs to identify recoverable states is computationally cheaper and potentially more accurate (less conservative) than existing approaches based on state-space exploration. We apply our technique to two hybrid systems: a water tank pump and a stop-sign-obeying controller for a car.

The verification and validation (V&V) of autonomous systems is a complex and difficult task, especially when artificial intelligence is used to achieve autonomy. However, without proper V&V, sufficient evidence to argue safety is not attainable. We propose in this work the use of a Safety Supervisor (SSV) to circumvent this issue. However, the design of an adequate SSV is a challenge in itself. To assist in this task, we present a conceptual framework and a corresponding metamodel, which are motivated and justified by existing work in the field. The conceptual framework supports the alignment of future research in the field of runtime safety monitoring. Our vision is for the different parts of the framework to be filled with exchangeable solutions so that a concrete SSV can be derived systematically and efficiently, and that new solutions can be embedded in it and get evaluated against existing approaches. To exemplify our vision, we present an SSV that is based on the ISO 22839 standard for forward collision mitigation.

When arguing safety for an autonomous road vehicle it is considered very hard to show that the sensing capability is sufficient for all possible scenarios that might occur. Already for today’s manually driven road vehicles equipped with advanced driver assistance systems (ADAS), it is far from trivial how to argue that the sensor systems are sufficiently capable of enabling a safe behavior. In this paper, we argue that the transition from ADAS to automated driving systems (ADS) enables new solution patterns for the safety argumentation dependent on the sensor systems. A key factor is that the ADS itself can compensate for a lower sensor capability, by for example lowering the speed or increasing the distances. The proposed design strategy allocates safety requirements on the sensors to determine their own capability. This capability is then to be balanced by the tactical decisions of the ADS equipped road vehicle.

We develop a notion of safety architecture, based on an extension to bow tie diagrams, to characterize the overall scope of the mitigation measures undertaken to provide safety assurance in the context of unmanned aircraft systems. We use a formal semantics as a basis for implementation in our assurance case tool, AdvoCATE. We also describe the functionality that a safety architecture affords to support both the related safety analysis and subsequent development activities. We motivate the need for a safety architecture through an example based upon a real safety case, whilst also illustrating its application and utility. Additionally, we discuss its role, when combined with structured arguments, in providing a more comprehensive basis for the associated safety case.

The availability of functionality is a crucial aspect of mission- and safety-critical systems. This is for instance demonstrated by the pursuit to automate road transportation. Here, the driver is not obligated to be part of the control loop, thereby requiring the underlying system to remain operational even after a critical component failure. Advances in the field of mixed-criticality research have allowed to address this topic of fail-operational system behaviour more efficiently. For instance, general purpose computing platforms may relinquish the need for dedicated backup units, as their purpose can be redefined at runtime. Based on this, a deterministic and resource-efficient reconfiguration mechanism is developed, in order to address safety concerns with respect to availability in a generic manner. To find a configuration for this mechanism that can ensure all availability-related safety properties, a design-time method to automatically generate schedules for different modes of operations from declaratively defined requirements is established. To cope with the inherent computational complexity, heuristics are developed to effectively narrow the problem space. Subsequently, this method’s applicability and scalability are respectively evaluated qualitatively within an automotive case study and quantitatively by means of a tool performance analysis.

This paper introduces a static analysis technique for computing formally verified round-off error bounds of floating-point functional expressions. The technique is based on a denotational semantics that computes a symbolic estimation of floating-point round-off errors along with a proof certificate that ensures its correctness. The symbolic estimation can be evaluated on concrete inputs using rigorous enclosure methods to produce formally verified numerical error bounds. The proposed technique is implemented in the prototype research tool PRECiSA (Program Round-off Error Certifier via Static Analysis) and used in the verification of floating-point programs of interest to NASA.

The Classification Tree Method (CTM) is a structured and diagrammatic modeling technique for combinatorial testing. CTM can express the notion of “parameter shielding”, the phenomenon that some system parameters become invalidated depending on another system parameter. The current form of CTM, however, is limited in its expressiveness: it can only express parameter shielding that depends on a single parameter. In this paper, we extend CTM with parameter shielding that depends on multiple parameters, proposing CTM\(_{\textit{shield}}\). We evaluate the proposed extension on several industrial systems. The evaluation finds that parameter shielding often depends on multiple parameters in real systems, and the effectiveness of the extension.

This paper introduces a new lightweight tool for simulative error propagation analysis of Simulink models. The tool allows a user to inject different types of faults that are common for embedded control systems and analyze error propagation to critical system parts and outputs. The intended workflow comprises the following three steps: (i) setup faulty and critical blocks of a Simulink model, (ii) setup and run simulations, and (iii) observe and examine the obtained results. The tool is implemented in MATLAB using the callback block functions from the Simulink API. The graphical user interface allows the injection of several types of faults including computing hardware faults such as single and multiple bit-flips, sensor faults such as offsets, stuck-at faults, and a noise, and network faults such as time delays and packet drops. The fault occurrence and duration can be specified either with the classical reliability metrics like mean time to failure and mean time to repair, or failure rates with classical (normal, exponential, Poisson, Weibull etc.) or custom user-defined probability distributions. The error propagation to the selected critical blocks is reported with several statistical metrics including the mean number of errors, failure rate, and mean error value, as well as performance indexes such as integral squared error, integral absolute error, and integral time-weighted absolute error. The reported numerical results support standard reliability and safety assessment methods such as fault tree analysis and failure mode and effects analysis. The paper demonstrates the tool with a case study Simulink model of fault-tolerant control for a passenger jet.

As road vehicles increase their autonomy and the driver reduces his role in the control loop, novel challenges on dependability assessment arise. Model-based design combined with a simulation-based fault injection technique and a virtual vehicle poses as a promising solution for an early safety assessment of automotive systems. To start with, the design, where no safety was considered, is stimulated with a set of fault injection simulations (fault forecasting). By doing so, safety strategies can be evaluated during early development phases estimating the relationship of an individual failure to the degree of misbehaviour on vehicle level. After having decided the most suitable safety concept, a second set of fault injection experiments is used to perform an early safety validation of the chosen architecture. This double-step process avoids late redesigns, leading to significant cost and time savings. This paper presents a simulation-based fault injection approach aimed at finding acceptable safety properties for model-based design of automotive systems. We focus on instrumenting the use of this technique to obtain fault effects and the maximum response time of a system before a hazardous event occurs. Through these tangible outcomes, safety concepts and mechanisms can be more accurately dimensioned. In this work, a prototype tool called Sabotage has been developed to set up, configure, execute and analyse the simulation results. The feasibility of this method is demonstrated by applying it to a Lateral Control system.

Statically proving the adherence of a system to its safety requirements specified at design-time provokes overcautious systems with limited performance. Contrarily, dynamically assessing the quality of sensor observations at run-time enables adapting a system’s performance accordingly while maintaining its safety. While this could be shown by Brade et al. [2] for a simulated scenario with one-dimensional sensors, we apply the proposed scheme, that is, the Validity Concept to 3D depth information. In this endeavour, we define a failure model covering the failure types Noise, Outlier, and Illumination Artefacts, define functions to estimate their severity at run-time and represent a 3D point cloud’s quality in terms of validity information. Furthermore, we show that calculated Validity Values correlate with sensor failures impairing sensor observations and enable estimating the quality of subsequent applications.

Safety-critical systems must satisfy safety requirements ensuring that catastrophic consequences of combined component failures are kept below a certain probability occurrence threshold. Therefore, designers must define a hardened architecture of the system, which fulfils the required safety level by integrating safety mechanisms. We propose an automatic SMT-based synthesis methodology to harden an initial architecture for a given safety objective. The proposed ideas are experimented on an avionics flight controller case-study and several benchmarks.

The automotive industry relies increasingly on computer technology in their cars, which malicious attackers can exploit. Therefore, the Original Equipment Manufacturers (OEMs) have to adopt security engineering practices in their development efforts, in addition to their safety engineering efforts. In particular, information assets that can undermine safety have to be identified and protected. Assessing the safety relevance of specific information assets is best done by safety engineers, who, unfortunately, often do not have the security expertise to do so. In this paper, we propose a technique for identifying information assets and protection goals that are relevant for safety. Our method is based on security guide-words, which allow a structured identification of possible attack scenarios. The method is similar to the Hazard and Operability Study (HAZOP) in safety for eliciting possible faults. The similarity of the approach shall ease the effort for non-security engineers to identify information assets and protection goals to allow an exchange between safety and security mindsets. In contrary to other proposed methods, we performed an evaluation of our technique to show their practical application. In our evaluation with a total of 30 employees of an automotive supplier and employees of the University of Applied Sciences in Karlsruhe, results show that all non-security engineers achieved for precision, productivity and sensitivity, on average, higher values than the security control group.

We present the proposed security architecture Deutsche Bahn plans to deploy to protect its trackside safety-critical signalling system against cyber-attacks. We first present the existing reference interlocking system that is built using standard components. Next, we present a taxonomy to help model the attack vectors relevant for the railway environment. Building upon this, we present the proposed “compartmentalized” defence concept for securing the upcoming signalling systems.

Future automotive systems will exhibit increased levels of automation as well as ever tighter integration with other vehicles, traffic infrastructure, and cloud services. From safety perspective, this can be perceived as boon or bane - it greatly increases complexity and uncertainty, but at the same time opens up new opportunities for realizing innovative safety functions. Moreover, cybersecurity becomes important as additional concern because attacks are now much more likely and severe. Unfortunately, there is lack of experience with security concerns in context of safety engineering in general and in automotive safety departments in particular. To remediate this problem, we propose a systematic pattern-based approach that interlinks safety and security patterns and provides guidance with respect to selection and combination of both types of patterns in context of system engineering. The application of a combined safety and security pattern engineering workflow is shown and demonstrated by an automotive use case scenario.