Computer Safety, Reliability, and Security

This paper describes the challenges involved in arguing the safety of highly automated driving functions which make use of machine learning techniques. An assurance case structure is used to highlight the systems engineering and validation considerations when applying machine learning methods for highly automated driving. Particular focus is placed on addressing functional insufficiencies in the perception functions based on convolutional neural networks and possible types of evidence that can be used to mitigate against such risks.

A thought experiment on evolution of assurance argument is performed on the basis of an interview with a manufacturer that applied for a certification of conformance of their in-house software life cycle to a safety standard. The working hypothesis of the experiment is that assurance cases help find problems in arguments on software life cycle and improve the life cycle. Based on the result of the thought experiment, questions for further empirical studies are generated and the ontology of relevant information items are analysed.

Assurance cases have been developed to reason and communicate about the trustworthiness of systems. Recently we have also been using them to support the development of policy and to assess the impact of security issues on safety regulation. In the example we present in this paper, we worked with a safety regulator (anonymised as A Regulatory Organisation (ARO) in this paper) to investigate the impact of cyber-security on safety regulation.

Assurance cases are developed and maintained in parallel with corresponding system models and therefore need to reference each other. Managing the correctness and consistency of interrelated safety argument and system models is essential for system dependability and is a nontrivial task. The model interface presented in this paper enables a uniform process of establishing and managing assurance case references to various types of system models. References to system metamodels are specified in an argument pattern and then used for assurance case instantiation. The proposed approach permits incremental development of assurance cases that maintain consistency with corresponding system models throughout the system development life cycle.

Tools for creating safety cases currently on the market target safety experts, whose main concern is the management of safety cases. However, for safety assurance, safety experts should collaborate with technical experts, who have better understanding of technical and operational hazards. Thus, there should be a closer collaboration between the management of safety cases and technical expertise. Technical expertise may be retrieved, among others, from model-based system artifacts and processes. In order to close the gap between safety and technical expertise, we present ExplicitCase, an open-source tool for semi-automatic modeling, maintenance, and verification of safety cases integrated with system models. The advantage of this tool is two-fold. First, it enables its users to capture safety relevant information from model-based artifacts into safety cases. Second, it makes the safety cases rationale available to engineers in order to help them reason about design choices, while minding safety concerns. We evaluate the approach and the implemented tool based on the experiences obtained in a project use case.

This paper presents “D-Case Communicator”, a web-based GSN editor which facilitates co-authoring of GSN diagrams by (possibly) remote stakeholders. D-Case Communicator is easy to use: it can be used in typical web-browsers such as Chrome, Firefox, and Safari; Editing is smooth as it is implemented using recent web technologies. This paper explains basic specification, usage, and design rationale of the tool. D-Case Communicator is available in https://mlab.ce.cst.nihon-u.ac.jp/dcase/.

A growing threat to the cyber-security of embedded safety-critical systems calls for a new look at the development methods for such systems. One alternative to address security and safety concerns jointly is to use the perspective of modeling using system theory. Systems-Theoretic Process Analysis (STPA) is a new hazard analysis technique based on an accident causality model. NIST SP 800-30 is a well-known framework that has been largely employed to aid in identifying threats event/source and vulnerabilities, determining the effectiveness security control, and evaluating the adverse impact of risks. Safety and security analyses, when performed independently, may generate conflicts of design constraints that result in an inconsistent design. This paper reports a novel integrated approach for safety analysis and security analysis of systems. In our approach, safety analysis is conducted with STPA while security analysis employs NIST SP800-30. It builds on a specification of security and safety constraints and outlines a scheme to automatically analyze and detect conflicts between and pairwise reinforcements of various constraints. Preliminary results show that the approach allows security and safety teams to perform a more efficient analysis.

Approaches to the safety analysis of software-intensive systems are being adapted to also provide security assurance. Extensions have been proposed to reflect the specific nature of security analysis by introducing intention as a causal factor to reaching unsafe state of the system, or by introducing new layers in the system modelling to model its surface of attack.In this paper we propose to extend these approaches by modelling the attacks perspective alongside the system. We explain how such modelling could be used to verify the coverage of the security analysis and facilitate its maintenance.

As safety-critical systems increasingly rely on computing, communication, and control, there have been a number of safety and security co-analysis methods put forth to identify, assess, and mitigate risks. However, there is an ideological gap between qualitative system-level methods that focus on control interactions, and more traditional methods based on component failure and/or vulnerability. The growing complexity of cyber-physical and socio-technical systems as well as their interactions with their environments seem to demand a systems-theoretic perspective. Yet, at the same time, more complex threats and failure modes imply a greater need for risk-based analysis to understand and prioritize the large volume of information. In this work we identify promising aspects from two existing safety/security co-analysis methods and outline a vision for reconciling them in a new analysis method.

This article proposes approaches supporting the analysis of code vulnerabilities based on overlapping machine instructions of variable length. For the purpose of focusing the search for potential malicious code it is suggested to apply first disassembling techniques allowing for a restriction of potentially exploitable memory space. Successively, testing based on heuristic optimization may be applied in order to evaluate dynamically the practicality of vulnerability exploitation.

Dependability is crucial in Safety Critical Cyber Physical Systems (CPS). In spite of the research carried out in recent years, implementation and certification of such systems remain costly and time consuming. In this paper, a framework for Statecharts based SW component development is presented. This framework called CRESC (C++ REflective StateCharts), in addition to assisting in transforming a Statechart model to code, uses reflection to make the model available at Run Time. Thus, the SW components can be monitored at Run Time in terms of model elements. Our framework helps the developer separate monitoring from functionality. Any monitoring strategy needed to increase dependability can be added independently from the functional part. The framework was implemented in C++ because this programming language, together with the Statechart formalism constitute widely used choices for the Safety Critical CPS domain.

In the sandbox world of cyber-physical systems and internet-of-things a number of applications is only eclipsed by a number of products that provide solutions for specific problem or set of problems. Initiatives like the European project $$EMC^2$$ serve as cross-disciplinary incubators for novel technologies and fuse them together with state-of-the-art industrial applications. This paper reflects on challenges in scope of hardware architectures and related technologies. It also provides a short overview of several technologies explored in the project that provide bridging solutions for these problems.

Application complexity in safety-critical systems is currently creating an immediate need to employ new model-based approaches to ensure system’s safe operation in high performances. At the same time, hardware evolution through multicore and hybrid architectures, while serving performance requirements, has not been realized as a safe and technology-ready solution to be employed in critical domains. In this paper, we report our experiences on the development of a model-based design workflow for safety assurance in mixed-critical applications executed on multicore platforms. Starting from our application specification, we develop intermediate models and extract configuration parameters that help us define a task optimization problem. Tasks composing the application will be weighted according to their criticality degree, allowing us to solve an optimization problem for safe resource and time partitioning at the available multicore resources. Based on code-generation techniques, we automatically generate an optimal and safe schema to be implemented in a real-time operating system, safeguarding the multicore resources from errors while executing the tasks. Indicative results are being presented by a prototype tool developed for a case study while we reason about the applicability of the approach.

Safety-critical applications could benefit from the standardisation, cost reduction and cross-domain suitability of current heterogeneous computing platforms. They are of particular interest for Mixed-Criticality Product Lines (MCPL) where safety- and non-safety functions can be deployed on a single embedded device using suitable isolation artefacts and development processes. The development of MCPLs can be facilitated by providing a reference architecture, a model-based design, analysis tools and Modular Safety Cases (MSC) to support the safety claims.In this paper, we present a method based on the MSCs to ease the certification of MCPLs. This approach consists of a semi-automated composition of layered argument fragments that trace the safety requirements argumentation to the supporting evidences. The core of the method presented in this paper is an argument database that is represented using the Goal Structuring Notation language (GSN). The defined method enables the concurrent generation of the arguments and the compilation of evidences, as well as the automated composition of safety cases for the variants of products. In addition, this paper exposes an industrial-grade case study consisting of a safety wind turbine system where the presented methodology is exemplified.

Not available services or service interruption could have different impact to our social life. Emails or messages which are not delivered in a proper time-frame could lead to omit a meeting or a discussion with colleagues. Interconnected CPS in different domains, like autonomous driving, smart grids, Industry 4.0, needs a guaranteed and safe delivery of information.Nowadays distributed application in critical infrastructures such as transportation (e.g. air traffic management, train control, traffic management), financial services, or electricity systems, are often implemented in dedicated network infrastructures not using the public Internet. This leads to high expenditures (CAPEX and OPEX) for the companies to maintain these separated and dedicated telecommunication infrastructure.Our approach in this work is to verify concepts to share the Internet, as a telecommunication infrastructure for critical and non-critical applications. This reduces the effort to implement and to manage different communication architectures. The present work develops and evaluates methods and procedures that enable high reliable communication between two endpoints over several shared telecommunication networks for future critical and uncritical applications. Our approach shows that it is possible to use the public Internet for future communication requirements in a converged network. Further innovations include the integration of novel network technologies, such as software-defined networks (SDN), programming protocol independent packet processors (P4), and self-adaptive and autonomous network management functions.

In collaborative automation systems, providing both security and safety assessments are getting increasingly important. As IoT systems gain momentum in the industrial domain, experts stress their concerns about security and safety. Improperly or carelessly deployed and configured systems hide security threats, and even raise issues on safety, as their behavior can threaten human life. The cloud based back-ends are getting used for processing sensor data – on the other hand, legacy equipment, which may contain sensitive information, is made interoperable with broader infrastructure. Safety risks can be triggered by attacks on the backend and confidential information is at risks by attacks on legacy equipment.In order to maintain safe and secure operations, safety and cyber-security assessment methods have been established. There is an increased demand in modern industrial systems to perform these regularly. These methods however require a lot of time and effort to complete. A solution to this problem would be combining the assessments. This requires that proper safety and security analysis methods must be selected – those that have compatible elements.In this paper we propose a method that combines the elements of existing methodologies, in order to make the safety and security analysis process more effective. Furthermore, we present a case study, where we verified the combined methodology.

The application of Industry 4.0 in automation systems leads to a higher interconnectivity among machines, devices, sensors, the cloud and humans. Nevertheless, this paradigm leaves open the possibility of new cyber-security threats and attacks against industrial control systems, even for those that perform safety-critical functions. Consequently, software updates are needed in order to fix the vulnerabilities and bugs discovered on these systems. This article presents a review of safety and security standards with respect to software updates. In addition to this, a roadmap of standards for the development of safe and secure systems is provided.

In response to the recent Jeep hacking and recalls based on information security vulnerability in 2015, the significance of secure system design has become increasingly important in the automotive industry. From this perspective, security guidelines such as JASO TP 15002 and SAE J3061 have been published. To realize future connected-car systems or the future autonomous driving in line with these guidelines, many automotive Original Equipment Manufacturers (OEMs) and their major suppliers are now developing key components such as central gateways (CGW), telematics, or end Electronic Control Units (ECUs), with theses security concerns in mind. In this paper, we focus on a security evaluation that consists of model definition, threat identification, and the risk analysis in JASO TP 15002. To do so we first identify gaps between an understanding of JASO TP15002 and implementation of secure system design based on it. We then present a detailed analysis which includes new methods to fill this gap using illustrative examples such as CGW. As a result, we provide a solution with an improvement in terms of work efficiency over typical methods according to the JASO TP 15002.

Establishing collaboration processes of systems in an open and dynamically changing environment like the automotive domain will inescapably lead to a varying availability of shared services. A vivid example is driving in a platoon, where smaller distances between vehicles are made possible due to additional safety related runtime guarantees provided by surrounding vehicles. In such collaboration scenarios environmental conditions can change, driving behavior from surrounding vehicles may not be adequate or hardware/software failure of involved systems may occur. For safety critical use cases like platooning, such degraded or even missing collaboration capabilities can rapidly lead to hazardous situations due to the highly dynamic context. When such events occur, only an immediate and situation adapted reaction behavior can prevent physical or material damage. For the certification of such described dynamic collaboration processes, it is therefore essential to develop a conclusive safety concept for each individual system, which also considers the return to a safe mode. The presented “Dynamic Safety Contracts” approach enables a systematic composition of available services at runtime to extend or reduce allowed degrees of freedom for a system involved in a dynamic collaboration scenario.

Medical Cyber Physical Systems of Systems (MCPSoS) refer to a set of systems that flexibly collaborate at runtime in order to render higher level functionality. Most systems in a MCPSoS offer a generic piece of functionality so that they can contribute to many totally different collaboration scenarios. Consequently, it is unknown at design time which systems will how collaborate at runtime. This unpredictability leads to new challenges for the assurance of safety, because established approaches always build on the assumption that systems and their environments are completely known. We believe that the safety research community has to pull together in order to tackle the challenge of unpredictability and that this requires an appropriate taxonomy in order to establish a common understanding of the challenge and related solutions. To this end, we propose enhancements based on a widely accepted taxonomy for dependable computing with respect to the system-of-systems aspect. Further, we will use the taxonomy to reflect on the new challenge of unpredictability and related solutions from the state-of-the-art, namely, safety contracts and dynamic risk assessment. Finally, we motivate an integration of the safety contracts and dynamic risk assessment and present some ideas on this integration. Throughout the paper, we use a real-world example to exemplify our proposed taxonomy and our thoughts.

In this paper, we propose a novel approach to ensuring safety while planning and controlling an operation of swarms of drones. We derive the safety constraints that should be verified both during the mission planning and at the run-time and propose an approach to safety-aware mission planning using evolutionary algorithms. High performance of the proposed algorithm allows us to use it also at run-time to predict and resolve in a safe and optimal way dynamically emerging hazards. The benchmarking of the proposed approach validate its efficiency and safety.

Understanding and following safety standards with their text can be difficult. Ambiguity and inconsistency, among other issues, can easily arise. As a solution, several authors argue for the explicit representation of the standards with models, which can be created with semantic technologies such as ontologies. However, this possibility has received little attention. The few authors that have addressed it have also only dealt with a subset of safety standard aspects and have used technologies not usually applied for critical systems engineering. As a first step towards addressing these issues, this position paper presents our initial work on the representation of safety standards with Knowledge Manager, a tool used in industrial environments that exploits semantic technologies to manage domain information. The proposal also builds on prior work on the specification of safety compliance needs with a holistic generic metamodel. We describe how to use Knowledge Manager to specify the concepts and relationships of the metamodel for a given safety standard, and discuss the application and benefits of the corresponding representation.

Currently developed automotive systems exhibit an increased level of automation as well as an ever-tighter integration with other vehicles, traffic infrastructure and cloud services. Thus, just as safety became a critical part of the development in the late 20th century, the automotive domain must now consider cyber-security as an integral part of the development of modern vehicles. Novel features, such as advanced driver assistance systems or automated driving functions drive the need for built-in security solutions and cyber-security aware system design. Unfortunately, there is still a lack of experience with security concerns in the context of safety engineering in general and in the automotive safety departments in particular. A European partnership developed a skill set, training materials and best practices for ISO 26262 in the context of the EU project SafEUr. This working party (SoQrates working group) shares knowledge and experiences and integrated the Automotive SPICE assessment model with functional safety requirements, which was further used in integrated Automotive SPICE and safety assessments. The members of the SoQrates working group are, to a large extent, certified Automotive SPICE assessors dealing with security-related project in practice. From 2016 onwards, the SoQrates working party started to analyse the SAE J3061 cyber-security guidebook and integrated the additional requirements of SAE J3061 into this assessment model. This paper will summarise the previous results and extensions of the assessment model and the working group’s vision, how an Automotive SPICE assessor can support also the auditing of projects with close security relation.

Automotive systems become increasingly complex due to their functional range and data exchange with the outside world. Until now, functional safety of such safety-critical electrical/electronic systems has been covered successfully. However, the data exchange requires interconnection across trusted boundaries of the vehicle. This leads to security issues like hacking and malicious attacks against interfaces, which could bring up new types of safety issues. Before mass-production of automotive systems, arguments supported by evidences are required regarding safety and security. Product engineering must be compliant to specific standards and must support arguments that the system is free of unreasonable risks.This paper shows a safety and security co-engineering framework, which covers standard compliant process derivation and management, and supports product specific safety and security co-analysis. Furthermore, we investigate process- and product-related argumentation and apply the approach to an automotive use case regarding safety and security.

Nuclear power plants set strict requirements for their suppliers. Need for digital systems containing software increases as analog technology is maintained and replaced. We have used process assessments to evaluate safety-related systems development and developed a tailored assessment method for that. Selection of a capable supplier is a key to successful system delivery and qualification. Process assessments are found to be a cost-efficient way to analyze systems development. This paper provides a practical example in applying a process assessment method in supplier evaluation. A similar approach could be exploited in other domains, where domain specific requirements are essential. Benefits of the approach are discussed based on the experiences so far.

Adaptivity is a consequential requirement for software systems that allow integration of components or devices at runtime. Dynamic integration of components and a subsequent reconfiguration during operation causes change in both functional and non-functional properties of the system. Since these systems often operate in Safety-Critical environment, safety becomes a crucial characteristic to be taken under consideration during reconfiguration. In this paper, we introduce a dynamic metrics based runtime risk assessment approach for safe reconfiguration in open adaptive systems. We combine design time safety analysis and runtime monitoring to evaluate risk factors of potential configurations of an adaptive component at runtime. Based on the evaluated risk factors the configurations are assigned a dynamic rank in an increasing order of their risk. During reconfiguration the adaptive component conforms to the ranking, thereby activating the configuration with lowest associated risk.

Automated cooperation is arriving in practice, for instance in vehicular automation like platoon driving. The development and safety assurance of those systems poses new challenges, as the participating nodes are not known at design time; they engage in communication at runtime and the system behaviour can be distorted at any time by failures in some participant or in the communication itself. When running on a highway, simply switching off the function is not an option, as this would also result in hazardous situations. Graceful degradation offer a systematic approach to define a partial-order of less and less acceptable operation modes, of which the best achievable is selected in presence of failures. In this work we propose an approach for assurance of the degradation cascades based on mode-specific assertions, captured by assumption/guarantee contracts. More specifically, we share our experiences and methodology for specifying the contracts for both the nominal safe behaviour as well as the less safe but acceptable behaviour in presence of failures. Furthermore, we present an argument pattern for adequacy of the degradation cascades for meeting the global safety goals based on the contracts. We illustrate our approach by a car platooning case study.

The European General Data Protection Regulation defines a set of obligations for personal data controllers and processors. Primary obligations include: obtaining explicit consent from the data subject for the processing of personal data, providing full transparency with respect to the processing, and enabling data rectification and erasure (albeit only in certain circumstances). At the core of any transparency architecture is the logging of events in relation to the processing and sharing of personal data. The logs should enable verification that data processors abide by the access and usage control policies that have been associated with the data based on the data subject’s consent and the applicable regulations. In this position paper, we: (i) identify the requirements that need to be satisfied by such a transparency architecture, (ii) examine the suitability of existing logging mechanisms in light of said requirements, and (iii) present a number of open challenges and opportunities.

It is well understood that security informatics is constrained by the availability of reliable data sources, which limits the development of robust methods for measuring the impact of data breaches. To date, empirical data breach analysis has largely relied upon the use of economic and financial data associated with an organisation as a measure of impact. To provide an alternative, complementary approach, we explore monetary fines resulting from data protection regulatory actions to understand how the data can inform the evaluation of data breaches. The results indicate where context matters and also provide information on the wider challenges faced by organisations managing personal data.

Legal compliance-by-design is the process of developing a software system that processes personal data in such a way that its ability to meet specific legal provisions is ascertained. In this paper, we describe techniques to automatically check the compliance of the security policies of a system against formal rules derived from legal provisions by re-using available tools for security policy verification. We also show the practical viability of our approach by reporting the experimental results of a prototype for checking compliance of realistic and synthetic policies against the European Data Protection Directive (EU DPD).

Testing access control policies relies on their execution on a security engine and the evaluation of the correct responses. Coverage measures can be adopted to know which parts of the policy are most exercised. This paper proposes an access control infrastructure for enabling the coverage criterion selection, the monitoring of the policy execution and the analysis of the policy coverage assessment. The framework is independent from the policy specification language and does not require the instrumentation of the evaluation engine. We show an instantiation of the proposed infrastructure for assessing the XACML policy testing.

Human gait seamless continuous authentication, based on wearable accelerometers, is a novel biometric instrument which can be exploited to identify the user of mobile and wearable devices. In this paper, we present a study on recognition of user identity, by analysis of gait data, collected through body inertial sensors from 175 different users. The mechanism used for identity recognition is based on deep learning machinery, specifically on a convolutional network, trained with readings from different sensors, and on filtering and buffering mechanism to increase the accuracy. Results show a very high accuracy in both recognizing known and unknown identities.

The burst in smartphone use, handy design in laptops and tablets as well as other smart products, like cars with the ability to drive you around, manifests the exponential growth of network usage and the demand of accessing remote data on a large variety of services. However, users notoriously struggle to maintain distinct accounts for every single service that they use. The solution to this problem is the use of a Single Sign On (SSO) framework, with a unified single account to authenticate user’s identity throughout the different services. In April 2007, AOL introduced OpenAuth framework. After several revisions and despite its wide adoption, OpenAuth 2.0 has still several flaws that need to be fixed in several implementations. In this paper, we present a thorough review about both benefits of this single token authentication mechanism and its open flaws.

In this paper we present PolEnA, an extension of the Android Security Framework (ASF). PolEnA enables a number of features that are not currently provided by the ASF. Among them, PolEnA allows for the definition of fine-grained security policies and their dynamic verification. The runtime enforcement of the policies is supported by a state-of-the-art SAT solver. One of the main features of our approach is the low invasiveness as it does not require modifications to the operating system.

Mobility data are an important proxy to understand the patterns of human movements, develop analytical services and design models for simulation and prediction of human dynamics. Unfortunately mobility data are also very sensitive, since they may contain personal information about the individuals involved. Existing frameworks for privacy risk assessment enable the data providers to quantify and mitigate privacy risks, but they suffer two main limitations: (i) they have a high computational complexity; (ii) the privacy risk must be re-computed for each new set of individuals, geographic areas or time windows. In this paper we explore a fast and flexible solution to estimate privacy risk in human mobility data, using predictive models to capture the relation between an individual’s mobility patterns and her privacy risk. We show the effectiveness of our approach by experimentation on a real-world GPS dataset and provide a comparison with traditional methods.

The automotive domain is undergoing a tremendous transformation in the speed and depth of technological development in recent years. Most of the innovations are based on electronics and ICT. As it is the case for most ICT-based systems, there are increasing concerns about security and privacy in the automotive domain. In this paper, we present a technical and social analysis of this issue using a methodological scenario building approach. We believe that current and future solutions must take both technical and social aspect into consideration. Our analysis provides stakeholders with such a view.

We present an approach to reduce the complexity of adjusting privacy preferences for multiple online social networks. To achieve this, we quantify the effect on privacy for choices that users make, and simplify configuration by introducing privacy configuration as a service. We present an algorithm that effectively measures privacy and adjusts privacy settings across social networks. The aim is to configure privacy with one click.

We demonstrate the application of a new domain-specific language (DSL) for modeling Hierarchical State Machines (HSMs) to the software that manages communications for the Curiosity Mars rover.

Synchronous (or static) dataflow (SDF) is deemed the most stable and mature model to represent streaming systems. It is useful, not only to reason about functional behavior and correctness of such systems, but also about non-functional aspects, in particular timing and performance constraints. When talking about performance, throughput is a key metric. Within the SDF domain, hierarchical SDF models are of special interest as they enable compositional modeling, which is a necessity in the design of large systems.Techniques exist to analyze throughput of synchronous dataflow models. If the model is hierarchical, it first needs to be flattened before these techniques can be applied (for exact analysis at least). Furthermore, all of these techniques are adversely affected by the increase in the graph’s repetition vector entries. In this paper, for a loosely defined class of hierarchical synchronous dataflow models, we argue that these dependence issues can be mitigated by taking advantage of the hierarchical structure rather than by flattening the graph. We propose a hierarchical extension to an existing technique that is based on the (max,+) algebraic semantics of SDF.