The 4+1 Principles of Software Safety Assurance and Their Implications for Scrum

This study is part of the research under the High Integrity System Engineering Group, Computer Science Department, of the University of York. This paper introduces the 4+1 Principles of Software Safety Assurance [1] and their implications for Scrum [2], specifically, the impact on the processes, roles and artefacts associated with Scrum development.

Historically, there has been a reluctance to adopt agile methods within safety-critical systems development. However, feedback from our initial research in this area suggests that there are benefits to be gained from the application of agile methods to safety critical systems [3, 8]. Following this feedback we have done further work to assess how the 4+1 principles of software safety assurance can be integrated with Scrum, and have developed an initial proposal for how Scrum could be modified to better address the principles. The aim of the proposed study at XP2016 is gain practitioner feedback on these proposals. The feedback we receive will ultimately be used to help refine the proposal before further empirical evaluation.

Our research, as a whole, is focused on answering the following questions:

This study, specifically furthers our investigation into RQ2 and 3. We now how have initial proposals for changes and a description of assurance case development integrated with Scrum. However, what we lack is a substantial and varied practitioner base to help assess the credibility, feasibility, and efficacy of our proposals.

Despite progress in the use of agile development methods in safety critical systems development (e.g. [4]), there are still those with doubts about the potential for successful integration. There are also reported experiences [5] that highlight the complementary nature of the iterative and incremental approach underlying many agile methods, and recognised best practice in risk management in safety critical systems development. Rather than start with a theoretical evaluation of the compatibility of the principles of agile development with software safety assurance, we decided to draw out these experiences, opinions (and possibly preconceptions) by means of a practitioner’s semi-structured interview. In particular, our first round of semi-structured interviews drew out specific responses relating to (possible) incremental and iterative nature of safety requirements development, hazard analysis and safety (assurance) case developments. The responses we received showed both the potential for benefits from agile development of safety-critical software, together with residual concerns about the ability to provide software safety assurance in a manner compatible with current software safety assurance standards. Rather than focusing on a single safety assurance standard (as some have done, e.g. [4]) we have used the framework of the 4+1 software safety assurance principles to tackle the common and broad issues of software safety assurance that exist across multiple industry domains and safety standards. These principles have been developed to highlight the commonality of purpose of multiple existing safety standards, and are being adopted by industry (e.g. in Defence Standard 00-55) as a framework against which software safety assurance can be judged.

The importance of this research is that it represents one step along the path of pushing beyond simplistic and over-generalised preconceptions of the compatibility of agile and safety-critical systems development, and potentially unlocking the benefits of agility within the safety domain.

It is not easy task to find practitioners with both experience in the field of Agile and Safety. However, XP2016 will involve various categories of experts (software engineers, industry and academia etc.) as well there being a significant opportunity to link this study with the XP2016 workshop Agile Development of Safety-critical Software (ASCS).

We would like to interview individuals who have been involved with Safety Critical-Systems, Agile methods, or both, during XP2016 in order to use their experience and insight to gain feedback on our proposed approach.

The study will be conducted as a qualitative survey using “semi-structured interviews” for data collection [6]. Shull et al. [7] illustrate the advantage and disadvantage of conducting semi-structured interview. The interview will include some simple (e.g. Likert scale-based) question, as well as more open-ended questions that allow for greater depth of response.

The responses received from XP2016 will also be compared with 1-to-1 semi-structured interviews conducted with some of the respondents from our initial survey [3]; the purpose of this interview study is to investigate the success of the proposed integration of 4+1 principles and assurance safety case development with Scrum.

Interviews will be conducted face-to-face at the XP2016 location. Further interviews (with further participants) may be conducted over phone. Interviews will be recorded and transcribed to facilitate subsequent analysis.

Participants would take part in an approximately 40 min interview to explore perceptions around the 4+1 Principles of Software Safety Assurance and their implications for Scrum.

The interview will start by introducing the research aims and the topics to be discussed. Then the 4+1 principles will be explained, together with an outline of the proposal for integrating these principles within a Scrum development. Questions will then be asked relating to the proposal – picking out specific features one-by-one (e.g. our recommendations for team composition). The questions will tackle both aspects of (a) whether the proposed approach challenges agility and (b) whether the proposed approach challenges safety assurance.

Documents that will be prepared for the interview:

Transcripts will be analysed using thematic analysis. One researcher will read all of the interview transcripts, and will code the transcripts using first-cycle coding (Open Coding or Initial Coding), supported by the NVivo 11 software package. Main categories (or topics) will be identified through clustering of codes in project review meetings. Codes and categories will be constantly compared with the data and revised or refined as appropriate.

The results of the thematic analysis will then be written up in a form suitable for sharing with participants and subsequent publication.

Initially, we will present the key findings within the High Integrity System Engineering Group at York. The findings will also potentially form part of the final thesis of the ongoing PhD research on Assurance Case Integration with An Agile Development Method. Our findings will be submitted, in the future, for publication in peer-reviewed journals.

Ultimately, the findings will be used to refine our proposed approach before proceeding to empirical case studies.