Elsevier

Safety Science

Volume 47, Issue 8, October 2009, Pages 1185-1194
Safety Science

Human and organizational error data challenges in complex, large-scale systems

https://doi.org/10.1016/j.ssci.2009.01.008Get rights and content

Abstract

In complex, large-scale systems, event analyses are constrained by the quality of the data gathered, the maturity of the associated reporting system, and the training and background of the investigator and reporter. Such constraints place limits on the adequacy and strength of analyses conducted with the data. In this paper, we focus on the challenges of measuring performance variability in complex systems, using the lens of human and organizational error modeling. This paper begins with an overview of human and organizational error assessments, and then introduces the particular challenges of data needs in human reliability analyses. A case study of human and organizational error analysis in a complex, large-scale system, marine transportation in Puget Sound, is used to illustrate the impact of the data challenges on risk assessment processes. Suggestions for future research conclude the paper.

Introduction

Risk may be defined as a measure of the probability and severity of an unwanted event. Risk events occur for a variety of reasons, as seen in Fig. 1 (Harrald et al., 1998, Reason, 1997). Sometimes risk events are the result of basic or root causes, such as inadequate operator knowledge, skills or abilities, or the lack of a safety management system in an organization. Risk events could also result from immediate causes, such as a failure to apply basic knowledge, skills, or abilities, or an operator impaired by drugs or alcohol. Incidents are unwanted events that may or may not result in accidents; accidents are unwanted events that have either immediate or delayed consequences. Immediate consequences could include injuries, loss of life, property damage, and persons in peril; delayed consequences could include further loss of life, environmental damage, and financial costs. Fig. 1 depicts the risk event error chain, and illustrates that risk events often occur because the error chain cascades: a basic cause can occur and an immediate cause and an incident will trigger an accident (Reason, 1997). Absent risk reduction measures to interrupt the error chain, basic causes can cascade into immediate causes, which can cascade into an incident, which can trigger an accident. The key to risk mitigation, therefore, is to introduce risk reduction interventions at appropriate points in the error chain so as to prevent the cascade.

Fig. 1 suggests a linear, cascading process that leads from basic and root causes to the onset of an accident. However, recent experience with events such as the BP Texas City oil refinery explosion on March 23, 2005, which resulted in 15 deaths and more than 170 injuries (Baker et al., 2007); the space shuttle Challenger and Columbia accidents (Columbia Accident Investigation Board (CAIB), 2003, Vaughan, 1996); and the recent allision of the container ship Cosco Busan with the San Francisco Bay Bridge in November 2007 that spilled 58,000 gallons of oil in San Francisco Bay (US Department of Justice, 2008), as well as recent research (Hollnagel, 2004, Hollnagel et al., 2008), suggest that the roots of accident causation may be more complex in large-scale systems, involving non-linear interdependencies among organizational, group and individual factors in the system. In addition, systemic views of risk and accident causation (Hollnagel, 2004, Hollnagel et al., 2008) view accidents as emergent phenomena that arise from the variability of organizational, group and individual performance, with roots in processes at each of those levels. In all of these approaches to evaluating and assessing risk, however, the need to consider the role of human and organizational error is clear (Popova and Sharpanskykh, 2007, Rasmussen, 1983, Rasmussen, 1986, Reason, 1990, Reason, 1997, Stroeve et al., 2007, Swain, 1987, Swain and Guttmann, 1983). Thus, in this paper we focus on the data challenges of measuring performance, using the lens of human and organizational error modeling. The paper begins with an overview of human and organizational error assessments, and then introduces the particular challenges of data needs in human reliability analyses. A case study of human and organizational error analysis in a complex, large-scale system, marine transportation in Puget Sound, is used to illustrate the impact of the data challenges on the risk assessment process. Suggestions for future research conclude the paper.

Section snippets

Human and organizational error assessment

A number of approaches for representing, modeling and assessing the role of human and organizational error in complex, large-scale systems have been developed. Turner, 1978, Perrow, 1984 provided early models and explanations for understanding the role of humans and organizations in accidents in complex, socio-technical systems; Reason (1990) advanced the Swiss Cheese model, which suggested that accidents occur when active failures and latent pathogens are aligned in a complex system. Swain,

Background

In June 2006, BP contracted with The George Washington University, Rensselaer Polytechnic University, and Virginia Commonwealth University to perform a maritime risk assessment with the intent of incorporating its results into the Section 10 permit environmental impact statement for the addition of a north dock to the BP Cherry Point, Washington facility. The purpose of the risk assessment was to analyze the effects on oil spill risk of potential incremental vessel traffic projected to call at

Challenges with accident, incident and human error data

Problems with data to support modeling and analysis in marine transportation are well-documented (National Research Council, 1983, National Research Council, 1990, National Research Council, 1994, National Research Council, 1999, National Research Council, 2003), having grown with the proliferation of electronic data; the data have varying storage requirements, exist in various formats, and are gathered and collected from various agencies and individuals, with varying degrees of compatibility (

Discussion: improving accident databases and systems to support human and organizational error analyses

The differences between the various data sources just described are expected challenges in data analysis; however, difficulties in the integrity, availability, reliability and maintainability of the data greatly influence the strength and validity of risk assessments conducted with such data. Hence, an examination of these data challenges has import for data and risk analyses in complex, large-scale systems such as marine transportation. In this section, we offer suggestions for improving

Acknowledgements

We are particularly indebted to Scott McCreery and CAPT Craig Lee of BP; to Captain Stephen Metruck, USCG; LCDR Jason Tama, USCG; and Ken Alger, of the US Coast Guard Sector Seattle; as well as to Bob Bohlman and John Veentjer of the Puget Sound Marine Exchange; to CAPT Mike Lynch, Dale Jensen and Jack Bartlett from the Washington State Department of Ecology; to CAPTs Richard McCurdy and Del Mackenzie from the Puget Sound Pilots; to the members of the Puget Sound Harbor Safety Committee; and to

References (40)

  • D. Mendonca

    Measures of resilient performance

  • J.R.W. Merrick et al.

    The Prince William Sound risk assessment

    Interfaces

    (2002)
  • National Research Council

    Ship Collisions with Bridges: The Nature of the Accidents, their Prevention and Mitigation

    (1983)
  • National Research Council

    Crew Size and Maritime Safety

    (1990)
  • National Research Council

    Minding the Helm: Marine Navigation and Piloting

    (1994)
  • National Research Council

    Applying Advanced Information Systems to Ports and Waterways Management

    (1999)
  • National Research Council, 2003. Special Report 273: Shipboard Automatic Identification System Displays: Meeting the...
  • Pacific States/British Columbia Oil Spill Task Force

    Recommendations to Prevent Oil Spills Caused by Human Error: 1995 Report to the Pacific States/BC Oil Spill Task Force

    (1995)
  • Pacific States/British Columbia Oil Spill Task Force

    Spill and Incident Data Collection Project Report

    (1997)
  • Pacific States/British Columbia Oil Spill Task Force

    Data Dictionary

    (2007)
  • Cited by (51)

    View all citing articles on Scopus
    1

    Tel.: +1 518 276 2954; fax: +1 518 276 8227.

    View full text