Top

AI & SOCIETY

Published in:

13-07-2021 | Curmudgeon Corner

From Blade Runners to Tin Kickers: what the governance of artificial intelligence safety needs to learn from air crash investigators

Author: Carl Macrae

Published in: AI & SOCIETY | Issue 5/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Excerpt

What should we do when artificial intelligence (AI) goes wrong? AI has huge potential to improve the safety of societally critical systems, such as healthcare and transport, but it also has the potential to introduce new risks and amplify existing ones. For instance, biases in widely deployed diagnostic AI systems could adversely affect the care of a large number of patients (Fraser et al. 2018), and hidden weaknesses in the perception systems of autonomous vehicles may regularly expose road users to significant risk (NTSB 2019). What are the most appropriate strategies for governing the safety of AI-based systems? One answer emerges from taking contrasting looks forwards to our imagined dystopian AI future and backwards to the progressive evolution of aviation safety. …

previous article Testing Turing

next article Artificial intelligence in marketing: friend or foe of sustainable consumption?

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

AAIB (2021) About us. Air Accidents Investigation Branch. https://www.gov.uk/government/organisations/air-accidents-investigation-branch/about

ATSB (2007) Analysis, causality and proof in safety investigations. Australian Transport Safety Bureau, Canberra

Braithwaite J (2011) The essence of responsive regulation. UBC Law Rev 44(3):475–520

Byrne G (2002) Flight 427: anatomy of an air disaster. Springer, LondonCrossRef

Cave S, Dihal K (2019) Hopes and fears for intelligent machines in fiction and reality. Nat Mach Intell 1:74–78CrossRef

Dick PK (1968) Do androids dream of electric sheep? Doubleday, London

Elish MC (2019) Moral crumple zones: cautionary tales in human–robot interaction. Engag Sci Technol Soc 5:40–60

Fraser H, Coiera E, Wong D (2018) Safety of patient-facing digital symptom checkers. Lancet 392(10161):2263–2264CrossRef

Hradecky S (2012) United Kingdom's Air Accident Investigation Board celebrates 100 years of air accident investigation. The Aviation Herald. http://avherald.com/h?article=450d2364&opt=1

Jobin A, Ienca M, Vayena E (2019) The global landscape of AI ethics guidelines. Nat Mach Intell 1:389–399CrossRef

Levin S (2016) Uber blames humans for self-driving car traffic offenses as California orders halt. The Guardian, 15 Dec 2016. https://www.theguardian.com/technology/2016/dec/14/uber-self-driving-cars-run-red-lights-san-francisco

Levin S (2020) Safety driver charged in 2018 incident where self-driving Uber car killed a woman. The Guardian, 16 Sep 2020. https://www.theguardian.com/us-news/2020/sep/16/uber-self-driving-car-death-safety-driver-charged

Macrae C (2014) Close calls: managing risk and resilience in airline flight safety. Palgrave Macmillan, LondonCrossRef

Macrae C (2019) Governing the safety of artificial intelligence in healthcare. BMJ Qual Saf 28:495–498CrossRef

Macrae C (2021) Learning from the failure of autonomous and intelligent systems: accidents, safety and sociotechnical sources of risk. SSRN. https://ssrn.com/abstract=3832621

Macrae C, Vincent C (2014) Learning from failure: the need for independent safety investigation in healthcare. J R Soc Med 107:439–443CrossRef

Macrae C, Vincent C (2017) A new national safety investigator for healthcare: the road ahead. J R Soc Med 110(3):90–92CrossRef

Nixon J, Braithwaite GR (2018) What do aircraft accident investigators do and what makes them good at it? Developing a competency framework for investigators using grounded theory. Saf Sci 103:153–161CrossRef

NTSB (2017) Collision between a car operating with automated vehicle control systems and a tractor-semitrailer truck near Willliston, Florida, May 7 2016. National Transportation Safety Board, Washington, DC

NTSB (2019) Collision between vehicle controlled by developmental Automated Driving System and Pedesterian, Tempe, Arizona, March 18, 2018: Accident Report NTSB/HAR-19/03. National Transportation Safety Board, Washington, DC

O’Kane S (2019) Uber debuts a new self-driving car with more fail-safes. The Verge. https://www.theverge.com/2019/6/12/18662626/uber-volvo-self-driving-car-safety-autonomous-factory-level

O’Kane S (2020) Tesla ignored safety board’s Autopilot recommendations, chairman says. The Verge. https://www.theverge.com/2020/2/25/21152984/tesla-autopilot-safety-recommendations-ignored-ntsb-crash-hearing

Volvo (2019) Volvo Cars and Uber present production vehicle ready for self-driving. Volvo Cars Global Newsroom. https://www.media.volvocars.com/global/en-gb/media/pressreleases/254697/volvo-cars-and-uber-present-production-vehicle-ready-for-self-driving

Waterson P, Jenkins DP, Salmon PM, Underwood P (2017) ‘Remixing Rasmussen’: the evolution of Accimaps within systemic accident analysis. Appl Ergon 59B:483–503CrossRef

Winfield AFT, Jirotka M (2017) The case for an ethical black box. In: Gao Y, Fallah S, Jin Y, Lekakou C (eds) Towards autonomous robotic systems. TAROS 2017. Lecture notes in computer science, vol 10454. Springer, Champaign. Doi: https://doi.org/10.1007/978-3-319-64107-2_21

Winfield AFT, Jirotka M (2018) Ethical governance is essential to building trust in robotics and artificial intelligence systems. Philos Trans R Soc A 376(2133):20180085CrossRef

Winfield AFT, Winkle K, Webb H, Lyngs U, Jirotka M, Macrae C (2021) Robot accident investigation: a case study in responsible robotics. In: Cavalcanti A, Dongol B, Hierons R, Timmis J, Woodcock J (eds) Software engineering for robotics. Springer, London

Title: From Blade Runners to Tin Kickers: what the governance of artificial intelligence safety needs to learn from air crash investigators
Author: Carl Macrae
Publication date: 13-07-2021
Publisher: Springer London
Published in: AI & SOCIETY / Issue 5/2023
Print ISSN: 0951-5666
Electronic ISSN: 1435-5655
DOI: https://doi.org/10.1007/s00146-021-01246-5

Springer Professional

Excerpt

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 5/2023

On professional skill in the age of digital technology

AI transparency: a matter of reconciling design with critique

The Internet of Bodies—alive, connected and collective: the virtual physical future of our bodies and our senses

Michael Polányi’s fiduciary program against fake news and deepfake in the digital age

Writing in water: dense responsive media in place of relational interfaces

Causal Reasoning and Meno’s Paradox

Premium Partner