Tobias Holstein
Gordana Dodig-Crnkovic
ABSTRACT
As an envisaged future of transportation, self-driving cars are being discussed from various perspectives, including social, economical, engineering, computer science, design, and ethical aspects. On the one hand, self-driving cars present new engineering problems that are being gradually successfully solved. On the other hand, social and ethical problems have up to now being presented in the form of an idealized unsolvable decision-making problem, the so-called "trolley problem", which is built on the assumptions that are neither technically nor ethically justifiable. The intrinsic unfairness of the trolley problem comes from the assumption that lives of different people have different values.
In this paper, techno-social arguments are used to show the infeasibility of the trolley problem when addressing the ethics of self-driving cars. We argue that different components can contribute to an "unfair" behaviour and features, which requires ethical analysis on multiple levels and stages of the development process. Instead of an idealized and intrinsically unfair thought experiment, we present real-life techno-social challenges relevant for the domain of software fairness in the context of self-driving cars.
- Moral Machine. http://moralmachine.mit.edu, 2016.Google Scholar
- Ethics commission on automated driving presents report: First guidelines in the world for self-driving computers. Technical report, Federal Ministry of Transport and Digital Infrastructure, 2017.Google Scholar
- J. Achenbach. Driverless cars are colliding with the creepy Trolley Problem. https://www.washingtonpost.com/news/innovations/wp/2015/12/29/will-self-driving-cars-ever-solve-the-famous-and-creepy-trolley-problem/, December 2015.Google Scholar
- E. Ackerman. People Want Driverless Cars with Utilitarian Ethics, Unless They're a Passenger. https://spectrum.ieee.org/cars-that-think/transportation/self-driving/people-want-driverless-cars-with-utilitarian-ethics-unless-theyre-a-passenger, June 2016.Google Scholar
- H. S. Alavi, F. Bahrami, H. Verma, and D. Lalanne. Is driverless car another weiserian mistake? In Proceedings of the 2017 ACM Conference Companion Publication on Designing Interactive Systems, DIS '17 Companion, pages 249--253, New York, NY, USA, 2017. ACM. Google ScholarDigital Library
- S. Applin. Autonomous vehicle ethics: Stock or custom? IEEE Consumer Electronics Magazine, 6(3):108--110, July 2017.Google ScholarCross Ref
- A. Bleske-Rechek, L. Nelson, J. P. Baker, M. Remiker, and S. J. Brandt. Evolution and the trolley problem: People save five over one unless the one is young, genetically related, or a romantic partner. 4:115--127, 01 2010.Google Scholar
- J.-F. Bonnefon, A. Shariff, and I. Rahwan. The social dilemma of autonomous vehicles. Science, 352(6293):1573--1576, 2016.Google ScholarCross Ref
- M. Broy, I. H. Kruger, A. Pretschner, and C. Salzmann. Engineering Automotive Software. Proceedings of the IEEE, 95(2):356--373, feb 2007.Google Scholar
- I. Coca-Vila. Self-driving cars in dilemmatic situations: An approach based on the theory of justification in criminal law. Criminal Law and Philosophy, Jan 2017.Google Scholar
- Coodriver GmbH. Schutzranzen App - anonymous, safe, commercial-free - makes it easier to see children near roads, September 2017. https://www.schutzranzen.com/files/2115/0478/2378/PM_Schutzranzen_IAA_2017_english.pdf.Google Scholar
- K. Deamer. What the First Driverless Car Fatality Means for Self-Driving Tech. https://www.scientificamerican.com/article/what-the-first-driverless-car-fatality-means-for-self-driving-tech/, 2016.Google Scholar
- L. Dennis, M. Fisher, M. Slavkovik, and M. Webster. Ethical Choice in Unforeseen Circumstances, pages 433--445. Springer Berlin Heidelberg, Berlin, Heidelberg, 2014.Google Scholar
- L. Dennis, M. Fisher, M. Slavkovik, and M. Webster. Formal verification of ethical choices in autonomous systems. Robotics and Autonomous Systems, 77(Supplement C):1 -- 14, 2016. Google ScholarDigital Library
- D. Dolgov. Google self-driving car project - monthly report - september 2016 - on the road. Technical report, Google, 2016.Google Scholar
- J. Dunbar and J. E. Gilbert. The human element in autonomous vehicles. In D. Harris, editor, Engineering Psychology and Cognitive Ergonomics: Cognition and Design, pages 339--362, Cham, 2017. Springer International Publishing.Google ScholarCross Ref
- S. I. Earth Imaging Journal (EIJ): Remote Sensing, Satellite Images. Lidar boosts brain power for self-driving cars, 2012.Google Scholar
- Ethics Commission. Automated and connected driving. Technical report, Federal Ministry of Transport and Digital Infrastructure, 2017.Google Scholar
- A. K. Faulhaber, A. Dittmer, F. Blind, M. A. Wächter, S. Timm, L. R. Sütfeld, A. Stephan, G. Pipa, and P. König. Human decisions in moral dilemmas are largely described by utilitarianism: Virtual car driving study provides guidelines for autonomous driving vehicles. Science and Engineering Ethics, Jan 2018.Google ScholarCross Ref
- P. Foot. The problem of abortion and the doctrine of double effect. Oxford Review, 5, 1967.Google Scholar
- A.-K. Frison, P. Wintersberger, and A. Riener. First person trolley problem: Evaluation of drivers' ethical decisions in a driving simulator. In Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI '16 Adjunct, pages 117--122, New York, NY, USA, 2016. ACM. Google ScholarDigital Library
- S. Galhotra, Y. Brun, and A. Meliou. Fairness testing: Testing software for discrimination. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, pages 498--510, New York, NY, USA, 2017. ACM. Google ScholarDigital Library
- General Motors. GM Self-Driving Safety Report, January 2018. https://www.gm.com/content/dam/gm/en_us/english/selfdriving/gmsafetyreport.pdf.Google Scholar
- G. Ghisio. Challenges for the Automotive Platform of the Future, 2016.Google Scholar
- N. J. Goodall. Vehicle automation and the duty to act. In Proceedings of the 21st world congress on intelligent transport systems, pages 7--11, 2014.Google Scholar
- N. J. Goodall. Can you program ethics into a self-driving car? IEEE Spectrum, 53(6):28--58, June 2016.Google ScholarDigital Library
- B. Goodman and S. Flaxman. European Union regulations on algorithmic decision-making and a "right to explanation". ArXiv e-prints, June 2016.Google Scholar
- Google. Google self-driving car project, 2016.Google Scholar
- J. D. Greene. Our driverless dilemma. Science, 352(6293):1514--1515, 2016.Google ScholarCross Ref
- L. Greenemeier. Driverless Cars Will Face Moral Dilemmas. https://www.scientificamerican.com/article/driverless-cars-will-face-moral-dilemmas/, 2016.Google Scholar
- F. Gustafsson. Automotive safety systems. IEEE Signal Processing Magazine, 26(4):32--47, July 2009.Google ScholarCross Ref
- T. Holstein, G. Dodig-Crnkovic, and P. Pelliccione. Ethical and Social Aspects of Self-Driving Cars. ArXiv e-prints, Feb. 2018.Google Scholar
- N. Kalra and S. M. Paddock. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 94(Supplement C):182 -- 193, 2016.Google Scholar
- K. Kirkpatrick. The moral challenges of driverless cars. Commun. ACM, 58(8):19--20, July 2015. Google ScholarDigital Library
- S. Kuchinskas. Crash Course: Training the Brain of a Driverless Car. https://www.scientificamerican.com/article/autonomous-driverless-car-brain/, 2013.Google Scholar
- B. MacKinnon. Ethics: Theory and Contemporary Issues, Concise Edition. Cengage Learning, 2012.Google Scholar
- N. McBride. The ethics of driverless cars. SIGCAS Comput. Soc., 45(3):179--184, Jan. 2016. Google ScholarDigital Library
- C. Metz. Former Apple Engineers Working on New Eyes for Driverless Cars, September 2017. https://www.nytimes.com/2017/09/20/technology/former-apple-engineers-driverless-cars.html.Google Scholar
- C. Metz. What Virtual Reality Can Teach a Driverless Car, October 2017. https://www.nytimes.com/2017/10/29/business/virtual-reality-driverless-cars.html.Google Scholar
- C. Mooney. Save the driver or save the crowd? Scientists wonder how driverless cars will 'choose'. https://www.washingtonpost.com/news/energy-environment/wp/2016/06/23/save-the-driver-or-save-the-crowd-scientists-wonder-how-driverless-cars-will-choose/, 2016.Google Scholar
- National Highway Traffic Safety Administration (NHTSA). Federal automated vehicles policy - accelerating the next revolution in roadway safety. Technical report, U.S. Department of Transportation, 2016.Google Scholar
- N. H. T. S. A. (NHTSA). "dot/nhtsa policy statement concerning automated vehicles" 2016 update to "preliminary statement of policy concerning automated vehicles". Technical report, National Highway Traffic Safety Administration (NHTSA).Google Scholar
- P. Oliver, B. Jan, and K. Sören. Automated driving on public roads: Experiences in real traffic, 2015.Google Scholar
- G. Orwell. 1984. Secker & Warburg, 1949.Google Scholar
- P. Pelliccione, E. Knauss, R. Heldal, S. M. Ågren, P. Mallozzi, A. Alminger, and D. Borgentun. Automotive architecture framework: The experience of volvo cars. Journal of Systems Architecture, 77(Supplement C):83 -- 100, 2017.Google Scholar
- M. Persson and S. Elfström. Volvo Car Group's first self-driving Autopilot cars test on public roads around Gothenburg, 2014.Google Scholar
- S. Pillath. Briefing: Automated vehicles in the EU. European Parliamentary Research Service (EPRS), (January):12, 2016.Google Scholar
- A. Riener, M. P. Jeon, I. Alvarez, B. Pfleging, A. Mirnig, M. Tscheligi, and L. Chuang. 1st workshop on ethically inspired user interfaces for automated driving. In Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI '16 Adjunct, pages 217--220, New York, NY, USA, 2016. ACM. Google ScholarDigital Library
- SAE. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. Global Ground Vehicle Standards, (J3016):30, 2016.Google Scholar
- A. Shashkevich. Stanford professors discuss ethics involving driverless cars. https://news.stanford.edu/2017/05/22/stanford-scholars-researchers-discuss-key-ethical-questions-self-driving-cars-present/, may 2017.Google Scholar
- J. D. Stoll. Gm executive credits silicon valley for accelerating development of self-driving cars, 2016.Google Scholar
- L. R. Sütfeld, R. Gast, P. König, and G. Pipa. Using virtual reality to assess ethical decisions in road traffic scenarios: Applicability of value-of-life-based models and influences of time pressure. Frontiers in Behavioral Neuroscience, 11:122, 2017.Google ScholarCross Ref
- Tesla. A tragic loss | tesla deutschland, 2016.Google Scholar
- Tesla. Upgrading Autopilot: Seeing the World in Radar | Tesla Deutschland, 2016.Google Scholar
- Toyota. New toyota test vehicle paves the way for commercialization of automated highway driving technologies | toyota global newsroom, 2015.Google Scholar
- D. WAKABAYASHI. WaymoâĂŹs Autonomous Cars Cut Out Human Drivers in Road Tests, November 2017. https://www.nytimes.com/2017/11/07/technology/waymo-autonomous-cars.html.Google Scholar
- M. M. Waldrop. Autonomous vehicles: No drivers required. Nature, 518:20--3, 2015.Google ScholarCross Ref
- Waymo. Waymo - Navigating city streets, December 2016. https://youtu.be/fbWeKhAPMig?t=10.Google Scholar
- Waymo. Technology - Waymo, 2017. https://waymo.com/tech/.Google Scholar
- Waymo. Waymo, September 2017. https://waymo.com.Google Scholar
- Waymo. Waymo Self-Driving Safety Report, 2017. https://storage.googleapis.com/sdc-prod/v1/safety-report/waymo-safety-report-2017.pdf.Google Scholar
- Waymo. Waymo - On the Road, February 2018. https://waymo.com/ontheroad/.Google Scholar
- P. Wintersberger, A. K. Prison, A. Riener, and S. Hasirlioglu. The experience of ethics: Evaluation of self harm risks in automated vehicles. In 2017 IEEE Intelligent Vehicles Symposium (IV), pages 385--391, June 2017.Google ScholarCross Ref
Index Terms
- Avoiding the intrinsic unfairness of the trolley problem
Recommendations
Trolled by the Trolley Problem: On What Matters for Ethical Decision Making in Automated Vehicles
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsAutomated vehicles have to make decisions, such as driving maneuvers or rerouting, based on environment data and decision algorithms. There is a question whether ethical aspects should be considered in these algorithms. When all available decisions ...
Real-world ethics for self-driving cars
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion ProceedingsEthical and social problems of the emerging technology of self-driving cars can best be addressed through an applied engineering ethical approach. However, currently social and ethical problems are typically being presented in terms of an idealized ...
First Person Trolley Problem: Evaluation of Drivers' Ethical Decisions in a Driving Simulator
AutomotiveUI '16 Adjunct: Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsNo matter how safe automated driving will be in the future, there will remain situations where an accident is unavoidable and vehicles will have to decide between options that all potentially result in lethal outcome. Empirical ethics is one method that ...
Comments