Product Development within Artificial Intelligence, Ethics and Legal Risk

With regard to safe product development in the dilemma between Artificial Intelligence, ethics and legal risk, Prof. Dr.-Ing. MBA Thomas Winkle first provides a meta-analysis on safety assessment based on accident data examples to show possible definitions of safety benefits and risk. With more than 20 years of research and consulting at three automobile manufacturers, Thomas Winkle also points to disasters from nuclear power plant accidents at Chernobyl and Fukushima. Further comparisons between global mortality rates of women and men to life expectancy of different countries of the world compared to the traffic mortality sensitize for ethical relevance and accepted risk today.As the first of its kind, the scientific analysis in this book uses one example to analyze all police-reported traffic accidents nationwide in Saxony (1.28 million) over a 10-year period. Relevant examples highlight technical, ethical, and legal requirements for the development and production of automated vehicles with image recognition and Artificial Intelligence. The inclusion of highly sensitive international court rulings and growing consumer expectations make the book a helpful guide for product development from the initial idea to market launch. Qualitative interviews within the development departments of automobile manufacturers show how a structured guideline-based process with expert feedback loops increases product quality in terms of safety in use and functional safety. Finally, there follows a development guide with a checklist of 101 questions on the requirements that contribute to the duty of care in the development of automated vehicles and fulfill the highest court rulings on product liability.

With regard to safe product development in the dilemma between Artificial Intelligence, ethics and legal risk, Thomas Winkle provides a meta-analysis for safety assessment using accident data to demonstrate potential safety benefits and risks. Thomas Winkle also refers to the disasters of the Chernobyl and Fukushima nuclear power plant accidents. They mark changes in acceptance about the use of nuclear energy. Comparisons are made between the global mortality rates of females and males on traffic mortality in relation to the life expectancy of various countries around the world, from Sierra Leone with the lowest life expectancy to Japan with the highest. The probability of dying from a traffic accident is highest in Venezuela and Sierra Leone. Another overview addresses the global mortality rate with exemplary causes of death in contrast to the ISO 26262 requirements of the Automotive Safety Integrity Level "ASIL D" and a hardware failure rate of less than 1 * 10^{-8 1}/_h. Furthermore, he uses traffic accident data examples of safety-enhancing automated vehicle systems with a low degree of automation that are already available on the market. For testing methods to develop and validate safe automated vehicles with reasonable expenditure, the author recommends combining worldwide traffic accident-, weather-, vehicle operation data and traffic simulations. Based on these findings, a realistic evaluation of internationally prospective, and statistically relevant real-world traffic scenarios as well as error processes and stochastic models can be analyzed (in combination with virtual tests in laboratories and driving simulators) to control critical driving situations.

Product development involving Artificial Intelligence, ethics and legal risks using the example of safe autonomous vehicles involves a number of challenges. These include dealing with complex traffic situations and reliable image recognition under difficult light and weather conditions, such as rain, snow, backlight, wet road surface, spray/splashing water, icing/contamination of windshield/sensors or road markings that are only partially visible. In order to develop and validate automated vehicles with reasonable expenditure, a first area-wide analysis based on 1.28 million police accident reports was conducted including all police reports in Saxony over 10 years concerning bad weather conditions.Based on this large database, 374 accidents were found with regard to perception limitations for the detailed investigation. These traffic scenarios are relevant for automated driving. They will form a key aspect for future development, validation and testing of machine perception including Artificial Intelligence within automated driving functions.

In the following chapter the author traces the technical improvements in vehicle safety over recent decades, including new sensor technologies with image recognition and Artificial Intelligence, factoring in growing consumer expectations. Through Federal Court of Justice rulings on product liability and economic risks, he depicts requirements that car manufacturers must meet. For proceedings from the first idea until development to sign-off, he recommends interdisciplinary, harmonized safety and testing procedures. He argues for further development of current internationally agreed-upon standards including tools, methodological descriptions, simulations, and guiding principles with checklists. These will represent and document the practiced state of science and technology, which has to be implemented technically suited and economically reasonable. Dilemma situations have always served to clarify ethical and legal principles, such as in the famous example of the so-called “trolley case”. The answer of the law is clear: the killing of a human being with the intention of saving others from certain death may be excused in a concrete case, but it remains illegal in any case. The solution is to avoid accidents at any rate by adapting and forward-looking driving. Relevant maneuvers of driving robots have to be defined and assessed for example using accident data and virtual methods. Further investigation of real driving situations in comparison with system specifications with tests on proving grounds, car clinics, field tests, human driver training or special vehicle studies are recommended. For the required exchange of information, storage of vehicle data and possible criminal attacks protective technical measures are necessary.

The previous chapters indicate that development approaches using innovative technology or Artificial Intelligence must be reviewed against the background of the increasing demands on interdisciplinary project teams as well as the growing complexity of functions. Interviews with engineers, executive managers and a psychologist from the development department of automobile manufacturers show that a structured guided process increases quality in respect of operational and functional safety. The surveys were conducted using the example of the “Code of Practice for the Design and Evaluation of ADAS” including ISO 26262 requirements. It focused on 1. Success and/or failure of guided development projects; 2. Different perceptions, expectations, ideas and conceptions about the optimal development process; 3. Liability-based product responsibility of the developers and 4. general developer’s attitude to the development process. As one of the insightful results, a practice-oriented guideline with supportive advice “forces” all participants involved in the product development process to sit around a table introducing and discussing their different aspects in a structured way. Through the surveys, the developers were sensitized to the advantages of a guideline-based development process. Often the employees themselves are the best advisors. Each expert contributes to the development of a reliable system through their special field of expertise. The developers concerned are the most aware of the weaknesses and can initiate innovations in companies from the “bottom-up”. A final consulting concept (checklist with 101 questions in Annex B) includes guidelines and requirements and will support the efficient, user-friendly development of new automated vehicle functions.

The survey in the development departments shows a great need for structured advice during the development process including a strong interest in supporting consulting services. A competent supervision from an independent consultant from outside the respective area is recommended for achieving continuous documentation throughout the development process according to the duty of care. Most respondents would like to have a point of contact or personal contact person, who will always be on hand with competent technical or legal advice and assistance for any questions or problems that arise. In the case of a developer, guidance, sense and purpose for the benefit of the individual developer are primary motivations. This means that a structured guideline will only be used with conviction if it is perceived as an advantage. The author’s experience in connection with the processing of product liability cases lead to the following general questions as a consultant to the development process: 1. How carefully are the tasks of development, production and marketing implemented? 2. What is expected beyond the legal requirements? 3. Will possible damage be avoided or its effect reduced if another design is used? 4. How does the system behave in comparison to the competitors (other car manufacturers)? 5. Were preventive and comprehensible warnings made available to prevent possible damage? A final consulting concept (a checklist of 101 questions in Appendix B) provides guidelines and requirements.

The automotive industry is in the progress of a fundamental change, as they no longer meet mobility requirements, especially in urban areas. As a result, many predict a disruptive change. Responses to this are new innovative developments. One answer to this are automated driving systems that offer great potential for increasing safety, comfort, environmental pollution and efficiency in road traffic.