Computational Intelligence in Automotive Applications

A popular definition of workload is given by O’Donnell and Eggmeir, which states that “The term workload refers to that portion of the operator’s limited capacity actually required to perform a particular task” [1]. In the vehicle environment, the “particular task” refers to both the vehicle control, which is the primary task, and other secondary activities such as listening to the radio. Three major types of driver workload are usually studied, namely, visual, manual, and cognitive. Auditory workload is not treated as a major type of workload in the driving context because the auditory perception is not considered as a major requirement to perform a driving task. Even when there is an activity that involves audition, the driver is mostly affected cognitively.

Among the three major types of driver workload, cognitive workload is the most difficult to measure. For example, withdrawing hands from the steering wheel to reach for a coffee cup requires extra manual workload. It also may require extra visual workload in that the position of the cup may need to be located. Both types of workload are directly measurable through such observations as hands-off-wheel and eyes-off- road time. On the other hand, engaging in thinking (the so-called minds-off-road phenomenon) is difficult to detect. Since the cognitive workload level is internal to the driver, it can only be inferred based on the information that is observable. In this chapter, we report some of our research results on driver's cognitive workload estimation.1 After the discussion of the existing practices, we propose a new methodology to design driver workload estimation systems, that is, using machine-learning techniques to derive optimized models to index workload. The advantage of this methodology will be discussed, followed by the presentation of some experimental results. This chapter concludes with discussion of future work.

The increasing number of traffic accidents due to driver inattention has become a serious problem for society. Every year, about 45,000 people die and 1.5 million people are injured in traffic accidents in Europe. These figures imply that one person out of every 200 European citizens is injured in a traffic accident every year and that around one out 80 European citizens dies 40 years short of the life expectancy. It is known that the great majority of road accidents (about 90–95%) are caused by human error. More recent data has identified inattention (including distraction and falling asleep at the wheel) as the primary cause of accidents, accounting for at least 25% of the crashes [15]. Road safety is thus a major European health problem. In the “White Paper on European Transport Policy for 2010,” the European Commission declares the ambitious objective of reducing by 50% the number of fatal accidents on European roads by 2010 (European Commission, 2001).

This chapter presents an original system for monitoring driver inattention and alerting the driver when he is not paying adequate attention to the road in order to prevent accidents. According to [40] the driver inattention status can be divided into two main categories: distraction detection and identifying sleepiness. Likewise, distraction can be divided in two main types: visual and cognitive. Visual distraction is straightforward, occurring when drivers look away from the roadway (e.g., to adjust a radio). Cognitive distraction occurs when drivers think about something not directly related to the current vehicle control task (e.g., conversing on a hands-free cell phone or route planning). Cognitive distraction impairs the ability of drivers to detect targets across the entire visual scene and causes gaze to be concentrated in the center of the driving scene. This work is focused in the sleepiness category. However, sleepiness and cognitive distraction partially overlap since the context awareness of the driver is related to both, which represent mental occurrences in humans [26].

Motivation for the use of statistical machine learning techniques in the automotive domain arises from our development of context aware intelligent driver assistance systems, specifically, Driver Workload Management systems. Such systems integrate, prioritize, and manage information from the roadway, vehicle, cockpit, driver, infotainment devices, and then deliver it through a multimodal user interface. This could include incoming cell phone calls, email, navigation information, fuel level, and oil pressure to name a very few. In essence, the workload manager attempts to get the right information to the driver at the right time and in the right way in order that driver performance is optimized and distraction is minimized.

In this chapter we describe three major efforts that have employed our machine learning approach. First, we discuss how we have utilized our machine learning approach to detect and classify a wide range of driving maneuvers, and describe a semi-automatic data annotation tool we have created to support our modeling effort. Second, we perform a large scale automotive sensor selection study towards intelligent driver assistance systems. Finally, we turn our attention to creating a system that detects driver inattention by using sensors that are available in the current vehicle fleet (including forwarding looking radar and video-based lane departure system) instead of head and eye tracking systems.

This approach resulted in the creation of two generations of our workload manager system called Driver Advocate, Driver Advocate that was based on data rather than just expert opinions. The described techniques helped reduce the research cycle times while resulting in broader insight. There was rigorous quantification of theoretical sensor subsystem performance limits and optimal subsystem choices given economic price points. The resulting system performance specs and architecture design created a workload manager that had a positive impact on driver performance [23, 33].

Accidents involving pedestrians is one of the leading causes of death and injury around the world. Intelligent driver support systems hold a promise to minimize accidents and save many lives. Such a system would detect the pedestrian, predict the possibility of collision, and then warn the driver or engage automatic braking or other safety devices. This chapter describes the framework and issues involved in developing a pedestrian protection system. It is emphasized that the knowledge of the state of the environment, vehicle, and driver are important for enhancing safety. Classification, clustering, and machine learning techniques for effectively detecting pedestrians are discussed, including the application of algorithms such as SVM, Neural Networks, and AdaBoost for the purpose of distinguishing pedestrians from background. Pedestrians unlike vehicles are capable of sharp turns and speed changes, therefore their future paths are difficult to predict. In order to estimate the possibility of collision, a probabilistic framework for pedestrian path prediction is described along with related research. It is noted that sensors in vehicle are not always sufficient to detect all the pedestrians and other obstacles. Interaction with infrastructure based systems as well as systems from other vehicles can provide a wide area situational awareness of the scene. Furthermore, in infrastructure based systems, clustering and learning techniques can be applied to identify typical vehicle and pedestrian paths and to detect anomalies and potentially dangerous situations. In order to effectively integrate information from infrastructure and vehicle sources, the importance of developing and standardizing vehicle-vehicle and vehicle-infrastructure communication systems is also emphasized.

The production pipeline of present day’s automobile manufacturers consists of a highly heterogeneous and intricate assembly workflow that is driven by a considerable degree of interdependencies between the participating instances as there are suppliers, manufacturing engineers, marketing analysts and development researchers. Therefore, it is of paramount importance to enable all production experts to quickly respond to potential on-time delivery failures, ordering peaks or other disturbances that may interfere with the ideal assembly process. Moreover, the fast moving evolvement of new vehicle models require well-designed investigations regarding the collection and analysis of vehicle maintenance data. It is crucial to track down complicated interactions between car components or external failure causes in the shortest time possible to meet customer-requested quality claims.

To summarize these requirements, let us turn to an example which reveals some of the dependencies mentioned in this chapter. As we will see later, a normal car model can be described by hundreds of variables each of which representing a feature or technical property. Since only a small number of combinations (compared to all possible ones) will represent a valid car configuration, we will present a means of reducing the model space by imposing restrictions. These restrictions enter the mathematical treatment in the form of dependencies since a restriction may cancel out some options, thus rendering two attributes (more) dependent. This early step produces qualitative dependencies like “engine type and transmission type are dependent.” To quantify these dependencies some uncertainty calculus is necessary to establish the dependence strengths. In our cases probability theory is used to augment the model, e.g., “whenever engine type 1 is ordered, the probability is 56% of having transmission type 2 ordered as well.” There is a multitude of sources to estimate or extract this information from. When ordering peaks occur like an increased demand of convertibles during the Spring, or some supply shortages arise due to a strike in the transport industry, the model is used to predict vehicle configurations that may run into delivery delays in order to forestall such a scenario by, e.g., acquiring alternative supply chains or temporarily shifting production load. Another part of the model may contain similar information for the aftercare, e.g., “whenever a warranty claim contained battery type 3, there is a 30% chance of having radio type 1 in the car.” In this case dependencies are contained in the quality assessment data and are not known beforehand but are extracted to reveal possible hidden design flaws.

Rule extraction for root cause analysis in manufacturing process optimization is an alternative to traditional approaches to root cause analysis based on process capability indices and variance analysis. Process capability indices alone do not allow to identify those process parameters which have the major impact on quality since these indices are only based on measurement results and do not consider the explaining process parameters. Variance analysis is subject to serious constraints concerning the data sample used in the analysis. In this work a rule search approach using Branch and Bound principles is presented, considering both the numerical measurement results and the nominal process factors. This combined analysis allows to associate the process parameters with the measurement results and therefore to identify the main drivers for quality deterioration of a manufacturing process.

Neural networks are making their ways into various commercial products across many industries. As in aerospace, in automotive industry they are not the main technology. Automotive engineers and researchers are certainly familiar with the buzzword, and some have even tried neural networks for their specific applications as models, virtual sensors, or controllers (see, e.g., [1] for a collection of relevant papers). In fact, a quick search reveals scores of recent papers on automotive applications of NN, fuzzy, evolutionary and other technologies of computational intelligence (CI); see, e.g., [2–4]. However, such technologies are mostly at the stage of research and not in the mainstream of product development yet. One of the reasons is “black-box” nature of neural networks. Other, perhaps more compelling reasons are business conservatism and existing/legacy applications (trying something new costs money and might be too risky) [5, 6].

The chapter deals with neural networks and learning machines for engine control applications, particularly in modeling for control. In the first section, basic features of engine control in a layered engine management architecture are reviewed. The use of neural networks for engine modeling, control and diagnosis is then briefly described. The need for descriptive models for model-based control and the link between physical models and black box models are emphasized by the grey box approach discussed in this chapter. The second section introduces the neural models frequently used in engine control, namely, MultiLayer Perceptrons (MLP) and Radial Basis Function (RBF) networks. A more recent approach, known as Support Vector Regression (SVR), to build models in kernel expansion form is also presented. The third section is devoted to examples of application of these models in the context of turbocharged Spark Ignition (SI) engines with Variable Camshaft Timing (VCT). This specific context is representative of modern engine control problems. In the first example, the airpath control is studied, where open loop neural estimators are combined with a dynamical polytopic observer. The second example considers modeling the in-cylinder residual gas fraction by Linear Programming SVR (LP-SVR) based on a limited amount of experimental data and a simulator built from prior knowledge. Each example demonstrates that models based on first principles and neural models must be joined together in a grey box approach to obtain effective and acceptable results.

Since 80s continuous government constraints have pushed car manufacturers towards the study of innovative technologies aimed at reducing automotive exhaust emissions and increasing engine fuel economy. As a result of this stringent legislation, the automotive engine technology has experienced continuous improvements in many areas. In the field of Engine Control Systems (ECS) innovative procedures have been proposed and major efforts have been devoted to the study of transient phenomena related to operation and design of engine control strategies. Particular attention has been given to the control of mixture strength excursions, which is a critical task to assure satisfactory efficiency of three-way catalytic converters and thus to meet exhaust emissions regulations. This goal has to be reached in both steady state and transient conditions by estimating the air flow rate at the injector location and delivering the fuel in the right amount and with the appropriate time dependence. Furthermore, the ECS designers have to face with the On Board Diagnostics (OBD) requirements that were introduced in 1996 in California and later in Europe and represent one of the most challenging targets in the field of Automotive Control. OBD requires a continuous monitoring of all powertrain components in order to prevent those faults that could result in a strong increase of exhaust emissions.

This chapter overviews the progress of vehicle power management technologies that shape the modern automobile. Some of these technologies are still in the research stage. Four in-depth case studies provide readers with different perspectives on the vehicle power management problem and the possibilities that intelligent systems research community can contribute towards this important and challenging problem.

The increased complexity and integration of vehicle systems has resulted in greater difficulty in the identification of malfunction phenomena, especially those related to cross-subsystem failure propagation and thus made system monitoring an inevitable component of future vehicles. Consequently, a continuous monitoring and early warning capability that detects, isolates and estimates size or severity of faults (viz., fault detection and diagnosis), and that relates detected degradations in vehicles to accurate remaining life-time predictions (viz., prognosis) is required to minimize downtime, improve resource management via condition-based maintenance, and minimize operational costs.

Resistance spot welding (RSW) is an important process in the automotive industry. The advantages of spot welding are many: an economical process, adaptable to a wide variety of materials (including low carbon steel, coated steels, stainless steel, aluminum, nickel, titanium, and copper alloys) and thicknesses, a process with short cycle times, and overall, a relatively robust process with some tolerance to fit-up variations. Although used in mass production for several decades, RSW poses several major problems, most notably, large variation in weld quality. Given the variation and uncertainty in weld quality (attributed to factors such as tip wear, sheet metal surface debris, and fluctuations in power supply), it is a common practice in industry to add a significant number of redundant welds to gain confidence in the structural integrity of the welded assembly [1]. In recent years, global competition for improved productivity and reduced nonvalue added activity, is forcing automotive OEMs and others to eliminate these redundant spot welds. The emphasis on reduction of the redundant welds significantly increases the need for monitoring of weld quality and minimizing weld process variability. Traditionally, destructive and nondestructive tests for weld quality evaluation are predominantly off-line or end-of-line processes. While this test information is useful and valuable for quality and process monitoring, it cannot be utilized in process control because of the significant delays that are associated with the off-line test analysis. In order to minimize the number of spot welds and still satisfy essential factors such as strength and surface integrity, weld quality has to be monitored and controlled in real-time. Advances over the last decade in the area of non-intrusive electronic sensors, signal processing algorithms, and computational intelligence, coupled with drastic reductions in computing and networking hardware costs, have now made it possible to develop non-intrusive intelligent resistance welding systems that overcome the above shortcomings.

The National Institute of Standards and Technology (NIST) Intelligent Control of Mobility Systems (ICMS) Program provides architectures and interface standards, performance test methods and data, and infrastructure technology needed by the U.S. manufacturing industry and government agencies in developing and applying intelligent control technology to mobility systems to reduce cost, improve safety, and save lives. The ICMS Program is made up of several areas including: defense, transportation, and industry projects, among others. Each of these projects provides unique capabilities that foster technology transfer across mobility projects and to outside government, industry and academia for use on a variety of applications. A common theme among these projects is autonomy and the Four Dimensional (3D + time)/Real-time Control System (4D/RCS) standard control architecture for intelligent systems that has been applied to these projects.

This chapter will briefly describe recent project advances within the ICMS Program including: goals, background accomplishments, current capabilities, and technology transfer that has or is planned to occur. Several projects within the ICMS Program have developed the 4D/RCS into a modular architecture for intelligent mobility systems, including: an Army Research Laboratory (ARL) Project currently studying onroad autonomous vehicle control, a Defense Advanced Research Project Agency (DARPA) Learning Applied to Ground Robots (LAGR) Project studying learning within the 4D/RCS architecture with road following application, and an Intelligent Systems Ontology project that develops the description of intelligent vehicle behaviors. Within the standards and performance measurements area of the ICMS program, a Transportation Project is studying components of intelligent mobility systems that are finding their way into commercial crash warning systems (CWS). In addition, the ALFUS (Autonomy Levels For Unmanned Systems) project determines the needs for metrics and standard definitions for autonomy levels of unmanned systems. And a JAUS (Joint Architecture for Unmanned Systems) project is working to set a standard for interoperability between components of unmanned robotic vehicle systems. Testbeds and frameworks underway at NIST include the PRIDE (Prediction in Dynamic Environments) framework to provide probabilistic predictions of a moving object's future position to an autonomous vehicle's planning system, as well as the USARSim/MOAST (Urban Search and Rescue Simulation/Mobility Open Architecture Simulation and Tools) framework that is being developed to provide a comprehensive set of open source tools for the development and evaluation of autonomous agent systems. A NIST Industrial Autonomous Vehicles (IAV) Project provides technology transfer from the defense and transportation projects directly to industry through collaborations with automated guided vehicles manufacturers by researching 4D/RCS control applications to automated guided vehicles inside facilities. These projects are each briefly described in this Chapter followed by Conclusions and continuing work.