Autonomous Driver Based on an Intelligent System of Decision-Making

A similar competition, which is intended for developing robotics and associated technologies, is the DARPA Grand Challenge, and in particular, its Urban version. Due to this competition, many autonomous vehicles and technologies related to them have evolved. The idea of unmanned ground vehicles (UGV) has a long history. The very first project of this kind was described in [11], where automobiles powered by electrical circuits and steered by radio were being engineered. It was though only a vision. Currently, several states and countries are introducing law regulations and highway code on the use of autonomous vehicles. Among them, the Google Car caused a sensation among people [28]. The scientists are thinking also about designing and creating cars controlled by the mind such as AutoNOMOS [12]. There are also concepts of a driver assistance using ideas derived from cognitive systems [14, 47], or augmented reality [34], for example. It cannot thus be doubted that this vision is becoming a reality now.

There are a great number of projects that develop the concept of unmanned vehicles [8, 25, 38]. Nevertheless, there are no real systems that outperform a human driver (who also is an imperfect system, but still the best one that is known). Many design methods are based on artificial intelligence, such as fuzzy systems, neural networks, evolutionary algorithms or rule-based methods.

The systems for autonomous driving are quite complex and can be divided into few subsystems (in abstract layers shown in Fig. 1): The perception system corresponds to extraction of the abstract data (e.g. positions and types of surrounding objects) from the raw data. The data are perceived by sensors from the environment, both inner (car) and outer (road and its neighbourhood via camera, laser, etc.) sensors. The traffic rules interpreter checks the current rules (extracted from abstract data) and points at reactions that are compatible with them. The role of the decision system is to decide what reaction the system should perform (e.g. the ones compatible with rules or other reactions, like emergency braking). The chosen reaction is then translated to language expressions that steer actuators via a behaviour interpreter.

From the viewpoint of the evolutionary theory, mammals represent the highest form of life that has amazing auto-adaptation features (e.g. squirrels can adapt to changeable labyrinth tasks in order to get food). The highest species of mammals is homo sapiens. Therefore, the human decision-making system can be considered the most efficient of all known such systems and can serve as a standard for auto-adaptation. Taking into account cybernetics theories [30], which advise following nature in modelling objectives, a control system based on human psychology models is expected to be able to adequately perform the task of steering a car.

An Intelligent System of Decision-making (ISD), as presented in [19‐23], can be such a system. It implements models of cognitive and personality (motivation) psychology for a control system of an agent [22]. It shows how people make a decision, from the incentives, to a reaction. The design of the whole ISD system is a result of a thorough modelling of human psychology based on an extensive literature study. This approach can thus also be used for controlling unmanned ground vehicles, including cars. The studies in this paper concern an adaptation of the ISD system to the role of an intelligent driver (xDriver), in a virtual environment.

One of the modern control ideas of adaptation to changing conditions is the reconfigurable control approach [4, 35, 45, 51] that is based on the methods and procedures of diagnosis. Factors and diagnostic indicators, in a way similar to the early adaptation mechanism known as the technique of scheduling variable, allow one to take the right strategy very quickly [18]. Similarly, they may be applied in database systems designed for error detection and diagnostics [51]. In the case of ISD, the role of such indicators is played by certain ideas taken from fuzzy logics and linguistics (as they are applied in expert systems).

Such a concept, which implements the reconfigurable control approach and uses a fuzzy logic switching technique for autonomous navigation of the car, is applied in [1, 49, 50], where the low-level (trajectory) controller is based on a rule-fuzzy expert system.

The idea of variable system configuration, due to external conditions, can also be deduced from the cybernetics paradigm. In living organisms, the possibility of the reconfigurable approach to control is provided by emotions [2, 32]. For instance, emotions make an immediate and concrete basis for producing an effective control for an agent in hazardous conditions [3, 15]. Emotions can eliminate reactions that are locally or temporarily inefficient, and unclog other movements (known or formerly learned) ’most’ suitable for the situation. Thus, emotions provide the system with a higher level of autonomy [27, 39], which allows the system (agent) to choose the correct behaviour, even in the most difficult circumstances. In practice, various computational models of emotions can be of use in autonomous agents [37].

Emotions in the ISD system perform their function at a higher level of control than the basic ISD control ruled by the need system. In our robotics (agent) practice, they allow us to narrow down the set of possible reactions to the movements that are most adequate (in the view of the designer) for the current time moment and state of the ISD system [23].

Note that from the viewpoint of psychology, in particular, according to the human motivation theory, emotions are one of the most important elements of human behaviour. Systems, based on the model of human psychology (both cognitive and motivative), without emotions would be lame and totally inadequate. Actually, the presented experiment does not take into account the idea of emotions. This concept is mentioned here only as an integral part of the ISD. The whole idea of the higher-level control system (including emotions) is coherent and ready for implementation in future work.

Needs and emotions constitute a crucial part of the ISD model, since they describe the human motivational system. In the same way, they allow us to ’control’ the xDriver’s desire to act. The symbol η represents the degree of unfulfilment of a certain need and hereinafter η will be called a need (Fig. 3). It is an abstract fuzzy value, which takes one or more (two) of three states: satisfaction (lowest), prealarm and alarm (highest) [22]. It can, for instance, be partially satisfied and partially prealarmed (according to its actual crisp value). A need is completely satisfied whenever its crisp value is equal to zero (the applied negative logics is more suitable for several implementational reasons). All needs (H) are grouped according to their importance in a Maslow pyramid (5 levels) [29]. Moreover, each need of the same pyramid level has its individual importance, which changes according to its current state of satisfaction. This importance is described by a weighting function \((\omega (\eta _i))\) [22], which takes the form of a sigmoid curve. The inflection point of the function starts at the foot (beginning) of the membership function of the alarm. The weighting emphasizes the importance of alarmed needs. On this basis, it is easier for xDriver to choose those needs that require immediate reaction and fulfilment.

The number of the needs in such an abstract control system depends on the system designer (creator) and may be different than in human standards (note that a child has about 26 basic needs). In the case of xDriver, the system of needs has been simplified to seven needs, as follows:

The energy optimization need corresponds to minimization of the quantity of the car manoeuvres. Each change of the speed of the vehicle (when braking, for instance) has an impact on this need: if the speed is changing, its respective quantity η is increasing (a corresponding positive/fulfilment level for this need would decrease) proportionally to the speed change. The need of goal achievement is connected to the travelled (virtual) section of the road. Its level of fulfilment wanes when xDriver recedes from the goal, and increases when it is getting closer to the goal. These two needs are at the bottom of the pyramid, as they are the most essential.

The purpose of safety (second) level of the pyramid is safe maintenance of the xDriver system. The security of the car is associated with accidents on the road (every little scratch reduces the fulfilment level of this need (increases its η), a major crash causes that this need goes to the highest state of alert). Furthermore, all traffic rules (including the speed restrictions) affect on the traffic regulations need. Any departure is sanctioned by waning its fulfilment level (in proportion to the quantity/importance of the departure).

The speed need is highly arguable, but from the psychological point of view, driving at full speed gives an esteem for an agent. This need is obviously opposite to the need of security. Thus, the driver must search for a (Pareto) optimum, when it could drive fast and responsibly. It seems very human. Moreover, in terms of Maslow, the higher the position in the Maslow pyramid the lower the importance, and vice versa, the lower the level of the pyramid, the grater the significance of the need group. According to the applied design formula, the need for the speed will only be considered when all other needs are already fulfilled. Due to the fact that the reactions which can satisfy the need of speed can also decrease the satisfaction with respect to the security situation/need, they can thus be easily excluded from the use in inappropriate conditions. This is illustrated by the behaviour of the xDriver, presented further in the article.

The social level of the Maslows pyramid is omitted here. Self-actualization needs are on the top of the pyramid, one can think of them only when the lower level needs are suitably satisfied. The need for creativity (its degree of unfulfilment) is growing in the case of frequent (and tedious) sequences of actions, and it decreases when a new action is implemented.

An important element of the ISD system is an emotion engine. The emotion ξ of the xDriver is associated with several factors: the level of satisfaction of all needs, the emotional context of perceived objects and the earlier state of emotion. There are nine states (colours) of emotion: neutral, anger, anticipation, joy, trust, fear, surprise, sadness and disgust, each in three different amplitudes [33]. When emotion is not in a neutral state, it unlocks reactions that have emotional context (e.g. in case of surprise, the reaction of braking down is unlocked). The higher the magnitude of the emotion, the more desirable is a reaction (within its emotional context).

It is thus transparent that the system of emotions used in the xDriver allows for faster decision-making in emotionally detectable conditions (especially, hazardous or dangerous circumstances), for which there are feasible emotionally designable strategies. In addition, the use of a changeable database of possible reactions allows us to eliminate the possibility of taking a relatively dangerous reaction under safe conditions and vice versa. Simple examples which illustrate the effects of such unsuitable or excessive reactions are the manoeuvres of heavy braking or too-delicate braking. Under normal conditions, for example, performing a quick manoeuvre may threaten the safety of other road users. However, when the situation begins to endanger life, or in the face of other dangers, such as the intersection of the trajectories of two vehicles, a rapid manoeuvre may become necessary. Thus, after detection of a collision situation, the xDriver changes its emotional state and next unlocks the reaction of rapid braking and blocks other, not suitable reactions (like slow acceleration). Note that the emotional part is only described here, as a basis for future works.

The reaction is a sequence of simple and natural (for humans) movements (e.g. movement of the hand, or one step). However, you need more complex reactions when driving a car. The applied set of reactions of the xDriver include:Reactions such as emergency braking, avoiding obstacles, speeding up and others have an emotional context (can be used only under certain circumstances). The system is also capable of creating new reaction, as a sequence of basics reactions (e.g. an overtaking reaction should consist of reduction of gear, turning light signal on, changing lane, speeding up and changing lane one more time), in an evolutionary way, according to a learning scheme, or even at random. Other reactions, such as changing the gear, are planned to be added in future studies.

The block diagram of the algorithm that chooses reactions is shown in Fig. 4. The environment block is responsible for all objects near the road (the outer environment) and for the interpretation of the traffic rules corresponding to the perceived signs (the inner environment). Changes in the current environment of the xDriver influence the states of H and ξ. Namely, the observed objects have their concrete emotional and need context and a specific representation written in the xDriver’s memory, and these data have a definite impact on the system state. Some objects have strictly crisp values of need change (e.g. a pedestrian perceived on a zebra crossing adds 20 to the need for car security—as it endangers the car security and thus raises the degree of unfulfilment). Other objects (such as cars and signs) require the calculation of this impact in relation to the current parameters of the xDriver. For example, if the current speed of xDriver is about 100 km/h, and it perceives the sign of the recommended speed of 50 km/h, the impact of the traffic regulations need is calculated as \((100 - 50 )/\sigma \), where \(\sigma \,=\,10\), thus the implemented need change equals five. A detailed description of this mechanism is presented in [19, 22, 23].

After updating the state of the ISD system (Fig. 4), the xDriver calculates the estimates \((\hat{if})\) of the impact factor of prospective reactions. Selection of the reaction (the third-level filter in psychological terms) is performed in the actual emotional and perceptional context, as it is based on two discriminants: feelings (emotion) and objects assigned to each reaction. Moreover, each reaction has its own assigned effect on the system (H, ξ) described in terms of state increments (i.e. the difference between the state after the reaction implementation and the initial states, before applying this reaction). The reaction impact is estimated based on the history/effectiveness of the application of the reaction. The impact estimator \((\hat{if})\) of a reaction is computed by using the following fuzzy formula:where η represents a need and {S P A} are the states of a need {satisfaction, prealarm and alarm}. It means that \(\hat{if} = 1\) when all needs are satisfied and none of them is in the prealarm or alarm state after the considered reaction. Formula (1) is illustrated in Fig. 5. The first (lower) neuron reflects fuzzy operations between the function of the membership of the needs and the fuzzy satisfaction set \((\mu _s(\eta _i))\) using the respective weights \((\omega (\eta _i))\) of these needs. The second neuron considers the fuzzy prealarm set \((\mu _p(\eta _i)\) and the third neuron reflects the fuzzy alarm set \((\mu _a(\eta _i))\). Both use the fuzzy membership and the weightings. The operations AND and OR are fuzzy neurons using the Einstein norms, and NOT means negation in the Yager sense [26].

The reaction is chosen on the basis of Eq. (1), which is equivalent to the network shown in Fig. 5. However, the inputs to the network/equation are not the values of the reactions or the needs. In fact, each input is the probable effect of a considered/potential reaction on the indicated need η _i in terms of the particular s-, p-, a-membership function \(\mu _x(\eta _i), x=s,p,a)\), computed from the current state of the need \(\eta _i\) and a learned correction \(\delta \eta _i\) in the need taken from the system database. According to this, the equation in fact predicts a potential future averaged fulfilment (impact factor \(\hat{if})\) of all the needs (an aggregate state of the needs) after the hypothetical execution of the reaction. In other words, it describes how much the need system would be satisfied after the execution of the reaction.

The input weights are basically weights of certain needs. They are calculated upon the current need value, its parameters (describing the membership functions) and the Maslow pyramid level [20, 21]. The hidden layers weights were selected experimentally [22].

After computing all impact factors (for a current situation), the ISD simply chooses the reaction which has the highest value of \(\hat{if}\). In the next cycle, the reaction parameters, namely the impacts of the reaction on the states H and ξ, are accordingly updated. For the purpose of updating the reaction database, calculation of the difference between the current and previous states of needs (when the reaction was initiated) is carried out. Certainly, the calculation takes into account the environmental influence (like the current evaluation of traffic condition). Mainly, the ISD system translates the current road conditions (detailed road signs and traffic conditions) to its own specific ’well-being’—to be more specific, to the system of its needs (one of the inner system states of the ISD, as described above). In such a way, the xDriver can consider future effects of its reactions.

The simulation scenario was relatively simple, and there were a few recommended speed signs and one command of lane change, without external objects. In the simulation, we relied only on the needs system (emotions were not involved). During the first part, the xDriver system acted like a cruise control system (CC), whereas the second part showed that the xDriver was something more than that. The road scenario is described in Table 1.

In this particular scenario case, there was no need to use all of the needs, especially the creativity and confidence needs (Table 2) . The visibility range was set at the level of 350 m. Thus, decisions that were performed by the system were dependent on all signs in that range. Furthermore, the environmental influence on the needs was based on the difference between the allowed speed and the speed of xDriver, with an additional random value. There were also several rules that control the changes of the xDriver’s need as shown in Table 3. Even though the possible effects of reaction were estimated only with a certain probability, the system drove quite well. The resulting sequence of its decisions is given in Table 2.

The decisions made by the xDriver system were not repetitious (fuzziness used in the system and quasi-random factors cause such behaviour). What is more important, the reactions of increasing/decreasing the speed were not precise (even with the use of the PI controller), and they ended after the velocity went through the first oscillation which caused the undershoot/overshoot. In such cases, the xDriver oscillated around a certain speed (for example, after slowing to 50 km/h, it tried to keep the speed, but the throttle position was too low, so its subsequent reaction was to increase the speed to 50 km/h). This phenomenon was a clear disadvantage of the xDriver system.

To correct this problem, we do not need reactions of changing the speed, but transitions to a given speed value. Thus, the reactions should include only braking, changing speed to a given value and steering the wheels and gears (if they are allowed by the model). Moreover, we need a reaction which manipulates the attention of the driver, especially the so-called cognitive beam (a phenomenon of drawing attention by the objects from the road). The reactions should be able to expand and narrow the cognitive beam in case of perceived objects (such as informative road signs, objects on the road and built-up area). This issue will be studied in future.

The results of simulation are shown in Figs. 6 and 7. The first reaction of the system was increasing the speed to 90 km/h. The reaction had been interrupted about 650 m, because the xDriver saw the speed limit sign. Thus, the next reaction was decreasing the speed to 50 km/h. In the next step, the xDriver tried to keep the speed at a certain level, but the inertia of the previous reaction was too large, so the speed was going down (the effect of an inexperienced driver). That caused the reaction of increasing the speed and oscillations around the speed of 50 km/h (due to consecutive repeated reactions).

When the xDriver saw the next road sign (the end of recommended speed), it increased the speed to 90 km/h and kept that speed during the next three reactions. Then, the oscillations showed up once again, due to the inertia of the previous reaction. The next reaction was changing the lane (at a given speed of −30 km/h) and series of reactions of keeping the speed followed. At the next point, after another lane change, the xDriver increased its speed to 90 km/h, according to the next sign (and with some delay).

Oscillations observed in the velocity curve (Fig. 6), resulting from the decisions of the xDriver, when driving on a straight road, are a typical behaviour of an inexperienced driver. In future studies, we would like to develop another variant of the xDriver, which would take into account the methods of machine learning, especially learning based on own mistakes (in this way, the oscillations should be eliminated). The presented simulation is a first attempt which proves that the system works correctly as an entity. We have not worked on the problem of obstacle avoidance, nor on the issue of optimization of the velocity curve. Also, driving on a winding and hilly terrain constitutes actually a very interesting challenge for the xDriver. Nevertheless, to implement this idea, the system would have to be substantially developed, taking into account a wide spectrum of possible road/driving scenarios. Note also that the emphasis of this restricted work is on modelling the driver and not on fully autonomous driving. Thus, the problem that concerns this case should be resolved in further research.

Interpretation of Fig. 7 is as follows. At a starting point, xDriver had two needs (of speed and goal achievement) in the alarm state, and one need (of energy optimization) rising from the satisfaction state. The degree of unfulfilment of the goal need decreased until the goal was reached. The energy optimization need went down when the xDriver’s speed did not change. When xDriver exceeded the recommended speed, its traffic regulation need (unfulfilment) started to increase and the value of the need of speed decreased. After it slowed down though, the behaviours of this two needs showed opposite trends. The recommended speed sign (noticed at 1300 m) and the reaction of xDriver changed the need for safety—traffic regulations (it started to rise to the top and stayed there until braking). Similarly, the next behaviour of xDriver was to change the lane—the speed need was completely unsatisfied, but the reaction of the system proves that the traffic regulation need was more important (xDriver’s speed was near to the recommended one). During the manoeuvre of lane changing, the need of car security started increasing. It happened due to the limited area of vision of xDriver, which resulted from travelling outside the lane.

Figure 6 shows that xDriver behaves like humans. It adheres to the speed regulations, though in a fuzzy way. To some extent, it is similar to cruise control (CC) systems. To adapt the xDriver system for the tasks of CC, we need to equip it with a different set of reactions and needs (allowing it to follow other cars, for instance).

For comparison, another simulation was performed at the same scenario but with the use of a PI controller. It had the same configuration settings as the controller used in the xDriver in terms of the increasing/decreasing reactions (in response to perceived road signs). The PI controller parameters had been set manually (\(k_p=0.4\), \(K_i=0.4\), based on a MATLAB tuner). Probably, it would be possible to reduce the overshoot with a better PI tuning, but it was not the objective of this work. The results are shown in Fig. 8.

The curve of the speed for the PI-controlled system is more smooth—there are no drastic speed changes. On the other hand, xDriver reacts faster to the new road signs (on the basis of the velocity deduced by itself). In short, the xDriver behaved like an inexperienced driver. Supposedly, it can be trained like most people, to achieve better results.