12. Reinforcing Trust in Autonomous Systems: A Quantum Cognitive Approach

Bad decisions can have dire consequences. From high exposure events such as an oil spill or a plane crash, to the smaller scale drama of a patient who dies on the operating table, the unavoidable question soon follows: Was this mechanical failure or human error? Yet, in a society where people increasingly base their decisions on autonomous systems such as search engines, recommender systems, or social media, the distinction becomes blurred. Although these systems are based on algorithms (less material, but nonetheless mechanical) people will have to process and consider the provided information, thus becoming the weakest link in the decision chain. In general, mechanical failure, once discovered, seems more easily addressed than human error. So if autonomous systems could be made aware of how humans judge information, they could become more judicious in advising humans, and more proactive in the way they present their information. Currently this is not the case. To change this, we have looked into decades of research about human judgement (For the case of how judgement of a particular system is shaped by some of its properties see Sect. 7.​6 of this book.). We found a whole range of human judgement that deviates substantially from what would be normatively correct according to logic and probability theory. As example take the famous experiment in which Tversky and Kahneman [35] presented participants with the following text:

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations Which is more probable:

(a) Linda is a bank teller, or

(b) Linda is a bank teller and is active in the feminist movement?

In this experiment, the participants consistently rated option (b) as more probable than (a). However, according to the axioms of probability theory, the conjunction of events is less probable than a single event, so (b) must be less probable than (a).

This type of judgement errors has been found so invariably that it became known as the conjunction fallacy (see [31, 34] for other such fallacies). These findings are not widely known, let alone implemented, by programmers designing the communication part of autonomous systems. The latter systems so far adhere to the laws of probability and logic, which is their strength. Their weakness, however, is to not account for how humans make decisions. For example, an intelligent system that would correctly answer (a) in the experiment above, might bewilder the human who was expecting answer (b). In turn this could erode that human’s trust in such a system.

Let us take another, more concrete, example from technology soon to become reality. Suppose you arrive late in the evening at a meeting. As the street is clearly indicated as a tow zone, you let your self-driving car find a parking space elsewhere to park in your stead. Later you come back and find that it parked in the first free spot in that same street. Would this not be the time to reconsider your trust in the autonomy of the car? And so you instruct it not to do this again. But then it politely explains [24] that the tow zone only applies during rush hour, thus restoring your trust. An even better scenario would be that when you leave the car to park itself, it would tell you that yes, this is a tow zone, but only during rush hour. Machines have become smart enough to do the first part, in this case finding a parking space and park. But then, is it not time to work on the second part, where the machine proactively explains its actions from the human’s point of view? Or the part where it can foresee a human error because it knows how a human would reason in a particular case? This stands to hugely help humans put trust in autonomous systems, and in the current presentation we show a direction one could go.

To elaborate the problem we want to address, before trying to solve it, consider Wittgenstein’s often cited remark “If a lion could talk, we could not understand him” ([37], p. 223). The remark has been food for much thought and speculation (that it is probably true) for half a century, notably in the philosophy of language. We wonder, however, that if we cannot even understand a lion, whence the confidence that we will understand machines when it comes to communicating with them verbally? Here we think that Wittgenstein’s own, less quoted, comment can lead the way [37]:

If language is to be a means of communication there must be agreement not only in definitions but also (queer as this may sound) in judgements. This seems to abolish logic, but does not do so — It is one thing to describe methods of measurement, and another to state results of measurement. (p. 391, our italics)

This is an important vantage point for the current presentation: first it emphasizes the role of judgement, second it distinguishes the method of measurement from its result, and third it challenges the role of logic.

We will express these notions in the language of quantum cognition , which derives terminology and computations from quantum mechanics. But whereas quantum mechanics deals with physical states, quantum cognition describes cognitive (or mental) states. The Linda experiment can be described in this language, as we showed already in the book we wrote about quantum cognition [10]. For further details we will refer the reader to that book, so that we can use the space here to describe a new experiment. The experiment is also modeled using quantum cognition, laying a foundation for its implementation in future autonomous systems. Incorporating such models can proactively help the user avoid mistakes that are inherent in human judgement and thus prevent an erosion of trust. We contend that in this way the interactions between humans and future autonomous system will become more effective.

The conjunction fallacy does not mean that people always judge the probability of a conjunction as higher than each of its conjuncts. That would indeed be counter to probability theory. Just imagine that the choice between (a) and (b) was presented without the story about Linda. Then one would expect everyone with some notion of probability to choose (a) over (b), as confirmed in [35]. But when asked the question after first hearing the story, even people schooled in statistics fall victim to the conjunction fallacy. Why is this? The question generated a host of publications with possible explanations over the last several decades (see [19] for an overview). Among the many kinds of explanations offered, two stand out in particular. One assumes that words such as ‘and’ and ‘probability’ are misunderstood by the participants, or at least not understood in their formal interpretation. The other assumes a reasoning bias. A recent overview [30] concludes that the latter has the best support of the two. But this answer begs the question: if there is a reasoning bias, where does that reasoning bias originate?

Indeed, we are not satisfied with just knowing there is such a bias, rather we would like to describe how that bias unfolds as a cognitive process. To do so, let us formulate the participants’ judgements as the outcome of a decision process. The explanations in the literature almost invariably involve two competing states, one in which Linda is a bankteller, and another where she is a feminist. To most participants in the experiment these states are not compatible, and whether bias or reasoning, each can in principle tip the scale in favor of one state or the other. In order to make headway, we have to take a closer look at the notion of compatible and incompatible states.

In this presentation we will formulate states in the language of quantum cognition. Especially for the reader who is not already familiar with this approach, we will first recall some concepts from quantum mechanics. One such concept is the (formal) notion of compatible and incompatible states. Incompatibility lies at the heart of Heisenberg’s famous uncertainty principle. It holds that when we are certain about a quantum particle’s momentum, we are necessarily uncertain about its position, and vice versa. Position and momentum are therefore called incompatible states. On the other hand, again given the momentum of that particle, we can still measure its kinetic energy with certainty, momentum and kinetic energy thus being compatible states.

There is a vision of nature that readers may have entertained themselves at one time or another, namely that “all phenomena could be explained mechanically if only we knew enough.” Those readers are in good company, as this is a direct quote from Leibniz’s writing of 1695 [29]. Yet the problems with this vision are several. First, we are not omniscient and second it is uncertain if we will ever have enough computing power to do the explaining. But even if is this may become an issue for intergalactic travel, why would it apply to the Linda experiment? After all, everything the participants in the experiment need to know is given in the instructions, and the computing power needed is an unassuming application of probability theory. So why is an explanation for the experimental findings still wanting?

There is a third, perhaps more fundamental, problem with Leibniz’s position: His mechanical view of the laws of nature has been drastically undermined with the advent of quantum mechanics. It turns out that some phenomena require a probability calculus, but different than given by classical probability theory. For example, probabilities may not always add up to 1.

The problem does not just apply to the description of physical systems, but also to cognitive systems, and more in particular those that play a role in interactions between humans and autonomous systems. We can safely assume that such interactions require decisions under uncertainty. For decades cognitive scientists have studied how humans make judgments under such conditions. The theories they produced can be roughly divided into a heuristic and a rational approach.

The heuristic approach is founded on so-called bounded rationality [33]. It assumes that in order to make decisions, humans use simple heuristics such as representativeness, anchoring-and-adjustment, and base-rate neglect. They support powerful (often inductive) processes that may depend on the environment [22]. In contrast, the rational approach conforms to rules drawn from a theory, most notably Bayes’ rule [13] or expected utility theory [32]. In this approach the same basic axioms can be used to derive inferences and utilities across all environmental conditions. Recently, a third approach, called quantum cognition has emerged [10, 11, 36]. In common with the heuristic approach, it assumes that the human decision maker is subject to bounded rationality. And in common with the rational approach, inferences used for decisions are derived from basic axioms that define a probability theory. However, the axioms are different from those employed by the Bayesian approach, and consequently, so are the decisions that follow from it.

Two core concepts underpinning quantum cognition are incompatibility and contextuality. We will briefly come back to ‘contextuality’ later in this chapter, but right now we will continue with ‘incompatibility’ from Sect. 12.2 in more detail. Both concepts play an important role in formalizing people’s perception of trust.

We could perhaps have presented participants with pieces of news of varying degrees of trustworthiness. However, note the many variables that should then be kept under control, such as the participant’s background knowledge or the ephemerality of news. Instead we used pictures of which they had to assess the trustworthiness on a five point scale from untrustworthy to very trustworthy [18]. (Trustworthy was defined as an image that gave an accurate representation of a situation, person or object.) The participants were also asked to supply the reasons for their rating. An example of one such a picture was a smiling face of Putin, which can be seen by following the URL in [7].

The image of Putin was deliberately chosen. Many participants will know that he is a former KGB agent and probably not predisposed to smiling. Would this then lead to uncertainty whether the image had been photoshopped, and a consequent lack of trust that the image is a true an accurate depiction of Putin? When analyzing the qualitative data a curious phenomenon emerged. A considerable number of participants appeared to confuse the decision regarding trust of the image with a decision on whether they trust the content of the image. For example, “This is Vladimir Putin, a world leader I associate with dishonesty and distrust, who works to his own agenda and doesn’t worry about other people”, “I wouldn’t trust the person, but the photo is fine”, and “I really could not separate what I know about this man from his image”.

Assuming the confounding of the trustworthiness decision is a robust cognitive effect, how can it be explained? Quantum cognition offers an explanation based on incompatible decision perspectives. Consider diagram (a) on the left in Fig. 12.1.

Quantum cognition is defined by quantum probabilities which are related geometrically, not by an underlying Boolean algebra over the event space. More formally, let T denote the decision that the participant trusts the image and let \(\overline{T}\) denote the decision that the participant does not trust the image. These are the basis vectors of the image subspace. Similarly, let C and \(\overline{C}\) denote the basis vectors of the content subspace. The cognitive state \(\psi \) is superposed between the two decisions in both the image and content subspaces. The probability that the participant decides they trust the content portrayed in the image is equal to the square of the projection of the cognitive state \(\psi \) onto the basis vector C, denoted \(\Vert \mathbf {P}_C\psi \Vert ^2\). We can see from the diagram that the length of this projection is small, hence the probability is small, which reflects the weak predisposition of the participant to trust Vladimir Putin. Conversely, the predisposition not to trust Putin is high because of the long projection of the cognitive state vector \(\psi \) onto the basis vector \(\overline{C}\). Hence the associated probability \(\Vert \mathbf {P}_{\overline{C}}\psi \Vert ^2= 1 - \Vert \mathbf {P}_C\psi \Vert ^2\) is high.

Consider diagram (b) in Fig. 12.1. Note that the cognitive state vector now lies on the basis vector. This models the situation in which the participants have decided that they do not trust the content of the image (Vladimir Putin). Therefore, the cognitive state \(\psi \) is no longer superposed with respect to the decision perspective regarding the trustworthiness of the content. Note, however, that the cognitive state vector is necessarily superposed with respect the decision perspective regarding the trustworthiness of the image. The reason for this is the degree of rotation between the two decision subspaces. Because these subspaces are not orthogonal to each other the decision perspectives are incompatible. Incompatibility between the decision perspectives explains why participants confuse the decision on the trustworthiness of the image versus content of the image. The reason is that in case of incompatibility the law of total probability does not hold. For example, the probability that the participant will trust the content of the image in terms of the two decision perspectives is as follows: The preceding rendered in standard probability theory looks like the law of total probability, which is being modified by extra term \(\textit{Int}\): \(\textit{Int}\) is referred to as the “interference term”. The interference term can positively or negatively contribute to the probability p(C). For this reason, incompatible subspaces have been put forward as a natural explanation why human beings do not adhere to the law of total probability like in the conjunction and other so called fallacies in human decision making [12]. When the interference term is zero, the law of total probability holds. This happens when the decision perspectives are compatible.

Incompatible decision perspectives are a recent development in cognitive modeling and their striking characteristic is the use of “quantum” probabilities. By quantum probabilities, we mean that the decision event space is modelled as a vector space rather than a Boolean algebra of sets. A key differentiator is the use of the interference term. When this term is non-zero, violations of the law of total probability occur. The interference term has been used in models of the perception of gestalt images [15, 28], models of the conjunction and other decision fallacies [12, 14], modeling violations of rational decision theory [6, 28, 31], modeling belief dynamics [34] and conceptual processing [3‐5, 20, 21]. In this presentation there is no need to go any deeper into the fine points where quantum cognition and the Bayesian approach part company. Therefore we only present Table 12.1 as a summary, and refer the interested reader to [9] for further details.

Placed in a psychological context, the uncertainty principle becomes relevant because a person’s understanding of two events, such as two different perspectives on a matter, requires changing from one point of view to another, and the two points of view can imply incompatibility. In other words, the uncertainty principle entails that it is not possible to be simultaneously decided on both the image and content with respect to assessing trustworthiness. Just like it is not possible to form a joint probability of both the position and momentum of a quantum particle, it is not possible for the human participant to form a joint probability across decisions, e.g., whether they both trust the image and the content of the image. Not being able to form joint probabilities signals the presence of contextuality. A well studied example of contextuality is the curious phenomenon of entanglement: Empirical observations are collected in four measurement settings of a system of two quantum particles such as photons. Each of the four settings yields a pairwise joint probability distribution which models the observations made in that measurement setting. An entangled system is deemed “contextual" because it is not possible to combine these four pairwise joint probability distributions into a single probabilistic model such that the four pairwise empirical distributions are marginal distributions of this global model. Even though contextuality manifests within the sub-atomic realm, there is a growing body of research which is exploring whether contextuality manifests in cognition and related areas (e.g., [1, 2, 8, 9, 16, 17]). In the context of our example, contextuality arises because the image and content decision perspectives cannot be meaningfully combined into a single joint distribution.

In the foreseeable future, humans and autonomous systems will engage in shared decision making. Given the discrepancies between the way they arrive at decisions, whose form of rationality should be given precedence: human or machine?

One form of rationality can be termed “Bayesian rationality” in which Bayesian probability theory provides such powerful models of both human and machine cognition, that it is sometimes called “Bayesian fundamentalism” [26]. We made the point in this chapter that humans often do not adhere to Bayesian rationality, but rather to a “quantum rationality,” as it is based on the the same Dirac-von Neumann axioms as quantum theory. Neither of these two rationalities should be given precedence. We argued that quantum rationality is often more suited to model human decision making, and Bayesian rationality more to model decision making by machines. So the more pertinent question is how to best align these rationalities so that shared decision making between human and machine becomes more effectual than that of each in isolation.

There are good examples where traditional (often brute force) mechanisms were unable to solve difficult problems, and which became tractable when augmented with cognitive elements. One famous example is the breaking of the Enigma code by augmenting brute force methods with simple aspects of human communication [23]. A second example is the incorporation of human reasoning in chess programs. And finally we have already seen this clearly confirmed in our own research in areas outside that of this chapter [24, 25].

Therefore, we propose to augment the rationality already available in the current systems with quantum rationality. We expect that future autonomous systems can compute decisions based on both rationalities and hence detect situations when these decisions do not align. In such cases the machine could make the human aware of the discrepancy, thus preventing a potential erosion of trust between human and machine.