Where Do Mistakes Lead? A Survey of Games with Incompetent Players

Chronologically, Beck and Filar [10] introduced incompetence to matrix games and Beck et al. [9] introduced incompetence to bimatrix games. Essentially, by quantifying a player’s tendency to accidentally deviate from their selected actions, these authors represent incompetence as stochastic matrices that can be used to account for these deviations. The application of incompetent classical games to military planning is discussed in [8] and [7].

We note that the notion of incompetence introduced here is superficially similar to several concepts used to measure the sensitivity of equilibria to changes in a game’s parameters. For example, Selten [63] imagines players having “trembling hands” that cause them to accidentally execute unintended actions with negligible probability. This is used to refine the equilibrium solution concept in extensive-form games by defining trembling hand perfect equilibria. However, unlike the notion of incompetence, a trembling hand is not intended to model players making mistakes with arbitrary or even prescribed probabilities (e.g. a tennis player who routinely places a ‘passing shot’ in the opponent’s hitting zone).

While the concepts and some of the results are generalisable to N-person non-cooperative games, the setting of classic two-player games—meaning matrix and bimatrix games—is a natural starting point for the introduction of incompetence to game-theoretic models. Larkey et al. [45], when discussing the game “Sum Poker”, observe that there are several cognitive and physical limitations that might prevent a player from finding or implementing optimal strategies. Then, seeking to classify the specific difficulties a player encounters, they propose a typology of skills consisting of:Although Larkey et al. [45] apply this typology to experimentally compare different strategies in “Sum Poker” under different skill limitations, a precise mathematical formulation of skill is not provided. The notion of incompetence mainly addresses the issue of execution skill as it quantifies accidental (ipso facto, unintended) deviations from a player’s chosen strategy.

The concept of incompetence is a useful modelling tool in traditional game-theoretic settings because it captures a player’s inability to precisely control the outcome of their actions. Necessarily, real-world strategic interactions are often complicated and a player’s intentions might not be perfectly realised due to noise from their environment. For instance, a tennis player is unable to control the exact trajectory of a shot and might sometimes make consequential mistakes. Similarly, in the economic context, it is conceivable that a firm in the classic Cournot oligopoly model is unable to guarantee the quantity of goods produced, perhaps due to sporadic errors occurring during production. The latter may reflect flaws in the firm’s quality control regime which in itself is related to its level of competence¹. The players in these situations must accept some degree of variability in the outcomes of their actions and incompetence is a method capable of accounting for this variability.

In this paper, in the context of classical games, we will review the introduction of incompetence and present several related, unpublished, results. Moreover, we will discuss and solve a simple model for incrementally learning to decrease incompetence during a repeatedly-played incompetent matrix game.

Game theory applied to populations of species has branched out from non-cooperative game theory and evolved into an independent field called evolutionary game theory [28, 56, 68]. The setup of evolutionary games differs from the classical games in the very basic assumption rationality of players. Obviously, one cannot expect rational behaviour from individual bacteria or lions. However, the selection strength still acts rationally, which follows from the classic prediction that the fittest survives [69]. Hence, in evolutionary games players, or animals, do not make conscious choice of strategies to play as humans in economic settings. Instead, individuals are born with a strategy predefined by their parents. The competition then happens at a genetic level, which encodes the strategy. Incidentally, the question of how cooperation evolves in biology is still open and, possibly, related to the coexistence of different strategies at equilibrium. We note that in a recent paper [2] thermodynamics was invoked to shed light on cooperation in evolutionary games.

The concept of incompetence was considered in biological settings when studying the evolution of social behaviour [36‐40]. That is, incompetence now acts at the selection level rather than organisms themselves. Then, incompetence can be seen as behavioural plasticity leading to mixed strategies executed at a genetic level. As a result, we do not require organisms to be aware of probability distributions of all genes as well as their mistakes. Instead, selection forces act upon these distributions driving the competition among types. Such plasticity can be of variable degree, depending on the environmental conditions and adaptations of organisms. The latter will correspond to the level of incompetence discussed above.

Naturally, the idea of behavioural plasticity or stochasticity is not novel in the field of evolutionary games and recently became one of the foci in the field. There are many approaches considered in biological settings such as genetic mutations [12, 57, 70, 72], learning processes [23, 29, 41, 42, 51, 57, 64], adaptation dynamics [47], phenotypic plasticity [15], noise in continuous and discrete-time replicator dynamics [4, 6, 19, 22] and environmental fluctuations [75, 81]. Thus, the notion of incompetence of players merely fills a new niche where behavioural stochasticity is only induced at the moment of interactions.

Let us demonstrate this concept on a well-studied example of a Hawk-Dove game [67]. Imagine that individuals in a well-mixed large population compete for some resource. Two behavioural strategies are available in a population: a Hawk (aggressive) strategy and a Dove (passive) strategy. That is, Hawks fight for the resource, while Doves prefer to share equally and flee when attacked. This game has a payoff matrix of the same structure as a Chicken or Snowdrift games. As in these classical examples, in an equilibrium, both strategies stably coexist. The game can be illustrated as an interaction between a naturally aggressive person and a naturally passive one. Of course, a counter-attack in response to aggression is not something one naturally expects from a passive person. However, behavioural plasticity induced by incompetence can lead to situations when a passive player responses aggressively or an aggressive player flees instead of fighting. While their strategies did not change as such, the behaviour exhibited by individuals was altered. In [37], it was shown that such an assumption may lead to different evolutionary outcomes and may change the way selection is realised. In this survey, we discuss these results and demonstrate them on an example of a biological game.

An \(m \times n\) bimatrix game \(\varGamma \) consists of a pair of action sets \({\mathcal {A}} = \{1, 2, \ldots , m\}\) and \({\mathcal {B}} = \{1, 2, \ldots , n\}\) and a pair of reward matrices \(R^1 = (r^1(i, j)) \in {\mathbb {R}}^{m \times n}\) and \(R^2 = (r^2(i, j)) \in {\mathbb {R}}^{m \times n}\). Here, throughout this section only, we use non-standard notation for matrix entries to accommodate additional symbols associated with incompetent games. After an action \(i \in {\mathcal {A}}\) is selected by Player 1 and an action \(j \in {\mathcal {B}}\) is selected by Player 2, they receive rewards according to the matrix entries \(r^1(i, j)\) and \(r^2(i, j)\), respectively. A (mixed) strategy extends this behaviour to allow for randomised action selection. Specifically, Player 1 chooses a strategy from the probability simplexover \({\mathcal {A}}\) and Player 2 chooses a strategy from the probability simplexover \({\mathcal {B}}\). The resulting strategy profile \((\mathbf {x}, \mathbf {y}) \in \mathbf {X} \times \mathbf {Y}\) yields an expected reward ofto Player \(k \in \{1, 2\}\) where \({\mathbf {y}}^T\) is the transpose of \({\mathbf {y}}\). We want to find the (Nash) equilibria of \(\varGamma \), which capture the notion of a stable strategy profile. Precisely, \((\mathbf {x}^*, \mathbf {y}^*) \in \mathbf {X} \times \mathbf {Y}\) is an equilibrium whenever it is resilient to unilateral deviations or, equivalently,for all \(\mathbf {x} \in \mathbf {X}\) and \(\mathbf {y} \in \mathbf {Y}\). Nash [54], in a seminal contribution to game theory, proves that every game with finitely-many players and actions has an equilibrium point. Vorob’ev [79] shows that, in bimatrix games, the set of equilibria is the union of a finite collection of convex sets. Specifically, these are called maximal Nash subsets and are the largest equilibrium-containing sets that are closed under interchanging a player’s strategies. The extreme points of a maximal Nash subset are called extreme equilibria and are associated with paired non-singular square submatrices of \(R^1\) and \(R^2\) called kernels (see Kuhn [43]).

A bimatrix game satisfying the zero-sum property: \(r^1(i, j) = - r^2(i, j)\) for all \(i = 1, 2, \ldots , m\) and \(j = 1, 2, \ldots , n\), is called a matrix game and is described by the single matrix \(R := R^1 = -R^2\). Note that, when \(\varGamma \) is a matrix game, every equilibrium \((\mathbf {x}^*, \mathbf {y}^*) \in \mathbf {X} \times \mathbf {Y}\) achieves the same reward \(\mathrm {val}(\varGamma ) := v^1(\mathbf {x}^*, \mathbf {y}^*) = - v^2(\mathbf {x}^*, \mathbf {y}^*)\), which is called the value of the game \(\varGamma \). Moreover, in recognition of von Neumann’s [55] celebrated minimax theorem showingthe component strategies \(\mathbf {x}^*\) and \(\mathbf {y}^*\) of an equilibrium are often called (minimax) optimal strategies.

A completely mixed equilibrium \((\mathbf {x}^*, \mathbf {y}^*) \in \mathbf {X} \times \mathbf {Y}\) of a bimatrix game \(\varGamma \) is an equilibrium under which every action is played with non-zero probability. If \(\varGamma \) has only completely mixed equilibria, then it is called a completely mixed game and has only a single (completely mixed) equilibrium (see Kaplansky [35] and Raghavan [62]). Additionally, if \(\varGamma \) has a maximal Nash subset containing only completely mixed equilibria, then it is called a weakly completely mixed game. Jurg et al. [34] prove that a weakly completely mixed game contains a unique completely mixed equilibrium.

Beck et al. [9] introduce incompetence to a bimatrix game \(\varGamma \) by allowing players to accidentally deviate from their intended actions. Specifically, after a pair of actions is selected from \({\mathcal {A}} \times {\mathcal {B}}\), incompetence randomly determines an executed action profile—also from \({\mathcal {A}} \times {\mathcal {B}}\)—according to a predefined probability distribution. This distribution is represented by the incompetence matrices \(Q^1 := (q^1(i, i')) \in {\mathbb {R}}^{m \times m}\) and \(Q^2 := (q^2(j, j')) \in {\mathbb {R}}^{n \times n}\). After Player 1 and Player 2 select the actions \(i \in {\mathcal {A}}\) and \(j \in {\mathcal {B}}\), they execute the actions \(i' \in {\mathcal {A}}\) and \(j' \in {\mathcal {B}}\) with probability \(q^1(i, i')\) and \(q^2(j, j')\), respectively. The notation \(\varGamma _{Q^1 Q^2}\) denotes the game \(\varGamma \) played under incompetence. If the incompetence matrices \(Q^1\) and \(Q^2\) are unambiguous, we will often replace the subscript “\(Q^1 Q^2\)” with simply “Q” (e.g. \(\varGamma _Q\) instead of \(\varGamma _{Q^1 Q^2}\)).

Note that the original incompetence framework described by Beck et al. [9] allows the players’ sets of selectable and executable actions to differ. This is especially useful for modelling actions that can be executed with variable quality. Beck and Filar [10] give an example of a capability acquisition game in which a defender, after selecting the action “Conventional Defence”, may execute either “Good Conventional Defence” or “Bad Conventional Defence”. Here, for the sake of notational simplicity, we assume that a player’s sets of selectable and executable actions coincide.

Suppose that Player 1 selects the action \(i \in {\mathcal {A}}\) and Player 2 selects the action \(j \in {\mathcal {B}}\). The expected reward received by Player \(k \in \{1, 2\}\) under incompetence isHence, \(\varGamma _Q\) can be treated as another bimatrix game with the incompetent reward matrix \(R^k_{Q} := (r^k_{Q}(i, j)) \in {\mathbb {R}}^{m \times n}\) belonging to Player \(k \in \{1, 2\}\). Clearly, we haveNote that, as an immediate consequence of (7), an incompetent game derived from a matrix game is also a matrix game. The expected reward granted to Player \(k \in \{1, 2\}\) in \(\varGamma _{Q}\) isfor each \((\mathbf {x}, \mathbf {y}) \in \mathbf {X} \times \mathbf {Y}\).

Beck et al. [9] are not only interested in games with static incompetence, but also dynamic games wherein players are able to vary their incompetence. This is captured by a pair of learning trajectories \(Q^1 : [0, 1] \rightarrow {\mathbb {R}}^{m \times m}\) and \(Q^2 : [0, 1] \rightarrow {\mathbb {R}}^{n \times n}\). Then, for each pair of learning parameters \(\lambda , \mu \in [0, 1]\), the corresponding incompetent game \(\varGamma _Q(\lambda , \mu )\) has Player 1’s incompetence matrix defined as \(Q^1(\lambda )\) and Player 2’s incompetence matrix defined as \(Q^2(\mu )\). Equivalently, \(\varGamma _Q(\lambda , \mu )\) is the bimatrix game with reward matricesfor each Player \(k \in \{1, 2\}\). Although the family of parameterised incompetent games \(\varGamma _Q(\lambda , \mu )\) is interesting in its own right (see Sect. 2.4), it is also an essential building-block used to construct dynamic learning games (see Sect. 2.5).

Example (attack-defence game with incompetence) Consider, as an example, a matrix game \(\varGamma \) played between two aeroplane pilots—labelled “Attacker” (Player 1) and “Defender” (Player 2)—competing over three sites. The attacker wants to destroy a site and the defender wants to prevent this from occurring. Precisely, we have the action sets \({\mathcal {A}} = \{1, 2, 3\}\) and \({\mathcal {B}} = \{1, 2, 3\}\) where, for each \(i, j \in \{1, 2, 3\}\), Player 1’s action i means “Attack Site i” and Player 2’s action j means “Defend Site j”. A successful attack occurs if and only if the defending pilot does not anticipate the attacking pilot’s destination or, equivalently, the executed action profile \((i, j) \in {\mathcal {A}} \times {\mathcal {B}}\) has \(i \ne j\). The attacker receives a reward \(\nu _1 = 3\) when Site 1 is destroyed, \(\nu _2 = 4\) when Site 2 is destroyed, and \(\nu _3 = 5\) when Site 3 is destroyed. The corresponding utility matrix iswhere \(R = R^1 = - R^2\). The game value of \(\varGamma \) is \(\nicefrac {120}{47} \approx 2.55\) and its (unique) equilibrium has the attacking strategy and the defending strategy .

We use incompetence to capture the pilots’ navigation skills and their propensity to arrive at an incorrect site after getting lost. Define their learning trajectories \(Q^1, Q^2 : [0, 1] \rightarrow {\mathbb {R}}^{3 \times 3}\) where, for each \(\lambda , \mu \in [0, 1]\), we setwhere \(J_n \in {\mathbb {R}}^{n \times n}\) is the \(n \times n\) all-ones matrix and \(I_n \in {\mathbb {R}}^{n \times n}\) is the \(n \times n\) identity matrix. Note that \(Q^k = \nicefrac {1}{n} J_n\) is called uniform incompetence and \(Q^k = I_n\) is called complete competence [9]. The resulting incompetent reward matrices at \(\lambda = \mu = 0\) and \(\lambda = \mu = \nicefrac {1}{2}\) arerespectively. The game value of \(\varGamma _Q(0, 0)\) is \(\nicefrac {8}{3}\) and the game value of \(\varGamma _Q(\nicefrac {1}{2}, \nicefrac {1}{2})\) is \(\nicefrac {835}{324} \approx 2.58\). Moreover, since complete competence is achieved at \(\lambda = \mu = 1\) and \(\varGamma _Q(1, 1) = \varGamma \), we already know that the game value of \(\varGamma _Q(1, 1)\) is \(\nicefrac {120}{47} \approx 2.55\). Thus, it appears that the parameterised incompetent games \(\varGamma _Q(\lambda , \mu )\) move in the defender’s favour as the learning parameters are increased along \(\lambda = \mu \).

A clearer picture of the game value’s dependence on these learning parameters is achieved in Fig. 1 by plotting the function \((\lambda , \mu ) \mapsto \mathrm {val}(\varGamma _Q(\lambda , \mu ))\) on the domain \([0, 1] \times [0, 1]\). Note that, in this specific example, \((\lambda , \mu ) \mapsto \mathrm {val}(\varGamma _Q(\lambda , \mu ))\) is piecewise linear and non-decreasing (or non-increasing) in the variable \(\lambda \) (or \(\mu \)). This means that learning is beneficial for the attacking and defending player when their opponent’s incompetence remains fixed. Furthermore, the game value plateaus on the region \([\nicefrac {11}{47}, 1] \times [\nicefrac {26}{47}, 1]\) indicating that the attacker reaches their “maximum useful skill’ at \(\lambda = \nicefrac {11}{47} \approx 0.23\) and the defender reaches their “maximum useful skill” at \(\mu = \nicefrac {26}{47} \approx 0.55\). Interestingly, \(\varGamma _Q(\lambda , \mu )\) is also completely mixed on \((\nicefrac {11}{47}, 1] \times (\nicefrac {26}{47}, 1]\), which suggests a connection between complete mixedness and this game value plateau. We will further explore the general properties of parameterised incompetent games in Sect. 2.4.

Although [9] and [10] view incompetence as modifying a player’s reward matrix, it is also possible to view it as modifying their strategy spaces. Here, we return to the setting of a static incompetent game \(\varGamma _Q\) with incompetence matrices \(Q^1 \in {\mathbb {R}}^{m \times m}\) and \(Q^2 \in {\mathbb {R}}^{n \times n}\). Note that, after Player 1 selects a strategy \(\mathbf {x} \in \mathbf {X}\) (or Player 2 selects a strategy \(\mathbf {y} \in \mathbf {Y}\)), the resulting executed strategy is \(\mathbf {x} Q^1\) (or \(\mathbf {y} Q^2\)) after incompetence is included. What strategies are the players able to execute? Well, Player 1 and Player 2 are able to execute the strategies inrespectively. We call \(\mathbf {E}^k(Q^k)\) the executable strategy space belonging to Player \(k \in \{1, 2\}\) and, when the incompetence matrices are unambiguous, we simply write \(\mathbf {E}^k\) instead. Importantly, from the perspective of an outside observer who only sees that players have executed strategies from \(\mathbf {E}^1\) and \(\mathbf {E}^2\), we would be unable to distinguish whether they are playing the competent game \(\varGamma \) or the incompetent game \(\varGamma _Q\). Figure 2 shows some of the executable strategy spaces within the previously discussed attack-defence game with incompetence. Note that the transition between executable strategy spaces can be more complicated than the “growing” seen in Fig. 2.

What is the connection between equilibria of \(\varGamma _Q\) in \(\mathbf {X} \times \mathbf {Y}\) and equilibria of \(\varGamma \) in \(\mathbf {E}^1 \times \mathbf {E}^2\)? Theorem 1 gives conditions under which an equilibrium in the competent game \(\varGamma \) can be converted into an equilibrium in the incompetent game \(\varGamma _Q\), and vice versa. Theorem 1(i) implies that an equilibrium of \(\varGamma \) in \(\mathbf {E}^1 \times \mathbf {E}^2\) is always executed by an equilibrium of \(\varGamma _Q\). Meanwhile, Theorem 1(ii) implies that there exists an equilibrium of \(\varGamma \) in the interior of \(\mathbf {E}^1 \times \mathbf {E}^2\) provided that \(\varGamma _Q\) is weakly completely mixed.

Corollary 1, which states that a completely mixed matrix game achieves the same game value as its competent counterpart, was originally presented by Beck and Filar [10]. Although they give a utility-centred argument based on Shapley and Snow’s [66] game value formula, we give an alternative strategy-centred argument based on Theorem 1(ii). Note that, a generalisation of this result to bimatrix games is presented in [9]; however, we still choose to highlight the matrix game version for later discussion.

Lastly, the result in Theorem 1(ii) can be extended to incompetent bimatrix games that are “almost” weakly completely mixed. Theorem 2 shows that, if \(\varGamma _Q\) can be approximated by a sequence of weakly completely mixed incompetent games, then \(\varGamma \) has an equilibrium in \(\mathbf {E}^1 \times \mathbf {E}^2\).

Now, we return to the dynamic setting where \(\varGamma _Q(\lambda , \mu )\) denotes a family of incompetent games parameterised by a pair of learning trajectories. A central focus in the development of incompetence has been the variational properties of \(\varGamma _Q(\lambda , \mu )\) when \(\varGamma \) is a matrix game or a bimatrix game. Here, we will summarise what is known about the behaviour of these incompetent games under variations in the players’ learning parameters.

Beck et al. [9] study the dependence of equilibrium-induced expected rewards on the players’ learning parameters. They present Theorem 3 and Theorem 4 showing that, under certain conditions on \(Q^1(\lambda )\) and \(Q^2(\mu )\), the expected rewards granted by a specific extreme equilibrium have useful representations.

Furthermore, in addition to proving specialisations of Theorem 3 and Theorem 4 for matrix games, Beck and Filar [10] establish several other properties regarding the game value of a parameterised incompetent matrix game \(\varGamma _Q(\lambda , \mu )\). Specifically, they prove that the function \((\lambda , \mu ) \mapsto \mathrm {val}(\varGamma _Q(\lambda , \mu ))\) is continuous and not-necessarily monotone in \(\lambda \) and \(\mu \). It is also shown that a player can never achieve a greater reward than under complete competence; that is,for all \(\lambda , \mu \in [0, 1]\). Beck and Filar [10] also briefly address the plateauing game values of some parameterised incompetent matrix games (see, for example, Fig. 1) by noting that Corollary 1 might apply when a player approaches complete competence. The tools developed in Sect. 2.3 allow us to further explore this observation. Consider the set of learning parameterson which \(\varGamma _Q(\lambda , \mu )\) is completely mixed. Assume that the learning trajectories \(Q^1(\lambda )\) and \(Q^2(\mu )\) are continuous. Then, given that the set of reward matrices belonging to completely mixed matrix games is open (see Jansen [33]), the set \({\mathcal {C}}\) is also open. Theorem 2 shows that, for each \((\lambda , \mu ) \in \overline{{\mathcal {C}}}\), the players are both able to execute a completely competent optimal strategy in \(\varGamma _Q(\lambda , \mu )\). This means that, by an identical argument to Corollary 1, the function \((\lambda , \mu ) \mapsto \mathrm {val}(\varGamma _Q(\lambda , \mu ))\) is constant on \(\overline{{\mathcal {C}}}\). Hence, we expect a game value plateau to emerge whenever \(\varGamma _Q(\lambda , \mu )\) becomes completely mixed.

Next, we will demonstrate a simple model of incremental learning in a parameterised family of incompetent matrix games \(\varGamma _Q(\lambda , \mu )\). This incremental learning game \(\varGamma _{\mathrm {inc}}\) is a stochastic game unfolding over an infinite time horizon \(T = \{0, 1, 2, \ldots \}\) in which, between repeated plays of an incompetent game, the players may choose to increment their learning parameters through the ordered sets \(\varLambda := \{\lambda _1, \lambda _2, \ldots , \lambda _M\}\) and \(M := \{\mu _1, \mu _2, \ldots , \mu _N\}\). It is assumed that \(\lambda _i < \lambda _{i + 1}\) and \(\mu _j < \mu _{j + 1}\) for each \(i = 1, 2, \ldots , M - 1\) and \(j = 1, 2, \ldots , N - 1\). This means that a player’s skill parameter can never be decreased or, informally, that a player can halt but never reverse the process of learning. Henceforth, we simplify notation by identifying i with \(\lambda _i\) and j with \(\mu _j\).

Now, we give a precise description of \(\varGamma _{\mathrm {inc}}\) using the language and notation associated with stochastic games in [18]. The state spaceis chosen to index the learning parameters \(\varLambda \times M\). Fix a stage \(t \in T\) and a state \(s = (i, j) \in {\mathcal {S}}\) such that \(\lambda _i\) and \(\mu _j\) are the learning parameters belonging to Player 1 and Player 2 at stage t.

Player 1 and Player 2 (optimally) play the incompetent game \(\varGamma _Q(i, j)\) and are given the option to advance their learning parameters to \(i + 1\) and \(j + 1\), respectively. The decision to increment a learning parameter might incur a state-dependent learning cost \(c^k(i, j)\) to Player \(k \in \{1, 2\}\). Formally, we say that the actions belonging to Player 1 and Player 2 at state s arewhere “0” means “Don’t Learn” and “1” means “Learn”. If Player 1 selects \(a \in {\mathcal {A}}(s)\) and Player 2 selects \(b \in {\mathcal {B}}(s)\), then they receive the stage-t immediate rewardswhere the \(\mathrm {val}(\varGamma _Q(i, j))\) term is the reward received after optimally playing \(\varGamma _Q(i, j)\). Moreover, before the subsequent \((t + 1)\)^th stage, the game transition to the state \((i + a, j + b)\) with (degenerate) transition probabilities given byfor every \(s' \in {\mathcal {S}}\). The general transition structure of this game is shown in Fig. 3.

Here, we will focus on stationary strategies, which are represented as block row vectors \(\mathbf {f} = (\mathbf {f}(s))_{s \in {\mathcal {S}}}\) for Player 1 and \(\mathbf {g} = (\mathbf {g}(s))_{s \in {\mathcal {S}}}\) for Player 2. The block \(\mathbf {f}(s) = (f(s, a))_{a \in {\mathcal {A}}(s)}\) stores the probability f(s, a) of choosing action \(a \in {\mathcal {A}}(s)\) and the block \(\mathbf {g}(s) = (g(s, b))_{b \in {\mathcal {B}}(s)}\) stores the probability of choosing action \(b \in {\mathcal {B}}(s)\). The sets of stationary strategies belonging to Player 1 and Player 2 are denoted by \(\mathbf {F}\) and \(\mathbf {G}\), respectively. The immediate rewards in (20) and the transition probabilities in (21) are extended to \(\mathbf {F} \times \mathbf {G}\) by definingandfor each \((\mathbf {f}, \mathbf {g}) \in \mathbf {F} \times \mathbf {G}\). If the stochastic process \(\{S_t\}_{t = 0}^\infty \) stores the state at each stage \(t \in T\), then it becomes a Markov chain under the dynamics induced by a strategy profile \((\mathbf {f}, \mathbf {g}) \in \mathbf {F} \times \mathbf {G}\). We use \({\mathbb {P}}_{s \mathbf {f} \mathbf {g}}\) and \({\mathbb {E}}_{s \mathbf {f} \mathbf {g}}\) to denote probabilities and expectations under these dynamics with the initial state \(S_0 = s \in {\mathcal {S}}\). The \(\beta \)-discounted value (\(\beta \in [0, 1)\)) of \((\mathbf {f}, \mathbf {g}) \in \mathbf {F} \times \mathbf {G}\) to Player \(k \in \{1, 2\}\) with the initial state \(s \in {\mathcal {S}}\) isThen, \((\mathbf {f}^*, \mathbf {g}^*) \in \mathbf {F} \times \mathbf {G}\) is a (Nash) equilibrium of the incremental learning game \(\varGamma _{\mathrm {inc}}\) wheneverfor all \(s \in {\mathcal {S}}\), \(\mathbf {f} \in \mathbf {F}\), and \(\mathbf {g} \in \mathbf {G}\).

Although \(\varGamma _{\mathrm {inc}}\) unfolds over an infinite time horizon, its transition structure admits a specialised backward induction algorithm for computing equilibria. We construct a suitable notion of “past” and “future” states by finding a sequence \(s_1, s_2, \ldots , s_L\) (where \(L := MN\)) such that \(\ell ' < \ell \) implies \(p(s_{\ell '} | \ell , a, b) = 0\) for all distinct \(\ell , \ell ' = 1, 2, \ldots , L\) and \((a, b) \in {\mathcal {A}}(s_\ell ) \times {\mathcal {B}}(s_\ell )\).

It is straightforward to verify that a suitable ordering exists—for example, the lexicographical ordering. So, we shall assume that an ordering has been fixed and write \(\ell \) instead of \(s_\ell \). Lemma 3 shows that the discounted value of a strategy profile at a specific state does not depend on the “past” states. This allows us to restrict the stochastic game \(\varGamma _{\mathrm {inc}}\) to the limited state space \(\{\ell , \ell + 1, \ldots , L\}\) while still being able to assess the value of strategies.

Next, we will show that a backward induction algorithm can solve this incremental learning game by working backward through the states \(1, 2, \ldots , L\). Fix a state \(\ell \in \{1, 2, \ldots , L - 1\}\) and define \(\mathbf {F}_{\ell + 1} := \{(\mathbf {f}(\ell '))_{\ell ' = \ell + 1}^L\}\) and \(\mathbf {G}_{\ell + 1} := \{(\mathbf {g}(\ell '))_{\ell ' = \ell + 1}^L\}\) to be the sets of stationary strategies over the “future” states \(\ell + 1, \ldots , L\). Assume that we have already found \(\mathbf {f}^*_{\ell + 1} \in \mathbf {F}_{\ell + 1}\) and \(\mathbf {g}^*_{\ell + 1} \in \mathbf {G}_{\ell + 1}\) solving (25) at each state \(\ell ' = \ell + 1, \ldots , L\). Can we extend this equilibrium to include the current state? Theorem 5 shows that it is sufficient to consider a simplified version of the equilibrium inequalities that only account for unilateral deviations at state \(\ell \). The sets \(\mathbf {F}_\ell (\mathbf {f}_{\ell + 1}^*) := \{(\mathbf {f}(\ell ), \mathbf {f}^*(\ell + 1), \ldots , \mathbf {f}^*(L))\}\) and \(\mathbf {G}_\ell (\mathbf {f}^*_{\ell + 1}) := \{(\mathbf {g}(\ell ), \mathbf {g}^*(\ell + 1), \ldots , \mathbf {g}^*(L))\}\) denote the spaces of stationary strategies that extend \(\mathbf {f}^*_{\ell + 1}\) and \(\mathbf {g}^*_{\ell + 1}\) to state \(\ell \).

The useful consequence of Theorem 5 is that, by solving a “local” problem at “previous” state \(\ell \), we can extend the equilibrium \((\mathbf {f}^*_{\ell + 1}, \mathbf {g}^*_{\ell + 1})\) to create \((\mathbf {f}^*_\ell , \mathbf {g}^*_\ell )\). This local problem resembles a repeated game with absorbing states. Namely, if the players both choose to forego learning, then the game remains at state \(\ell \). Otherwise, if either of the players choose to learn, then the game transitions into a new state where the expected future rewards are fixed by \((\mathbf {f}^*_{\ell + 1}, \mathbf {g}^*_{\ell + 1})\). The rewards given to Player \(k \in \{1, 2\}\) in this repeated game with absorbing states arefor each \((a, b) \in {\mathcal {A}}(\ell ) \times {\mathcal {B}}(\ell )\). An immediate consequence of Lemma 3 is that, for each \(k \in \{1, 2\}\) and \((\mathbf {f}_\ell , \mathbf {g}_\ell ) \in \mathbf {F}_\ell (\mathbf {f}^*_{\ell + 1}) \times \mathbf {G}_\ell (\mathbf {g}^*_{\ell + 1})\), we havewhere \(p_0 = 1 - p_1 = f(\ell , 0)\) and \(q_0 = 1 - q_1 = g(\ell , 0)\). Hence, to ensure that \((\mathbf {f}^*_\ell , \mathbf {g}^*_\ell ) \in \mathbf {F}_\ell (\mathbf {f}^*_{\ell + 1}) \times \mathbf {G}_\ell (\mathbf {g}^*_{\ell + 1})\) satisfies the inequalities in (27), we need to solve the coupled pair of maximisation problemswhere \(p^*_0 = 1 - p^*_0 = f^*(\ell , 0)\) and \(q^*_0 = 1 - q^*_1 = g^*(\ell , 0)\). Under the additional assumption that this repeated game with absorbing states is non-degenerate, the solutions are either both pure strategies (\(p^*_0, q^*_0 \in \{0, 1\}\)) or both completely mixed strategies (\(p^*_0, q^*_0 \in (0, 1)\)). The pure strategy solutions can be found by imposing restriction \(p_0, p^*_0, q_0, q^*_0 \in \{0, 1\}\) in (30); that is, by comparing the payoffs of every possible pure strategy profile. Moreover, by setting the appropriate partial derivatives (with respect to \(p_0\) and \(q_0\), respectively)

of the functions being maximised in (30) equal to zero, we obtainThe solutions to (31) with \(p^*_0, q^*_0 \in (0, 1)\) give the completely mixed strategy solutions to \(\varGamma _{\mathrm {inc}}\). This shows that we are always able to extend \((\mathbf {f}^*_{\ell + 1}, \mathbf {g}^*_{\ell + 1})\) to an equilibrium \((\mathbf {f}^*_{\ell }, \mathbf {g}^*_{\ell })\) of \(\varGamma _{\mathrm {inc}}\) restricted to \(\ell , \ldots , L\). Hence, since \((\mathbf {f}^*_L, \mathbf {g}^*_L)\) where \(f^*(L, 0) = g^*(L, 0) = 1\) is the only strategy profile available at state L, we can work backwards through the states \(L-1, L-2, \ldots , 2,1\) and repeatedly extend it until obtaining an equilibrium \((\mathbf {f}^*, \mathbf {g}^*)\) of \(\varGamma _{\mathrm {inc}}\).

Example (attack-defence game with incremental learning) Lastly, recalling the attack-defence game \(\varGamma _Q(\lambda , \mu )\) previously introduced in Sect. 2.2, suppose that the attacking and defending pilots have the option to undergo navigation training between engagements. We might model this as an incremental learning game \(\varGamma _{\mathrm {inc}}\) in which training allows the pilots to advance their skill parameters throughafter paying learning costs of \(c^1(i, j) = c^2(i, j) = \nicefrac {1}{10}\) at state \(s = (i, j) \in {\mathcal {S}}\). Moreover, assume that the pilots have far-sighted discounted strategy valuations with a discount factor of \(\beta = \nicefrac {99}{100}\). What are the best strategies to reduce incompetence throughout this game?

The aforementioned backward induction algorithm produces a unique equilibrium of \(\varGamma _{\mathrm {inc}}\) shown graphically in Fig. 4. A node indicates a pair of learning parameters and an arc indicates a transition realised by the equilibrium. So, a vertical arrow means that only Player 1 learns, a horizontal arrow means that only Player 2 learns, a diagonal arrow means that both players learn, and a loop means that neither player learns.

Note that, under the equilibrium shown in Fig. 4, the attacker learns until their skill reaches the interval \([\nicefrac {11}{47}, 1]\) and the defender learns until they reach the interval \([\nicefrac {26}{47}, 1]\). We know that the underlying parametrised incompetent game \(\varGamma _Q(\lambda , \mu )\) is completely mixed on \((\nicefrac {11}{47}, 1] \times (\nicefrac {26}{47}, 1]\). So, by the observations in Sect. 2.4, both players are able to execute completely competent optimal strategies when \((\lambda , \mu ) \in [\nicefrac {11}{47}, 1] \times [\nicefrac {26}{47}, 1]\). This means that, once the players have achieved learning parameters within these intervals, the game value plateaus (see Fig. .1) and there is no incentive to learn further. Therefore, it is not always necessary to achieve complete competence so long as the players are able to “mimic” competence by executing an optimal strategy from the completely competent game \(\varGamma \).

Naturally, game assumptions in biological settings differ from the classic games since rationality of each individual behaviour might not always be natural to assume. Consider a population of species consisting of N individual organisms. At every time step, individuals interact in a pair-wise manner, where they have to choose one action out of n distinct available actions. Outcomes of these interactions determine fitness of individuals based on the fitness matrix \(R\in {\mathbb {R}}^{n \times n}\). In the evolutionary settings, all individuals in the population obtain the same fitness matrix R, however, during the interaction, Player 1’s fitness is determined by R, while Player 2’s fitness is determined by \(R^T\). Furthermore, the sets of selectable and executable actions coincide for all players. Let \({\mathbf {x}}=(x_1,\ldots ,x_n)\), where \(x_i\) denotes the frequency of the (pure) strategy i. We assume that in a given population, all individuals have the same set of selectable actions \({\mathcal {A}}\), fitness matrix R, and the mixed strategy of the entire population \({\mathbf {x}}\).

The main focus of evolutionary games is to predict the strategy \({\mathbf {x}}\) that will be adopted by the population. Since we assume that n actions are available to each individual, the resulting mixed strategies lie in the simplex \(\varDelta _n\) defined bywhere \(x_i = \frac{N_i}{N}\) with \(N_i\) being a number of individuals adopting strategy i and N being a total number of individuals in the population. Then, an evolutionary game \( \varGamma ^e\) can be denoted byWe say that the population adopts a pure \(i^{\text {th}}\) strategy if all individuals are behaving as the \(i^{\text {th}}\) type and, hence, their behavioural frequency vector is the unit basis vector \({\mathbf {e}}_i\). However, this may not always be the case. If not, we are in the case of mixed strategies \({\mathbf {x}}\), and hence we are interested in finding a mixture \({\mathbf {x}}^*\) which is a stable outcome of the evolution.

It was shown, that the concept of Nash equilibria is not sufficient when taking into account the evolution of populations [67]. As a result, a new equilibrium concept was proposed. The evolutionary stable strategy (ESS) ensures that population’s strategy is resistant against random mutations and is defined, more precisely, below.

Here, \({\mathbf {x}}^*R({\mathbf {x}}^{*})^T\) measures the frequency-dependent fitness of the entire population, given that everyone adopts strategy \({\mathbf {x}}\), whereas \({\mathbf {y}}R({\mathbf {x}}^{*})^T\) measure fitness of a population adopting strategy \({\mathbf {y}}\) in a population of individuals using strategy \({\mathbf {x}}\). In the long run, an ESS guarantees that selection prefers \({\mathbf {x}}^*\) to any other arising strategy. Note that the ESS is a special case of a Nash equilibrium [56].

However, besides equilibria, we are usually interested in how these equilibria can be reached, bringing us to the concept of evolutionary dynamics. Given that biological populations not only interact, but also reproduce, there is a need to take into account the reproduction process. The first classic evolutionary dynamics model was proposed by Taylor and Jonker in [74], and is called replicator dynamics. These dynamics assume well-mixed infinitely large populations which is, of course, a simplification. Subsequently, many new concepts of dynamics were suggested in order to capture mutations [12, 57, 70, 72], finite size of populations and stochasticity [31, 53, 73, 76‐78], adaptation [13, 14, 17, 25, 58], and a population structure [5, 49, 59, 60]. However, to date, the concept of incompetence was only considered in a classic setting of replicator dynamics. In Conclusions section, we discuss possible extensions to other forms of dynamics for incompetent games.

Replicator dynamics captures a frequency-dependent selection, where the evolution of population’s strategy depends on the current frequencies of all strategies in the population. That is, the fitness of a particular strategy is compared to the mean fitness of the entire population and is determined by the adopted strategies. With respect to a mixed strategy \({\mathbf {x}} \in \varDelta _n\), the expected fitness of a (pure) strategy i is defined byThe mean fitness payoff of the population is then defined by the scalarThen, the dynamics of strategy i’s frequency in the population is defined byor in a matrix form,In the folk theorem of evolutionary game theory, it was shown that any equilibrium of the replicator dynamics is a Nash equilibrium of the game \(\varGamma ^e\) and that a strict Nash equilibrium is asymptotically stable [28]. Moreover, any ESS is an asymptotically stable equilibrium of the replicator dynamics. Hence, when considering evolutionary games, it is frequently sufficient to find equilibria of a static game \(\varGamma ^e\). This simplification is useful when trying to predict how the behaviour of the game changes under the assumption that interacting individuals are incompetent. We shall next consider how incompetence changes the game setup.

When introducing an assumption that individuals are prone to making behavioural mistakes in an evolutionary game, one can interpret such mistakes as a form of behavioural plasticity. In some ways, this can be seen as phenotypic plasticity (for instance, in microbes). However, in application to more sophisticated organisms, behavioural plasticity need not relate to genetic background of the organism. These behavioural mistakes can be driven by migration to a new environment or any other form of environmental change and are reflected in the incompetence matrix Q analogous to that introduced in Sect.  2.2.

Since we assume that the entire population obtains only one fitness matrix, we also assume that the incompetence matrix is given for the entire population. Then, a new incompetent fitness matrix is determined in a similar manner to (7) asIn line with previous sections, we assume that players’ ability for improving their strategy execution is determined by some parameter. Since here we consider one population of players all of whom obtain the same measure of incompetence, we only need one incompetence parameter \(\lambda \in [0,1]\). Then, the incompetent fitness matrix is defined asThroughout this section, we make a specific assumption on the functional form of learning. We assume that \(Q(\lambda )\) is linear and defined aswhere S is the staring level of incompetence and I is the identity matrix. When \(\lambda =1\), the population does not make any execution errors and has a perfect strategy execution. Now we can define the evolutionary incompetent game asWe can further simplify the analysis by utilising the property of replicator dynamics that it is invariant under a linear positive transformation [27]. This allows us to reduce the fitness matrix by subtracting diagonal elements of R from the corresponding columns. Mathematically speaking, such transformation can be defined aswhere \({\mathbf {d}}_{R}\) is a vector consisting of the diagonal elements of R and \(\mathbf {1}_n\) is a vector consisting of ones. Throughout the manuscript, we shall denote any matrix \({\tilde{R}}\) as a canonical form of matrix R as in (40). Then, according to (33)-(35), for a new game under incompetence \(\varGamma _Q^e\), we re-write the expected fitness for strategy i asand for the mean fitness payoff of the population,Hence, the incompetent replicator dynamics can be written asIn a strict sense, the new system given by (43) is a perturbed evolutionary game, and perturbations depend on the parameter \(\lambda \). As \(\lambda \) tends to 1 for all i, the game under incompetence approaches the original game given by R. In the following section, we summarise the main results obtained for incompetent evolutionary games.

Here, we are mostly interested in behaviours that dynamics exhibit under changes in parameter values \(\lambda \) given the starting level of incompetence S and the fitness matrix R. These behaviours may arise for different values of \(\lambda \) and the dynamics change their behaviour at critical levels \(\lambda ^c\) of \(\lambda \), referred to as bifurcation points, where equilibria emerge, disappear or change their stability properties.

Under incompetence, behaviour of the game dynamics may exhibit several bifurcations [37]. Since by design the incompetence parameter approaches 1 when incompetence decreases, the incompetent fitness matrix \(R(\lambda )\) is approaching the original fitness matrix R. As a result, in the limit of perfect competence, behaviour of the incompetent game approaches the behaviour of the original game. That is, there exists a maximal critical value of \(\lambda \), that preserves robust properties of the game. We recall this result in the following theorem.

A natural question arises of how these bifurcation values of the incompetence parameter can be determined and the behaviour of dynamics. The larger the game (the more available strategies it has), the harder it becomes to define all possible bifurcations. However, even for an arbitrary number of strategies, we can find bifurcations of special equilibria, such as, interior equilibria or pure-strategies equilibria using analysis presented in [11]. Let us first focus on the bifurcations of the interior equilibria.

Then, the point of bifurcation for an interior equilibrium can be found by considering a determinant of the incompetent fitness matrix.

Next, we recall the special canonical form of the matrix \({\tilde{R}}(\lambda )\) that is defined through a rank-one transformation of an incompetent fitness matrix \(R(\lambda )\). By [24], its determinant can be written asHence, critical values of the incompetence parameter can be found by finding zeroes of either \(\det (Q(\lambda )),\) or \([1-\mathbf {1}_n^TR(\lambda )^{-1}{\mathbf {d}}_{R(\lambda )}]\).

In a special case of a rock-paper-scissors game [40], stability of the interior equilibrium is determined by the sign of the determinant of the fitness matrix [80], which gives rise to three cases: (a) if \(\det (R)<0\), then an unstable interior equilibrium exists resulting in a heteroclinic cycle; (b) if \(\det (R)>0\), then such an equilibrium is a stable mixed equilibrium; (c) if \(\det (R)=0\), then there exists a centre and periodic orbits around it.

However, games \({\tilde{R}}(\lambda )\) and \(R(\lambda )\) exhibit the same behaviour [27]. Since the determinant of the fitness matrix \(R(\lambda )\) always preserves the same sign as \(\det (R)\), then \(\det ({\tilde{R}}(\lambda ))\) also cannot change its sign while the interior equilibrium exists.

Deriving a general form of equilibria depending on \(\lambda \) is complex and depends on the form of matrices R and S. For a special case of uniform incompetence, which implies that everybody makes mistakes with the same probability \(\nicefrac {1}{n}\), we can sometimes find a closed-form expression for the interior equilibrium. A uniform incompetence can be interpreted as a form of plasticity in biological populations. For instance, phenotypic plasticity, when different types might have slight variations in the exact degree of each gene expression. We provide this result in the following theorem.

In [11], it was shown that pure-strategy equilibrium’s stability properties can be determined from the sign of the j-th column of matrix \({\tilde{R}}(\lambda )\). Hence, given the maximal level of incompetence, we can determine which of the vertices will be stable and when this stability will change.

This result can be generalised for any level of incompetence, where we will have to considerfor all levels of \(\lambda \), where \({\mathbf {q}}_i\) is the \(i^{\text {th}}\) row of \(Q(\lambda )\). Generally speaking, this condition implies that for a pure strategy to be stable under incompetence, it is necessary for it to be the best response to itself given all other pure strategies.

As in [38], in Fig. 5, we provide some cases illustrating how equilibria stability can change as competence level of the population changes, for an example of an unstable rock-paper-scissors game. In every panel, the colour-coded bar at the top indicates which of the three vertices is stable, while in the main plot we depict the interior equilibrium components as functions of \(\lambda \). Even in this simple game, the behaviour exhibited by the replicator dynamics in response to changing \(\lambda \) can be very rich. As shown in these three examples, the interior equilibrium may or may not exist for different values of the incompetence parameter. Similarly, one, two, three or none of the vertices may be stable at the same time.

As in the case of classical games, in evolutionary settings it is natural to consider decreasing levels of incompetence, a process we called learning. Note that increasing \(\lambda \) corresponds to greater skill level and decreasing incompetence.

In the evolutionary games setup, dynamic incompetence was interpreted from two different perspectives: as an environmental shift that requires adaptation from organisms before the stable equilibrium is reached and as a learning process designed to maximise the fitness of the population after the population stabilised at some equilibrium.

Behaviour exhibited by the population dynamics when the process of learning is treated as a function of time, \(\lambda (t)\), was considered in [36, 38]. There are many options possible when choosing the functional form. So far, two functional forms of \(\lambda (t)\) were analysed: a sigmoid and a periodic function. The sigmoid form of learning implies that organisms are capable of learning faster in the beginning of the process and slower when they reach sufficient competence. An assumption of a slowing rate for high enough levels of \(\lambda \) is motivated by the absence of necessity to learn fast since the evolutionary stable outcome can already be reached (see Theorem 6).

Analysing parameters of \(\lambda (t)\) one can determine how long will it take for species to be fully recovered in behavioural sense and act as in the environment they are familiar with. However, while the functional form of the adaptation trajectory captures the pace and steepness of the learning process, the starting level of incompetence can be seen as a measure of the magnitude of the changes in the environment. That is, the further the new habitat is from the previous one, the longer it may take for organisms to fully recover.

Since natural habitats are prone to some form of regular stochasticity, in [36], it was also considered how periodic environmental fluctuations due to the seasonal or daily changes affect the evolutionary dynamics. It appeared that periodicity of environmental changes leads to periodic behaviour in the evolutionary dynamics as well. Specifically, if the original game possesses a stable equilibrium, then the solution of the incompetent game with periodic form of incompetence will converge to a stable periodic orbit around this stable equilibrium.

Let us now demonstrate how the concept of incompetence can be applied to a more specific biological setting. In the next section, we formulate a game of two foraging strategies of marine bacteria and try to analyse it from the perspective of incompetence.

Evolutionary game theory has been widely applied to studying the evolution of microbes. Despite their primitivism and small sizes, marine bacteria are among the most ubiquitous forms of marine organisms, playing a central role in governing health of marine ecosystems and regulating global biosphere [52]. Understanding how cells make decisions and interact has implications for both biology of bacterial communities and our exploitation of these communities [1, 20, 21, 44, 46, 48]. Fundamental to nutrient competition among bacteria is the choice of motility (chemotactic) strategies. Chemotaxis—the ability to sense environmental signals, and react to the stimuli accordingly—has been studied since the late 1800s [16, 61].

However, a deterministic game theoretic approach misses an essential feature of the bacterial population dynamics: these populations and their interactions are highly stochastic. For instance, stochastic environmental fluctuations often affect ecological systems [21]. In order to at least partially allow for this, the concept of incompetence was applied to study foraging strategies of bacteria in [36] by incorporating behavioural stochasticity in a matrix game that captures interactions between different strategic types of microbes. The aim is to identify the most efficient strategy for given environmental conditions. We consider two possible strategies: nonmotile or chemotactic. Nonmotile bacteria cannot induce active swimming and only drift with the water flow, whereas chemotaxis allows for active choice of direction. The fitness matrix can be constructed aswhere c is the cost of swimming and m is the reward for being able to efficiently determine the direction of swimming and both parameters are normalised so that \(c,m \in [0,1]\). Depending on the exact values of the parameters, the game might exhibit four different behaviours, as proposed in [82] in relation to the signs of matrix elements in a canonical form from (40): We shall focus on two cases: when chemotactic strategy dominates nonmotile strategy and when a stable mixed equilibrium exists. The mixed equilibrium is given byWhen introducing incompetence in a model, one has to take into account biological limitations of the strategies. For instance, there exist no conditions under which a nonmotile bacterium can exhibit chemotaxis because it lacks receptors and flagella required for such a strategy. However, a chemotactic bacterium can be both nonmotile and chemotactic. Hence, the starting incompetence matrix S for this example, may have the following formThen, the resulting induced incompetent fitness matrix \({\tilde{R}}(\lambda )\) is given byNote that the relative fitness of chemotactic strategy is not affected by incompetence. However, the advantage of nonmotile strategy depends on the level of incompetence induced by chemotactic bacteria. By using Lemma 4, we can determine the critical value of the incompetence parameter, which is determined as the solution of \(\det ({\tilde{R}}(\lambda ))=0\) or \({\tilde{r}}_{21}(\lambda ^c)=0\), and given byDepending on the stability properties of the dynamics in the original game, the behaviour of equilibria under incompetence will differ. For instance, if chemotactic strategy was dominating in a fully competent game, then for \(\lambda <\lambda ^c\) both strategies will stably co-exist. If a stable mixed equilibrium existed, then for \(\lambda <\lambda ^c\) chemotactic strategy will dominate nonmotile strategy.

Additionally, turbulence affects life of marine bacteria [71]. Mathematically, this can be modelled via a stochastic adaptation process. Construct a stochastic learning process where each point \(\lambda (t)\) is a random variable with distribution that is determined by the species’ migration process. This assumption provides a more realistic interpretation of the species’ behaviour when we take into account migration and environmental stochasticity. However, games with an ESS are well-known to be robust [11].

We shall compare population dynamics for two types of the learning processes: deterministic sigmoid learning and a stochastic migration process. Any learning process, either deterministic or stochastic, will lead to the ESS in terms of a species’ choice (Fig. 6). Observations of the real behaviour depend on the incompetence matrix. Population’s behavioural observations for different learning processes will be different depending on the learning dynamics.

The stochastic learning brings us to a situation where the majority of bacteria in a population are able to perform chemotaxis if chemotactic strategy was dominating (Fig. 6, left), as in the deterministic case. However, if there exists a stable mixed equilibrium, then dynamics under stochastic incompetence converges to the stable frequency of chemotactic bacteria as well (Fig. 6, right). This is yet different from the equilibrium of the game without incompetence.

Due to incompetence, extinct strategies may still reappear in the behaviour of individuals as a manifestation of mistakes that cause a revival of the extinct types. This randomisation may become beneficial as a changing environment may require flexibility from individuals in their adaptation.

Even if the adaptive peak has been reached (i.e. \(\lambda =1\)), behavioural randomisation may become essential in preparedness to unforeseen changes. This is supported by the existing research in stochastic phenotype switching, when bacteria perform behavioural stochasticity even in stable environments [30].

When considering incompetent games, the main focus of the analysis is on where the dynamics will stabilise and whether a stable equilibrium will be reached. However, what if the change in the environment happened after the stable equilibrium was already reached? Is there an optimal way to re-learn effective strategies that is least costly in terms of fitness losses? We discuss results answering this question in the next section.

When allowing for learning after the stable equilibrium is reached, the focus of the analysis is the population’s need to re-learn its effective strategies in an optimal manner. Hence, in [39] the learning under incompetence was considered with respect to maximising the fitness over the learning path.

When addressing learning, one needs to distinguish whether the entire population is learning with the rate \(\lambda \) or whether each strategy has its own learning rate \(\lambda _i \in [0,1]\). This decision depends on the specific situation under consideration. For this section, let us assume a more general case with \(\varvec{\lambda }=(\lambda _1,\ldots ,\lambda _n)\) to define an evolutionary game under incompetence. Then, a performance measure of the learning path over fitness can be thought of aswhere C is a learning path that can be taken and \(\phi _C(\varvec{\lambda })\) is the mean-fitness of the population. Note that here it is explicitly assumed that every strategy i has its own incompetence parameter \(\lambda _i\), which implies that \(\varvec{\lambda }=(\lambda _1,\ldots ,\lambda _n)\). Since the complexity of the problem grows with the number of strategies, this model was considered in its simplest possible setup: when only two strategies compete. That is, the fitness matrix R has the canonical formand the starting level of incompetence S can be denoted asIf the initial game has one stable pure equilibrium, then the optimal learning will simply imply reduction of frequency of execution of the unwanted strategy. However, if the game possesses a mixed stable equilibrium, it is no longer obvious what the learning path should look like. Note that an interior equilibrium in a 2-strategy game has the following form \(\hat{{\mathbf {p}}}=({\hat{a}},{\hat{b}})\), whereIn order to maximise the fitness of the population, it is sufficient to consider the mean fitness function [74], which has the following formwhich under incompetence is reduced to the analysis of two parametersAn important aspect stemming from this model is the understanding of fitness and learning advantages. Given that relative fitness of each strategy is positive, that is, \(a,b>0\), we say that the strategy with higher relative fitness obtains a fitness advantage.

In addition, we say that strategies may obtain a learning advantage. This concept is induced by incompetence and implies that the strategy with more variability in the behaviour, that is, with a higher probability of mistakes, has a higher potential fitness advantage which might be achieved by reducing incompetence. Hence, the lower \(\eta \) or \(\gamma \), the greater the learning advantage. A new parameter \(\delta :={\tilde{a}}-{\tilde{b}}\) was defined to measure the relative strategic advantage of one strategy over another. We summarise and compare these concepts in Table 1.

Interestingly, the sign of \(\delta \) fully determines which strategy has to be learnt first. That is, it cannot be determined separately based on either fitness or learning advantages of strategies. The naive suggestion would be that the most advantageous skill in terms of fitness has to be learnt first. However, the strategy with lower relative strategic advantage is learnt first in the optimal learning path.

We suggest that natural selection tries to compensate the most disrupted strategy first even if its fitness is not the highest. Nonetheless, if the fitness difference is high enough to overcome the effect of incompetence, then the optimal learning will demand that the better strategy is learned first. Another possible interpretation would be to consider the mixed equilibrium as mixed strategies used by players. Then, by learning the less-advantageous strategy first, individuals are reaching the nearest optimal mixed strategy.

In the next section we shall demonstrate results from three previous sections on a reduced 2-strategy game based on the foraging strategies of marine bacteria as presented in [36].

Let us now assume that the population has stabilised at the mixed equilibrium defined in (47). Assume that the environmental conditions have changed leading to deviations in strategy executions for both bacterial strategies. For this, assume that the new starting incompetence matrix is defined asSince nonmotile bacteria can exhibit chemotactic behaviour only as a random noise, it is natural to assume that \(\epsilon _1\le \epsilon _2\). Furthermore, let us allow for each strategy to be learnt at a different pace as in Sect. 3.5. In order to determine the optimal path that maximises the fitness over learning, we first calculate advantages of nonmotile and chemotactic strategies from (49) asThen, the strategic advantage of nonmotile strategy over chemotactic strategy equals toNote that if \(\epsilon _1=\epsilon _2=\epsilon \), then \(\delta =\nicefrac {1}{\epsilon }>0\) and chemotactic strategy has to be learnt first in one step (see Fig. 7 (left)). Generally, chemotactic strategy has to be learnt first whenever \(\delta >0\) or equivalentlywhich together with the condition \(\epsilon _1<\epsilon _2\) requires that \(m < 2c - \frac{1}{2}\). We plot a special case when \(\delta =0\) in Fig. 7 (right).

Currently, in the setting of classical nooncooperative game theory, incompetence has been studied mainly in matrix and bimatrix games. However, there are clearly several other, more general, classes of games to which this approach could be extended. Below, we name just four, out of many possible, generalisations.

It should be clear that the work done so far in studying incompetent evolutionary games constitutes merely a beginning. As above, in this section we briefly describe just three, out of many, possible continuations of this research.

In particular, the effect of the finite population size and inherent stochasticity of the reproduction process were addressed in finite population dynamics like Moran birth-death process, ability of others to imitate successful behavioural aspects of neighbours were addressed in imitation dynamics and the effect of interaction with neighbours was addressed in many different dynamics on networks. Hence, as a natural extension, one should consider how relaxed assumptions on the population dynamics affect the dynamics of games with incompetence.

b) Generalising prioritised learning by exploiting the power of simulations. In recent years, evolutionary models adopted methods of computer simulations to facilitate exploration of more realistic complex models that would have been intractable using analytical methods. Hence, one could extend our prioritised learning of Sect. 3.5 to allow more than two strategies to compete at the same time. Furthermore, it is tempting to allow every individual organism their own learning parameter so as to closer approximate natural scenarios.

While the complexity of such a complex model will render it intractable analytically, simulating specific setups may shed light on many puzzling biological problems. For instance, the problem of determining how niches emerge and are filled by organisms while interacting under many different environmental conditions with multiple organisms.

c) Learning as a function of frequency of strategies. One simplifying assumption we made so far in all models of incompetence applied in biology is that of the separation of the learning process from the reproduction process. However, evolution of learning or levels of incompetence might be frequency dependent, which could lead to intricate co-evolution. Setups that follow similar logic were considered in [81] and [75]. While results presented in [75] can be seen as more general, the form of the exact dynamics of the learning process in the settings of incompetence was not addressed in previous works. Hence, we believe it would be worthwhile to consider co-dependence of \({\mathbf {x}}(\lambda )\) and \(\lambda (t,{\mathbf {x}})\).