Open Access 07122021
Markov Perfect Equilibria in MultiMode Differential Games with Endogenous Timing of Mode Transitions
Published in: Dynamic Games and Applications  Issue 2/2022
Abstract
We study Markov perfect equilibria (MPE) of twoplayer multimode differential games with controlled state dynamics, where one player controls the transition between modes. Different types of MPE are characterized distinguishing between delay equilibria, inducing for some initial conditions mode switches after a positive finite delay, and now or never equilibria, under which, depending on the initial condition, a mode switch occurs immediately or never. These results are applied to analyze the MPE of a game capturing the dynamic interaction between two incumbent firms among which one has to decide when to extend its product range by introducing a new product. The market appeal of the new product can be influenced over time by both firms through costly investments. Under a wide range of market introduction costs a now or never equilibrium coexists with a continuum of delay equilibria, each of them inducing a different time of product introduction.
1 Introduction
The main agenda of this paper are to improve our understanding of strategic effects arising in dynamic economic and managerial settings characterized by potential structural breaks, which induce jumps in the payoff functions of the economic actors, in the law governing the dynamics of relevant state variables, or both. Areas of application where such structural breaks, which we will refer to as mode changes, arise include environmental economics dealing with potential catastrophic transitions (e.g., [8]), financial portfolio and real investment problems incorporating potential crashes and macroeconomic regime shifts (e.g., [7, 13]) or analyses of innovation dynamics capturing abrupt changes in the market structure due to the adoption of new technologies or the introduction of new products by some market participants ([1, 3, 4]). Whereas all these different problems can be formulated as multimode models, crucial differences arise with respect to the way mode transitions are triggered. In particular, the timing and the type of mode transition might be deterministic or stochastic and might be purely exogenous or directly, respectively, indirectly (through the state dynamics) controlled by some economic agent(s).
In this paper, we focus on multimode settings with deterministic, controlled mode transitions and strategic interaction. In particular, we are interested in characterizing Markov perfect equilibria (MPE) in multimode differential games with a finite number of continuously evolving states, the dynamics of which are controlled by the actions of all players, and a set of modes, where the time of the transition between the modes is determined by one of the players; for easier exposition we assume this is player 1. Problems of this kind arise, for example, in dynamic competition models, like capital accumulation games or dynamic models of reputation formation, where one of the competitors through the introduction of new products or technologies to the market can change the demand structure. Whereas the literature on MPE in timing games with stochastic uncontrolled state dynamics is large and well established (see, e.g., [9, 10, 17]), there is considerably less work dealing with situations with controlled mode transitions where the players can also influence the state dynamics. Intuitively, characterizing MPE in such a setting generates intricate strategic effects. The Markovian strategy determining the time of the mode switch induces a split of the state space in a given mode in regions with and without an immediate jump to an alternative mode.^{1} In a setting where all players through their controls can influence the state dynamics, this implies that each player might influence the timing of the mode switch and this has to be taken into account when determining the optimal strategies. At the same time, player 1, when determining the optimal mode switching strategy, has to take into account the state dynamics under the equilibrium strategies of all players.
Advertisement
Few contributions have addressed these issues. Reddy et al. [15] and Gromov and Gromova [6] study multimode games with controlled state dynamics, where, however, the mode transition is not directly controlled by players but determined by state or time constraints. Hence, players can influence the timing of the mode transition only indirectly by steering the state variable through their control. In contrast to these papers, we consider a scenario where the timing of the mode transition is directly controlled by one of the two players and additionally both players can influence the dynamics of a continuously evolving state variable. Closest to our paper is Long et al. [14], where in a differential game model with multiple regimes, the concept of piecewiseclosedloop Nash equilibria (PCNE) is introduced. They consider a twoplayer multimode differential where both players can induce a change of the regime of the game and study piecewiseclosedloop Nash equilibria (PCNE). Under this equilibrium concept the state at which a player carries out a mode switch is derived from the condition, that it is optimal for the corresponding player to switch at that point, and the timing of the mode switches is determined as the point in time when the state variable under the equilibrium controls arrives at that switching state. However, in their setting, it is assumed that firms commit to their switching time in the sense, that they do not alter that time even if the other firm would deviate from its equilibrium control path and hence also the state variable would deviate from its equilibrium path. Therefore, the considered equilibrium is not fully Markov perfect with respect to the timing decision. Also in Dawid and Gezer [2], a multimode differential game with controlled state dynamics and mode switches is considered, but again the strategic interactions arising under Markovian strategies are not fully captured. In particular, the authors consider a strategy space where the players’ controls influencing the state dynamics are determined by Markovian strategies, whereas the time of the regime switch is determined with full commitment at time zero and hence can be seen as following an openloop strategy. To our best knowledge so far, no characterizations of full MPE in a setting similar to that considered in this paper are available in the literature.
The innovation of this paper relative to the literature discussed above is that we characterize a (simultaneous move) Nash equilibrium of multimode differential games where both the controls of the players steering the state variable and the decision, whether a mode switch occurs at given point in time, are determined by Markovian strategies and therefore depend solely on the current value of the state and the mode. Put differently, in contrast to the existing literature, we do not assume any kind of commitment of player 1 with respect to the timing of a mode switch. Since we consider an infinite horizon problem, where objective functions and state dynamics do not explicitly depend on time, these equilibrium profiles are also Markov Perfect. The equilibrium strategy profiles therefore correspond to Nash equilibrium behavior for any combination of values of the state and mode and this in particular implies, that, whenever player 1 triggers a mode switch, this change is (intertemporally) optimal for player 1 in light of the current state.
In light of the complexity associated with the characterization of the interplay of Markovian strategies determining controls that steer the state variable and such that determine the timing of mode switches, in this paper we restrict attention to a relative simple setting with two players, two modes and a onedimensional state space in order to obtain first insights into the structure of MPE in multimode games with controlled states and mode switches. This is done mainly for reasons of tractability and easier exposition. Moreover, we believe that the main strategic effects occurring in such a setting and the types of equilibria arising can already be seen in such a relatively simple setup. We derive a set of sufficient conditions for a strategy profile to be a MPE of the game and, based on these conditions, identify different possible types of equilibria. There are delay equilibria, with the property that for some initial conditions the mode switch occurs after some finite delay, and now or never equilibria, under which the state space is partitioned in two areas with the property that for initial conditions in one area the game stays in the first mode forever, whereas for initial conditions in the other area player 1 already at \(t=0\) induces the switch to the second mode. Furthermore, we show that among the delay equilibria only for a special class of MPE, labeled as maximum delay equilibria, a standard smooth pasting condition holds for player 1. Such equilibria always exist if the game has any delay equilibria, but generically in addition to a maximum delay equilibrium there also exists a continuum of forced switch delay equilibria in which the mode transition is triggered earlier compared to the maximum delay equilibrium. For these equilibria, no smooth pasting conditions hold at the mode switching threshold. Intuitively, the mode switch by player 1 is triggered by the fact that, if the state would cross the switching threshold, while the game is in the initial mode, the action of player 2 would jump to an action with strong adversarial effects for player 1. Since under such an equilibrium strategy profile the game never stays for a positive amount of time at a state above the threshold in the first mode, any action of player 2 in this part of the state space and the first mode can be supported in a MPE. These arguments show that in the considered settings the fact that a strategy profile constitutes a Markov perfect equilibrium does not prevent the occurrence of what could be described as ‘incredible threats’ in the sense that a player’s strategy prescribes an action which would be suboptimal for the player to implement if forced to do so for a time interval with positive measure.
We illustrate these findings by studying the optimal timing of new product introduction for a producer (firm 1) facing on an established market a competitor (firm 2), which has not developed a new product yet and therefore does not have the option of a new product introduction. The new product introduction corresponds to a mode change, and the state variable of the considered differential game is the appeal of the new product with consumers, which is influenced by the two firms through advertising (firm 1) and negative campaigning (firm 2).^{2} Furthermore, the market introduction of the new product is associated with lumpsum costs. We show that for a wide range of values of the market introduction costs all different types of equilibria mentioned above coexist, and therefore, for a substantial subset of initial values of the new product’s market appeal no clear cut prediction about the time of market introduction of the new product is possible. Furthermore, we determine a threshold such that for values of the market introduction costs above this value only now or never equilibria exist. Overall, this analysis highlights that multimode timing games of this kind almost generically give rise to multiplicity of Markov perfect equilibria. A potential implication of this insight is that more refined equilibrium concepts than MPE should be considered to analyze such games.
Advertisement
The remainder of the paper is organized as follows. In Sect. 2, we introduce the type of multimode games we are considering, and in Sect. 3, we derive necessary conditions for MPE in such a setting and also some additional results on equilibrium profiles. In Sect. 4, we study the optimal timing of new product introduction, thereby illustrating our general findings. A discussion of our results and conclusions is provided in Sect. 5. ‘Appendix’ contains all proofs and some additional analysis.
2 The Model
We consider a differential game between two players \(i=1,2\), in which each player intends to maximize an infinite horizon discounted payoff stream of the formwhere \(x \in {{\mathbb {R}}}\) is a onedimensional state and \(m(t) \in \{m_1,m_2\}\) is the mode of the game. The interval \(X=[x_l,x_u]\) with \(x_l < x_u\) is the statespace of the game.^{3} Here, \(\mathbb {1}\) denotes the indicator function and \(\tau = \inf \{t \ge 0 \vert m(t) = m_2\}\) is the point in time in which the mode process moves from \(m_1\) to \(m_2\). In case no transition to \(m_2\) occurs, we set \(\tau = \infty \). At time \(\tau \), transition costs of \(\kappa \ge 0\) arise for player 1. The vector \(u = (u_1,u_2)\) denotes the controls of both players with \(u_i \in {{{\mathcal {U}}}}_i \subseteq {{\mathbb {R}}}^n\). We assume that \(F_i(x,u,m)\) is continuous and differentiable with respect to x and u for each \(m \in \{m_1,m_2 \}\). The state evolves according towhere f(x, u, m) is Lipschitz continuous and differentiable with respect to x and u for all \(m \in \{m_1,m_2\}\). The mode process m(t) initially is in \(m(0)=m_1\) and is controlled by player 1. More precisely, player 1 can determine at which point in time the process jumps from \(m_1\) to \(m_2\). Once \(m(t) = m_2\), no additional mode transitions are possible.^{4}
$$\begin{aligned} J_i = \int _{0}^{\infty } e^{r t} F_i(x(t), u(t), m(t)) dt  \mathbb {1}_{[i=1]} e^{r \tau } \kappa , \end{aligned}$$
(1)
$$\begin{aligned} \dot{x} = f(x,u,m(t)), \quad x(0) = x_{ini} \in X \end{aligned}$$
(2)
In what follows, we restrict attention to timeinvariant Markovian strategies. More precisely, we consider strategy profiles \(((\Phi _1(x,m),\Psi ),\Phi _2(x,m))\) with \(\Phi _i: X \times \{m_1,m_2\} \rightarrow {{\mathbb {R}}}^{n_i}\) and \(\Psi \subseteq X\) such that \(u_i(t) = \Phi _i(x(t), m(t)), i=1,2\) for all t and the process \(m(\cdot )\) jumps from \(m_1\) to \(m_2\) at t if and only if \(m(t) = m_1\) and \(x(t) \in \Psi \). Intuitively, the switching set \(\Psi \), which is chosen by player 1, determines the set of states at which player 1 induces a switch from \(m_1\) to \(m_2\) and hence represents a Markovian strategy of that player.^{5} A Markov perfect equilibrium of the (simultaneous move) differential game is a strategy profile such that each player solves the dynamic optimization problem defined by (1) and (2) given that the other player determines her control using her equilibrium strategy.
3 Markov Perfect Equilibria
In this section, we derive necessary conditions to be satisfied by a Markov Perfect equilibrium profile of the problem described in Sect. 2. The fact that one of the players controls the transition from mode \(m_1\) to \(m_2\) implies that, in addition to standard conditions characterizing the value functions and optimal strategies in each of the two modes, the effect of the endogenous timing of the mode transition has to be taken into account. Intuitively, this means that on the one hand the time of the mode transition has to be optimal for player 1, thereby fulfilling standard conditions for optimal stopping problems. On the other hand, for a given mode switching strategy \(\Psi \) of player 1, the opponent player 2 can also influence the dynamics of x, and thereby the time of the mode switch, by choosing her controls in mode \(m_1\). The interplay of these effects has to be taken into account when characterizing a MPE profile. In what follows, we focus on equilibria in which the mode switching set \(\Psi \) is of threshold type, i.e., it has the form \(\Psi = [{{\bar{x}}}, x_u]\), where player 1 chooses the value of \({{\bar{x}}}\).^{6} We denote such a threshold strategy characterized by \(\bar{x}\) as \(\Psi _{{{\bar{x}}}}\). The following set of sufficient conditions characterizes MPE of such threshold type. (The proof of this and all following propositions is given in ‘Appendix’ A.)
Proposition 1
Consider a multimode differential game described in Sect. 2. If there exists a set of except at \(({{\bar{x}}}, m_1)\) everywhere continuous and continuously differentiable value functions \(V_i: X \times \{m_1,m_2\} \rightarrow {{\mathbb {R}}}\) and a profile of almost everywhere continuous Markovian strategies \(((\Phi _1(x,m),\Psi _{{{\bar{x}}}}),\Phi _2(x,m))\) such that the conditions below are satisfied for \(i=1,2\), then this profile constitutes to a Markov perfect equilibrium of the game:
(i)
$$\begin{aligned} \begin{array}{ll} V_i(x,m_2) = \frac{1}{r} \Big ( F_i(x,(\Phi _1(x,m_2),\Phi _2(x,m_2)),m_2) + &{} \\ + \frac{\partial V_i(x,m_2)}{\partial x} f(x,(\Phi _1(x,m_2),\Phi _2(x,m_2)),m_2) \Big ) \quad &{} x \in X, \end{array} \end{aligned}$$
(ii)
$$\begin{aligned} \begin{array}{ll} \Phi _i(x,m_2) \in argmax_{u_i \in {{{\mathcal {U}}}}_i} \Big ( F_i(x,(u_i,\Phi _j(x,m_2)),m_2) + &{} \\ + \frac{\partial V_i(x,m_2)}{\partial x} f(x,(u_i,\Phi _j(x,m_2)),m_2) \Big ) \quad &{} x \in X, j\not =i, \end{array} \end{aligned}$$
(iii)
$$\begin{aligned} \lim \sup _{t\rightarrow \infty } e^{rt} V_i(x,m_2) \le 0, i=1,2 \end{aligned}$$
(iv)
$$\begin{aligned} V_i(x,m_1) = \left\{ \begin{array}{ll} \frac{1}{r} \Big ( F_i(x,(\Phi _1(x,m_1),\Phi _2(x,m_1)),m_1) +&{}\\ + \frac{\partial V_i(x,m_1)}{\partial x} f(x,(\Phi _1(x,m_1),\Phi _2(x,m_1)),m_1) \Big ) \quad &{} x < {{\bar{x}}},\\ V_i(x,m_2)  \mathbb {1}_{[i=1]} \kappa &{} x \ge {{\bar{x}}}, \end{array} \right. \end{aligned}$$
(v)
where the inequality for player 2 has to hold as equality if there exists an \(\epsilon > 0\) such that \(f(x,(\Phi _1(x,m_1),\Phi _2(x,m_1)),m_1) > 0\) for all \(x \in ({{\bar{x}}}  \epsilon , {{\bar{x}}})\).
$$\begin{aligned} \begin{array}{ll} \lim _{x \rightarrow {{\bar{x}}}} V_1(x,m_1) &{}= V_1({{\bar{x}}},m_1)\\ \lim _{x \rightarrow {{\bar{x}}}} V_2(x,m_1) &{}\ge V_2({{\bar{x}}},m_1), \end{array} \end{aligned}$$
(vi)
$$\begin{aligned} V_1(x,m_2) \kappa< V_1(x,m_1), \quad \forall x < {{\bar{x}}} \\ \end{aligned}$$
(vii)
$$\begin{aligned} \begin{array}{ll} r (V_1(x,m_2) \kappa ) &{}> \max _{u_1 \in {{{\mathcal {U}}}}_1} \Big [F_1(x, (u_1, \Phi _2(x,m_1)),m_1) +\\ &{} + \frac{\partial V_1(x,m_2)}{\partial x} f(x,(u_1, \Phi _2(x,m_1)),m_1) \Big ], \quad \forall x > {{\bar{x}}} \end{array} \end{aligned}$$
(viii)
for some functions \({{\tilde{\Phi }}}_i: [{{\bar{x}}}, x_u] \rightarrow {\mathbb R}^{n_i}, \ i=1,2\).
$$\begin{aligned} \Phi _i(x,m_1) \left\{ \begin{array}{ll} \in argmax_{u_i \in {{{\mathcal {U}}}}_i} \Big ( F_i(x,(u_i,\Phi _j(x,m_1)),m_1) +&{}\\ + \frac{\partial V_i(x,m_1)}{\partial x} f(x,(u_i,\Phi _j(x,m_1)),m_1) \Big ) \quad &{} x < {{\bar{x}}}, \ j \not = i,\\ = {{\tilde{\Phi }}}_i(x) &{} x \ge {{\bar{x}}} \end{array} \right. \end{aligned}$$
In order to interpret the conditions listed in the proposition, we first observe that conditions (i) and (ii) are standard conditions for a MPE of the game in mode \(m_2\). Since for any value of the state x above the threshold \({\bar{x}}\) player 1 immediately switches to mode \(m_2\), the value functions in mode \(m_1\) in this part of the state space coincide with that in mode \(m_2\) net of the costs associated with the switch from \(m_1\) to \(m_2\). For values of x below \({{\bar{x}}}\), the value function and the equilibrium strategies are characterized by standard Hamilton–Jacobi–Bellman (HJB) equations (see (iv) and (viii)). The boundary condition for these HJB equations is given by the value matching condition in (v) which guarantees that the value function of player 1 is continuous at the threshold \({{\bar{x}}}\), whereas the value function of player 2 might exhibit a jump at the threshold \({{\bar{x}}}\) at which player 1 switches to mode \(m_2\). However, as we will illustrate below, a noncontinuous value function for player 2 can arise only if the state dynamics in equilibrium is such that the switch to mode \(m_2\) occurs either immediately or never, depending on the initial state, i.e., if the dynamics in mode \(m_1\) leads the state away from the threshold \({{\bar{x}}}\) (cf. condition (v)). The optimality of the threshold \({{\bar{x}}}\) from the perspective of player 1 is guaranteed by conditions (vi) and (vii), where condition (vi) ensures that it is not optimal for player 1 to switch to mode \(m_2\) at any \(x \in [x_l,{{\bar{x}}})\) and (vii) guarantees that for any \(x > {{\bar{x}}}\) it is optimal for player 1 to switch immediately to mode \(m_2\) rather than remaining in mode \(m_1\).
Finally, we like to point out that the sufficient conditions for an MPE, provided in Proposition 1, impose hardly any restrictions on the strategies \(\phi _i(x,m_1)\) for \(x > {{\bar{x}}}\). This implies that in spite of the fact that we consider Markov perfect equilibria, which constitute equilibria of every subgame defined by the current state and mode, the players might use strategies which for \(x > {{\bar{x}}}\) in mode \(m_1\) induce actions that would not be optimal for that player if implemented for a positive amount of time. This is still optimal because for these states the game immediately switches to mode \(m_2\). We will discuss this issue in more detail below and illustrate that this feature might give rise to a wide range of coexisting equilibria.
For our further discussion, it is helpful to distinguish different types of equilibria that can arise in our setting. The main property to be considered is whether the state dynamics under the equilibrium strategies \((\Phi _1(x,m_1), \Phi _2(x,m_1))\) for \(x < {{\bar{x}}}\) in the neighborhood of \({{\bar{x}}}\) points toward the threshold \({{\bar{x}}}\) , i.e., whether \(f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) > 0\) for \(x \in ({{\bar{x}}}  \epsilon , {{\bar{x}}})\). If this condition holds, then for \(x^{ini} \in ({{\bar{x}}}  \epsilon , {{\bar{x}}})\) the game switches to \(m_2\) with some positive delay. We refer to such equilibria as delay equilibria.^{7} On the other hand, if \(f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) < 0\) for all \(x \in ({{\bar{x}}}  \epsilon , {{\bar{x}}})\), then the game either switches immediately to mode \(m_2\), for \(x(0) \ge {{\bar{x}}}\) or remains in mode \(m_1\) forever (for \(x(0) < {{\bar{x}}}\)). We denote such equilibria as now or never equilibria. The following definition puts these notions more formally.
Definition 1
Consider a MPE strategy profile \({{{\mathcal {P}}}} = ((\Phi _1(.),\Psi _{\bar{x}}), \Phi _2(.))\) satisfying the conditions of Proposition 1 and denote by \(\tau (x^{ini}) = \inf \{t \ge 0: \tilde{x}(t) \ge {{\bar{x}}}\}\) with \({{\tilde{x}}}(0)=x^{ini}\) and \(\dot{ {{\tilde{x}}}} = f({{\tilde{x}}}, (\Phi _1({{\tilde{x}}},m_1), \Phi _2({{\tilde{x}}},m_1)),m_1)\) and \(\tau (x^{ini}) = \infty \) if \({{\tilde{x}}}(t) < {{\bar{x}}} \ \forall t\). Then,

the MPE profile \({{{\mathcal {P}}}}\) is called a now or never equilibrium if for every initial state \(x^{ini} \in X\) either \(\tau (x^{ini}) = 0\) or \(\tau (x^{ini}) = \infty \).

the MPE profile \({{{\mathcal {P}}}}\) is called a delay equilibrium if there exist initial states \(x^{ini} \in X\) such that \(0< \tau (x^{ini}) < \infty \).

a delay equilibrium profile \({{{\mathcal {P}}}} = ((\Phi _1(.),\Psi _{\bar{x}}), \Phi _2(.))\) is called a forced switch delay equilibrium if there is a small \(\epsilon > 0\) such that for any alternative strategy \({{\tilde{\Phi }}}_2(.)\) of player 2 with \({{\tilde{\Phi }}}_2(x,m_1) = \Phi _2(x,m_1) \ \forall x \not \in [{{\bar{x}}}, {{\bar{x}}}+\epsilon ), \ {{\tilde{\Phi }}}_2(x,m_2) = \Phi _2(x,m_2) \ \forall x \in X\) and \(\tilde{\Phi }_2(x,m_1)\) continuous and continuously differentiable on \((\bar{x}  \epsilon , {{\bar{x}}} + \epsilon )\) there exists a strategy \((\tilde{\Phi }_1(.),\Psi _{{{\tilde{x}}}})\) with \({{\tilde{x}}} > {{\bar{x}}}\) which is a better response for player 1 to \({{\tilde{\Phi }}}_2(.)\) than \((\Phi _1(.),\Psi _{{{\bar{x}}}})\).

a delay equilibrium which is not a forced switch delay equilibrium is called maximum delay equilibrium.
Intuitively, a forced switch delay equilibrium has the property that the switch of player 1 to mode \(m_2\) at the state \({{\bar{x}}}\) is triggered by the threat of a jump in the control player 2 would implement at \({{\bar{x}}}\) if the mode would stay in \(m_1\).^{8} Formally in Definition 1, this is expressed by the fact that under a forced switch delay equilibrium, any continuous extension of player 2’s strategy \(\Phi _2(.,m_1)\) beyond \({{\bar{x}}}\) would make it a best response for player 1 to switch to mode \(m_2\) at some threshold above \({{\bar{x}}}\). Delay equilibria, where the mode switch by player 1 is not triggered by such a threat, are called maximum delay equilibria. In any delay equilibrium, the fact that player 1 has incentives to delay the switch to mode \(m_2\) for any \(x \in ({{\bar{x}}}  \epsilon , {{\bar{x}}})\) induces that the inequalityholds for \(x \in ({{\bar{x}}}  \epsilon , {{\bar{x}}})\). The inequality has the standard interpretation that the lefthand side gives the (flow) profit player 1 receives in mode \(m_1\) from marginally delaying the mode switch and the righthand side the associated (discounted) loss in value received in \(m_2\). In general, (3) might hold as a strict inequality even in the limit as x converges to \({{\bar{x}}}\), in particular if the strategy \(\Phi _2(.,m_1)\) of player 2 exhibits a jump at \({{\bar{x}}}\). In an equilibrium with this property, player 1 has strict incentives to delay the switch to mode \(m_2\) for any value of the state x up to \({{\bar{x}}}\), and the switch to \(m_2\) at \({{\bar{x}}}\) is then triggered by the jump the action of player 2 would exhibit if the game would remain in mode \(m_1\). From the perspective of player 2, the jump in \(\Phi _2(.,m_1)\) at \({{\bar{x}}}\) in such a setting, however, is optimal only because player 1 switches the mode to \(m_2\) for any \(x \ge {{\bar{x}}}\) such that player 2 is never required to implement the actions induced by \(\Phi _2(x,m_1), x \ge {{\bar{x}}}\) for a positive amount of time. In such a situation, if player 2 would move the jump of its control from \({{\bar{x}}}\) to \({{\bar{x}}} + \epsilon \), this would also induce a change in the optimal switching strategy of player 1 in the sense that the optimal mode switching threshold for player 1 would move upward. These are the forced switch delay equilibria. If an equilibrium has the maximum delay property, and therefore the choice of \({{\bar{x}}}\) by player 1 is not triggered by the threat of a jump of the action of player 2, standard optimal stopping arguments imply that player 1 delays the switch until a value of x is reached where (3) holds as equality. In such equilibria, also a standard smooth pasting condition for player 1 holds in the sense that the slope of the value function of player 1 at \(x = {{\bar{x}}}\) in mode \(m_1\) coincides with the slope in mode \(m_2\). In the following lemma, we state these observations more formally.
$$\begin{aligned} \begin{array}{ll} F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) &{} > r (V_1(x,m_2) \kappa ) \\ &{}  \frac{\partial V_1(x,m_2)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \end{array} \end{aligned}$$
(3)
Lemma 1
If \({{{\mathcal {P}}}} = ((\Phi _1(x,m),\Psi _{{{\bar{x}}}}),\Phi _2(x,m))\) is a maximum delay equilibrium profile, thenand
$$\begin{aligned}&\lim _{x \rightarrow {{\bar{x}}}} \left[ F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) + \frac{\partial V_1(x,m_2)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \right] \nonumber \\&\quad = r (V_1({{\bar{x}}},m_2) \kappa ) \end{aligned}$$
(4)
$$\begin{aligned} \lim _{x \rightarrow {{\bar{x}}}} \frac{\partial V_1(x,m_1)}{\partial x} = \frac{\partial V_1({{\bar{x}}},m_2)}{\partial x}. \end{aligned}$$
(5)
In the remainder of this section, we discuss the implications of the different types of equilibria for the continuity of the players’ controls over time. First, we observe that in case of now or never equilibria with continuous strategies \(\Phi _i(.,m_1)\) and \(\Phi _i(.,m_2)\) the trajectories of the actions of both players are continuous over time, since no switch from \(m_1\) to \(m_2\) occurs for \(t > 0\). Hence, we focus here on delay equilibria and the question whether controls are continuous at the point of time when the mode is changed by player 1. Given that we consider deterministic models, where for a given strategy profile both player can perfectly predict the time of the switch to mode \(m_1\), intuitively one could expect that in equilibrium controls are continuous as long as the marginal effects of the controls on the players instantaneous profits and on the state dynamics is the same in both modes. As we show below, this intuition holds true for player 1 as long as we consider a maximum delay equilibrium where the smooth pasting condition holds for player 1 at the time of the mode switch and the optimal action of player 1 is not affected by potential jumps of the control of player 2. To show this formally, we introduce the following notions.
Definition 2
A differential game of the structure described in Sect. 2 has

mode independent control effects if and only if$$\begin{aligned} \frac{\partial F_i(\cdot ,m_1)}{\partial u_i} =\frac{\partial F_i(\cdot ,m_2)}{\partial u_i} \quad \text{ and } \quad \frac{\partial f(\cdot ,m_1)}{\partial u_i} =\frac{\partial f(\cdot ,m_2)}{\partial u_i}, \ i=1,2. \end{aligned}$$(6)

separable control effects ifdo not depend on \(u_j\) for all \(x \in X\) and \(m \in \{m_1,m_2\}\).$$\begin{aligned} \frac{\partial F_i(x,(u_i,u_j),m)}{\partial u_i} \quad \text{ and } \quad \frac{\partial f(x,(u_i,u_j),m)}{\partial u_i} \end{aligned}$$
For twice differentiable functions \(F_i\) and f, separable control effects are equivalent to the condition that the crossderivatives of these functions with respect to the two controls vanish. In many types of games, including, for example, different variants of investment games, direct effect of the control of one player on the incentives of the other player indeed does not exist.
Even if we consider games that have both of these properties, equilibrium profiles, as characterized by Proposition 1, in general might exhibit jumps in the feedback strategies of both players such that \( \lim _{x \rightarrow {{\bar{x}}}} \Phi _i(x,m_1) \not = \Phi _i({{\bar{x}}},m_2)\). More precisely, in the case of forced switch delay equilibria, the actions of both players jump at the point in time when the modes switch to \(m_2\). Concerning the investment of player 2, who cannot directly control the switch from mode \(m_1\) to \(m_2\), in general jumps in the control occur as the game moves from mode \(m_1\) to mode \(m_2\) in all types of delay equilibria. In equilibrium, player 2 perfectly predicts the time of the mode switch and also the value of its investment in mode \(m_1\) for her future profits in mode \(m_2\). Therefore, at first sight the discontinuity of the investment of player 2 might be surprising. The jump in the action of player 2 results from the fact that in mode \(m_1\) player 2 can influence the time of the switch to mode \(m_2\), since the choice of her control affects the dynamics of the state and thereby the time the state hits the threshold \({{\bar{x}}}\) determined by the equilibrium strategy of player 1. This effect, which influences the optimal choice of the control of player 2, immediately disappears, at the point in time when the mode changes to \(m_2\). Hence, the equilibrium strategy of player 2 in general exhibits a jump at the state \({{\bar{x}}}\) in any delay equilibrium even if the game exhibits mode independent and separable control effects.
Considering the action of player 1, we have to distinguish between maximal delay equilibria and forced switch delay equilibria. Whereas Proposition 2 shows that the control of player 1 is continuous at the time of the regime switch in maximal delay equilibria, in general this does not hold for delay equilibria not having this property. Intuitively, in such an equilibrium player 1 has strict incentives to delay the jump to mode \(m_2\) for all values of the state below the threshold \({{\bar{x}}}\), and, differently to the maximal delay equilibrium, this incentive to delay does not converge to zero as x approaches \({{\bar{x}}}\). The reason for player 1 to switch to mode \(m_2\) at \({{\bar{x}}}\) is that player 2 threatens to discontinuously change its action at \({{\bar{x}}}\) if the game would stay in mode \(m_1\). Hence, for a given strategy of player 2, the threshold \({{\bar{x}}}\) at which player 1 switches to mode \(m_2\) is determined by the state at which player 2 threatens to change its control. Based on this, player 1 has to adjust its investment in order to influence the duration of the game in mode \(m_1\). Similarly to what we described for player 2 above, this effect disappears as soon as the mode is \(m_2\) and hence also the action of player 1 in general exhibits a jump at the mode switch. More formally, in the absence of a smooth pasting condition the state derivative of the value function of player 1 in general exhibits a jump as the mode switches from \(m_1\) to \(m_2\). Therefore, the marginal return of investment for player 1 jumps as the game switches from mode \(m_1\) to \(m_2\) at \({{\bar{x}}}\), which induces a jump in the control of player 1. In Sect. 4, we will illustrate these scenarios using a simple example analyzing the optimal timing of new product introduction in a duopoly.
The following proposition formally establishes that under maximum delay equilibria the action of player 1 is continuous over time although the mode changes at some positive t from \(m_1\) to \(m_2\).
Proposition 2
If \(((\Phi _1(x,m),\Psi _{{{\bar{x}}}}(x)),\Phi _2(x,m))\) is a maximum delay equilibrium profile of a game with mode independent and separable control effects such that in each mode m the strategies \(\Phi _i, i=1,2\) are continuous with respect to the state x and the righthand side of condition (ii) in Proposition 1 has a unique maximizer at \(x = {{\bar{x}}}\), then
$$\begin{aligned} \lim _{x \rightarrow {{\bar{x}}}} \Phi _1(x,m_1) = \Phi _1({{\bar{x}}},m_2). \end{aligned}$$
The intuition for this result is straightforward. For the maximal delay equilibrium, the smooth pasting condition holds for player 1 at the threshold \({{\bar{x}}}\). This ensures that the slope with respect to the state of her value function is identical in modes \(m_1\) and \(m_2\) at this point. If the marginal effects of the own control on the instantaneous payoff and the state dynamics are not affected directly by the change in mode (mode independent control effect) and by the potential change in the other player’s control (separable control effect) due to the mode switch, then the optimization problem which player 1 faces at state \({{\bar{x}}}\) is equivalent, no matter whether the firm is in mode \(m_1\) or in \(m_2\). Hence, its equilibrium feedback strategy is continuous at \({{\bar{x}}}\).
4 An Illustrative Example: Optimal Timing of New Product Introduction
To illustrate our general findings, we now consider the timing problem of a firm, denoted as firm 1, which has to decide when to introduce a new product, that it has developed. We assume that the firm is already active producing an established product and competes with a second firm (firm 2) on the market for the established product. Only firm 1 has developed the new product and therefore has the option to introduce that product at any point in time \(t \ge 0\).
Before the new product is introduced (mode \(m_1\)), the inverse demand for the established product is given bywhere \(p_o\) denotes the price of established product and \(q_{io}\) the quantity of that product supplied by firm \(i, \ i=1,2\). After the introduction of the new product (mode \(m_2\)), the inverse demand changes toHere, \(\alpha _o\) is the reservation price of the established product, whereas \(\alpha _n^0 + \alpha _n\) denotes the reservation price of the new product. We write this reservation price as the sum of a constant parameter \(\alpha _n^0\), giving the minimal value of the reservation price, and the state variable \(\alpha _n(t) \ge 0\) capturing the effects of the efforts of the two competitors to influence the (inverse) demand for the new product. Firm 1, as the (potential) innovator, tries to increase the willingness to pay for the new product, whereas the competitor firm 2 might invest in reducing this willingness, e.g., by providing information to consumers about negative features of the new product or by negative campaigning. Both firms choose nonnegative effort \(h_i\), measured e.g., in work hours, to do advertising (firm 1), respectively, negative campaigning (firm 2) for the new product. The unit cost of such effort is denoted by \(c_i > 0, i=1,2\) ,and we assume that there are decreasing returns to effort in terms of the size of the effect on the inverse demand for the new product. More precisely, we assume that the change in reservation price is given byHere, \(\gamma > 0\) is a parameter, and \(\delta \) is the rate by which the effect of firms’ effort on inverse demand vanishes. We assume that the equation governing the dynamics of \(\alpha _n\) is the same in both modes, which captures a situation where firms might invest effort in influencing the reservation price of the new product also before the product is actually introduced to the market.^{9} For analytical convenience, we transform the firms’ control variables according to \(u_i = \sqrt{2 h_i}\) to obtain the following formulation with linear state dynamicsand quadratic cost functions \(\xi _i(u_i) = \frac{c_i}{2} u_i^2\). In slight abuse of notation, we will refer to the controls \(u_i\) as firm effort in the further analysis. Production costs for the established product are assumed to be symmetric across firms and given by \(\frac{\nu _o}{2} q_{io}^2, i=1,2\) and analogously for the new product firm 1 has production costs \(\frac{\nu _n}{2} q_{1n}^2\).
$$\begin{aligned} p_o = \alpha _o  (q_{1o} + q_{2o}), \end{aligned}$$
$$\begin{aligned} p_o(q_{1o}+q_{2o}, q_{1n})&= \alpha _o  (q_{1o} + q_{2o})  \eta q_{1n} \\ p_n(q_{1o}+q_{2o}, q_{1n})&= \alpha _n^0 + \alpha _n  \eta (q_{1o} + q_{2o})  q_{1n}. \end{aligned}$$
$$\begin{aligned} {{\dot{\alpha }}}_n = \sqrt{2 h_1}  \gamma \sqrt{2 h_2}  \delta \alpha _n, \ m \in \{m_1,m_2\}. \end{aligned}$$
$$\begin{aligned} {{\dot{\alpha }}}_n = f(\alpha _n,(u_1,u_2),m) := u_1  \gamma u_2  \delta \alpha _n, \ m \in \{m_1,m_2\} \end{aligned}$$
(7)
Firms maximize profits by choosing the production quantities as well as the effort \(u_i\) at every point in time t. In addition, firm 1 decides about the time at which the new product is introduced. The introduction of the new product is associated with lumpsum costs of \(\kappa > 0\).
Since the quantity choice in this setting does not have any intertemporal effects, firms at t choose quantities according to the Cournot equilibrium, which depends on the value of \(\alpha _n(t)\) for all t after the introduction of the new product. Standard calculations (see ‘Appendix’ B for a derivation of the Cournot equilibrium quantities and profits in both modes) show that before the introduction of the new product we haveand the market profit of each firm is given by \(\pi _{io}^{m_1} = \left( 1+\frac{\nu _o}{2} \right) (q_{io}^{m_1})^2\). Hence, we obtain for the instantaneous profit in mode \(m_1\)In order to guarantee that after the introduction of the new product also a positive amount of this good is produced in equilibrium regardless of the value of \(\alpha _n\), we assume that \(\alpha _n^0 > \frac{3 \eta \alpha _o}{3 + \nu _o}\). Furthermore, we restrict attention to scenarios where in equilibrium both firms also sell a positive quantity of the established product. This is true as long as \(\alpha _n < \alpha _n^{UB} := \frac{(2 + \nu _n)(1+\nu _o) + \eta ^2}{\eta (3+2\nu _o)} \alpha _o  \alpha _n^0\) (see ‘Appendix’ B).
$$\begin{aligned} q_{io}^{m_1} = \frac{\alpha _o}{3 + \nu _o} \quad i=1,2, \end{aligned}$$
(8)
$$\begin{aligned} F_i(\alpha _n,u_i,m_1) = \left( 1+\frac{\nu _o}{2} \right) (q_{io}^{m_1})^2  \xi _i(u_i) \quad i=1,2. \end{aligned}$$
(9)
For values of \(\alpha _n \in [0, \alpha _n^{UB}]\), we obtain as the equilibrium quantities in mode \(m_2\):The corresponding instantaneous profit in mode \(m_2\) isOverall, the dynamic strategic interaction between the two firms constitutes a twomode differential game of the form considered in Sect. 2 with the single state \(\alpha _n\) evolving according to (7) and the instantaneous profits in mode \(m_1\) given by (9) and those in mode \(m_2\) given by (11). Both firms decide on their effort \(u_i\) as a function of the current state and mode and additionally firm 1 determines the timing of the switch from \(m_1\) to \(m_2\), which means that firm 1 decides on a threshold \({{\bar{\alpha }}}_n\) such that it introduces the new product as soon as the reservation price of the new product \(\alpha _n^0 + \alpha _n\) is larger than \(\alpha _n^0 + {{\bar{\alpha }}}_n\). Formally, in accordance with Sect. 3 we consider profiles of Markovian strategies of the form \(((\Phi _1(\alpha _n,m), \Psi _{{{\bar{\alpha }}}_n}), \Phi _2(\alpha _n,m))\) and in what follows characterize Markov perfect equilibria of this game.
$$\begin{aligned} q_{1o}^{m_2}(\alpha _n)= & {} \frac{\alpha _o ((1+ \nu _o) (2 + \nu _n) + \eta ^2) \eta (\alpha _n^0+\alpha _n) (3 + 2 \nu _o)}{(1\eta ^2) (6 + 5 \nu _o) + 3 (\nu _o + \nu _n) + (2+\nu _n) \nu _o^2 + 4 \nu _o \nu _n} \nonumber \\ q_{2o}^{m_2}(\alpha _n)= & {} \frac{\alpha _o ((1+ \nu _o) (2 + \nu _n)2 \eta ^2)  \eta (\alpha _n^0+\alpha _n) \nu _o }{(1\eta ^2) (6 + 5 \nu _o) + 3 (\nu _o + \nu _n) + (2+\nu _n) \nu _o^2 + 4 \nu _o \nu _n} \nonumber \\ q_{1n}^{m_2}(\alpha _n)= & {} \frac{(1+\nu _o)((\alpha _n^0+\alpha _n)(3+\nu _o) 3 \alpha _o \eta )}{(1\eta ^2) (6 + 5 \nu _o) + 3 (\nu _o + \nu _n) + (2+\nu _n) \nu _o^2 + 4 \nu _o \nu _n} \end{aligned}$$
(10)
$$\begin{aligned} F_1(\alpha _n, u_1, m_2)&= \left( 1 + \frac{\nu _o^2}{2} \right) (q_{1o}^{m_2}(\alpha _n))^2 + \left( 1 + \frac{\nu _n^2}{2} \right) (q_{1n}^{m_2}(\alpha _n))^2\nonumber \\ {}&\quad + 2 \eta q_{1o}^{m_2}(\alpha _n) q_{1n}^{m_2}(\alpha _n)  \xi _1(u_1) \nonumber \\ F_2(\alpha _n, u_2, m_2)&= \left( 1 + \frac{\nu _o^2}{2} \right) (q_{2o}^{m_2}(\alpha _n))^2  \xi _2(u_2) \end{aligned}$$
(11)
4.1 MPE in Mode \(m_2\)
In order to characterize the different types of equilibria in our game, we first consider the final mode \(m_2\). Since the instantaneous profits given in (11) are quadratic functions of state and controls and the state Eq. (7) is linear, the game in mode \(m_2\) is of linear quadratic structure. Following a wide range of the literature about such games (see, e.g., [5]), we assume that the game has a MPE with linear feedback strategies^{10}, giving rise to quadratic value functions of both players. The value and feedback functions can then be determined by a guess and verify approach using the firstorder conditions and Hamilton–Jacobi–Bellman equations, see ‘Appendix’ C for details.
×
Figure 1 shows the value functions of both players in the MPE for our benchmark parameter setting.^{11} The green arrows indicate the direction of the state dynamics under the equilibrium feedback strategies. There is a unique stable steady state \(\alpha _n^{*,m_2}\), and it is easy to verify that \(\alpha _n^{*,m_2} < \alpha _n^{UB}\) under our parameter setting such that both firms sell positive quantities of the old product in the steady state. Figure 1 clearly shows that, as expected, the value of firm 1 increases with \(\alpha _n\), whereas firm 2 is hurt by an increase in the attractiveness of the new product.
4.2 Different Types of MPEs in Mode \(m_1\)
In order to characterize MPE strategies in mode \(m_1\), we first determine the value functions of both players under the extreme scenarios in which the new product, regardless of the initial condition, is either immediately or never introduced. We denote the value function of firm i in the former case of immediate introduction by \(V_i^0(\alpha _n,m_1)\). Clearly, we haveIf the new product is never introduced, positive investment is never optimal for either firm, i.e., \(u_i(t) = 0, \ i=1,2, \ t \ge 0\), and hence, the value functions readNote that \(V_i^{\infty }\) is constant with respect to \(\alpha _n\) because the attractiveness of the new product is irrelevant if this product is never introduced. Given that \(V_1^0(\alpha _n,m_1)\) increases with \(\alpha _n\), it follows that \(V_1^{0} (0,m_1) > V_1^{\infty } (0,m_1)\) implies \(V_1^{0} (\alpha _n,m_1) > V_1^{\infty } (\alpha _n,m_1) \ \forall \alpha \in [0, \alpha _n^{UB}]\) and it is optimal for firm 1 to introduce the new product at some finite point in time t. If this inequality is violated, then in general it might depend on the initial value \(\alpha _n(0)\) whether the new product is introduced. In order to gain a better understanding of the potential structure of equilibria under which the new product introduction depends on the initial state, we first characterize the properties of equilibrium state dynamics on \((0,{{\bar{\alpha }}}_n)\).
$$\begin{aligned} V_i^0(\alpha _n,m_1) = V_i(\alpha _n, m_2)  \mathbb {1}_{[i=1]} \kappa . \end{aligned}$$
$$\begin{aligned} V_i^{\infty } (\alpha _n,m_1) = \left( 1+\frac{\nu _o}{2} \right) \frac{(q_{io}^{m_1})^2}{r}, \ i=1,2. \end{aligned}$$
The following proposition shows that under an MPE profile the state \(\alpha _n\) is either monotonously increasing or decreasing on the entire interval below the threshold value at which firm 1 introduces the new product (i.e., for \(\alpha \in (0,{{\bar{\alpha }}}_n)\)).
Proposition 3
If \(((\Phi _1(\alpha _n,m), \Psi _{{{\bar{\alpha }}}_n}),\Phi _2(\alpha _n,m))\) is a MPE profile of the game, then it holds for all \(\alpha _n^1, \alpha _n^2 \in (0, {{\bar{\alpha }}}_n)\) that
$$\begin{aligned} \text{ sgn } \left[ f(\alpha _n^1, (\Phi _1(\alpha _n^1,m_1),\Phi _2(\alpha _n^1,m_1))) \right] = \text{ sgn } \left[ f(\alpha _n^2, (\Phi _1(\alpha _n^2,m_1),\Phi _2(\alpha _n^2,m_1))) \right] . \end{aligned}$$
The observation that in equilibrium the state \(\alpha _n\) is either strictly increasing or strictly decreasing on the entire interval \((0,{{\bar{\alpha }}}_n)\) has several important implications. If in a given equilibrium the new product is introduced after a positive and finite delay for some initial value \(\alpha _n(0)\), then under this equilibrium profile the product is introduced after finite time regardless of the initial state \(\alpha _n(0)\). Conversely, if in a given equilibrium the product is never introduced for some initial value of the new market size, then under this equilibrium a positive finite delay in product introduction can never occur, regardless of \(\alpha _n(0)\). Relating to the different types of equilibria introduced in Sect. 3, the proposition shows that the property ‘delay equilibrium,’ which in Definition 1 was defined locally in the sense that for some initial condition the mode switches after some delay, in the framework of this model is actually a global property in the sense that in any delay equilibrium the switch to \(m_2\) occurs after a positive finite delay for all \(\alpha _n \in [0, {{\bar{\alpha }}}_n)\). Similarly, in this model in any now or never equilibrium the switch never occurs for any initial value of the state below the threshold \({{\bar{\alpha }}}_n\).
In what follows, we illustrate our general results from Sect. 3 by showing that, depending on the size of the new product introduction costs \(\kappa \), qualitatively different types of MPE constellations exist in in our model.
4.2.1 Small Costs of Market Introduction
If \(\kappa \) is sufficiently small, i.e., \(\kappa < {\underline{\kappa }} := V_1(0,m_2)  V_1^{\infty }(0,m_1)\), then it follows directly that immediate introduction is more profitable for firm 1 than no introduction regardless of the value of \(\alpha _n\).^{12} Hence, in equilibrium the new product is always introduced. Concentrating first on a maximum delay equilibrium, we need to characterize the threshold value of \(\bar{\alpha }_n\), above which the product is introduced. Denoting the threshold in the maximum delay equilibrium by \({{\bar{\alpha }}}_n^{md}\), we obtain that the following condition has to be satisfied for some positive value of \(u_2^{m_1}\) (see ‘Appendix’ D):Whereas the first equation captures the smooth pasting condition, the second is the HJB equation of firm 2 at \(\alpha _n^{md}\). To obtain these equations, we have used the fact that the equilibrium value function of firm 2 has to coincide between the two modes for all \(\alpha _n \ge {{\bar{\alpha }}}_n\) (see Proposition 1) and that the control of firm 1 is identical in both modes for \(\alpha _n = {{\bar{\alpha }}}_n^{md}\) since all conditions of Proposition 2 are satisfied in our model.
$$\begin{aligned} r \left( V_1({{\bar{\alpha }}}_n^{md}, m_2)  \kappa \right)= & {} F_1(\bar{\alpha }_n^{md}, \Phi _1({{\bar{\alpha }}}_n^{md},m_2), m_1)\nonumber \\&+ \frac{\partial V_1({{\bar{\alpha }}}_n^{md},m_2)}{\partial \alpha _n} f(\bar{\alpha }_n^{md}, \Phi _1({{\bar{\alpha }}}_n^{md},m_2), u_2^{m_1}), \nonumber \\ r V_2({{\bar{\alpha }}}_n^{md}, m_2)= & {} F_2({{\bar{\alpha }}}_n^{md}, u_2^{m_1}, m_1)  \frac{c_2 u_2^{m_1}}{\gamma } f({{\bar{\alpha }}}_n^{md}, \Phi _1({{\bar{\alpha }}}_n^{md},m_2), u_2^{m_1}) \end{aligned}$$
(12)
×
×
For our benchmark parametrization together with market introduction costs of \(\kappa = 0.095\), the system (12) has a unique positive solution given by \({{\bar{\alpha }}}_n^{md} = 0.036\) and \(u_2^{m_1} = 0.0047\). The corresponding maximum delay equilibrium is illustrated in Figs. 2 and 3a. The solid lines in the two panels of Fig. 2 show the value functions of the two firms in mode \(m_1\) under such an equilibrium. More precisely, these functions have been determined as the solutions of the HJB equations for mode \(m_1\) on \([0,{{\bar{\alpha }}}_n^{md}]\) under the boundary conditions that \(V_{i}({{\bar{\alpha }}}_n^{md},m_1) = V_{i}(\bar{\alpha }_n^{md},m_2)  \mathbb {1}_{[i=1]} \kappa , i=1,2\). It can be easily checked that these value functions in combination with feedback functions derived from the maximization of the righthand side of the HJB equation on \([0,{{\bar{\alpha }}}_n^{md}]\) satisfy all conditions of Proposition 1 for any profile \((\Phi _1, \Phi _2)\) satisfying \( \Phi _2(\alpha _n, m_1) \ge \Phi _2(\bar{\alpha }_n^{md}, m_1)\) for all \(\alpha _n \ge {{\bar{\alpha }}}_n^{md}\) (in order to ensure condition (vii) of Proposition 1). The equilibrium investment functions under the maximum delay equilibrium are shown in Fig. 3a. It can be clearly seen that the investment of firm 1 in mode \(m_1\) (solid black line) intersects with the equilibrium investment in mode \(m_2\) (black dashed line) exactly at the threshold \({{\bar{\alpha }}}_n^{md}\) above which firm 1 immediately introduces the new product. We do not show any values for the function \(\Phi _1(\alpha _n, m_1)\) for \(\alpha _n \ge {{\bar{\alpha }}}_n^{md}\) since any choice of the function on this interval is compatible with equilibrium. The figure also illustrates the downward jump of the investment of firm 2 at the threshold \(\alpha _n = {{\bar{\alpha }}}_n^{md}\) where the mode switches from \(m_1\) to \(m_2\). In mode \(m_1\), there is an additional incentive for player 2 to invest. Such investment decreases the speed of the increase of \(\alpha _n\) and delays the point in time when the state variable arrives at \({\bar{\alpha }}_n^{md}\) and the new product is introduced. Firm 2 has an incentive to delay the new product introduction, which provides additional investment incentives. Once the new product is introduced, this additional incentive vanishes which results in a downward jump in player 2’s investment.
However, the maximum delay equilibrium is not the only MPE in our setting. Actually, for any threshold \({{\bar{\alpha }}}_n^d \in [0, \bar{\alpha }_n^{md})\) there is a forced switch delay equilibrium such that firm 1 introduces the new product immediately for all \(\alpha _n \ge {{\bar{\alpha }}}_n^d\). In Fig. 2, the value functions corresponding to such an equilibrium with \({{\bar{\alpha }}}_n^d = 0.01\) are illustrated with coarsely dashed lines. It can be clearly seen that both value functions have a kink at \(\alpha _n = \bar{\alpha }_n^d\) such that also for firm 1 the smooth pasting condition does not hold at this threshold where the firm introduces the new product. The equilibrium feedback function corresponding to this MPE is shown in Fig. 3b. The figure illustrates that the investments of both firms jump at the point in time when the new product is introduced, where the jump is upward for firm 1 and downward for firm 2. Whereas the intuition for the downward jump of firm 2 is analogous to that developed for the maximum delay equilibrium, the upward jump for firm 1 is due to the fact that after the introduction of the new product an increase of \(\alpha _n\) has a positive impact on the instantaneous profit of firm 1, whereas in mode \(m_1\) such an increase only has impact on the remaining time till the new product is introduced. If the new product is introduced at a level of \(\alpha _n\), where the value function in mode \(m_2\) is still strictly steeper than that in mode \(m_1\), the investment incentives jump upward at the point of market introduction. A crucial feature of this equilibrium profile is that \(\Phi _2(\alpha _n,m_1)\) for \(\alpha _n \ge {{\bar{\alpha }}}_n^d\) is sufficiently large such that it is optimal for firm 1 to introduce the new product immediately at \({{\bar{\alpha }}}_n^d\). In our example, this is guaranteed by setting \(\Phi _2({{\bar{\alpha }}}_n^d, m_1) > \frac{1}{\gamma } \Phi _1({{\bar{\alpha }}}_n^d, m_1)\), which implies that under optimal investment by firm 1 the state variable \(\alpha _n\) does not move above \({{\bar{\alpha }}}_n^d\). As discussed in Sect. 3, although such high investments by firm 2 would not be optimal, if it were to be carried out for a time interval with positive measure, in equilibrium the firm is never required to carry out such investment, regardless of the value of \(\alpha _n(0)\).
×
4.2.2 Intermediate Costs of Market Introduction
We now consider the case where \(\kappa > {\underline{\kappa }}\), which implies that for \(\alpha _n(0) = 0\) immediate introduction of the new product yields a lower value for firm 1 than never introducing the new product and abstaining from any investment into the buildup of \(\alpha _n\). Although immediate introduction of the new product is not optimal, an introduction with some delay might still be more profitable than no introduction. Taking into account that the largest value for firm 1 is obtained under the maximum delay equilibrium, \(V_1^{md}(0,m_1) > V_1^{\infty }(0,m_1)\) is a necessary and sufficient condition for the existence of delay equilibria in our setting. It should be noted that \(V_1^{md}(0,m_1)\) is a decreasing function of \(\kappa \) such that this condition implies an upper bound \({{\bar{\kappa }}}\) for the costs of market introduction such that for all \(\kappa \le {{\bar{\kappa }}}\) there exists an equilibrium such that for all \(\alpha _n(0) \ge 0\) the new product is introduced to the market after finite time. For this subsection, we assume that \(\kappa \in [{\underline{\kappa }}, {{\bar{\kappa }}}]\), i.e., the market introduction costs are in an intermediate range such that for \(\alpha _n(0) = 0\) immediate introduction is not optimal but there exists an equilibrium which induces new product introduction with some delay for this initial value.
In Fig. 4, this case is illustrated by showing the value functions for immediate market introduction (dashed lines), the maximum delay equilibrium (solid line) and no market introduction (dotted line). We denote by \({{\bar{\alpha }}}_n^{nn}\) the largest value of \(\alpha _n\) such that \(V_1^{\infty }(\alpha _n,m_1) \ge V_1(\alpha _n,m_2)  \kappa \). Clearly, for all values of \(\alpha _n < {{\bar{\alpha }}}_n^{nn}\) immediate market introduction is not optimal; therefore, the range of threshold values \({{\bar{\alpha }}}_n^d\) for which delay equilibria can exist is restricted to \(\bar{\alpha }_n^d \in [{{\bar{\alpha }}}_n^{nn}, {{\bar{\alpha }}}_n^{md}]\). However, for a delay equilibrium to exist, the associated value function must also satisfy \(V_1^d(0,m_1;{{\bar{\alpha }}}_n^d) \ge V_1^{\infty }(0,m_1)\), where we denote by \(V_1^d(0,m_1;\bar{\alpha }_n^d)\) the value function of firm 1 under a (candidate for a) delay equilibrium with threshold \({{\bar{\alpha }}}_n^d\). If this inequality does not hold, condition (ii) in Proposition 1 would be violated at \(\alpha _n = 0\). This can be seen by realizing that by choosing \(u_i = 0\) for \(\alpha _n = 0\) we have \({{\dot{\alpha }}} = 0\) and the value of the righthand side of the HJB equation becomesSince under the equilibrium feedback functions in the delay equilibrium the righthand side of the HJB equation has to be equal to \(r V_1^d(0,m_1;{{\bar{\alpha }}}_n^d)\) (see condition (i) in Proposition 1), the feedback function \(\Phi ^d(\alpha _n,m_1;{{\bar{\alpha }}}_n^d)\) in that equilibrium candidate does not maximize the righthand side of the HJB equation. This shows that no delay equilibrium with \(V_1^d(0,m_1;{{\bar{\alpha }}}_n^d) < V_1^{\infty }(0,m_1)\) can exist with the property that the new product is introduced after a positive delay for \(\alpha _n(0)=0\). Furthermore, we know from Proposition 3 that in any equilibrium the sign of the state dynamics cannot change on the interval \([0, {{\bar{\alpha }}}_n]\). This rules out any equilibrium profile under which the new product is never introduced (and hence \(\alpha _n\) decreases over time) for a small value of \(\alpha _n(0)\), but is introduced after a delay for larger values of \(\alpha _n(0)\). Put together these arguments establish that for any threshold \(\bar{\alpha }_n^d\) with \(V_1^d(0,m_1;{{\bar{\alpha }}}_n^d) < V_1^{\infty }(0,m_1)\) no delay equilibrium can exist. However, there exists a continuum of delay equilibria where at the market introduction threshold this inequality does not hold. In particular, there exists a unique \({\underline{\alpha }}_n^d \in (\bar{\alpha }_n^{nn}, {{\bar{\alpha }}}_n^{md}]\) such that \(V_1^d(0,m_1;\bar{\alpha }_n^{d}) \ge V_1^{\infty }(0,m_1)\) for all \({{\bar{\alpha }}}_n^d \in [{{\underline{\alpha }}}_n^{d}, {{\bar{\alpha }}}_n^{md}]\). To see this, note that, since the smooth pasting condition is satisfied at \(\bar{\alpha }_n^{md}\), for all \({{\bar{\alpha }}}_n^d < {{\bar{\alpha }}}_n^{md}\) an equilibrium with a later switch to \(m_2\) leads to a higher value for firm 1. Hence, \(V_1^d(0,m_1;{{\bar{\alpha }}}_n^d)\) is an increasing and continuous function of \({{\bar{\alpha }}}_n^d\). Furthermore, in light of the definition of \({{\bar{\alpha }}}_n^{nn}\) we haveSince without market introduction of the new product the value of firm 1 does not depend on \(\alpha _n\), we have \(V_1^{\infty }(0,m_1) = V_1^{\infty }({{\bar{\alpha }}}_n^{nn},m_1) > V_1^d(0,m_1;\bar{\alpha }_n^{nn})\). Due to \(\kappa < {{\bar{\kappa }}}\), we also have \(V_1^{\infty }(0,m_1) < V_1^d(0,m_1;{{\bar{\alpha }}}_n^{md})\) and the intermediate value theorem implies the existence of \(\underline{\alpha }_n^{d} \in ({{\bar{\alpha }}}_n^{nn},{{\bar{\alpha }}}_n^{md})\) with \(V_1^d(0,m_1;{{\underline{\alpha }}}_n^{d}) = V_1^{\infty }(0,m_1)\). In the two panels of Fig. 4, we illustrate this observation by showing, as coarsely dashed lines, the value functions of the two firms under the (forced switch) delay equilibrium with threshold \({{\bar{\alpha }}}_n^d = {\underline{\alpha }}_n^d\). For any threshold value \({{\bar{\alpha }}}_n^d\) in the interval \([{{\underline{\alpha }}}_n^{d}, \bar{\alpha }_n^{md}]\), a delay equilibrium profile can be constructed in the same way as discussed in the previous subsection.
$$\begin{aligned} \left( 1+\frac{\nu _o}{2} \right) (q_{io}^{m_1})^2 = r V_1^{\infty }(0,m_1) > r V_1^d(0,m_1;{{\bar{\alpha }}}_n^d). \end{aligned}$$
$$\begin{aligned} V_1^d(0,m_1;{{\bar{\alpha }}}_n^{nn}) < V_1^d({{\bar{\alpha }}}_n^{nn},m_1;\bar{\alpha }_n^{nn}) = V_1({{\bar{\alpha }}}_n^{nn},m_2)  \kappa = V_1^{\infty }({{\bar{\alpha }}}_n^{nn},m_1). \end{aligned}$$
×
However, in the case of intermediate values of market introduction costs considered here, there also exists a now or never equilibrium, as discussed in Sect. 3. Similarly to the forced switch delay equilibria, also this equilibrium is characterized by a threat of a strong investment of player 2 in \(m_1\) as soon as the state variable \(\alpha _n\) is larger or equal than \(\bar{\alpha }_n^{nn}\). Hence, firm 1 introduces the product immediately if \(\alpha _n \ge {{\bar{\alpha }}}_n^{nn}\). As discussed above, under a potential delay equilibrium in which firm 1 invests positive amounts we would havefor all \(\alpha _n < {{\bar{\alpha }}}_n^{nn}\) and therefore setting \(\Phi _1(\alpha _n,m_1) = 0\) is the optimal strategy for firm 1 on this interval. Hence, in equilibrium we also must have \(\Phi _2(\alpha _n,m_1) = 0\) for all \(\alpha _n < {{\bar{\alpha }}}_n^{nn}\). We obtain a MPE profile under which for all \(\alpha _n(0) < \bar{\alpha }_n^{nn}\) both firms invest nothing and the new product is never introduced, whereas for \(\alpha _n(0) \ge {{\bar{\alpha }}}_n^{nn}\) the product is introduced at \(t=0\) and both firms afterward invest according to the MPE strategies in mode \(m_2\). The equilibrium feedback functions underlying this equilibrium are illustrated in Fig. 5, where, as in Fig. 3, we do not show any values of \(\Phi _1(\alpha _n,m_1)\) for \(\alpha _n \ge {{\bar{\alpha }}}_n^{nn}\) since any choice is compatible with an equilibrium profile.
$$\begin{aligned} V_1^d(\alpha _n,m_1;{{\bar{\alpha }}}_n^{nn}) < V_1^d(\bar{\alpha }_n^{nn},m_1;{{\bar{\alpha }}}_n^{nn}) = V_1^{\infty }(\bar{\alpha }_n^{nn},m_1) = V_1^{\infty }(\alpha _n,m_1) \end{aligned}$$
×
Figure 6 illustrates the dynamics emerging under the different types of equilibria that coexist for an intermediate level of the market introduction costs. In particular, the trajectories of the market attractiveness (\(\alpha _n\)) and of the investments of both firms (\(u_1, u_2\)) are depicted for a small initial value of the market attractiveness. The dotted lines corresponding to the now or never equilibrium show that in such an equilibrium, due to absence of any investments the attractiveness of the new product decreases toward zero and the product is never introduced. The dashed and the solid lines correspond to the dynamics under the (forced switch) delay equilibrium with the minimal possible threshold, i.e., \({{\bar{\alpha }}}_n^d = {\underline{\alpha }}_n^d\), and the maximum delay equilibrium. The points in time at which the new product is introduced under the forced switch delay and the maximum delay equilibrium are denoted by \(\tau ^d\) and \(\tau ^{md}\), where \(\tau ^d < \tau ^{md}\). The figure illustrates that under both equilibria the state \(\alpha _n\) increases over time. In the time interval \([0,\tau ^d]\), the increase is slower under the delay equilibrium than under the maximum delay equilibrium, because the investments of firm 2, which decrease the speed of growth of \(\alpha _n\), are larger under the delay than under the maximum delay equilibrium. The investments of firm 1, fostering the growth of \(\alpha _n\), are virtually identical under both equilibria. Intuitively, firm 2 invests more under the forced switch delay equilibrium, because the downward jump of its instantaneous profit associated with the switch to mode \(m_2\) is much closer timewise in the forced switch delay equilibrium than in the maximum delay equilibrium and therefore less heavily discounted. Hence, the incentive to invest in delaying the switch is stronger in the forced switch delay equilibrium. For firm 1, no such effect occurs and the incentive to invest during mode \(m_1\) is hardly affected by the type of the delay equilibrium. Under the forced switch delay equilibrium, both controls exhibit a jump at \(\tau ^d\) with the investment of firm 1 jumping upward and those of firm 2 jumping downward. As a result, \(\alpha _n\) grows faster after \(\tau ^d\) compared to the time before the new product introduction and we observe a higher attractiveness of the new market under the forced switch delay equilibrium than under the maximum delay equilibrium. It should, however, be noticed that under both equilibria the state \(\alpha _n\) converges to the steady state in mode \(m_2\) (\(\alpha _n^{*,m_2})\) and therefore this difference between the two equilibria disappears in the long run. Finally, Fig. 6b illustrates again that in the maximum delay equilibrium the investment of firm 1 is continuous throughout the entire trajectory, i.e., also at period \(\tau ^{md}\) at which the mode switches from \(m_1\) to \(m_2\).
×
4.2.3 Large Costs of Market Introduction
If market introduction costs are large, i.e., \(\kappa > {{\bar{\kappa }}}\), then we have \(V_1^{md}(0,m_1) < V_1^{\infty }(0,m_1)\), which means that any candidate for a delay equilibrium yields a smaller value for firm 1 at \(\alpha _n(0)=0\) than not investing in the buildup of \(\alpha _n\) and never introducing the new product. We illustrate the value functions of both firms for this case in Fig. 7. Due to \(V_1^{md}(0,m_1) < V_1^{\infty }(0,m_1)\), there can be no equilibria where for the initial condition \(\alpha _n(0)=0\) firm 1 invests in the buildup of \(\alpha _n\) and eventually introduces the new product. Using the same arguments as developed in the previous subsection, this rules out the existence of any MPE under which the product is introduced with delay for some initial value \(\alpha _n(0)\). Hence, the only equilibrium that still exists in this scenario is the now or never equilibrium, where firm 1 introduces the new product immediately for \(\alpha _n(0) \ge \bar{\alpha }_n^{nn}\) and never introduces it for \(\alpha _n(0) < \bar{\alpha }_n^{nn}\). The value functions of both firms corresponding to this equilibrium are indicated in bold in Fig. 7. It should be noted that under this equilibrium the value function for firm 1 has a kink at \({{\bar{\alpha }}}_n^{nn}\) (panel (a)), while the value of the game for firm 2 exhibits a downward jump as the initial state \(\alpha _n(0)\) crosses the threshold \({{\bar{\alpha }}}_n^{nn}\) (panel (b)).
5 Discussion and Conclusions
In this paper, we study Markov perfect equilibria in twoplayer multimode differential games where both players can influence the dynamics of the state variable and one player controls the timing of the mode switches. We identify different types of equilibria which can occur in such games and our analysis of a simple model of dynamic competition under potential new product introduction illustrates the properties of these different types of equilibria. In particular, we show that forced switch delay equilibria, maximum delay equilibria and now or never equilibria might coexist for some parameter constellations. More precisely, we have demonstrated that the following three scenarios arise: In scenario (iii) for each initial condition, a unique prediction about the occurrence and timing of the new product introduction can be made if we restrict attention to the MPE analyzed here. This is not true in the first two cases. In case (ii), there is a set of initial conditions \(\alpha _n(0) \in [0, {{\bar{\alpha }}}_n^{nn}]\) such that MPEs under which the new product is eventually introduced coexist with the now or never equilibrium under which no introduction of the new product occurs. For \(\alpha _n(0) > \bar{\alpha }_n^{nn}\), the new product is introduced under all existing equilibria although the time of the introduction varies across equilibria. Similarly, under scenario (i) the product is eventually introduced for all initial conditions, where the time of the introduction depends on the chosen equilibrium.
(i)
\(\kappa \le {\underline{\kappa }}\): There exists a maximum delay equilibrium with product introduction at \(\alpha = \alpha _n^{md}\) and a continuous investment path of player 1 across both modes of the game. Furthermore, for any \({{\bar{\alpha }}}_n^d \in [0,\alpha _n^{md})\) there exists a forced switch delay equilibrium with new product introduction at \({{\bar{\alpha }}}_n^d\). In each of these equilibria, the investments of both players jump as the mode switches from \(m_1\) to \(m_2\).
(ii)
\({\underline{\kappa }} < \kappa \le {\bar{\kappa }}\): There exists a maximum delay equilibrium plus for each \({{\bar{\alpha }}}_n^d \in [{{\underline{\alpha }}}_n^{d}, {{\bar{\alpha }}}_n^{md})\) there is also a forced switch delay equilibrium. Furthermore, in this case also a now or never MPE exists.
(iii)
\(\kappa > {\bar{\kappa }}\): Whereas strategy profiles inducing market introduction with delay for some initial value of \(\alpha _n\) do not constitute a MPE, the now or never equilibrium still exists.
Whereas all equilibria considered here are Markov perfect, intuitively the forced switch delay equilibria seem less plausible compared to the maximum delay and the now or never equilibrium. As discussed above, the equilibrium profiles underlying these equilibria include strategies for firm 2, under which in mode \(m_1\) for \(\alpha _n \ge {{\bar{\alpha }}}_n^d\) investments are chosen which would not be optimal if the game would stay in mode \(m_1\) for an amount of time with positive measure even after the state \(\alpha _n\) has crossed the threshold \({{\bar{\alpha }}}_n^d\). Since in equilibrium the state is never larger or equal than \({{\bar{\alpha }}}_n^d\) in mode \(m_1\) for a positive amount of time, this feature does not contradict the optimality of the strategy of firm 2. However, it nevertheless constitutes an ‘incredible threat’ in the sense that it would not be optimal for firm 2 to stick to this investment strategy, in case firm 1 would deviate from its own equilibrium strategy by increasing the threshold at which it switches to mode \(m_2\).
A main insight of our analysis is that a strategy profile might be Markov perfect even though the strategy of some players induces actions in some parts of the state space which would not be optimal for the player if carried out for a positive amount of time. Although this phenomenon has been demonstrated in this paper only for multimode games, it seems that it might arise also in other classes of differential games. For example, in games in which the state dynamics is controlled by potentially singular controls of both players MPEs might exist in which the strategy of one player in a certain region of the state space make it optimal for some other player to induce a jump of the state out of that region by means of a singular control. Similar to our setup, Markov perfection does not put any restrictions on the regular control of that player in the region in which the other player chooses a singular control.^{13} Intuitively, under such a MPE profile the state always immediately jumps out of that singular control region such that the amount of time for which the actions induced by the players’ strategies in that region are actually implemented has measure zero. Hence, also in the framework of such differential games incredible threats, in the sense that actions are chosen which would be suboptimal if implemented for a positive amount of time, can occur as part of Markov perfect equilibrium profiles. An interesting question for future research might be to explore whether such Markov perfect equilibria with ‘incredible threats’ in multimode or singular control games can be eliminated by appropriate equilibrium refinements. Developing such a refinement would also provide a valuable theoretical basis for alleviating the problem of nonuniqueness of predictions about equilibrium timing of regime switches in applications like the dynamic innovation model considered in this paper.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.