main-content

Swipe to navigate through the articles of this issue

Published in:

Open Access 07-12-2021

Markov Perfect Equilibria in Multi-Mode Differential Games with Endogenous Timing of Mode Transitions

Authors: Herbert Dawid, Serhat Gezer

Published in: Dynamic Games and Applications | Issue 2/2022

print
PRINT
insite
SEARCH

Abstract

We study Markov perfect equilibria (MPE) of two-player multi-mode differential games with controlled state dynamics, where one player controls the transition between modes. Different types of MPE are characterized distinguishing between delay equilibria, inducing for some initial conditions mode switches after a positive finite delay, and now or never equilibria, under which, depending on the initial condition, a mode switch occurs immediately or never. These results are applied to analyze the MPE of a game capturing the dynamic interaction between two incumbent firms among which one has to decide when to extend its product range by introducing a new product. The market appeal of the new product can be influenced over time by both firms through costly investments. Under a wide range of market introduction costs a now or never equilibrium coexists with a continuum of delay equilibria, each of them inducing a different time of product introduction.
Disclaimer
The authors gratefully acknowledge very helpful comments from two anonymous referees. Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—SFB 1283/2 2021—317210226.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

1 Introduction

The main agenda of this paper are to improve our understanding of strategic effects arising in dynamic economic and managerial settings characterized by potential structural breaks, which induce jumps in the payoff functions of the economic actors, in the law governing the dynamics of relevant state variables, or both. Areas of application where such structural breaks, which we will refer to as mode changes, arise include environmental economics dealing with potential catastrophic transitions (e.g., [8]), financial portfolio and real investment problems incorporating potential crashes and macroeconomic regime shifts (e.g., [7, 13]) or analyses of innovation dynamics capturing abrupt changes in the market structure due to the adoption of new technologies or the introduction of new products by some market participants ([1, 3, 4]). Whereas all these different problems can be formulated as multi-mode models, crucial differences arise with respect to the way mode transitions are triggered. In particular, the timing and the type of mode transition might be deterministic or stochastic and might be purely exogenous or directly, respectively, indirectly (through the state dynamics) controlled by some economic agent(s).
In this paper, we focus on multi-mode settings with deterministic, controlled mode transitions and strategic interaction. In particular, we are interested in characterizing Markov perfect equilibria (MPE) in multi-mode differential games with a finite number of continuously evolving states, the dynamics of which are controlled by the actions of all players, and a set of modes, where the time of the transition between the modes is determined by one of the players; for easier exposition we assume this is player 1. Problems of this kind arise, for example, in dynamic competition models, like capital accumulation games or dynamic models of reputation formation, where one of the competitors through the introduction of new products or technologies to the market can change the demand structure. Whereas the literature on MPE in timing games with stochastic un-controlled state dynamics is large and well established (see, e.g., [9, 10, 17]), there is considerably less work dealing with situations with controlled mode transitions where the players can also influence the state dynamics. Intuitively, characterizing MPE in such a setting generates intricate strategic effects. The Markovian strategy determining the time of the mode switch induces a split of the state space in a given mode in regions with and without an immediate jump to an alternative mode.1 In a setting where all players through their controls can influence the state dynamics, this implies that each player might influence the timing of the mode switch and this has to be taken into account when determining the optimal strategies. At the same time, player 1, when determining the optimal mode switching strategy, has to take into account the state dynamics under the equilibrium strategies of all players.
Few contributions have addressed these issues. Reddy et al. [15] and Gromov and Gromova [6] study multi-mode games with controlled state dynamics, where, however, the mode transition is not directly controlled by players but determined by state or time constraints. Hence, players can influence the timing of the mode transition only indirectly by steering the state variable through their control. In contrast to these papers, we consider a scenario where the timing of the mode transition is directly controlled by one of the two players and additionally both players can influence the dynamics of a continuously evolving state variable. Closest to our paper is Long et al. [14], where in a differential game model with multiple regimes, the concept of piecewise-closed-loop Nash equilibria (PCNE) is introduced. They consider a two-player multi-mode differential where both players can induce a change of the regime of the game and study piecewise-closed-loop Nash equilibria (PCNE). Under this equilibrium concept the state at which a player carries out a mode switch is derived from the condition, that it is optimal for the corresponding player to switch at that point, and the timing of the mode switches is determined as the point in time when the state variable under the equilibrium controls arrives at that switching state. However, in their setting, it is assumed that firms commit to their switching time in the sense, that they do not alter that time even if the other firm would deviate from its equilibrium control path and hence also the state variable would deviate from its equilibrium path. Therefore, the considered equilibrium is not fully Markov perfect with respect to the timing decision. Also in Dawid and Gezer [2], a multi-mode differential game with controlled state dynamics and mode switches is considered, but again the strategic interactions arising under Markovian strategies are not fully captured. In particular, the authors consider a strategy space where the players’ controls influencing the state dynamics are determined by Markovian strategies, whereas the time of the regime switch is determined with full commitment at time zero and hence can be seen as following an open-loop strategy. To our best knowledge so far, no characterizations of full MPE in a setting similar to that considered in this paper are available in the literature.
The innovation of this paper relative to the literature discussed above is that we characterize a (simultaneous move) Nash equilibrium of multi-mode differential games where both the controls of the players steering the state variable and the decision, whether a mode switch occurs at given point in time, are determined by Markovian strategies and therefore depend solely on the current value of the state and the mode. Put differently, in contrast to the existing literature, we do not assume any kind of commitment of player 1 with respect to the timing of a mode switch. Since we consider an infinite horizon problem, where objective functions and state dynamics do not explicitly depend on time, these equilibrium profiles are also Markov Perfect. The equilibrium strategy profiles therefore correspond to Nash equilibrium behavior for any combination of values of the state and mode and this in particular implies, that, whenever player 1 triggers a mode switch, this change is (intertemporally) optimal for player 1 in light of the current state.
In light of the complexity associated with the characterization of the interplay of Markovian strategies determining controls that steer the state variable and such that determine the timing of mode switches, in this paper we restrict attention to a relative simple setting with two players, two modes and a one-dimensional state space in order to obtain first insights into the structure of MPE in multi-mode games with controlled states and mode switches. This is done mainly for reasons of tractability and easier exposition. Moreover, we believe that the main strategic effects occurring in such a setting and the types of equilibria arising can already be seen in such a relatively simple setup. We derive a set of sufficient conditions for a strategy profile to be a MPE of the game and, based on these conditions, identify different possible types of equilibria. There are delay equilibria, with the property that for some initial conditions the mode switch occurs after some finite delay, and now or never equilibria, under which the state space is partitioned in two areas with the property that for initial conditions in one area the game stays in the first mode forever, whereas for initial conditions in the other area player 1 already at $$t=0$$ induces the switch to the second mode. Furthermore, we show that among the delay equilibria only for a special class of MPE, labeled as maximum delay equilibria, a standard smooth pasting condition holds for player 1. Such equilibria always exist if the game has any delay equilibria, but generically in addition to a maximum delay equilibrium there also exists a continuum of forced switch delay equilibria in which the mode transition is triggered earlier compared to the maximum delay equilibrium. For these equilibria, no smooth pasting conditions hold at the mode switching threshold. Intuitively, the mode switch by player 1 is triggered by the fact that, if the state would cross the switching threshold, while the game is in the initial mode, the action of player 2 would jump to an action with strong adversarial effects for player 1. Since under such an equilibrium strategy profile the game never stays for a positive amount of time at a state above the threshold in the first mode, any action of player 2 in this part of the state space and the first mode can be supported in a MPE. These arguments show that in the considered settings the fact that a strategy profile constitutes a Markov perfect equilibrium does not prevent the occurrence of what could be described as ‘incredible threats’ in the sense that a player’s strategy prescribes an action which would be suboptimal for the player to implement if forced to do so for a time interval with positive measure.
We illustrate these findings by studying the optimal timing of new product introduction for a producer (firm 1) facing on an established market a competitor (firm 2), which has not developed a new product yet and therefore does not have the option of a new product introduction. The new product introduction corresponds to a mode change, and the state variable of the considered differential game is the appeal of the new product with consumers, which is influenced by the two firms through advertising (firm 1) and negative campaigning (firm 2).2 Furthermore, the market introduction of the new product is associated with lump-sum costs. We show that for a wide range of values of the market introduction costs all different types of equilibria mentioned above coexist, and therefore, for a substantial subset of initial values of the new product’s market appeal no clear cut prediction about the time of market introduction of the new product is possible. Furthermore, we determine a threshold such that for values of the market introduction costs above this value only now or never equilibria exist. Overall, this analysis highlights that multi-mode timing games of this kind almost generically give rise to multiplicity of Markov perfect equilibria. A potential implication of this insight is that more refined equilibrium concepts than MPE should be considered to analyze such games.
The remainder of the paper is organized as follows. In Sect. 2, we introduce the type of multi-mode games we are considering, and in Sect. 3, we derive necessary conditions for MPE in such a setting and also some additional results on equilibrium profiles. In Sect. 4, we study the optimal timing of new product introduction, thereby illustrating our general findings. A discussion of our results and conclusions is provided in Sect. 5. ‘Appendix’ contains all proofs and some additional analysis.

2 The Model

We consider a differential game between two players $$i=1,2$$, in which each player intends to maximize an infinite horizon discounted payoff stream of the form
\begin{aligned} J_i = \int _{0}^{\infty } e^{-r t} F_i(x(t), u(t), m(t)) dt - \mathbb {1}_{[i=1]} e^{-r \tau } \kappa , \end{aligned}
(1)
where $$x \in {{\mathbb {R}}}$$ is a one-dimensional state and $$m(t) \in \{m_1,m_2\}$$ is the mode of the game. The interval $$X=[x_l,x_u]$$ with $$x_l < x_u$$ is the state-space of the game.3 Here, $$\mathbb {1}$$ denotes the indicator function and $$\tau = \inf \{t \ge 0 \vert m(t) = m_2\}$$ is the point in time in which the mode process moves from $$m_1$$ to $$m_2$$. In case no transition to $$m_2$$ occurs, we set $$\tau = \infty$$. At time $$\tau$$, transition costs of $$\kappa \ge 0$$ arise for player 1. The vector $$u = (u_1,u_2)$$ denotes the controls of both players with $$u_i \in {{{\mathcal {U}}}}_i \subseteq {{\mathbb {R}}}^n$$. We assume that $$F_i(x,u,m)$$ is continuous and differentiable with respect to x and u for each $$m \in \{m_1,m_2 \}$$. The state evolves according to
\begin{aligned} \dot{x} = f(x,u,m(t)), \quad x(0) = x_{ini} \in X \end{aligned}
(2)
where f(xum) is Lipschitz continuous and differentiable with respect to x and u for all $$m \in \{m_1,m_2\}$$. The mode process m(t) initially is in $$m(0)=m_1$$ and is controlled by player 1. More precisely, player 1 can determine at which point in time the process jumps from $$m_1$$ to $$m_2$$. Once $$m(t) = m_2$$, no additional mode transitions are possible.4
In what follows, we restrict attention to time-invariant Markovian strategies. More precisely, we consider strategy profiles $$((\Phi _1(x,m),\Psi ),\Phi _2(x,m))$$ with $$\Phi _i: X \times \{m_1,m_2\} \rightarrow {{\mathbb {R}}}^{n_i}$$ and $$\Psi \subseteq X$$ such that $$u_i(t) = \Phi _i(x(t), m(t)), i=1,2$$ for all t and the process $$m(\cdot )$$ jumps from $$m_1$$ to $$m_2$$ at t if and only if $$m(t) = m_1$$ and $$x(t) \in \Psi$$. Intuitively, the switching set $$\Psi$$, which is chosen by player 1, determines the set of states at which player 1 induces a switch from $$m_1$$ to $$m_2$$ and hence represents a Markovian strategy of that player.5 A Markov perfect equilibrium of the (simultaneous move) differential game is a strategy profile such that each player solves the dynamic optimization problem defined by (1) and (2) given that the other player determines her control using her equilibrium strategy.

3 Markov Perfect Equilibria

In this section, we derive necessary conditions to be satisfied by a Markov Perfect equilibrium profile of the problem described in Sect. 2. The fact that one of the players controls the transition from mode $$m_1$$ to $$m_2$$ implies that, in addition to standard conditions characterizing the value functions and optimal strategies in each of the two modes, the effect of the endogenous timing of the mode transition has to be taken into account. Intuitively, this means that on the one hand the time of the mode transition has to be optimal for player 1, thereby fulfilling standard conditions for optimal stopping problems. On the other hand, for a given mode switching strategy $$\Psi$$ of player 1, the opponent player 2 can also influence the dynamics of x, and thereby the time of the mode switch, by choosing her controls in mode $$m_1$$. The interplay of these effects has to be taken into account when characterizing a MPE profile. In what follows, we focus on equilibria in which the mode switching set $$\Psi$$ is of threshold type, i.e., it has the form $$\Psi = [{{\bar{x}}}, x_u]$$, where player 1 chooses the value of $${{\bar{x}}}$$.6 We denote such a threshold strategy characterized by $$\bar{x}$$ as $$\Psi _{{{\bar{x}}}}$$. The following set of sufficient conditions characterizes MPE of such threshold type. (The proof of this and all following propositions is given in ‘Appendix’ A.)
Proposition 1
Consider a multi-mode differential game described in Sect. 2. If there exists a set of except at $$({{\bar{x}}}, m_1)$$ everywhere continuous and continuously differentiable value functions $$V_i: X \times \{m_1,m_2\} \rightarrow {{\mathbb {R}}}$$ and a profile of almost everywhere continuous Markovian strategies $$((\Phi _1(x,m),\Psi _{{{\bar{x}}}}),\Phi _2(x,m))$$ such that the conditions below are satisfied for $$i=1,2$$, then this profile constitutes to a Markov perfect equilibrium of the game:
(i)
\begin{aligned} \begin{array}{ll} V_i(x,m_2) = \frac{1}{r} \Big ( F_i(x,(\Phi _1(x,m_2),\Phi _2(x,m_2)),m_2) + &{} \\ + \frac{\partial V_i(x,m_2)}{\partial x} f(x,(\Phi _1(x,m_2),\Phi _2(x,m_2)),m_2) \Big ) \quad &{} x \in X, \end{array} \end{aligned}

(ii)
\begin{aligned} \begin{array}{ll} \Phi _i(x,m_2) \in argmax_{u_i \in {{{\mathcal {U}}}}_i} \Big ( F_i(x,(u_i,\Phi _j(x,m_2)),m_2) + &{} \\ + \frac{\partial V_i(x,m_2)}{\partial x} f(x,(u_i,\Phi _j(x,m_2)),m_2) \Big ) \quad &{} x \in X, j\not =i, \end{array} \end{aligned}

(iii)
\begin{aligned} \lim \sup _{t\rightarrow \infty } e^{-rt} V_i(x,m_2) \le 0, i=1,2 \end{aligned}

(iv)
\begin{aligned} V_i(x,m_1) = \left\{ \begin{array}{ll} \frac{1}{r} \Big ( F_i(x,(\Phi _1(x,m_1),\Phi _2(x,m_1)),m_1) +&{}\\ + \frac{\partial V_i(x,m_1)}{\partial x} f(x,(\Phi _1(x,m_1),\Phi _2(x,m_1)),m_1) \Big ) \quad &{} x < {{\bar{x}}},\\ V_i(x,m_2) - \mathbb {1}_{[i=1]} \kappa &{} x \ge {{\bar{x}}}, \end{array} \right. \end{aligned}

(v)
\begin{aligned} \begin{array}{ll} \lim _{x \rightarrow {{\bar{x}}}-} V_1(x,m_1) &{}= V_1({{\bar{x}}},m_1)\\ \lim _{x \rightarrow {{\bar{x}}}-} V_2(x,m_1) &{}\ge V_2({{\bar{x}}},m_1), \end{array} \end{aligned}
where the inequality for player 2 has to hold as equality if there exists an $$\epsilon > 0$$ such that $$f(x,(\Phi _1(x,m_1),\Phi _2(x,m_1)),m_1) > 0$$ for all $$x \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$.

(vi)
\begin{aligned} V_1(x,m_2) -\kappa< V_1(x,m_1), \quad \forall x < {{\bar{x}}} \\ \end{aligned}

(vii)
\begin{aligned} \begin{array}{ll} r (V_1(x,m_2) -\kappa ) &{}> \max _{u_1 \in {{{\mathcal {U}}}}_1} \Big [F_1(x, (u_1, \Phi _2(x,m_1)),m_1) +\\ &{} + \frac{\partial V_1(x,m_2)}{\partial x} f(x,(u_1, \Phi _2(x,m_1)),m_1) \Big ], \quad \forall x > {{\bar{x}}} \end{array} \end{aligned}

(viii)
\begin{aligned} \Phi _i(x,m_1) \left\{ \begin{array}{ll} \in argmax_{u_i \in {{{\mathcal {U}}}}_i} \Big ( F_i(x,(u_i,\Phi _j(x,m_1)),m_1) +&{}\\ + \frac{\partial V_i(x,m_1)}{\partial x} f(x,(u_i,\Phi _j(x,m_1)),m_1) \Big ) \quad &{} x < {{\bar{x}}}, \ j \not = i,\\ = {{\tilde{\Phi }}}_i(x) &{} x \ge {{\bar{x}}} \end{array} \right. \end{aligned}
for some functions $${{\tilde{\Phi }}}_i: [{{\bar{x}}}, x_u] \rightarrow {\mathbb R}^{n_i}, \ i=1,2$$.

In order to interpret the conditions listed in the proposition, we first observe that conditions (i) and (ii) are standard conditions for a MPE of the game in mode $$m_2$$. Since for any value of the state x above the threshold $${\bar{x}}$$ player 1 immediately switches to mode $$m_2$$, the value functions in mode $$m_1$$ in this part of the state space coincide with that in mode $$m_2$$ net of the costs associated with the switch from $$m_1$$ to $$m_2$$. For values of x below $${{\bar{x}}}$$, the value function and the equilibrium strategies are characterized by standard Hamilton–Jacobi–Bellman (HJB) equations (see (iv) and (viii)). The boundary condition for these HJB equations is given by the value matching condition in (v) which guarantees that the value function of player 1 is continuous at the threshold $${{\bar{x}}}$$, whereas the value function of player 2 might exhibit a jump at the threshold $${{\bar{x}}}$$ at which player 1 switches to mode $$m_2$$. However, as we will illustrate below, a non-continuous value function for player 2 can arise only if the state dynamics in equilibrium is such that the switch to mode $$m_2$$ occurs either immediately or never, depending on the initial state, i.e., if the dynamics in mode $$m_1$$ leads the state away from the threshold $${{\bar{x}}}$$ (cf. condition (v)). The optimality of the threshold $${{\bar{x}}}$$ from the perspective of player 1 is guaranteed by conditions (vi) and (vii), where condition (vi) ensures that it is not optimal for player 1 to switch to mode $$m_2$$ at any $$x \in [x_l,{{\bar{x}}})$$ and (vii) guarantees that for any $$x > {{\bar{x}}}$$ it is optimal for player 1 to switch immediately to mode $$m_2$$ rather than remaining in mode $$m_1$$.
Finally, we like to point out that the sufficient conditions for an MPE, provided in Proposition 1, impose hardly any restrictions on the strategies $$\phi _i(x,m_1)$$ for $$x > {{\bar{x}}}$$. This implies that in spite of the fact that we consider Markov perfect equilibria, which constitute equilibria of every subgame defined by the current state and mode, the players might use strategies which for $$x > {{\bar{x}}}$$ in mode $$m_1$$ induce actions that would not be optimal for that player if implemented for a positive amount of time. This is still optimal because for these states the game immediately switches to mode $$m_2$$. We will discuss this issue in more detail below and illustrate that this feature might give rise to a wide range of coexisting equilibria.
For our further discussion, it is helpful to distinguish different types of equilibria that can arise in our setting. The main property to be considered is whether the state dynamics under the equilibrium strategies $$(\Phi _1(x,m_1), \Phi _2(x,m_1))$$ for $$x < {{\bar{x}}}$$ in the neighborhood of $${{\bar{x}}}$$ points toward the threshold $${{\bar{x}}}$$ , i.e., whether $$f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) > 0$$ for $$x \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$. If this condition holds, then for $$x^{ini} \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$ the game switches to $$m_2$$ with some positive delay. We refer to such equilibria as delay equilibria.7 On the other hand, if $$f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) < 0$$ for all $$x \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$, then the game either switches immediately to mode $$m_2$$, for $$x(0) \ge {{\bar{x}}}$$ or remains in mode $$m_1$$ forever (for $$x(0) < {{\bar{x}}}$$). We denote such equilibria as now or never equilibria. The following definition puts these notions more formally.
Definition 1
Consider a MPE strategy profile $${{{\mathcal {P}}}} = ((\Phi _1(.),\Psi _{\bar{x}}), \Phi _2(.))$$ satisfying the conditions of Proposition 1 and denote by $$\tau (x^{ini}) = \inf \{t \ge 0: \tilde{x}(t) \ge {{\bar{x}}}\}$$ with $${{\tilde{x}}}(0)=x^{ini}$$ and $$\dot{ {{\tilde{x}}}} = f({{\tilde{x}}}, (\Phi _1({{\tilde{x}}},m_1), \Phi _2({{\tilde{x}}},m_1)),m_1)$$ and $$\tau (x^{ini}) = \infty$$ if $${{\tilde{x}}}(t) < {{\bar{x}}} \ \forall t$$. Then,
• the MPE profile $${{{\mathcal {P}}}}$$ is called a now or never equilibrium if for every initial state $$x^{ini} \in X$$ either $$\tau (x^{ini}) = 0$$ or $$\tau (x^{ini}) = \infty$$.
• the MPE profile $${{{\mathcal {P}}}}$$ is called a delay equilibrium if there exist initial states $$x^{ini} \in X$$ such that $$0< \tau (x^{ini}) < \infty$$.
• a delay equilibrium profile $${{{\mathcal {P}}}} = ((\Phi _1(.),\Psi _{\bar{x}}), \Phi _2(.))$$ is called a forced switch delay equilibrium if there is a small $$\epsilon > 0$$ such that for any alternative strategy $${{\tilde{\Phi }}}_2(.)$$ of player 2 with $${{\tilde{\Phi }}}_2(x,m_1) = \Phi _2(x,m_1) \ \forall x \not \in [{{\bar{x}}}, {{\bar{x}}}+\epsilon ), \ {{\tilde{\Phi }}}_2(x,m_2) = \Phi _2(x,m_2) \ \forall x \in X$$ and $$\tilde{\Phi }_2(x,m_1)$$ continuous and continuously differentiable on $$(\bar{x} - \epsilon , {{\bar{x}}} + \epsilon )$$ there exists a strategy $$(\tilde{\Phi }_1(.),\Psi _{{{\tilde{x}}}})$$ with $${{\tilde{x}}} > {{\bar{x}}}$$ which is a better response for player 1 to $${{\tilde{\Phi }}}_2(.)$$ than $$(\Phi _1(.),\Psi _{{{\bar{x}}}})$$.
• a delay equilibrium which is not a forced switch delay equilibrium is called maximum delay equilibrium.
Intuitively, a forced switch delay equilibrium has the property that the switch of player 1 to mode $$m_2$$ at the state $${{\bar{x}}}$$ is triggered by the threat of a jump in the control player 2 would implement at $${{\bar{x}}}$$ if the mode would stay in $$m_1$$.8 Formally in Definition 1, this is expressed by the fact that under a forced switch delay equilibrium, any continuous extension of player 2’s strategy $$\Phi _2(.,m_1)$$ beyond $${{\bar{x}}}$$ would make it a best response for player 1 to switch to mode $$m_2$$ at some threshold above $${{\bar{x}}}$$. Delay equilibria, where the mode switch by player 1 is not triggered by such a threat, are called maximum delay equilibria. In any delay equilibrium, the fact that player 1 has incentives to delay the switch to mode $$m_2$$ for any $$x \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$ induces that the inequality
\begin{aligned} \begin{array}{ll} F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) &{} > r (V_1(x,m_2) -\kappa ) -\\ &{} - \frac{\partial V_1(x,m_2)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \end{array} \end{aligned}
(3)
holds for $$x \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$. The inequality has the standard interpretation that the left-hand side gives the (flow) profit player 1 receives in mode $$m_1$$ from marginally delaying the mode switch and the right-hand side the associated (discounted) loss in value received in $$m_2$$. In general, (3) might hold as a strict inequality even in the limit as x converges to $${{\bar{x}}}$$, in particular if the strategy $$\Phi _2(.,m_1)$$ of player 2 exhibits a jump at $${{\bar{x}}}$$. In an equilibrium with this property, player 1 has strict incentives to delay the switch to mode $$m_2$$ for any value of the state x up to $${{\bar{x}}}$$, and the switch to $$m_2$$ at $${{\bar{x}}}$$ is then triggered by the jump the action of player 2 would exhibit if the game would remain in mode $$m_1$$. From the perspective of player 2, the jump in $$\Phi _2(.,m_1)$$ at $${{\bar{x}}}$$ in such a setting, however, is optimal only because player 1 switches the mode to $$m_2$$ for any $$x \ge {{\bar{x}}}$$ such that player 2 is never required to implement the actions induced by $$\Phi _2(x,m_1), x \ge {{\bar{x}}}$$ for a positive amount of time. In such a situation, if player 2 would move the jump of its control from $${{\bar{x}}}$$ to $${{\bar{x}}} + \epsilon$$, this would also induce a change in the optimal switching strategy of player 1 in the sense that the optimal mode switching threshold for player 1 would move upward. These are the forced switch delay equilibria. If an equilibrium has the maximum delay property, and therefore the choice of $${{\bar{x}}}$$ by player 1 is not triggered by the threat of a jump of the action of player 2, standard optimal stopping arguments imply that player 1 delays the switch until a value of x is reached where (3) holds as equality. In such equilibria, also a standard smooth pasting condition for player 1 holds in the sense that the slope of the value function of player 1 at $$x = {{\bar{x}}}$$ in mode $$m_1$$ coincides with the slope in mode $$m_2$$. In the following lemma, we state these observations more formally.
Lemma 1
If $${{{\mathcal {P}}}} = ((\Phi _1(x,m),\Psi _{{{\bar{x}}}}),\Phi _2(x,m))$$ is a maximum delay equilibrium profile, then
\begin{aligned}&\lim _{x \rightarrow {{\bar{x}}}-} \left[ F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) + \frac{\partial V_1(x,m_2)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \right] \nonumber \\&\quad = r (V_1({{\bar{x}}},m_2) -\kappa ) \end{aligned}
(4)
and
\begin{aligned} \lim _{x \rightarrow {{\bar{x}}}-} \frac{\partial V_1(x,m_1)}{\partial x} = \frac{\partial V_1({{\bar{x}}},m_2)}{\partial x}. \end{aligned}
(5)
In the remainder of this section, we discuss the implications of the different types of equilibria for the continuity of the players’ controls over time. First, we observe that in case of now or never equilibria with continuous strategies $$\Phi _i(.,m_1)$$ and $$\Phi _i(.,m_2)$$ the trajectories of the actions of both players are continuous over time, since no switch from $$m_1$$ to $$m_2$$ occurs for $$t > 0$$. Hence, we focus here on delay equilibria and the question whether controls are continuous at the point of time when the mode is changed by player 1. Given that we consider deterministic models, where for a given strategy profile both player can perfectly predict the time of the switch to mode $$m_1$$, intuitively one could expect that in equilibrium controls are continuous as long as the marginal effects of the controls on the players instantaneous profits and on the state dynamics is the same in both modes. As we show below, this intuition holds true for player 1 as long as we consider a maximum delay equilibrium where the smooth pasting condition holds for player 1 at the time of the mode switch and the optimal action of player 1 is not affected by potential jumps of the control of player 2. To show this formally, we introduce the following notions.
Definition 2
A differential game of the structure described in Sect. 2 has
• mode independent control effects if and only if
\begin{aligned} \frac{\partial F_i(\cdot ,m_1)}{\partial u_i} =\frac{\partial F_i(\cdot ,m_2)}{\partial u_i} \quad \text{ and } \quad \frac{\partial f(\cdot ,m_1)}{\partial u_i} =\frac{\partial f(\cdot ,m_2)}{\partial u_i}, \ i=1,2. \end{aligned}
(6)
• separable control effects if
\begin{aligned} \frac{\partial F_i(x,(u_i,u_j),m)}{\partial u_i} \quad \text{ and } \quad \frac{\partial f(x,(u_i,u_j),m)}{\partial u_i} \end{aligned}
do not depend on $$u_j$$ for all $$x \in X$$ and $$m \in \{m_1,m_2\}$$.
For twice differentiable functions $$F_i$$ and f, separable control effects are equivalent to the condition that the cross-derivatives of these functions with respect to the two controls vanish. In many types of games, including, for example, different variants of investment games, direct effect of the control of one player on the incentives of the other player indeed does not exist.
Even if we consider games that have both of these properties, equilibrium profiles, as characterized by Proposition 1, in general might exhibit jumps in the feedback strategies of both players such that $$\lim _{x \rightarrow {{\bar{x}}}-} \Phi _i(x,m_1) \not = \Phi _i({{\bar{x}}},m_2)$$. More precisely, in the case of forced switch delay equilibria, the actions of both players jump at the point in time when the modes switch to $$m_2$$. Concerning the investment of player 2, who cannot directly control the switch from mode $$m_1$$ to $$m_2$$, in general jumps in the control occur as the game moves from mode $$m_1$$ to mode $$m_2$$ in all types of delay equilibria. In equilibrium, player 2 perfectly predicts the time of the mode switch and also the value of its investment in mode $$m_1$$ for her future profits in mode $$m_2$$. Therefore, at first sight the discontinuity of the investment of player 2 might be surprising. The jump in the action of player 2 results from the fact that in mode $$m_1$$ player 2 can influence the time of the switch to mode $$m_2$$, since the choice of her control affects the dynamics of the state and thereby the time the state hits the threshold $${{\bar{x}}}$$ determined by the equilibrium strategy of player 1. This effect, which influences the optimal choice of the control of player 2, immediately disappears, at the point in time when the mode changes to $$m_2$$. Hence, the equilibrium strategy of player 2 in general exhibits a jump at the state $${{\bar{x}}}$$ in any delay equilibrium even if the game exhibits mode independent and separable control effects.
Considering the action of player 1, we have to distinguish between maximal delay equilibria and forced switch delay equilibria. Whereas Proposition 2 shows that the control of player 1 is continuous at the time of the regime switch in maximal delay equilibria, in general this does not hold for delay equilibria not having this property. Intuitively, in such an equilibrium player 1 has strict incentives to delay the jump to mode $$m_2$$ for all values of the state below the threshold $${{\bar{x}}}$$, and, differently to the maximal delay equilibrium, this incentive to delay does not converge to zero as x approaches $${{\bar{x}}}$$. The reason for player 1 to switch to mode $$m_2$$ at $${{\bar{x}}}$$ is that player 2 threatens to discontinuously change its action at $${{\bar{x}}}$$ if the game would stay in mode $$m_1$$. Hence, for a given strategy of player 2, the threshold $${{\bar{x}}}$$ at which player 1 switches to mode $$m_2$$ is determined by the state at which player 2 threatens to change its control. Based on this, player 1 has to adjust its investment in order to influence the duration of the game in mode $$m_1$$. Similarly to what we described for player 2 above, this effect disappears as soon as the mode is $$m_2$$ and hence also the action of player 1 in general exhibits a jump at the mode switch. More formally, in the absence of a smooth pasting condition the state derivative of the value function of player 1 in general exhibits a jump as the mode switches from $$m_1$$ to $$m_2$$. Therefore, the marginal return of investment for player 1 jumps as the game switches from mode $$m_1$$ to $$m_2$$ at $${{\bar{x}}}$$, which induces a jump in the control of player 1. In Sect. 4, we will illustrate these scenarios using a simple example analyzing the optimal timing of new product introduction in a duopoly.
The following proposition formally establishes that under maximum delay equilibria the action of player 1 is continuous over time although the mode changes at some positive t from $$m_1$$ to $$m_2$$.
Proposition 2
If $$((\Phi _1(x,m),\Psi _{{{\bar{x}}}}(x)),\Phi _2(x,m))$$ is a maximum delay equilibrium profile of a game with mode independent and separable control effects such that in each mode m the strategies $$\Phi _i, i=1,2$$ are continuous with respect to the state x and the right-hand side of condition (ii) in Proposition 1 has a unique maximizer at $$x = {{\bar{x}}}$$, then
\begin{aligned} \lim _{x \rightarrow {{\bar{x}}}-} \Phi _1(x,m_1) = \Phi _1({{\bar{x}}},m_2). \end{aligned}
The intuition for this result is straightforward. For the maximal delay equilibrium, the smooth pasting condition holds for player 1 at the threshold $${{\bar{x}}}$$. This ensures that the slope with respect to the state of her value function is identical in modes $$m_1$$ and $$m_2$$ at this point. If the marginal effects of the own control on the instantaneous payoff and the state dynamics are not affected directly by the change in mode (mode independent control effect) and by the potential change in the other player’s control (separable control effect) due to the mode switch, then the optimization problem which player 1 faces at state $${{\bar{x}}}$$ is equivalent, no matter whether the firm is in mode $$m_1$$ or in $$m_2$$. Hence, its equilibrium feedback strategy is continuous at $${{\bar{x}}}$$.

4 An Illustrative Example: Optimal Timing of New Product Introduction

To illustrate our general findings, we now consider the timing problem of a firm, denoted as firm 1, which has to decide when to introduce a new product, that it has developed. We assume that the firm is already active producing an established product and competes with a second firm (firm 2) on the market for the established product. Only firm 1 has developed the new product and therefore has the option to introduce that product at any point in time $$t \ge 0$$.
Before the new product is introduced (mode $$m_1$$), the inverse demand for the established product is given by
\begin{aligned} p_o = \alpha _o - (q_{1o} + q_{2o}), \end{aligned}
where $$p_o$$ denotes the price of established product and $$q_{io}$$ the quantity of that product supplied by firm $$i, \ i=1,2$$. After the introduction of the new product (mode $$m_2$$), the inverse demand changes to
\begin{aligned} p_o(q_{1o}+q_{2o}, q_{1n})&= \alpha _o - (q_{1o} + q_{2o}) - \eta q_{1n} \\ p_n(q_{1o}+q_{2o}, q_{1n})&= \alpha _n^0 + \alpha _n - \eta (q_{1o} + q_{2o}) - q_{1n}. \end{aligned}
Here, $$\alpha _o$$ is the reservation price of the established product, whereas $$\alpha _n^0 + \alpha _n$$ denotes the reservation price of the new product. We write this reservation price as the sum of a constant parameter $$\alpha _n^0$$, giving the minimal value of the reservation price, and the state variable $$\alpha _n(t) \ge 0$$ capturing the effects of the efforts of the two competitors to influence the (inverse) demand for the new product. Firm 1, as the (potential) innovator, tries to increase the willingness to pay for the new product, whereas the competitor firm 2 might invest in reducing this willingness, e.g., by providing information to consumers about negative features of the new product or by negative campaigning. Both firms choose nonnegative effort $$h_i$$, measured e.g., in work hours, to do advertising (firm 1), respectively, negative campaigning (firm 2) for the new product. The unit cost of such effort is denoted by $$c_i > 0, i=1,2$$ ,and we assume that there are decreasing returns to effort in terms of the size of the effect on the inverse demand for the new product. More precisely, we assume that the change in reservation price is given by
\begin{aligned} {{\dot{\alpha }}}_n = \sqrt{2 h_1} - \gamma \sqrt{2 h_2} - \delta \alpha _n, \ m \in \{m_1,m_2\}. \end{aligned}
Here, $$\gamma > 0$$ is a parameter, and $$\delta$$ is the rate by which the effect of firms’ effort on inverse demand vanishes. We assume that the equation governing the dynamics of $$\alpha _n$$ is the same in both modes, which captures a situation where firms might invest effort in influencing the reservation price of the new product also before the product is actually introduced to the market.9 For analytical convenience, we transform the firms’ control variables according to $$u_i = \sqrt{2 h_i}$$ to obtain the following formulation with linear state dynamics
\begin{aligned} {{\dot{\alpha }}}_n = f(\alpha _n,(u_1,u_2),m) := u_1 - \gamma u_2 - \delta \alpha _n, \ m \in \{m_1,m_2\} \end{aligned}
(7)
and quadratic cost functions $$\xi _i(u_i) = \frac{c_i}{2} u_i^2$$. In slight abuse of notation, we will refer to the controls $$u_i$$ as firm effort in the further analysis. Production costs for the established product are assumed to be symmetric across firms and given by $$\frac{\nu _o}{2} q_{io}^2, i=1,2$$ and analogously for the new product firm 1 has production costs $$\frac{\nu _n}{2} q_{1n}^2$$.
Firms maximize profits by choosing the production quantities as well as the effort $$u_i$$ at every point in time t. In addition, firm 1 decides about the time at which the new product is introduced. The introduction of the new product is associated with lump-sum costs of $$\kappa > 0$$.
Since the quantity choice in this setting does not have any intertemporal effects, firms at t choose quantities according to the Cournot equilibrium, which depends on the value of $$\alpha _n(t)$$ for all t after the introduction of the new product. Standard calculations (see ‘Appendix’ B for a derivation of the Cournot equilibrium quantities and profits in both modes) show that before the introduction of the new product we have
\begin{aligned} q_{io}^{m_1} = \frac{\alpha _o}{3 + \nu _o} \quad i=1,2, \end{aligned}
(8)
and the market profit of each firm is given by $$\pi _{io}^{m_1} = \left( 1+\frac{\nu _o}{2} \right) (q_{io}^{m_1})^2$$. Hence, we obtain for the instantaneous profit in mode $$m_1$$
\begin{aligned} F_i(\alpha _n,u_i,m_1) = \left( 1+\frac{\nu _o}{2} \right) (q_{io}^{m_1})^2 - \xi _i(u_i) \quad i=1,2. \end{aligned}
(9)
In order to guarantee that after the introduction of the new product also a positive amount of this good is produced in equilibrium regardless of the value of $$\alpha _n$$, we assume that $$\alpha _n^0 > \frac{3 \eta \alpha _o}{3 + \nu _o}$$. Furthermore, we restrict attention to scenarios where in equilibrium both firms also sell a positive quantity of the established product. This is true as long as $$\alpha _n < \alpha _n^{UB} := \frac{(2 + \nu _n)(1+\nu _o) + \eta ^2}{\eta (3+2\nu _o)} \alpha _o - \alpha _n^0$$ (see ‘Appendix’ B).
For values of $$\alpha _n \in [0, \alpha _n^{UB}]$$, we obtain as the equilibrium quantities in mode $$m_2$$:
\begin{aligned} q_{1o}^{m_2}(\alpha _n)= & {} \frac{\alpha _o ((1+ \nu _o) (2 + \nu _n) + \eta ^2) -\eta (\alpha _n^0+\alpha _n) (3 + 2 \nu _o)}{(1-\eta ^2) (6 + 5 \nu _o) + 3 (\nu _o + \nu _n) + (2+\nu _n) \nu _o^2 + 4 \nu _o \nu _n} \nonumber \\ q_{2o}^{m_2}(\alpha _n)= & {} \frac{\alpha _o ((1+ \nu _o) (2 + \nu _n)-2 \eta ^2) - \eta (\alpha _n^0+\alpha _n) \nu _o }{(1-\eta ^2) (6 + 5 \nu _o) + 3 (\nu _o + \nu _n) + (2+\nu _n) \nu _o^2 + 4 \nu _o \nu _n} \nonumber \\ q_{1n}^{m_2}(\alpha _n)= & {} \frac{(1+\nu _o)((\alpha _n^0+\alpha _n)(3+\nu _o)- 3 \alpha _o \eta )}{(1-\eta ^2) (6 + 5 \nu _o) + 3 (\nu _o + \nu _n) + (2+\nu _n) \nu _o^2 + 4 \nu _o \nu _n} \end{aligned}
(10)
The corresponding instantaneous profit in mode $$m_2$$ is
\begin{aligned} F_1(\alpha _n, u_1, m_2)&= \left( 1 + \frac{\nu _o^2}{2} \right) (q_{1o}^{m_2}(\alpha _n))^2 + \left( 1 + \frac{\nu _n^2}{2} \right) (q_{1n}^{m_2}(\alpha _n))^2\nonumber \\ {}&\quad + 2 \eta q_{1o}^{m_2}(\alpha _n) q_{1n}^{m_2}(\alpha _n) - \xi _1(u_1) \nonumber \\ F_2(\alpha _n, u_2, m_2)&= \left( 1 + \frac{\nu _o^2}{2} \right) (q_{2o}^{m_2}(\alpha _n))^2 - \xi _2(u_2) \end{aligned}
(11)
Overall, the dynamic strategic interaction between the two firms constitutes a two-mode differential game of the form considered in Sect. 2 with the single state $$\alpha _n$$ evolving according to (7) and the instantaneous profits in mode $$m_1$$ given by (9) and those in mode $$m_2$$ given by (11). Both firms decide on their effort $$u_i$$ as a function of the current state and mode and additionally firm 1 determines the timing of the switch from $$m_1$$ to $$m_2$$, which means that firm 1 decides on a threshold $${{\bar{\alpha }}}_n$$ such that it introduces the new product as soon as the reservation price of the new product $$\alpha _n^0 + \alpha _n$$ is larger than $$\alpha _n^0 + {{\bar{\alpha }}}_n$$. Formally, in accordance with Sect. 3 we consider profiles of Markovian strategies of the form $$((\Phi _1(\alpha _n,m), \Psi _{{{\bar{\alpha }}}_n}), \Phi _2(\alpha _n,m))$$ and in what follows characterize Markov perfect equilibria of this game.

4.1 MPE in Mode $$m_2$$

In order to characterize the different types of equilibria in our game, we first consider the final mode $$m_2$$. Since the instantaneous profits given in (11) are quadratic functions of state and controls and the state Eq. (7) is linear, the game in mode $$m_2$$ is of linear quadratic structure. Following a wide range of the literature about such games (see, e.g., [5]), we assume that the game has a MPE with linear feedback strategies10, giving rise to quadratic value functions of both players. The value and feedback functions can then be determined by a guess and verify approach using the first-order conditions and Hamilton–Jacobi–Bellman equations, see ‘Appendix’ C for details.
Figure 1 shows the value functions of both players in the MPE for our benchmark parameter setting.11 The green arrows indicate the direction of the state dynamics under the equilibrium feedback strategies. There is a unique stable steady state $$\alpha _n^{*,m_2}$$, and it is easy to verify that $$\alpha _n^{*,m_2} < \alpha _n^{UB}$$ under our parameter setting such that both firms sell positive quantities of the old product in the steady state. Figure 1 clearly shows that, as expected, the value of firm 1 increases with $$\alpha _n$$, whereas firm 2 is hurt by an increase in the attractiveness of the new product.

4.2 Different Types of MPEs in Mode $$m_1$$

In order to characterize MPE strategies in mode $$m_1$$, we first determine the value functions of both players under the extreme scenarios in which the new product, regardless of the initial condition, is either immediately or never introduced. We denote the value function of firm i in the former case of immediate introduction by $$V_i^0(\alpha _n,m_1)$$. Clearly, we have
\begin{aligned} V_i^0(\alpha _n,m_1) = V_i(\alpha _n, m_2) - \mathbb {1}_{[i=1]} \kappa . \end{aligned}
If the new product is never introduced, positive investment is never optimal for either firm, i.e., $$u_i(t) = 0, \ i=1,2, \ t \ge 0$$, and hence, the value functions read
\begin{aligned} V_i^{\infty } (\alpha _n,m_1) = \left( 1+\frac{\nu _o}{2} \right) \frac{(q_{io}^{m_1})^2}{r}, \ i=1,2. \end{aligned}
Note that $$V_i^{\infty }$$ is constant with respect to $$\alpha _n$$ because the attractiveness of the new product is irrelevant if this product is never introduced. Given that $$V_1^0(\alpha _n,m_1)$$ increases with $$\alpha _n$$, it follows that $$V_1^{0} (0,m_1) > V_1^{\infty } (0,m_1)$$ implies $$V_1^{0} (\alpha _n,m_1) > V_1^{\infty } (\alpha _n,m_1) \ \forall \alpha \in [0, \alpha _n^{UB}]$$ and it is optimal for firm 1 to introduce the new product at some finite point in time t. If this inequality is violated, then in general it might depend on the initial value $$\alpha _n(0)$$ whether the new product is introduced. In order to gain a better understanding of the potential structure of equilibria under which the new product introduction depends on the initial state, we first characterize the properties of equilibrium state dynamics on $$(0,{{\bar{\alpha }}}_n)$$.
The following proposition shows that under an MPE profile the state $$\alpha _n$$ is either monotonously increasing or decreasing on the entire interval below the threshold value at which firm 1 introduces the new product (i.e., for $$\alpha \in (0,{{\bar{\alpha }}}_n)$$).
Proposition 3
If $$((\Phi _1(\alpha _n,m), \Psi _{{{\bar{\alpha }}}_n}),\Phi _2(\alpha _n,m))$$ is a MPE profile of the game, then it holds for all $$\alpha _n^1, \alpha _n^2 \in (0, {{\bar{\alpha }}}_n)$$ that
\begin{aligned} \text{ sgn } \left[ f(\alpha _n^1, (\Phi _1(\alpha _n^1,m_1),\Phi _2(\alpha _n^1,m_1))) \right] = \text{ sgn } \left[ f(\alpha _n^2, (\Phi _1(\alpha _n^2,m_1),\Phi _2(\alpha _n^2,m_1))) \right] . \end{aligned}
The observation that in equilibrium the state $$\alpha _n$$ is either strictly increasing or strictly decreasing on the entire interval $$(0,{{\bar{\alpha }}}_n)$$ has several important implications. If in a given equilibrium the new product is introduced after a positive and finite delay for some initial value $$\alpha _n(0)$$, then under this equilibrium profile the product is introduced after finite time regardless of the initial state $$\alpha _n(0)$$. Conversely, if in a given equilibrium the product is never introduced for some initial value of the new market size, then under this equilibrium a positive finite delay in product introduction can never occur, regardless of $$\alpha _n(0)$$. Relating to the different types of equilibria introduced in Sect. 3, the proposition shows that the property ‘delay equilibrium,’ which in Definition 1 was defined locally in the sense that for some initial condition the mode switches after some delay, in the framework of this model is actually a global property in the sense that in any delay equilibrium the switch to $$m_2$$ occurs after a positive finite delay for all $$\alpha _n \in [0, {{\bar{\alpha }}}_n)$$. Similarly, in this model in any now or never equilibrium the switch never occurs for any initial value of the state below the threshold $${{\bar{\alpha }}}_n$$.
In what follows, we illustrate our general results from Sect. 3 by showing that, depending on the size of the new product introduction costs $$\kappa$$, qualitatively different types of MPE constellations exist in in our model.

4.2.1 Small Costs of Market Introduction

If $$\kappa$$ is sufficiently small, i.e., $$\kappa < {\underline{\kappa }} := V_1(0,m_2) - V_1^{\infty }(0,m_1)$$, then it follows directly that immediate introduction is more profitable for firm 1 than no introduction regardless of the value of $$\alpha _n$$.12 Hence, in equilibrium the new product is always introduced. Concentrating first on a maximum delay equilibrium, we need to characterize the threshold value of $$\bar{\alpha }_n$$, above which the product is introduced. Denoting the threshold in the maximum delay equilibrium by $${{\bar{\alpha }}}_n^{md}$$, we obtain that the following condition has to be satisfied for some positive value of $$u_2^{m_1}$$ (see ‘Appendix’ D):
\begin{aligned} r \left( V_1({{\bar{\alpha }}}_n^{md}, m_2) - \kappa \right)= & {} F_1(\bar{\alpha }_n^{md}, \Phi _1({{\bar{\alpha }}}_n^{md},m_2), m_1)\nonumber \\&+ \frac{\partial V_1({{\bar{\alpha }}}_n^{md},m_2)}{\partial \alpha _n} f(\bar{\alpha }_n^{md}, \Phi _1({{\bar{\alpha }}}_n^{md},m_2), u_2^{m_1}), \nonumber \\ r V_2({{\bar{\alpha }}}_n^{md}, m_2)= & {} F_2({{\bar{\alpha }}}_n^{md}, u_2^{m_1}, m_1) - \frac{c_2 u_2^{m_1}}{\gamma } f({{\bar{\alpha }}}_n^{md}, \Phi _1({{\bar{\alpha }}}_n^{md},m_2), u_2^{m_1}) \end{aligned}
(12)
Whereas the first equation captures the smooth pasting condition, the second is the HJB equation of firm 2 at $$\alpha _n^{md}$$. To obtain these equations, we have used the fact that the equilibrium value function of firm 2 has to coincide between the two modes for all $$\alpha _n \ge {{\bar{\alpha }}}_n$$ (see Proposition 1) and that the control of firm 1 is identical in both modes for $$\alpha _n = {{\bar{\alpha }}}_n^{md}$$ since all conditions of Proposition 2 are satisfied in our model.
For our benchmark parametrization together with market introduction costs of $$\kappa = 0.095$$, the system (12) has a unique positive solution given by $${{\bar{\alpha }}}_n^{md} = 0.036$$ and $$u_2^{m_1} = 0.0047$$. The corresponding maximum delay equilibrium is illustrated in Figs. 2 and 3a. The solid lines in the two panels of Fig. 2 show the value functions of the two firms in mode $$m_1$$ under such an equilibrium. More precisely, these functions have been determined as the solutions of the HJB equations for mode $$m_1$$ on $$[0,{{\bar{\alpha }}}_n^{md}]$$ under the boundary conditions that $$V_{i}({{\bar{\alpha }}}_n^{md},m_1) = V_{i}(\bar{\alpha }_n^{md},m_2) - \mathbb {1}_{[i=1]} \kappa , i=1,2$$. It can be easily checked that these value functions in combination with feedback functions derived from the maximization of the right-hand side of the HJB equation on $$[0,{{\bar{\alpha }}}_n^{md}]$$ satisfy all conditions of Proposition 1 for any profile $$(\Phi _1, \Phi _2)$$ satisfying $$\Phi _2(\alpha _n, m_1) \ge \Phi _2(\bar{\alpha }_n^{md}, m_1)$$ for all $$\alpha _n \ge {{\bar{\alpha }}}_n^{md}$$ (in order to ensure condition (vii) of Proposition 1). The equilibrium investment functions under the maximum delay equilibrium are shown in Fig. 3a. It can be clearly seen that the investment of firm 1 in mode $$m_1$$ (solid black line) intersects with the equilibrium investment in mode $$m_2$$ (black dashed line) exactly at the threshold $${{\bar{\alpha }}}_n^{md}$$ above which firm 1 immediately introduces the new product. We do not show any values for the function $$\Phi _1(\alpha _n, m_1)$$ for $$\alpha _n \ge {{\bar{\alpha }}}_n^{md}$$ since any choice of the function on this interval is compatible with equilibrium. The figure also illustrates the downward jump of the investment of firm 2 at the threshold $$\alpha _n = {{\bar{\alpha }}}_n^{md}$$ where the mode switches from $$m_1$$ to $$m_2$$. In mode $$m_1$$, there is an additional incentive for player 2 to invest. Such investment decreases the speed of the increase of $$\alpha _n$$ and delays the point in time when the state variable arrives at $${\bar{\alpha }}_n^{md}$$ and the new product is introduced. Firm 2 has an incentive to delay the new product introduction, which provides additional investment incentives. Once the new product is introduced, this additional incentive vanishes which results in a downward jump in player 2’s investment.
However, the maximum delay equilibrium is not the only MPE in our setting. Actually, for any threshold $${{\bar{\alpha }}}_n^d \in [0, \bar{\alpha }_n^{md})$$ there is a forced switch delay equilibrium such that firm 1 introduces the new product immediately for all $$\alpha _n \ge {{\bar{\alpha }}}_n^d$$. In Fig. 2, the value functions corresponding to such an equilibrium with $${{\bar{\alpha }}}_n^d = 0.01$$ are illustrated with coarsely dashed lines. It can be clearly seen that both value functions have a kink at $$\alpha _n = \bar{\alpha }_n^d$$ such that also for firm 1 the smooth pasting condition does not hold at this threshold where the firm introduces the new product. The equilibrium feedback function corresponding to this MPE is shown in Fig. 3b. The figure illustrates that the investments of both firms jump at the point in time when the new product is introduced, where the jump is upward for firm 1 and downward for firm 2. Whereas the intuition for the downward jump of firm 2 is analogous to that developed for the maximum delay equilibrium, the upward jump for firm 1 is due to the fact that after the introduction of the new product an increase of $$\alpha _n$$ has a positive impact on the instantaneous profit of firm 1, whereas in mode $$m_1$$ such an increase only has impact on the remaining time till the new product is introduced. If the new product is introduced at a level of $$\alpha _n$$, where the value function in mode $$m_2$$ is still strictly steeper than that in mode $$m_1$$, the investment incentives jump upward at the point of market introduction. A crucial feature of this equilibrium profile is that $$\Phi _2(\alpha _n,m_1)$$ for $$\alpha _n \ge {{\bar{\alpha }}}_n^d$$ is sufficiently large such that it is optimal for firm 1 to introduce the new product immediately at $${{\bar{\alpha }}}_n^d$$. In our example, this is guaranteed by setting $$\Phi _2({{\bar{\alpha }}}_n^d, m_1) > \frac{1}{\gamma } \Phi _1({{\bar{\alpha }}}_n^d, m_1)$$, which implies that under optimal investment by firm 1 the state variable $$\alpha _n$$ does not move above $${{\bar{\alpha }}}_n^d$$. As discussed in Sect. 3, although such high investments by firm 2 would not be optimal, if it were to be carried out for a time interval with positive measure, in equilibrium the firm is never required to carry out such investment, regardless of the value of $$\alpha _n(0)$$.

4.2.2 Intermediate Costs of Market Introduction

We now consider the case where $$\kappa > {\underline{\kappa }}$$, which implies that for $$\alpha _n(0) = 0$$ immediate introduction of the new product yields a lower value for firm 1 than never introducing the new product and abstaining from any investment into the buildup of $$\alpha _n$$. Although immediate introduction of the new product is not optimal, an introduction with some delay might still be more profitable than no introduction. Taking into account that the largest value for firm 1 is obtained under the maximum delay equilibrium, $$V_1^{md}(0,m_1) > V_1^{\infty }(0,m_1)$$ is a necessary and sufficient condition for the existence of delay equilibria in our setting. It should be noted that $$V_1^{md}(0,m_1)$$ is a decreasing function of $$\kappa$$ such that this condition implies an upper bound $${{\bar{\kappa }}}$$ for the costs of market introduction such that for all $$\kappa \le {{\bar{\kappa }}}$$ there exists an equilibrium such that for all $$\alpha _n(0) \ge 0$$ the new product is introduced to the market after finite time. For this subsection, we assume that $$\kappa \in [{\underline{\kappa }}, {{\bar{\kappa }}}]$$, i.e., the market introduction costs are in an intermediate range such that for $$\alpha _n(0) = 0$$ immediate introduction is not optimal but there exists an equilibrium which induces new product introduction with some delay for this initial value.
In Fig. 4, this case is illustrated by showing the value functions for immediate market introduction (dashed lines), the maximum delay equilibrium (solid line) and no market introduction (dotted line). We denote by $${{\bar{\alpha }}}_n^{nn}$$ the largest value of $$\alpha _n$$ such that $$V_1^{\infty }(\alpha _n,m_1) \ge V_1(\alpha _n,m_2) - \kappa$$. Clearly, for all values of $$\alpha _n < {{\bar{\alpha }}}_n^{nn}$$ immediate market introduction is not optimal; therefore, the range of threshold values $${{\bar{\alpha }}}_n^d$$ for which delay equilibria can exist is restricted to $$\bar{\alpha }_n^d \in [{{\bar{\alpha }}}_n^{nn}, {{\bar{\alpha }}}_n^{md}]$$. However, for a delay equilibrium to exist, the associated value function must also satisfy $$V_1^d(0,m_1;{{\bar{\alpha }}}_n^d) \ge V_1^{\infty }(0,m_1)$$, where we denote by $$V_1^d(0,m_1;\bar{\alpha }_n^d)$$ the value function of firm 1 under a (candidate for a) delay equilibrium with threshold $${{\bar{\alpha }}}_n^d$$. If this inequality does not hold, condition (ii) in Proposition 1 would be violated at $$\alpha _n = 0$$. This can be seen by realizing that by choosing $$u_i = 0$$ for $$\alpha _n = 0$$ we have $${{\dot{\alpha }}} = 0$$ and the value of the right-hand side of the HJB equation becomes
\begin{aligned} \left( 1+\frac{\nu _o}{2} \right) (q_{io}^{m_1})^2 = r V_1^{\infty }(0,m_1) > r V_1^d(0,m_1;{{\bar{\alpha }}}_n^d). \end{aligned}
Since under the equilibrium feedback functions in the delay equilibrium the right-hand side of the HJB equation has to be equal to $$r V_1^d(0,m_1;{{\bar{\alpha }}}_n^d)$$ (see condition (i) in Proposition 1), the feedback function $$\Phi ^d(\alpha _n,m_1;{{\bar{\alpha }}}_n^d)$$ in that equilibrium candidate does not maximize the right-hand side of the HJB equation. This shows that no delay equilibrium with $$V_1^d(0,m_1;{{\bar{\alpha }}}_n^d) < V_1^{\infty }(0,m_1)$$ can exist with the property that the new product is introduced after a positive delay for $$\alpha _n(0)=0$$. Furthermore, we know from Proposition 3 that in any equilibrium the sign of the state dynamics cannot change on the interval $$[0, {{\bar{\alpha }}}_n]$$. This rules out any equilibrium profile under which the new product is never introduced (and hence $$\alpha _n$$ decreases over time) for a small value of $$\alpha _n(0)$$, but is introduced after a delay for larger values of $$\alpha _n(0)$$. Put together these arguments establish that for any threshold $$\bar{\alpha }_n^d$$ with $$V_1^d(0,m_1;{{\bar{\alpha }}}_n^d) < V_1^{\infty }(0,m_1)$$ no delay equilibrium can exist. However, there exists a continuum of delay equilibria where at the market introduction threshold this inequality does not hold. In particular, there exists a unique $${\underline{\alpha }}_n^d \in (\bar{\alpha }_n^{nn}, {{\bar{\alpha }}}_n^{md}]$$ such that $$V_1^d(0,m_1;\bar{\alpha }_n^{d}) \ge V_1^{\infty }(0,m_1)$$ for all $${{\bar{\alpha }}}_n^d \in [{{\underline{\alpha }}}_n^{d}, {{\bar{\alpha }}}_n^{md}]$$. To see this, note that, since the smooth pasting condition is satisfied at $$\bar{\alpha }_n^{md}$$, for all $${{\bar{\alpha }}}_n^d < {{\bar{\alpha }}}_n^{md}$$ an equilibrium with a later switch to $$m_2$$ leads to a higher value for firm 1. Hence, $$V_1^d(0,m_1;{{\bar{\alpha }}}_n^d)$$ is an increasing and continuous function of $${{\bar{\alpha }}}_n^d$$. Furthermore, in light of the definition of $${{\bar{\alpha }}}_n^{nn}$$ we have
\begin{aligned} V_1^d(0,m_1;{{\bar{\alpha }}}_n^{nn}) < V_1^d({{\bar{\alpha }}}_n^{nn},m_1;\bar{\alpha }_n^{nn}) = V_1({{\bar{\alpha }}}_n^{nn},m_2) - \kappa = V_1^{\infty }({{\bar{\alpha }}}_n^{nn},m_1). \end{aligned}
Since without market introduction of the new product the value of firm 1 does not depend on $$\alpha _n$$, we have $$V_1^{\infty }(0,m_1) = V_1^{\infty }({{\bar{\alpha }}}_n^{nn},m_1) > V_1^d(0,m_1;\bar{\alpha }_n^{nn})$$. Due to $$\kappa < {{\bar{\kappa }}}$$, we also have $$V_1^{\infty }(0,m_1) < V_1^d(0,m_1;{{\bar{\alpha }}}_n^{md})$$ and the intermediate value theorem implies the existence of $$\underline{\alpha }_n^{d} \in ({{\bar{\alpha }}}_n^{nn},{{\bar{\alpha }}}_n^{md})$$ with $$V_1^d(0,m_1;{{\underline{\alpha }}}_n^{d}) = V_1^{\infty }(0,m_1)$$. In the two panels of Fig. 4, we illustrate this observation by showing, as coarsely dashed lines, the value functions of the two firms under the (forced switch) delay equilibrium with threshold $${{\bar{\alpha }}}_n^d = {\underline{\alpha }}_n^d$$. For any threshold value $${{\bar{\alpha }}}_n^d$$ in the interval $$[{{\underline{\alpha }}}_n^{d}, \bar{\alpha }_n^{md}]$$, a delay equilibrium profile can be constructed in the same way as discussed in the previous subsection.
However, in the case of intermediate values of market introduction costs considered here, there also exists a now or never equilibrium, as discussed in Sect. 3. Similarly to the forced switch delay equilibria, also this equilibrium is characterized by a threat of a strong investment of player 2 in $$m_1$$ as soon as the state variable $$\alpha _n$$ is larger or equal than $$\bar{\alpha }_n^{nn}$$. Hence, firm 1 introduces the product immediately if $$\alpha _n \ge {{\bar{\alpha }}}_n^{nn}$$. As discussed above, under a potential delay equilibrium in which firm 1 invests positive amounts we would have
\begin{aligned} V_1^d(\alpha _n,m_1;{{\bar{\alpha }}}_n^{nn}) < V_1^d(\bar{\alpha }_n^{nn},m_1;{{\bar{\alpha }}}_n^{nn}) = V_1^{\infty }(\bar{\alpha }_n^{nn},m_1) = V_1^{\infty }(\alpha _n,m_1) \end{aligned}
for all $$\alpha _n < {{\bar{\alpha }}}_n^{nn}$$ and therefore setting $$\Phi _1(\alpha _n,m_1) = 0$$ is the optimal strategy for firm 1 on this interval. Hence, in equilibrium we also must have $$\Phi _2(\alpha _n,m_1) = 0$$ for all $$\alpha _n < {{\bar{\alpha }}}_n^{nn}$$. We obtain a MPE profile under which for all $$\alpha _n(0) < \bar{\alpha }_n^{nn}$$ both firms invest nothing and the new product is never introduced, whereas for $$\alpha _n(0) \ge {{\bar{\alpha }}}_n^{nn}$$ the product is introduced at $$t=0$$ and both firms afterward invest according to the MPE strategies in mode $$m_2$$. The equilibrium feedback functions underlying this equilibrium are illustrated in Fig. 5, where, as in Fig. 3, we do not show any values of $$\Phi _1(\alpha _n,m_1)$$ for $$\alpha _n \ge {{\bar{\alpha }}}_n^{nn}$$ since any choice is compatible with an equilibrium profile.
Figure 6 illustrates the dynamics emerging under the different types of equilibria that coexist for an intermediate level of the market introduction costs. In particular, the trajectories of the market attractiveness ($$\alpha _n$$) and of the investments of both firms ($$u_1, u_2$$) are depicted for a small initial value of the market attractiveness. The dotted lines corresponding to the now or never equilibrium show that in such an equilibrium, due to absence of any investments the attractiveness of the new product decreases toward zero and the product is never introduced. The dashed and the solid lines correspond to the dynamics under the (forced switch) delay equilibrium with the minimal possible threshold, i.e., $${{\bar{\alpha }}}_n^d = {\underline{\alpha }}_n^d$$, and the maximum delay equilibrium. The points in time at which the new product is introduced under the forced switch delay and the maximum delay equilibrium are denoted by $$\tau ^d$$ and $$\tau ^{md}$$, where $$\tau ^d < \tau ^{md}$$. The figure illustrates that under both equilibria the state $$\alpha _n$$ increases over time. In the time interval $$[0,\tau ^d]$$, the increase is slower under the delay equilibrium than under the maximum delay equilibrium, because the investments of firm 2, which decrease the speed of growth of $$\alpha _n$$, are larger under the delay than under the maximum delay equilibrium. The investments of firm 1, fostering the growth of $$\alpha _n$$, are virtually identical under both equilibria. Intuitively, firm 2 invests more under the forced switch delay equilibrium, because the downward jump of its instantaneous profit associated with the switch to mode $$m_2$$ is much closer time-wise in the forced switch delay equilibrium than in the maximum delay equilibrium and therefore less heavily discounted. Hence, the incentive to invest in delaying the switch is stronger in the forced switch delay equilibrium. For firm 1, no such effect occurs and the incentive to invest during mode $$m_1$$ is hardly affected by the type of the delay equilibrium. Under the forced switch delay equilibrium, both controls exhibit a jump at $$\tau ^d$$ with the investment of firm 1 jumping upward and those of firm 2 jumping downward. As a result, $$\alpha _n$$ grows faster after $$\tau ^d$$ compared to the time before the new product introduction and we observe a higher attractiveness of the new market under the forced switch delay equilibrium than under the maximum delay equilibrium. It should, however, be noticed that under both equilibria the state $$\alpha _n$$ converges to the steady state in mode $$m_2$$ ($$\alpha _n^{*,m_2})$$ and therefore this difference between the two equilibria disappears in the long run. Finally, Fig. 6b illustrates again that in the maximum delay equilibrium the investment of firm 1 is continuous throughout the entire trajectory, i.e., also at period $$\tau ^{md}$$ at which the mode switches from $$m_1$$ to $$m_2$$.

4.2.3 Large Costs of Market Introduction

If market introduction costs are large, i.e., $$\kappa > {{\bar{\kappa }}}$$, then we have $$V_1^{md}(0,m_1) < V_1^{\infty }(0,m_1)$$, which means that any candidate for a delay equilibrium yields a smaller value for firm 1 at $$\alpha _n(0)=0$$ than not investing in the build-up of $$\alpha _n$$ and never introducing the new product. We illustrate the value functions of both firms for this case in Fig. 7. Due to $$V_1^{md}(0,m_1) < V_1^{\infty }(0,m_1)$$, there can be no equilibria where for the initial condition $$\alpha _n(0)=0$$ firm 1 invests in the buildup of $$\alpha _n$$ and eventually introduces the new product. Using the same arguments as developed in the previous subsection, this rules out the existence of any MPE under which the product is introduced with delay for some initial value $$\alpha _n(0)$$. Hence, the only equilibrium that still exists in this scenario is the now or never equilibrium, where firm 1 introduces the new product immediately for $$\alpha _n(0) \ge \bar{\alpha }_n^{nn}$$ and never introduces it for $$\alpha _n(0) < \bar{\alpha }_n^{nn}$$. The value functions of both firms corresponding to this equilibrium are indicated in bold in Fig. 7. It should be noted that under this equilibrium the value function for firm 1 has a kink at $${{\bar{\alpha }}}_n^{nn}$$ (panel (a)), while the value of the game for firm 2 exhibits a downward jump as the initial state $$\alpha _n(0)$$ crosses the threshold $${{\bar{\alpha }}}_n^{nn}$$ (panel (b)).

5 Discussion and Conclusions

In this paper, we study Markov perfect equilibria in two-player multi-mode differential games where both players can influence the dynamics of the state variable and one player controls the timing of the mode switches. We identify different types of equilibria which can occur in such games and our analysis of a simple model of dynamic competition under potential new product introduction illustrates the properties of these different types of equilibria. In particular, we show that forced switch delay equilibria, maximum delay equilibria and now or never equilibria might coexist for some parameter constellations. More precisely, we have demonstrated that the following three scenarios arise:
(i)
$$\kappa \le {\underline{\kappa }}$$: There exists a maximum delay equilibrium with product introduction at $$\alpha = \alpha _n^{md}$$ and a continuous investment path of player 1 across both modes of the game. Furthermore, for any $${{\bar{\alpha }}}_n^d \in [0,\alpha _n^{md})$$ there exists a forced switch delay equilibrium with new product introduction at $${{\bar{\alpha }}}_n^d$$. In each of these equilibria, the investments of both players jump as the mode switches from $$m_1$$ to $$m_2$$.

(ii)
$${\underline{\kappa }} < \kappa \le {\bar{\kappa }}$$: There exists a maximum delay equilibrium plus for each $${{\bar{\alpha }}}_n^d \in [{{\underline{\alpha }}}_n^{d}, {{\bar{\alpha }}}_n^{md})$$ there is also a forced switch delay equilibrium. Furthermore, in this case also a now or never MPE exists.

(iii)
$$\kappa > {\bar{\kappa }}$$: Whereas strategy profiles inducing market introduction with delay for some initial value of $$\alpha _n$$ do not constitute a MPE, the now or never equilibrium still exists.

In scenario (iii) for each initial condition, a unique prediction about the occurrence and timing of the new product introduction can be made if we restrict attention to the MPE analyzed here. This is not true in the first two cases. In case (ii), there is a set of initial conditions $$\alpha _n(0) \in [0, {{\bar{\alpha }}}_n^{nn}]$$ such that MPEs under which the new product is eventually introduced coexist with the now or never equilibrium under which no introduction of the new product occurs. For $$\alpha _n(0) > \bar{\alpha }_n^{nn}$$, the new product is introduced under all existing equilibria although the time of the introduction varies across equilibria. Similarly, under scenario (i) the product is eventually introduced for all initial conditions, where the time of the introduction depends on the chosen equilibrium.
Whereas all equilibria considered here are Markov perfect, intuitively the forced switch delay equilibria seem less plausible compared to the maximum delay and the now or never equilibrium. As discussed above, the equilibrium profiles underlying these equilibria include strategies for firm 2, under which in mode $$m_1$$ for $$\alpha _n \ge {{\bar{\alpha }}}_n^d$$ investments are chosen which would not be optimal if the game would stay in mode $$m_1$$ for an amount of time with positive measure even after the state $$\alpha _n$$ has crossed the threshold $${{\bar{\alpha }}}_n^d$$. Since in equilibrium the state is never larger or equal than $${{\bar{\alpha }}}_n^d$$ in mode $$m_1$$ for a positive amount of time, this feature does not contradict the optimality of the strategy of firm 2. However, it nevertheless constitutes an ‘incredible threat’ in the sense that it would not be optimal for firm 2 to stick to this investment strategy, in case firm 1 would deviate from its own equilibrium strategy by increasing the threshold at which it switches to mode $$m_2$$.
A main insight of our analysis is that a strategy profile might be Markov perfect even though the strategy of some players induces actions in some parts of the state space which would not be optimal for the player if carried out for a positive amount of time. Although this phenomenon has been demonstrated in this paper only for multi-mode games, it seems that it might arise also in other classes of differential games. For example, in games in which the state dynamics is controlled by potentially singular controls of both players MPEs might exist in which the strategy of one player in a certain region of the state space make it optimal for some other player to induce a jump of the state out of that region by means of a singular control. Similar to our setup, Markov perfection does not put any restrictions on the regular control of that player in the region in which the other player chooses a singular control.13 Intuitively, under such a MPE profile the state always immediately jumps out of that singular control region such that the amount of time for which the actions induced by the players’ strategies in that region are actually implemented has measure zero. Hence, also in the framework of such differential games incredible threats, in the sense that actions are chosen which would be suboptimal if implemented for a positive amount of time, can occur as part of Markov perfect equilibrium profiles. An interesting question for future research might be to explore whether such Markov perfect equilibria with ‘incredible threats’ in multi-mode or singular control games can be eliminated by appropriate equilibrium refinements. Developing such a refinement would also provide a valuable theoretical basis for alleviating the problem of non-uniqueness of predictions about equilibrium timing of regime switches in applications like the dynamic innovation model considered in this paper.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://​creativecommons.​org/​licenses/​by/​4.​0/​.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
print
PRINT
Appendix

Appendix A: Proofs

Proof of Proposition 1
In order to prove that the considered profile is an equilibrium, we first observe that conditions (i) to (iii) are standard conditions implying that the profile in mode $$m_2$$ is a Markov perfect equilibrium of the infinite horizon game in this mode (see Theorem 4.4 in Dockner et al. [5]).
Consider now the optimization problem of player 2 in mode $$m_1$$ for a given strategy $$\Phi _1(x,m_1), \Psi _{{{\bar{x}}}}$$ of player 1. Given that player 1 switches to mode $$m_2$$ instantaneously for $$x > \bar{x}$$, the choice of $$\Phi _2(x,m_1)$$ for all $$x > {{\bar{x}}}$$ does not affect the objective function of firm 2. Hence, any choice for $${{\tilde{\Phi }}}_2(x)$$ is optimal.
We now turn to the problem of player 2 for an arbitrary initial state $$x^{ini} \in [x_l, {{\bar{x}}})$$. Define an auxiliary value function $${{\hat{V}}}_2(x)$$ on $$[x_l, {{\bar{x}}}]$$ as follows
\begin{aligned} {{\hat{V}}}_2(x) = \left\{ \begin{array}{ll} V_2(x, m_1) &{} x \in [x_l, {{\bar{x}}}), \\ \lim _{x \rightarrow {{\bar{x}}}-} V_2(x,m_1) &{} x= {{\bar{x}}}. \end{array} \right. \end{aligned}
By definition, this auxiliary value function is continuous and differentiable on $$[x_l, {{\bar{x}}}]$$ and due to (iv) satisfies the Hamilton–Jacobi–Bellman equation for the optimal control problem of player 2 on that interval. Standard results show that this is therefore the value function of the auxiliary control problem of player 2, in which player 2 receives the payoff $$\lim _{x \rightarrow \bar{x}-} V_2(x,m_1)$$ once the state hits $${{\bar{x}}}$$. Furthermore, because of condition (viii), $$\Phi _2(x,m_1)$$ is the optimal feedback function for player 2 with respect to this auxiliary problem. It should be noted that any control path under which the state does not hit $${{\bar{x}}}$$ yields the same value for player 2 under the auxiliary and the original problem. Any control path under which the state hits $${{\bar{x}}}$$ due to condition (v) yields a larger or equal value for player 2 under the auxiliary problem than under the original problem. In particular, this implies that, if a path under which $${{\bar{x}}}$$ is not hit is optimal under the auxiliary problem, it is also optimal under the original problem.
Consider now an arbitrary $$x^{ini} \in [x_l, {{\bar{x}}})$$. If the optimal path under the auxiliary problem induced by $$\Phi _2(x,m_1)$$ does not hit $${{\bar{x}}}$$, then it is also optimal under the original problem. If the optimal path under the auxiliary problem induced by $$\Phi _2(x,m_1)$$ hits $${{\bar{x}}}$$, then we must have $$f(x,(\Phi _1(x,m_1),\Phi _2(x,m_1)),m_1) > 0$$ for all $$x \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$ for some $$\epsilon > 0$$. By condition (v), this implies that $${{\hat{V}}}_2({{\bar{x}}}) = V_2({{\bar{x}}}, m_1)$$ and therefore the auxiliary problem coincides with the original problem. Hence, $$\Phi _2(x^{ini},m_1)$$ is optimal for player 2 under the original problem for any $$x^{ini} \in [x_l, {{\bar{x}}})$$.
We now consider the optimal control problem of player 1 in mode $$m_1$$ for a given strategy $$\Phi _2(x,m_1)$$ of player 2. Considering the switching strategy $$\Psi _{{{\bar{x}}}}$$ as given, the same arguments as just applied to player 2 establish the optimality of $$\Phi _1(x,m_1)$$. Hence, what remains to be shown is that switching to mode $$m_2$$ for all $$x \ge {{\bar{x}}}$$ is optimal for player 1 given $$\Phi _2(x,m_1)$$. It follows from (iv) and (v) and by continuity that any path from the initial state $$x^{ini} = {{\bar{x}}}$$, which stays in the interval $$[x_l, {{\bar{x}}}]$$, yields a value for player 1 that is not larger than $$\lim _{x \rightarrow {{\bar{x}}}-} V_1(x,m_1) = V_1({{\bar{x}}}, m_2) - \kappa$$. Hence, switching to mode $$m_2$$ at $${{\bar{x}}}$$ yields at least the same value as any such path staying in $$[x_l,{{\bar{x}}}]$$.
Consider now a potentially optimal path from $${{\bar{x}}}$$ which stays in mode $$m_1$$ and has $$x(t) > {{\bar{x}}}$$ for all $$t > 0$$. Taking into account the compactness of the state space and the monotonicity of any optimal path, there must exist a steady state $${{\tilde{x}}} > \bar{x}$$, associated with a steady-state control $${{\tilde{u}}}_1$$, of such a path and we must have
\begin{aligned} \frac{F_1({{\tilde{x}}}, ({{\tilde{u}}}_1, \Phi _2({{\tilde{x}}}, m_1)))}{r} \ge V({{\tilde{x}}}, m_2) - \kappa , \end{aligned}
since otherwise it would be optimal for player 1 to switch to mode $$m_2$$ at $${{\tilde{x}}}$$. Taking into account that $$f({{\tilde{x}}}, (\tilde{u}_1, \Phi _2({{\tilde{x}}}, m_1) )) = 0$$, we obtain
\begin{aligned} F_1({{\tilde{x}}}, ({{\tilde{u}}}_1, \Phi _2({{\tilde{x}}}, m_1))) + \frac{\partial V_1({{\tilde{x}}}, m_2)}{\partial x} f({{\tilde{x}}}, ({{\tilde{u}}}_1, \Phi _2({{\tilde{x}}}, m_1) )) \ge r (V({{\tilde{x}}}, m_2) - \kappa ), \end{aligned}
which contradicts condition (vii). This shows that under the given conditions no path from $${{\bar{x}}}$$ which never jumps to mode $$m_2$$ can be strictly better than switching to mode $$m_2$$ at $${{\bar{x}}}$$. The same arguments establish that this holds also for any $$x^{ini} > {{\bar{x}}}$$.
Focusing on paths on which player 1 switches to mode $$m_2$$, it can be shown that (vii) implies that by for $$x > {{\bar{x}}}$$ switching immediately is strictly better than marginally delaying the switch to mode $$m_2$$.
To see this, consider for some $$x(t) > {{\bar{x}}}$$ the value in mode $$m_1$$ of delaying the switch to mode $$m_2$$ from t to $$t+\epsilon$$ which is denoted by $$V_1^{\epsilon }(x(t),m_1)$$:
\begin{aligned} V^{\epsilon }_1(x(t),m_1)= & {} \max _{u_1 \in {{{\mathcal {U}}}}_1} \int _t^{t+\epsilon }{F_1(x(s), (u_1,m_1), \Phi _2(x(s),m_1)),m_1)ds} \\&+e^{-r \epsilon } (V_1(x(t+\epsilon ),m_2)-\kappa ) . \end{aligned}
We need to show that $$V^{\epsilon }_1(x(t),m_1) < V_1(x(t),m_2) - \kappa$$ for all (small) $$\epsilon$$. Direct calculations yield
\begin{aligned} \lim _{\epsilon \rightarrow 0}&\frac{V^{\epsilon }_1(x,m_1) - (V_1(x,m_2) - \kappa )}{\epsilon } \\&= \max _{u_1 \in {{{\mathcal {U}}}}_1} \left[ F_1(x, (u_1, \Phi _2(x,m_1)),m_1) \right. \\&\left. \quad + \frac{\partial V_1(x,m_2)}{\partial x} f(x,(u_1, \Phi _2(x,m_1)),m_1) \right] - r (V_1(x,m_2) - \kappa )\\&< 0, \end{aligned}
where the last inequality follows from condition (vii). Hence, for all $$x > {{\bar{x}}}$$ it is optimal for player 1 to immediately switch to mode $$m_2$$.
Consider now a value $$x^{ini} < {{\bar{x}}}$$. Applying $$\Phi _1(x,m_1)$$ and switching to mode $$m_2$$, once the generated path hits, $${{\bar{x}}}$$ yields a value given by $$V_1(x^{ini},m_1)$$. Due to (vi), we have $$V_1(x^{ini},m_1) > V_1(x^{ini},m_2) - \kappa$$ and therefore switching to mode $$m_2$$ at $$x^{ini}$$ is not optimal. This shows that the switching strategy $$\Psi _{{{\bar{x}}}}$$ is indeed optimal for player 1. $$\square$$
Proof of Lemma 1
Assume that for a delay equilibrium profile $${{{\mathcal {P}}}}$$ we have
\begin{aligned}&\lim _{x \rightarrow {{\bar{x}}}-} \left[ F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \nonumber \right. \\&\left. \qquad + \frac{\partial V_1(x,m_2)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \right] \nonumber \\&\quad > r (V_1({{\bar{x}}},m_2) -\kappa ) \end{aligned}
(13)
We show that this implies that $${{{\mathcal {P}}}}$$ is a forced switch delay equilibrium. More precisely, we show that in this situation for sufficiently small $$\epsilon > 0$$ and any strategy variation $$\tilde{\Phi }_2(.)$$ for player 2, satisfying the conditions given in the definition of a forced switch delay equilibrium in Definition 1, there is an alternative strategy for player 1 with a later mode switch which is a better response than $$(\Phi _1(.), \Psi _{{{\bar{x}}}})$$ for player 1. To show this, we first observe that sticking to the strategy profile $$(\Phi _1(.), \Psi _{{{\bar{x}}}})$$ player 1 would switch to $$m_2$$ at $${{\bar{x}}}$$ and hence the value at $$x = \bar{x}$$ would be given by $$V_1({{\bar{x}}}, m_2) - \kappa$$ since $$\tilde{\Phi }_2(x,m_2) = \Phi _2(x,m_2) \ \forall x \in X$$. Now, consider an alternative strategy $$({{\tilde{\Phi }}}_1(.), \Psi _{{{\tilde{x}}}})$$ for player 1 with $${{\tilde{x}}} = {{\bar{x}}} + \epsilon$$ and
\begin{aligned} {{\tilde{\Phi }}}_1(x,m) = \left\{ \begin{array}{ll} \Phi _1(x,m_1) &{} x \not \in [{{\bar{x}}}, {{\bar{x}}} + \epsilon ) , m=m_1, \\ {{\tilde{\Phi }}}_1^{{{\bar{x}}}} := \lim _{z \rightarrow {{\bar{x}}}-} \Phi _1(z,m_1) &{} x \in [{{\bar{x}}}, {{\bar{x}}} + \epsilon ) , m=m_1, \\ \Phi _1(x,m_2) &{} x \in X, m=m_2. \end{array} \right. \end{aligned}
The value for player 1 under this strategy for an initial state $$x = {{\bar{x}}}$$ and mode $$m_1$$ is given by
\begin{aligned} {{\tilde{V}}}_1({{\bar{x}}}, m_1) = \int _{0}^{{{\tilde{\tau }}}(\epsilon )} e^{-rt} F_1(x, ({{\tilde{\Phi }}}_1^{{{\bar{x}}}}, {{\tilde{\Phi }}}_2(x,m_1)),m_1) + e^{-{{\tilde{\tau }}}(\epsilon )} \left( V(x({{\tilde{\tau }}}(\epsilon )),m_2) \right) , \end{aligned}
where $${{\tilde{\tau }}}(\epsilon )$$ is the time when the state hits $${{\tilde{x}}}$$ under the profile $$({{\tilde{\Phi }}}_1(x,m_1),\tilde{\Phi }_2(x,m_1))$$ of player controls. Since $${{{\mathcal {P}}}}$$ is a delay equilibrium, we have $$f(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) > 0$$ for all $$x \in ({{\bar{x}}} - \epsilon , {{\bar{x}}})$$ and by continuity for sufficiently small $$\epsilon$$, we therefore must have $$f(x, (\tilde{\Phi }_1(x,m_1), {{\tilde{\Phi }}}_2(x,m_1)),m_1) > 0$$ on the interval $$[{{\bar{x}}}, {{\bar{x}}} + \epsilon )$$. Hence, $${{\tilde{\tau }}}(\epsilon )$$ is finite and $$\lim _{\epsilon \rightarrow 0} {{\tilde{\tau }}}(\epsilon ) = 0$$. Defining $$\Delta (\epsilon ) = {{\tilde{V}}}_1({{\bar{x}}}, m_1) - V_1({{\bar{x}}}, m_1)$$, it is easy to see that this function is continuous in $$\epsilon$$ with $$\Delta (0) = 0$$ and
\begin{aligned} \Delta '(0)&= F_1(x, (\Phi _1^{{{\bar{x}}}}, {{\tilde{\Phi }}}_2(x,m_1)),m_1)\\ {}&\quad + \frac{\partial V_1({{\bar{x}}},m_2)}{\partial x} f({{\bar{x}}},(\Phi _1^{\bar{x}}, {{\tilde{\Phi }}}_2(x,m_1)),m_1) - r (V_1({{\bar{x}}},m_2) -\kappa )\\&= \lim _{x \rightarrow {{\bar{x}}}-} \Bigg [ F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \\&\quad + \frac{\partial V_1({{\bar{x}}}, m_2)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \Bigg ] \\&\qquad - r (V_1({{\bar{x}}},m_2) - \kappa ) \\&> 0, \end{aligned}
where the second line follows from the continuity of $${{\tilde{\Phi }}}_1, {{\tilde{\Phi }}}_2$$ at $${{\bar{x}}}$$ and the inequality in the last line from (13). Note that it is assumed here that $$V_1(x,m_2)$$ is continuously differentiable at $${{\bar{x}}}$$. This shows that $$\Delta (\epsilon ) > 0$$ for sufficiently small $$\epsilon$$ and that for such value of $$\epsilon$$ the profile $$({{\tilde{\Phi }}}_1(.), \Psi _{{{\tilde{x}}}})$$ is a better response for player 1 than $$(\Phi _1(.), \Psi _{{{\bar{x}}}})$$ to all strategies $${{\tilde{\Phi }}}_2$$ which are locally continuous at $${{\bar{x}}}$$. Hence, $${{{\mathcal {P}}}}$$ must be a forced switch delay equilibrium. This implies that for any maximum delay equilibrium (4) has to hold. Furthermore, since $$V_1({{\bar{x}}}, m_1) = V_1({{\bar{x}}},m_2) -\kappa$$, we also have
\begin{aligned} r V_1({{\bar{x}}}, m_1)&= \lim _{x \rightarrow {{\bar{x}}}-} \Bigg [ F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1)\nonumber \\ {}&\quad + \frac{\partial V_1({{\bar{x}}}, m_2)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \Bigg ]. \end{aligned}
(14)
The HJB equation (iv) in mode $$m_1$$ for $$x < {{\bar{x}}}$$ implies
\begin{aligned} r V_1(x, m_1) =&F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) +\\& + \frac{\partial V_1(x,m_1)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1). \end{aligned}
which by continuity of $$V_1(.,m_1)$$ implies
\begin{aligned} r V_1({{\bar{x}}}, m_1)&= \lim _{x \rightarrow {{\bar{x}}}-} \Bigg [ F_1(x, (\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1)\nonumber \\ {}&\quad + \frac{\partial V_1(x, m_1)}{\partial x} f(x,(\Phi _1(x,m_1), \Phi _2(x,m_1)),m_1) \Bigg ]. \end{aligned}
(15)
Equations (14) and (15) together imply that the smooth pasting condition (5) holds. $$\square$$
Proof of Proposition 2
Considering conditions (ii) and (viii) of Proposition 1, we have to show that
\begin{aligned} \lim _{x \rightarrow {{\bar{x}}}-} \arg \max _{u_1 \in {{{\mathcal {U}}}}_1} g_1(x,u_1) = \arg \max _{u_1 \in {{{\mathcal {U}}}}_1} g_2(u_1) \end{aligned}
(16)
with
\begin{aligned} g_1(x,u_1)&= F_1(x,(u_1,\Phi _2(x,m_1)),m_1) + \frac{\partial V_1(x,m_1)}{\partial x} f(x,(u_1,\Phi _2(x,m_1)),m_1) \end{aligned}
(17)
\begin{aligned} g_2(u_1)&= F_1({{\bar{x}}},(u_1,\Phi _2({{\bar{x}}},m_2)),m_2) + \frac{\partial V_1({{\bar{x}}},m_2)}{\partial x} f(\bar{x},(u_1,\Phi _2({{\bar{x}}},m_2)),m_2). \end{aligned}
(18)
The derivatives with respect to $$u_1$$ for the expressions to be maximized in the two modes read:
\begin{aligned}&\frac{\partial F_1(x,(u_1,\Phi _2(x,m_1)),m_1)}{\partial u_1} + \frac{\partial V_1(x,m_1)}{\partial x} \frac{\partial f(x,(u_1,\Phi _2(x,m_1)),m_1)}{\partial u_1} \\&\frac{\partial F_1({{\bar{x}}},(u_1,\Phi _2({{\bar{x}}},m_2)),m_2)}{\partial u_1} + \frac{\partial V_1({{\bar{x}}},m_2)}{\partial x} \frac{\partial f({{\bar{x}}},(u_1,\Phi _2({{\bar{x}}},m_2)),m_2)}{\partial u_1}. \end{aligned}
Due to the assumption of mode independent and separable control effects, we have
\begin{aligned}&\frac{\partial F_1(x,(u_1,\Phi _2({{\bar{x}}},m_1)),m_1)}{\partial u_1} = \frac{\partial F_1(x,(u_1,\Phi _2({{\bar{x}}},m_2)),m_2)}{\partial u_1}\\&\text{ and }\\&\frac{\partial f(x,(u_1,\Phi _2({{\bar{x}}},m_1)),m_1)}{\partial u_1} = \frac{\partial f(x,(u_1,\Phi _2({{\bar{x}}},m_2)),m_2)}{\partial u_1}. \end{aligned}
for all $$u_1 \in {{{\mathcal {U}}}}_1$$. Denote by
\begin{aligned} g_3(u_1) = \Big ( F_1({{\bar{x}}},(u_1, {{\hat{u}}}_2), m_1) + \frac{\partial V_1({{\bar{x}}},m_2)}{\partial x} f({{\bar{x}}},(u_1,{{\hat{u}}}_2),m_1) \Big ) \end{aligned}
with $${{\hat{u}}}_2 = \lim _{x \rightarrow {{\bar{x}}}-} \Phi _2(x,m_1)$$. Recalling $$g_1$$ and $$g_2$$ from (17) and (18), it follows directly from our arguments above that $$g_2'(u_1) = g_3'(u_1)$$ for all $$u_1 \in {{{\mathcal {U}}}}_1$$. Therefore,
\begin{aligned} \arg \max _{u_1 \in {{{\mathcal {U}}}}_1} g_2(u_1) = \arg \max _{u_1 \in \mathcal{U}_1} g_3(u_1). \end{aligned}
The assumption that the right-hand side of condition (ii) in Proposition 1 has a single maximizer implies that the argmax of $$g_2$$ is unique, which means that also the argmax of $$g_3$$ has only a single element. Furthermore, using (5) we obtain
\begin{aligned}&\!\!\!\lim _{x \rightarrow {{\bar{x}}}-} g_1(x,u_1)\\&\!\!\!\quad = F_1({{\bar{x}}},(u_1, \lim _{x \rightarrow {{\bar{x}}}-} \Phi _2(x,m_1)),m_1) \!+\! \lim _{x \rightarrow {{\bar{x}}}-} \frac{\partial V_1(x,m_1)}{\partial x} f(\bar{x},(u_1,\lim _{x \rightarrow {{\bar{x}}}-} \Phi _2(x,m_1)),m_1) \\&\!\!\!\quad = F_1({{\bar{x}}},(u_1, {{\hat{u}}}_2),m_1) + \frac{\partial V_1(\bar{x},m_2)}{\partial x} f({{\bar{x}}},(u_1,{{\hat{u}}}_2),m_1)\\&\!\!\!\quad = g_3(u_1). \end{aligned}
Since both $$g_1$$ and $$g_3$$ are continuous with respect to x and $$u_1$$, it follows that
\begin{aligned} \lim _{x \rightarrow {{\bar{x}}}-} \arg \max _{u_1 \in {{{\mathcal {U}}}}_1} g_1(x,u_1) = \arg \max _{u_1 \in {{{\mathcal {U}}}}_1} g_3(u_1), \end{aligned}
where we have used that the argmax of $$g_3$$ has only one element. Hence,
\begin{aligned} \lim _{x \rightarrow {{\bar{x}}}-} \arg \max _{u_1 \in {{{\mathcal {U}}}}_1} g_1(x,u_1) = \arg \max _{u_1 \in {{{\mathcal {U}}}}_1} g_2(u_1), \end{aligned}
and since $$\phi _1(x,m_1)$$ is a maximizer of $$g_1$$ and $$\phi _1(\bar{x}, m_2)$$ is the maximizer of $$g_2$$, this proves the proposition. $$\square$$
Proof of Proposition 3
We define as $$g(\alpha _n) = f(\alpha _n, (\Phi _1(\alpha _n,m_1),\Phi _2(\alpha _n,m_1)))$$ the right-hand side of the state dynamics under the equilibrium profile. Assume first that there exists a state $$\alpha _n^* \in (0, {{\bar{\alpha }}}_n)$$ such that in a neighborhood around $$\alpha _n^*$$ we have $$g(\alpha _n) > 0$$ for all $$\alpha _n < \alpha _n^*$$ and $$g(\alpha _n) < 0$$ for all $$\alpha _n > \alpha _n^*$$. Then, this neighborhood is invariant under the state dynamics and for initial values $$\alpha _n(0)$$ in this neighborhood the threshold $${{\bar{\alpha }}}_n$$ is never reached and therefore the game never switches to mode $$m_2$$. Given that $$m_2$$ is never reached, any positive investment $$u_1 > 0$$ is clearly suboptimal for player 1, which contradicts $$g(\alpha _n) > 0$$ for $$\alpha _n < \alpha _n^*$$.
Therefore, the only remaining possibility for a scenario in which the direction of the state dynamics changes in the interval $$[0, \bar{\alpha }_n)$$ is that there exists a unique point $$\alpha _n^* \in (0, {{\bar{\alpha }}}_n)$$ such that $$g(\alpha _n)> 0 \ \forall \alpha _n > \alpha _n^*$$ and $$g(\alpha _n)< 0 \ \forall \alpha _n < \alpha _n^*$$. Assume that such an MPE exists and $$(\tilde{\Phi }_1(\alpha _n^1,m_1),{{\tilde{\Phi }}}_2(\alpha _n^1,m_1)))$$ is the strategy profile giving rise to this pattern. We denote the value functions of the two firms corresponding to this profile by $$\tilde{V}_i(\alpha _n, m_1), i=1,2$$.
The proof now proceeds by showing first that $$\lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_2(\alpha _n^* - \epsilon ,m_1) > \lim _{\epsilon \rightarrow 0} \tilde{V}_2(\alpha _n^* + \epsilon ,m_1)$$ and, second, that in light of this inequality the optimal strategy of player 2 at a state $$\alpha _n^* + \epsilon$$ for sufficiently small $$\epsilon$$ has to be such that $$g(\alpha _n^* + \epsilon ) < 0$$, which contradicts our assumption and therefore rules out the existence of a state $$\alpha _n^*$$ with the properties given above.
To show that
\begin{aligned} \lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_2(\alpha _n^* - \epsilon ,m_1) > \lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_2(\alpha _n^* + \epsilon ,m_1) \end{aligned}
(19)
, we first observe that in light of $$g(\alpha _n) < 0$$ for all $$\alpha _n \in [0,\alpha _n^*)$$ and $$\alpha _n^* < {{\bar{\alpha }}}_n$$ the new product is never introduced under this investment profile. Therefore, any optimal investment strategy for firm 2 must have $${{\tilde{\Phi }}}_2(\alpha _n,m_1) = 0$$ for all $$\alpha _n \in [0,\alpha _n^*)$$ and therefore for any $$\alpha _n < \alpha _n^*$$ we must have
\begin{aligned} {{\tilde{V}}}_2(\alpha _n,m_1) = V_2^{\infty }(\alpha _n,m_1) = \left( 1+\frac{\nu _o}{2} \right) \frac{(q_{2o}^{m_1})^2}{r}. \end{aligned}
Hence,
\begin{aligned} \lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_2(\alpha _n^* - \epsilon ,m_1) = \left( 1+\frac{\nu _o}{2} \right) \frac{(q_{2o}^{m_1})^2}{r}. \end{aligned}
Furthermore, taking into account that for $$\alpha _n(0) > \alpha _n^*$$, the threshold $${{\bar{\alpha }}}_n$$ is reached in finite time under the strategies $$({{\tilde{\Phi }}}_1(\alpha _n,m_1), \tilde{\Phi }_2(\alpha _n,m_1))$$, at which point the game switches to mode $$m_2$$. Hence, we obtain
\begin{aligned} \lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_2(\alpha _n^* + \epsilon )= & {} \int _{t=0}^{\tau } e^{-rt} \left( 1+\frac{\nu _o}{2} \right) (q_{2o}^{m_1})^2 - \xi _2(\Phi _2(\alpha _n,m_1)) dt\\&+ \int _{t=\tau }^{\infty } e^{-rt} \left( 1+\frac{\nu _o}{2} \right) q_{2o}^{m_2}(\alpha _n)^2 - \xi _2(\Phi _2(\alpha _n,m_2)) dt, \end{aligned}
where $$\tau$$ is the point in time when the mode switches from $$m_1$$ to $$m_2$$. Taking into account that $$\xi _2(u) \ge 0 \ \forall u \ge 0$$, it is sufficient for proving (19) to show that $$q_{2o}^{m_2}(\alpha _n) < q_{2o}^{m_1}$$ for all $$\alpha _n \ge 0$$. To see this, we first observe that inserting $$\alpha _n = 0$$ and $$\alpha _n^0 = \frac{3 \eta \alpha _o}{3 + \nu _o}$$ into (10) gives $$q_{2o}^{m_2}(0) = q_{2o}^{m_1}$$. Furthermore,
\begin{aligned} \frac{\partial q_{2o}^{m_2}}{\partial \alpha _n} = \frac{\partial q_{2o}^{m_2}}{\partial \alpha _n^0} = - \frac{\eta \nu _o}{(1-\eta ^2) (6 + 5 \nu _o) + 3 (\nu _o + \nu _n) + (2+\nu _n) \nu _o^2 + 4 \nu _o \nu _n} < 0, \end{aligned}
and therefore, due to our assumption that $$\alpha _n^0 > \frac{3 \eta \alpha _o}{3 + \nu _o}$$ we have $$q_{2o}^{m_2}(\alpha _n) < q_{2o}^{m_1}$$ for all $$\alpha _n \ge 0$$. This establishes that the inequality (19) holds.
To complete the proof, we show that there exists an alternative strategy $${{\hat{\Phi }}}_2(\alpha _n, m_1)$$ such that for an initial value $$\alpha _n(0) = \alpha _n^*+ {{\hat{\epsilon }}}$$ for sufficiently small $${{\hat{\epsilon }}}$$ the generated value for firm 2 is larger than $${{\tilde{V}}}_2(\alpha _n^*+{{\hat{\epsilon }}})$$ if firm 1 sticks to $$\tilde{\Phi }_1(\alpha _n, m_1)$$. In particular, we define
\begin{aligned} {{\hat{\Phi }}}_2(\alpha _n, m_1) = \left\{ \begin{array}{ll} {{\tilde{\Phi }}}_2(\alpha _n, m_1) &{} \alpha _n \not \in [\alpha _n^*, \alpha _n^*+{{\hat{\epsilon }}}] \\ \frac{1}{\gamma } {{\tilde{\Phi }}}_1(\alpha _n, m_1) &{} \alpha _n \in [\alpha _n^*, \alpha _n^*+{{\hat{\epsilon }}}] \end{array} \right. \end{aligned}
Under the strategy profile $$({{\tilde{\Phi }}}_1, {{\hat{\Phi }}}_2)$$, we have $${{\dot{\alpha }}}_n = - \delta \alpha _n < 0$$ for $$\alpha _n \in [\alpha _n^*, \alpha _n^*+{{\hat{\epsilon }}}]$$. Hence, for $$\alpha _n(0) = \alpha _n^*+{{\hat{\epsilon }}}$$ the state $$\alpha _n^*$$ is reached at $$t = \tau ({{\hat{\epsilon }}}):= \frac{\ln (\alpha _n^*+{{\hat{\epsilon }}}) - \alpha _n^*}{\delta }$$. The value for firm 2 generated by this strategy for $$\alpha _n(0) = \alpha _n^*+{{\hat{\epsilon }}}$$ therefore reads
\begin{aligned} {{\hat{V}}}_2(\alpha _n^*+{{\hat{\epsilon }}})= & {} \int _{0}^{\tau ({{\hat{\epsilon }}})} e^{-rt} \left( 1+\frac{\nu _o}{2} \right) (q_{2o}^{m_1})^2 \nonumber \\&- \xi _2({{\hat{\Phi }}}_2(\alpha _n,m_1)) dt + e^{-r \tau ({{\hat{\epsilon }}})} {{\hat{V}}}_2(\alpha _n^*,m_1) \end{aligned}
(20)
Furthermore, since under this profile we have $${{\dot{\alpha }}}_n < 0$$ at $$\alpha _n = \alpha _n^*$$, therefore $${{\hat{V}}}_2(\alpha _n^*,m_1) = \lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_{2}(\alpha _n^* - \epsilon ,m_1)$$. Taking into account that $$\lim _{{{\hat{\epsilon }}} \rightarrow 0} \tau (\hat{\epsilon }) = 0$$, we therefore obtain from (19) and (20) that
\begin{aligned} \lim _{{{\hat{\epsilon }}} \rightarrow 0} {{\hat{V}}}_2(\alpha _n^*+{{\hat{\epsilon }}},m_1) = \lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_{2}(\alpha _n^* - \epsilon ,m_1) > \lim _{\epsilon \rightarrow 0} {{\tilde{V}}}_2(\alpha _n^*+ \epsilon , m_1) \end{aligned}
holds. Accordingly, for sufficiently small $${{\hat{\epsilon }}}$$ we have $${{\hat{V}}}_2(\alpha _n^*+{{\hat{\epsilon }}},m_1) > {{\tilde{V}}}_2(\alpha _n^*+ {{\hat{\epsilon }}}, m_1)$$, which contradicts our assumption that $$\tilde{\Phi }_2(\alpha _n,m_1)$$ is the optimal feedback strategy of firm 2. This completes the proof of the proposition. $$\square$$

Appendix B: Derivation of the Cournot Equilibrium Quantities and Profits

We first treat mode $$m_2$$. In order to determine the market profit of firm 1 in $$m_2$$, we observe that from
\begin{aligned} \pi _{1}^{m_2} = p_o^{m_2} q_{1o}^{m_2} + p_n^{m_2} q_{1n}^{m_2} - \frac{\nu _o}{2} \left( q_{1o}^{m_2}\right) ^2 - \frac{\nu _n}{2} \left( q_{1n}^{m_2}\right) ^2 \end{aligned}
the three following first-order conditions are obtained
\begin{aligned} q_{1o}^{m_2}= & {} \frac{\alpha _o - q_{2o}^{m_2} - 2\eta q_{1n}^{m_2}}{2+\nu _o} \end{aligned}
(21)
\begin{aligned} q_{2o}^{m_2}= & {} \frac{\alpha _o - q_{1o}^{m_2} - 2\eta q_{1n}^{m_2}}{2+\nu _o} \end{aligned}
(22)
\begin{aligned} q_{1n}^{m_2}= & {} \frac{\alpha _n^0 +\alpha _n - q_{1o}^{m_2} - \eta q_{2o}^{m_2} - \eta q_{1o}^{m_2}}{2+\nu _n} \end{aligned}
(23)
The equilibrium quantities (10) are obtained by solving this set of equations for $$q_{1o}^{m_2}, q_{2o}^{m_2}, q_{1n}^{m_2}$$. In order to obtain the expression (11) for the profit, we observe that the first-order conditions for firm 1 (21,23) can be rewritten as
\begin{aligned} p_o^{m_2}= & {} (1+\nu _o) q_{1o}^{m_2}+ \eta q_{1n}^{m_2} \\ p_n^{m_2}= & {} (1+\nu _n) q_{1n}^{m_2}+ \eta q_{1o}^{m_2}. \end{aligned}
Using this, we obtain
\begin{aligned} \pi _{1}^{m_2}= & {} p_o^{m_2} q_{1o}^{m_2} + p_n^{m_2} q_{1n}^{m_2} - \frac{\nu _o}{2} \left( q_{1o}^{m_2}\right) ^2 - \frac{\nu _n}{2} \left( q_{1n}^{m_2}\right) ^2 \\= & {} \left( 1 + \frac{\nu _o^2}{2} \right) (q_{1o}^{m_2})^2 + \left( 1 + \frac{\nu _n^2}{2} \right) (q_{1n}^{m_2})^2 + 2 \eta q_{1o}^{m_2} q_{1n}^{m_2}, \end{aligned}
which coincides with (11). The expression for the profit of firm 2 is obtained analogously. To obtain the value of $$\alpha _n^{UB}$$, we take into account that firm 1, which is active on both markets, produces less of the established product than firm 2. Hence, the relevant condition is $$q_{1o}^{m_2}(\alpha _n) > 0$$, which implies $$\alpha _n < \alpha _n^{UB} := \frac{(2 + \nu _n)(1+\nu _o) + \eta ^2}{\eta (3+2\nu _o)} \alpha _o - \alpha _n^0$$.
The equilibrium quantities (8) in mode $$m_1$$ are obtained by setting $$q_{1n}^{m_2}=0$$ in (21,22) and solving for $$q_{1o},q_{2o}$$. The expression (9) for the profit is obtained in the same way as described for mode $$m_2$$ above.

Appendix C: Analysis of the MPE in Linear Strategies in Mode $$m_2$$

In mode $$m_2$$, the two firms interact through a linear quadratic differential game with a one-dimensional state. Standard arguments (see [5]) establish that a pair of functions $$V_i(.,m_2), i=1,2$$ satisfying the Hamilton–Jacobi–Bellman equations
\begin{aligned} r V_i(\alpha _n,m_2)=\max _{u_i}\Big [F_i(\alpha _n, u_i, m_2)+\frac{\partial V_i}{\partial \alpha _n}(u_1-\gamma u_2-\delta \alpha _n))\Big ], \ i=1,2 \end{aligned}
(24)
and the transversality conditions
\begin{aligned} \lim _{t\rightarrow \infty }e^{-rt}V_i(\alpha _n)=0, \ i=1,2 \end{aligned}
(25)
constitute value function of a MPE. Maximizing the right-hand side of the HJB equations yields
\begin{aligned} u_i=\frac{1}{c_i}\frac{\partial V_i}{\partial \alpha _n}, \quad i=1,2 . \end{aligned}
(26)
Due to the linear-quadratic structure, the infinite time horizon and the time-autonomous nature of the game, we assume the following form for the value function:
\begin{aligned} V_i= C_i+D_i \alpha _n+E_i \alpha _n^2, \ i=1,2 . \end{aligned}
(27)
Comparison of coefficients yields the following system of 6 algebraic equations which are solved by standard numerical methods:
\begin{aligned} \begin{aligned} r C_1=&\ \frac{1}{2}\left( \frac{D_1^2}{c_1}+\frac{2D_1 D_2 \gamma ^2}{c_2} \frac{\alpha _o^2(2+\nu _n+\eta ^2(-2+\nu _o))(1+\nu _o)^2(-4\eta ^2+(2+\nu _n)(2+\nu _o))}{K}\right) \\ r D_1=&\ -D_1 \delta + \frac{2 D_1 E_1}{c_1} + \frac{2 (D_2 E_1 +D_1 E_2) \gamma ^2}{c_2}\\&-\frac{\alpha _o \eta \nu _o (1+\nu _o)(2+\nu _o)(4 \eta ^2-(2+\nu _n)(2+\nu _o))}{K} \\ r E_1=&\ \frac{2 E_1^2}{c_1}+\frac{4 E_1 E_2 \gamma ^2}{c_2}\\&-\frac{-(2+\nu _n)(1+\nu _o)^2(3+\nu _o)^2+\eta ^2(3+2\nu _o)(6+\nu _o(9+2\nu _o))+4\delta E_1 K}{2K} \\ r C_2=&\ \frac{1}{2}\left( \frac{2D_1 D_2 }{c_1}+\frac{D_2^2\gamma ^2}{c_2} + \frac{\alpha _o^2(2+\nu _o)\left( (2+\nu _n)(1+\nu _o)-\eta ^2(2+\nu _o)\right) ^2}{K}\right) \\ r D_2=&\ -D_2 \delta + \frac{2 (D_2 E_1+D_1 E_2)}{c_1} + \frac{2 D_2 E_2 \gamma ^2}{c_2}\\&+\frac{\alpha _o \eta \nu _o (2+\nu _o)(-(2+\nu _n)(1+\nu _o)+\eta ^2(2+\nu _o))}{K} \\ r E_2=&\ 2 E_2(-\delta +\frac{2 E_1}{c_1}+\frac{E_2 \gamma ^2}{c_2})+\frac{\eta ^2 \nu _o^2(2+\nu _o)}{2K} \end{aligned} \end{aligned}
(28)
where
\begin{aligned} K=\left( (2+\nu _n)(1+\nu _o)(3+\nu _o)-\eta ^2(6+5\nu _o)\right) ^2 \end{aligned}
(29)
The corresponding equilibrium feedback functions in mode $$m_2$$ are then given by
\begin{aligned} \Phi _i(\alpha _n,m_2) = \frac{1}{c_i}(D_i + 2 E_i \alpha _n). \end{aligned}

Appendix D: Characterization of the Maximum Delay Equilibrium

In the maximum delay equilibrium, the unknown variables to be determined are the threshold $${{\bar{\alpha }}}_n$$ and the control of player 2 in mode $$m_1$$ for $$\alpha _n = {{\bar{\alpha }}}_n$$. In particular, we denote by $$u_2^{m_1}$$ equilibrium feedback of player 2 in mode $$m_1$$ at $$\alpha _n = {{\bar{\alpha }}}_n^{md}$$. Then, requiring that inequality (3) holds as an equality yields
\begin{aligned} r (V_1({{\bar{\alpha }}}_n^{md},m_2) -\kappa )= & {} \lim _{\epsilon \rightarrow 0} \Big [ F_1\left( {{\bar{\alpha }}}_n^{md}, \Phi _1\left( \bar{\alpha }_n^{md}-\epsilon ,m_1\right) ,m_1 \right) +\\&+ \frac{\partial V_1({{\bar{\alpha }}}_n^{md},m_2)}{\partial \alpha _n} f \left( {{\bar{\alpha }}}_n^{md}, \Phi _1\left( {{\bar{\alpha }}}_n^{md} - \epsilon ,m_1\right) , \Phi _2\left( {{\bar{\alpha }}}_n^{md} - \epsilon ,m_1\right) \right) \Big ] \\= & {} F_1\left( {{\bar{\alpha }}}_n^{md}, \Phi _1\left( {{\bar{\alpha }}}_n^{md},m_2 \right) , m_1 \right) +\\&+ \frac{\partial V_1({{\bar{\alpha }}}_n^{md}, m_2)}{\partial \alpha _n} f({{\bar{\alpha }}}_n^{md},\Phi _1\left( {{\bar{\alpha }}}_n^{md}, m_2 \right) , u_2^{m_1}), \end{aligned}
where we have used that due to the smooth pasting condition the control of player 1 is continuous at $$\alpha _n = \bar{\alpha }_n^{md}$$.
Moreover, considering the limit of the HJB equation of player 2 in mode $$m_1$$ for $$\alpha _n \rightarrow {{\bar{\alpha }}}_n^{md}$$ yields, again using the continuity of the control of player 1,
\begin{aligned}&\lim _{\epsilon \rightarrow 0} \Big [ r V_2({{\bar{\alpha }}}_n^{md} - \epsilon ,m_1) - F_2({{\bar{\alpha }}}_n^{md} - \epsilon , \Phi _2\left( \bar{\alpha }_n^{md} - \epsilon ,m_1 \right) ,m_1) \nonumber \\&\qquad - \frac{\partial V_2({{\bar{\alpha }}}_n^{md} - \epsilon ,m_1)}{\partial \alpha _n} f({{\bar{\alpha }}}_n^{md} - \epsilon ,\Phi _1\left( {{\bar{\alpha }}}_n^{md} - \epsilon ,m_1\right) , \Phi _2\left( {{\bar{\alpha }}}_n^{md} - \epsilon ,m_1\right) ) \Big ] \nonumber \\&\quad = r V_2({{\bar{\alpha }}}_n^{md},m_2) - F_2({{\bar{\alpha }}}_n^{md}, u_2^{m_1},m_1) \nonumber \\&\qquad - \lambda ({{\bar{\alpha }}}_n^{md}, u_2^{m1}) f(\bar{\alpha }_n^{md},\Phi _1\left( {{\bar{\alpha }}}_n^{md}, m_2\right) , u_2^{m_1})=0 \end{aligned}
(30)
with $$\lambda ({{\bar{\alpha }}}_n^{md}, u_2^{m1}) = \lim _{\epsilon \rightarrow 0} \frac{\partial V_2({{\bar{\alpha }}}_n^{md} - \epsilon ,m_1)}{\partial \alpha _n}$$. To determine $$\lambda ({{\bar{\alpha }}}_n^{md}, u_2^{m1})$$, we use that the first-order condition for the optimal control of player 2 in mode $$m_1$$ for $$\alpha _n < {{\bar{\alpha }}}_n^{md}$$ is given by
\begin{aligned} \frac{\partial F_2(\alpha _n, u_2,m_1}{\partial u_2} + \frac{\partial V_2(\alpha _n,m_1)}{\partial \alpha _n} \frac{\partial f(\alpha _n, \Phi _1(\alpha _n, m_1), u_2)}{\partial u_2} = 0, \end{aligned}
which yields $$\lambda ({{\bar{\alpha }}}_n^{md}, u_2^{m1}) = - \left( \frac{\partial F_2({{\bar{\alpha }}}_n^{md}, u_2^{m_1},m_1)}{\partial u_2} \right) / \left( \frac{\partial f({{\bar{\alpha }}}_n^{md}, \Phi _1(\bar{\alpha }_n^{md}, m_2), u_2^{m_1})}{\partial u_2} \right)$$, where again we have used the equality of the control of player 1 across the two modes at $$\alpha _n = {{\bar{\alpha }}}_n^{md}$$. Taking into account the functional forms of $$F_2$$ and f in our model, we obtain $$\lambda ({{\bar{\alpha }}}_n^{md}, u_2^{m1}) = - \frac{c_2 u_2^{m_1}}{\gamma }$$. Inserting this expression yields the second line in (12).
Footnotes
1
Such a split is well known from the optimal stopping or the real-options literature, where typically ‘continuation regions’ and ‘stopping regions’ are distinguished.

2
Our modeling approach is embedded in a large literature using differential game models to study advertising under dynamic competition, see Jorgensen and Zaccour [11] for a survey.

3
This representation of the state space includes the cases $$x_l = - \infty$$ and $$x_u = \infty$$, where in these cases the corresponding boundary of the state space interval is open.

4
The assumption that no additional mode transitions are possible, in particular rules out that the mode jumps back to $$m_1$$ and therefore so-called Zeno points (see, e.g., [16]), where the state gets stuck at a certain state, while the mode keeps switching, are not possible in our setting.

5
In optimal stopping or real option problems, this set is typically referred to as the ‘stopping region.’

6
The fact that we assume that the mode switch happens for all states above the threshold, rather than below the threshold, is not restrictive, since our equilibrium characterization also applies after transforming the state from x to $$-x$$.

7
To avoid any confusion, it should be noted that the notion of delay equilibria in our setting refers to a delay in the mode switch and hence is not related to work on Nash equilibria in differential games with delays in the state equation.

8
The label of ‘forced switch delay equilibrium’ should not be misinterpreted in a Stackelberg sense, that player 2 first announces to implement a jump in its control at $${{\bar{x}}}$$ and player 1 then reacts to this by choosing its mode switching threshold at $${{\bar{x}}}$$. Since we consider a MPE with simultaneous strategy choices by both players, the incentives for player 1 to switch mode at $${{\bar{x}}}$$ stem from the fact that in equilibrium player 1 has rational expectations about the strategy of player 2.

9
This assumption seems quite realistic since advertising for (and against) new products often starts after the product introduction has been announced but well before the product is actually introduced.

10
It should be noted that the game might as well have additional MPEs with nonlinear feedback strategies.

11
Since the purpose of the analysis of this model is to illustrate our theoretical findings, we abstain from carrying out a systematic calibration exercise for our model. The parameters have been chosen in a way that they give rise to the different types of equilibria discussed in the previous section. The values are $$\eta = 0.5, \ c_1=c_2=45, \ r= 0.04, \ \gamma = 0.5, \ \alpha _o = 1, \ \alpha _n^0=0.5, \ \delta = 0.1, \ \nu _o = 0.4, \ \nu _n= 0.2$$.

12
For our benchmark parameter constellation, we have $$V_1(0,m_2) = 2.7$$ and $$V_1^{\infty }(0,m_1) = 2.6$$ such that for all values $$\kappa < 0.1$$ this condition is fulfilled.

13
This can, for example, be seen in Theorem 1 of Kwon [12], characterizing best responses in a dynamic common goods game in continuous time.

Literature
1.
Chronopoulos M, Lumbreras S (2017) Optimal regime switching under risk aversion and uncertainty. Eur J Oper Res 256:543–555
2.
Dawid H, Gezer S (2021) Delaying product introduction in a duopoly: a strategic dynamic analysis. In: Veliov V, Haunschmied J, Kovacevic R, Semmler W (eds) Dynamic Economic Problems with Regime Switches. Springer, NY, pp 227–248 CrossRef
3.
Dawid H, Keoula M, Kopel M, Kort PM (2015) Product innovation incentives by an incumbent firm: a dynamic analysis. J Econ Behav Organizat 117:411–438 CrossRef
4.
Dawid H, Keoula M, Kort P (2017) Numerical analysis of Markov-Perfect equilibria with multiple stable steady states: a duopoly application with innovative firms. Dynam Games Appl 7:555–577
5.
Dockner EJ, Jorgensen S, Long NV, Sorger G (2000) Differential Games in Economics and Management Science. Cambridge University Press, NY CrossRef
6.
Gromov D, Gromova E (2017) On a Class of Hybrid Differential Games. Dynam Games Appl 7(2):266–288
7.
Guo X, Miao J, Morellec E (2005) Irreversible investment with regime shifts. Journal of Economic Theory 122:37–59
8.
Haurie A, Roche M (1994) Turnpikes and computing of piecewise open-loop equilibria in stochastic differential games. J Econ Dynam Control 18:317–344 CrossRef
9.
Hoppe HC, Lehmann-Grube U (2005) Innovation timing games: a general framework with applications. J Econ Theory 121(1):30–50
10.
Huisman KJM, Kort PM (2015) Strategic capacity investment under uncertainty. RAND J Econ 46(2):376–408 CrossRef
11.
Jorgensen S, Zaccour G (2004) Differential Games in Marketing. Kluwer, Dordrecht CrossRef
12.
Kwon HD (2021) Game of Variable Contributions to the Common Good Under Uncertainty. forthcoming in Operations Research
13.
Liu H, Loewenstein M (2013) Market crashes, correlated illiquidity, and portfolio choice. Manag Sci 59:715–732 CrossRef
14.
Long NV, Prieur F, Tidball M, Puzon K (2017) Piecewise closed-loop equilibria in differential games with regime switching strategies. J Econ Dynam Control 76:264–284
15.
Reddy PV, Schumacher JM, Engwerda JC (2015) Optimal management with hybrid dynamics-the shallow lake problem. Lecture Notes in Control and Information Sciences, vol 461. Springer, Cham, pp 111–136
16.
Seidl A (2019) Zeno points in optimal control models with endogenous regime switching. J Econ Dynam Control 100:353–368
17.
Steg J-H (2018) Preemptive investment under uncertainty. Games Econ Behav 110:90–119
Title
Markov Perfect Equilibria in Multi-Mode Differential Games with Endogenous Timing of Mode Transitions
Authors
Herbert Dawid
Serhat Gezer
Publication date
07-12-2021
Publisher
Springer US
Published in
Dynamic Games and Applications / Issue 2/2022
Print ISSN: 2153-0785
Electronic ISSN: 2153-0793
DOI
https://doi.org/10.1007/s13235-021-00408-w

Go to the issue