Skip to main content
Erschienen in: Dynamic Games and Applications 1/2019

03.03.2018

Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies

verfasst von: David González-Sánchez, Fernando Luque-Vásquez, J. Adolfo Minjárez-Sosa

Erschienen in: Dynamic Games and Applications | Ausgabe 1/2019

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This work deals with a class of discrete-time zero-sum Markov games under a discounted optimality criterion with random state-actions-dependent discount factors of the form \(\tilde{\alpha }(x_{n},a_{n},b_{n},\xi _{n+1})\), where \(x_{n}, a_{n}, b_{n}\), and \(\xi _{n+1}\) are the state, the actions of players, and a random disturbance at time n, respectively, taking values in Borel spaces. Assuming possibly unbounded payoff, we prove the existence of a value of the game as well as a stationary pair of optimal strategies.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Ash RB, Doléans-Dade C (2000) Probability and measure theory, 2nd edn. Academic Press, LondonMATH Ash RB, Doléans-Dade C (2000) Probability and measure theory, 2nd edn. Academic Press, LondonMATH
2.
Zurück zum Zitat Bäuerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, BerlinMATHCrossRef Bäuerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, BerlinMATHCrossRef
3.
Zurück zum Zitat Carmon Y, Shwartz A (2009) Markov decision processes with exponentially representable discounting. Oper Res Lett 37(1):51–55MathSciNetMATHCrossRef Carmon Y, Shwartz A (2009) Markov decision processes with exponentially representable discounting. Oper Res Lett 37(1):51–55MathSciNetMATHCrossRef
4.
Zurück zum Zitat Cruz-Suárez H, Ilhuicatzi-Roldán R, Montes-de Oca R (2014) Markov decision processes on Borel spaces with total cost and random horizon. J Optim Theory Appl 162(1):329–346MathSciNetMATHCrossRef Cruz-Suárez H, Ilhuicatzi-Roldán R, Montes-de Oca R (2014) Markov decision processes on Borel spaces with total cost and random horizon. J Optim Theory Appl 162(1):329–346MathSciNetMATHCrossRef
5.
Zurück zum Zitat Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2(2):197–214MathSciNetMATHCrossRef Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2(2):197–214MathSciNetMATHCrossRef
6.
Zurück zum Zitat Dynkin EB, Yushkevich AA (1979) Controlled Markov processes. Springer, BerlinCrossRef Dynkin EB, Yushkevich AA (1979) Controlled Markov processes. Springer, BerlinCrossRef
7.
Zurück zum Zitat Engwerda J (2005) LQ dynamic optimization and differential games. Wiley, New York Engwerda J (2005) LQ dynamic optimization and differential games. Wiley, New York
8.
Zurück zum Zitat Feinberg EA, Shwartz A (1999) Constrained dynamic programming with two discount factors: applications and an algorithm. IEEE Trans Autom Control 44(3):628–631MathSciNetMATHCrossRef Feinberg EA, Shwartz A (1999) Constrained dynamic programming with two discount factors: applications and an algorithm. IEEE Trans Autom Control 44(3):628–631MathSciNetMATHCrossRef
9.
Zurück zum Zitat Filar J, Vrieze K (2012) Competitive Markov decision processes. Springer, BerlinMATH Filar J, Vrieze K (2012) Competitive Markov decision processes. Springer, BerlinMATH
10.
Zurück zum Zitat González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2008) Adaptive policies for stochastic systems under a randomized discounted cost criterion. Bol Soc Mat Mexicana 3(14):149–163MathSciNetMATH González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2008) Adaptive policies for stochastic systems under a randomized discounted cost criterion. Bol Soc Mat Mexicana 3(14):149–163MathSciNetMATH
11.
Zurück zum Zitat González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2009) Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion. Kybernetika 45(5):737–754MathSciNetMATH González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2009) Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion. Kybernetika 45(5):737–754MathSciNetMATH
12.
Zurück zum Zitat González-Hernández J, López-Martínez RR, Minjárez-Sosa JA, Gabriel-Arguelles JR (2013) Constrained Markov control processes with randomized discounted cost criteria: occupation measures and extremal points. Risk Decis Anal 4(3):163–176MATHCrossRef González-Hernández J, López-Martínez RR, Minjárez-Sosa JA, Gabriel-Arguelles JR (2013) Constrained Markov control processes with randomized discounted cost criteria: occupation measures and extremal points. Risk Decis Anal 4(3):163–176MATHCrossRef
13.
14.
Zurück zum Zitat Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria, vol 30. Springer, BerlinMATHCrossRef Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria, vol 30. Springer, BerlinMATHCrossRef
15.
Zurück zum Zitat Huang Y, Guo X (2012) Constrained optimality for first passage criteria in semi-Markov decision processes. In: Hernández-Hernández D, Minjárez-Sosa JA (eds) Optimization, control, and applications of stochastic systems, systems & control: foundations & applications. Birkhauser, Boston, pp 181–202 chap. 11CrossRef Huang Y, Guo X (2012) Constrained optimality for first passage criteria in semi-Markov decision processes. In: Hernández-Hernández D, Minjárez-Sosa JA (eds) Optimization, control, and applications of stochastic systems, systems & control: foundations & applications. Birkhauser, Boston, pp 181–202 chap. 11CrossRef
16.
Zurück zum Zitat Huang Y, Wei Q, Guo X (2013) Constrained Markov decision processes with first passage criteria. Ann Oper Res 206(1):197–219MathSciNetMATHCrossRef Huang Y, Wei Q, Guo X (2013) Constrained Markov decision processes with first passage criteria. Ann Oper Res 206(1):197–219MathSciNetMATHCrossRef
17.
18.
Zurück zum Zitat Jaśkiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45(3):773–789MathSciNetMATHCrossRef Jaśkiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45(3):773–789MathSciNetMATHCrossRef
20.
Zurück zum Zitat Luque-Vásquez F (2002) Zero-sum semi-Markov games in Borel spaces: discounted and average payoff. Bol Soc Mat Mexicana 8:227–241MathSciNetMATH Luque-Vásquez F (2002) Zero-sum semi-Markov games in Borel spaces: discounted and average payoff. Bol Soc Mat Mexicana 8:227–241MathSciNetMATH
22.
Zurück zum Zitat Maschler M, Solan E, Zamir S (2013) Game theory. Cambridge University Press, CambridgeMATHCrossRef Maschler M, Solan E, Zamir S (2013) Game theory. Cambridge University Press, CambridgeMATHCrossRef
23.
Zurück zum Zitat Minjárez-Sosa JA (2015) Markov control models with unknown random state-action-dependent discount factors. Top 23(3):743–772MathSciNetMATHCrossRef Minjárez-Sosa JA (2015) Markov control models with unknown random state-action-dependent discount factors. Top 23(3):743–772MathSciNetMATHCrossRef
24.
Zurück zum Zitat Minjárez-Sosa JA, Luque-Vásquez F (2008) Two person zero-sum semi-Markov games with unknown holding times distribution on one side: a discounted payoff criterion. Appl Math Optim 57(3):289–305MathSciNetMATHCrossRef Minjárez-Sosa JA, Luque-Vásquez F (2008) Two person zero-sum semi-Markov games with unknown holding times distribution on one side: a discounted payoff criterion. Appl Math Optim 57(3):289–305MathSciNetMATHCrossRef
25.
Zurück zum Zitat Minjárez-Sosa JA, Vega-Amaya O (2009) Asymptotically optimal strategies for adaptive zero-sum discounted Markov games. SIAM J Control Optim 48(3):1405–1421MathSciNetMATHCrossRef Minjárez-Sosa JA, Vega-Amaya O (2009) Asymptotically optimal strategies for adaptive zero-sum discounted Markov games. SIAM J Control Optim 48(3):1405–1421MathSciNetMATHCrossRef
26.
Zurück zum Zitat Minjárez-Sosa JA, Vega-Amaya O (2013) Optimal strategies for adaptive zero-sum average Markov games. J Math Anal Appl 402(1):44–56MathSciNetMATHCrossRef Minjárez-Sosa JA, Vega-Amaya O (2013) Optimal strategies for adaptive zero-sum average Markov games. J Math Anal Appl 402(1):44–56MathSciNetMATHCrossRef
27.
Zurück zum Zitat Neyman A, Sorin S (2003) Stochastic games and applications, vol 570. Kluwer, DordrechtMATHCrossRef Neyman A, Sorin S (2003) Stochastic games and applications, vol 570. Kluwer, DordrechtMATHCrossRef
28.
Zurück zum Zitat Nowak AS (1984) On zero-sum stochastic games with general state space. I. Prob Math Stat 4(1):13–32MathSciNetMATH Nowak AS (1984) On zero-sum stochastic games with general state space. I. Prob Math Stat 4(1):13–32MathSciNetMATH
29.
Zurück zum Zitat Nowak AS (1985) Measurable selection theorems for minimax stochastic optimization problems. SIAM J Control Optim 23(3):466–476MathSciNetMATHCrossRef Nowak AS (1985) Measurable selection theorems for minimax stochastic optimization problems. SIAM J Control Optim 23(3):466–476MathSciNetMATHCrossRef
30.
Zurück zum Zitat Nowak AS (1987) Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure. J Optim Theory Appl 52(3):429–441MathSciNetMATHCrossRef Nowak AS (1987) Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure. J Optim Theory Appl 52(3):429–441MathSciNetMATHCrossRef
31.
Zurück zum Zitat Nowak AS, Szajowski K (1999) Nonzero-sum stochastic games. In: Stochastic and differential games. Annals of the international society of dynamic games, vol 4, chap 7. Springer, Berlin, pp 297–342 Nowak AS, Szajowski K (1999) Nonzero-sum stochastic games. In: Stochastic and differential games. Annals of the international society of dynamic games, vol 4, chap 7. Springer, Berlin, pp 297–342
32.
Zurück zum Zitat Osborne MJ, Rubinstein A (1994) A course in game theory. MIT Press, CambridgeMATH Osborne MJ, Rubinstein A (1994) A course in game theory. MIT Press, CambridgeMATH
33.
Zurück zum Zitat Rieder U (1991) Non-cooperative dynamic games with general utility functions. In: Raghavan TES, Ferguson TS, Parthasarathy T, Vrieze OJ (eds) Stochastic games and related topics, theory and decision library, vol 7. Springer, Berlin, pp 161–174CrossRef Rieder U (1991) Non-cooperative dynamic games with general utility functions. In: Raghavan TES, Ferguson TS, Parthasarathy T, Vrieze OJ (eds) Stochastic games and related topics, theory and decision library, vol 7. Springer, Berlin, pp 161–174CrossRef
34.
Zurück zum Zitat Schäl M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Probab Theory Rel Fields 32(3):179–196MathSciNetMATH Schäl M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Probab Theory Rel Fields 32(3):179–196MathSciNetMATH
36.
Zurück zum Zitat Shimkin N, Shwartz A (1995) Asymptotically efficient adaptive strategies in repeated games. Part I: certainty equivalence strategies. Math Oper Res 20(3):743–767MathSciNetMATHCrossRef Shimkin N, Shwartz A (1995) Asymptotically efficient adaptive strategies in repeated games. Part I: certainty equivalence strategies. Math Oper Res 20(3):743–767MathSciNetMATHCrossRef
37.
Zurück zum Zitat Shimkin N, Shwartz A (1996) Asymptotically efficient adaptive strategies in repeated games. Part II: asymptotic optimality. Math Oper Res 21(2):487–512MathSciNetMATHCrossRef Shimkin N, Shwartz A (1996) Asymptotically efficient adaptive strategies in repeated games. Part II: asymptotic optimality. Math Oper Res 21(2):487–512MathSciNetMATHCrossRef
38.
Zurück zum Zitat Wei Q, Guo X (2011) Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Oper Res Lett 39(5):369–374MathSciNetMATH Wei Q, Guo X (2011) Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Oper Res Lett 39(5):369–374MathSciNetMATH
Metadaten
Titel
Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies
verfasst von
David González-Sánchez
Fernando Luque-Vásquez
J. Adolfo Minjárez-Sosa
Publikationsdatum
03.03.2018
Verlag
Springer US
Erschienen in
Dynamic Games and Applications / Ausgabe 1/2019
Print ISSN: 2153-0785
Elektronische ISSN: 2153-0793
DOI
https://doi.org/10.1007/s13235-018-0248-8

Weitere Artikel der Ausgabe 1/2019

Dynamic Games and Applications 1/2019 Zur Ausgabe