Abstract
We provide a bound for the variation of the function that assigns to every competitive Markov decision process and every discount factor its discounted value. This bound implies that the undiscounted value of a competitive Markov decision process is continuous in the relative interior of the space of transition rules.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amir, R. (1987). Sequential Games of Resource Extraction: Existence of Nash Equilibrium, Cowles Foundation D.P. #825.
Arapostathis, A., Borkar, V. S., Fernández-Gaucherand, E.,Ghosh, M. K., and Marcus, S. I. (1993). Discrete-time controlled Markov processes with average cost criterion: A survey. SIAM J. Control Optim. 31,282–344.
Bewley, T., and Kohlberg, E. (1976). The asymptotic theory of stochastic games. Math. Oper. Res. 1, 197–208.
Catoni, O. (1999). Simulated Annealing Algorithms and Markov Chains with Rare Transitions, Séminaire de Probabilités, XXXIII, 69–119, Lecture Notes in Mathematics, 1709, Springer, Berlin.
Filar, J. A. (1985). Player aggregation in the traveling inspector model, IEEE Trans. Automatic Control AC-30, 723–729.
Filar, J. A., and Vrieze, K. (1997). Competitive Markov Decision Processes, Springer-Verlag, New York.
Freidlin, M. I., and Wentzell, A. D. (1984). Random Perturbations of Dynamical Systems, Springer-Verlag, Berlin.
Levhari, D., and Mirman, L. (1980). The great fish war: An example using a dynamic Cournot–Nash solution. Bell J. Econ. 11, 322–334.
Mertens, J. F., and Neyman, A. (1981). Stochastic games. Int. J. Game theory 10, 53–66.
Milman, E. (2002). The semi-algebraic theory of stochastic games. Math. Oper. Res. 27, 401–418.
Schweizer, P. J. (1968). Perturbation theory and finite Markov chains. J. Applied Probab. 5, 401–413.
Shapley, L. S. (1953). Stochastic games. Proc. Nat. Acad. Sci. U.S.A. 39, 1095–1100.
Sorin, S. (2002). A First Course on Zero-Sum Repeated Games, Mathématiques et Applications, Vol. 37, Springer-Verlag, Berlin.
Winston, W. (1978). A stochastic game model of a weapons development competition. SIAM J. Control Optim. 16, 411–419.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Solan, E. Continuity of the Value of Competitive Markov Decision Processes. Journal of Theoretical Probability 16, 831–845 (2003). https://doi.org/10.1023/B:JOTP.0000011995.28536.ef
Issue Date:
DOI: https://doi.org/10.1023/B:JOTP.0000011995.28536.ef