Introduction

Since the 1950s two of the classes of dynamic, stochastic, decision models that have been studied extensively by applied mathematicians, operations researchers, electrical engineers, and mathematical economists are Stochastic Games and Markov Decision Processes, respectively. The name Stochastic Games stems from the seminal paper by Shapley (1953) even though some authors use the name Markov Games, which probably dates back to Zachrisson (1964). Markov decision processes are also called Controlled Markov Chains by the engineers, and it appears that their evolution was stimulated by the books of Bellman (1957) and Howard (1960).