Achievements and Future Tasks in Applied Stochastic Decision Processes

In my opinion the most important achievements (from an applied viewpoint) in the area of stochastic decision processes, especially stationary Markov and semi-Markov decision processes, have been: 1.Solving the finite horizon problem by backward induction.2.Approximating a large finite horizon by an infinite horizon thus obtaining a simpler problem, especially a fixed-point equation.2a.Solving this fixed point equation by successive approximations, by policy iteration, by linear programming and by a lot of mixed procedures, transformations and reordering methods.2b.Constructing bounds (especially two sided) for the infinite horizon value and thus stopping rules for the successive approximation.2c.Forecasting and eliminating actions which are non-optimal for the infinite horizon or for the next step of successive approximation.3.Direct approximation of large finite horizon problems by small- horizon problems using the methods of 2b and 2c.4.Indirect approximation of large-horizon problems by interpolation between small finite and infinite horizons, by the same methods.5.Approximation by smaller state and action spaces: either one fixed partition, or sequential refining of partitions.6.Results for structured problems, especially the optimality or almost optimality of simple policies.