On sets of occupational measures generated by a deterministic control system on an infinite time horizon
Section snippets
Introduction and preliminaries
It is well known that nonlinear optimal control problems can be equivalently reformulated as infinite dimensional linear programming problems considered on spaces of occupational measures generated by control-state trajectories. Having many attractive features and being applicable in both stochastic and deterministic settings, the linear programming (LP) based approaches to optimal control problems have been intensively studied in the literature. Important results justifying the use of LP
A representation of the set of discounted occupational measures
Lemma 2.1 The following inclusion is validwhere stands for the closed convex hull, and
Proof Take arbitrary . By definition, there exists a relaxed -admissible pair such that is the discounted occupational measure generated by this pair. Using the fact that (11) is valid for any continuous function , one can obtain
Proof of Theorem 2.2
We will divide the proof into four steps.
(i) Auxiliary relaxed admissible pairs. Let the multivalued function be defined by the equation It is easy to verify that is upper semicontinuous. Hence, its graph is compact.
Let and be closed balls in such that and
Let be measurable and let satisfy the equation
Proof of Lemma 2.4
To prove (22), it is enough to show that (28) implies (32). Note that from (28) (and from (25)) it follows that, for any sequence , there exist a sequence of and a sequence of -admissible pairs such that From Lemma 3.5(ii) in [28] it follows that there exists a sequence , , such that ( being a constant) and such that
Acknowledgments
The work of the first author was partially funded by the Australian Research Council Discovery-Project Grants DP0664330, DP120100532, and DP130104432 and by the Linkage International Grant LX0560049. The second author was partially supported by project SADCO, FP7-PEOPLE-2010-ITN, No. 264735 and he was also supported partially by the French National Research AgencyANR-10-BLAN 0112.
References (29)
- et al.
The problem of optimal control with reflection studied through a linear optimization problem stated on occupational measures
Nonlinear Anal.
(2010) - et al.
Filippov’s and Filippov–Wazewski’s theorems on closed domains
Journal of Differential Equations
(2000) On the relation between discounted and average optimal value functions
Journal of Differential Equations
(1998)- et al.
Occupation measures for controlled markov processes: characterization and optimality
Annals of Probability
(1996) - et al.
Convex duality approach to the optimal control of diffusions
SIAM Journal on Control and Optimization
(1989) - et al.
Existence of Markov controls and characterization of optimal markov controls
SIAM Journal on Control and Optimization
(1998) Time-Average Control of a Martingale Problem. Existence of a Stationary Solution
Annals of Probability
(1990)- R.H. Stockbridge, Time-average control of a martingale problem: a linear programming formulation, Annals of...
- et al.
Ergodic control for constrained diffusions: characterization using HJB equations. (English summary)
SIAM Journal on Control and Optimization
(2004/05) - et al.
Stochastic optimal control and linear programming approach
Applied Mathematics and Optimization
(2011)
On the existence of strict optimal controls for constrained, controlled Markov processes in continuous-time
Stochastics
A note on linearization methods and dynamic programming principles for stochastic discontinuous control problems
Electronic Communications in Probability
The linear programming approach to deterministic optimal control problems
Applicationes Mathematicae
Markov Chains and Invariant Probabilities
Cited by (31)
On representation formulas for long run averaging optimal control problem
2015, Journal of Differential EquationsLINEAR PROGRAMMING ESTIMATES FOR CESÀRO AND ABEL LIMITS OF OPTIMAL VALUES IN OPTIMAL CONTROL PROBLEMS
2022, Discrete and Continuous Dynamical Systems - Series BUnique Ergodicity of Deterministic Zero-Sum Differential Games
2021, Dynamic Games and ApplicationsLP Based Bounds for Cesàro and Abel Limits of the Optimal Values in Non-ergodic Stochastic Systems
2021, 2021 European Control Conference, ECC 2021