Optimal control problems with continuous value functions: restricted state space

In this chapter we continue the study of optimal control problems with continuous value functions and consider cost functionals involving the exit time from a given domain, in particular time-optimal control, and infinite horizon problems with constraints on the state variables. The continuity of the value function for these problems is not as easy as in the previous chapter. For time-optimal control this is essentially the problem of small-time local controllability. We give the proof of just a few simple results on this topic, and state without proof several others. For each problem we characterize the value function as the unique viscosity solution of the appropriate Hamilton-Jacobi-Bellman equation and boundary conditions. We do not give all the applications of this theory, as verification functions and conditions of optimality: most of them can be obtained by the arguments of Chapter III and are left as exercises for the reader.