6. Dynamic Programming

This chapter introduces the discrete dynamic programming methodology. Dynamic programming is an appealing approach for the solution of (hybrid) optimal control problems. The theoretical foundation is based on the Hamilton–Jacobi–Bellman equation and is relatively easy to understand compared to the much more involved indirect methods. The general algorithm can be implemented in a simple form using only elementary operations compared with the more complex operations in direct methods. The dynamic programming paradigm is fairly general: it is easy to apply to purely continuous optimal control problems and with some minor reformulations it is also well suited for hybrid optimal control problems. Despite these appealing attributes, dynamic programming suffers from some serious drawbacks.