Skip to main content
Top

2017 | OriginalPaper | Chapter

4. The Minimum Principle and Hamilton–Jacobi–Bellman Equation

Authors : Thomas J. Böhme, Benjamin Frank

Published in: Hybrid Systems, Optimal Control and Hybrid Vehicles

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This chapter is devoted upon optimality, a topic in which the central result is the Pontryagin’s minimum principle. This important result is briefly approached from the classical calculus of variation. We show that the classical calculus of variation has some major limitations to modern control problems and motivate Pontryagin’s minimum principle. The Hamilton–Jacobi–Bellman method is discussed as an alternative approach to gain first-order necessary conditions for optimality. It is shown that both approaches correspond to each other under restrictive assumptions. The original Pontryagin’s minimum principle for continuous optimal control problems is not suitable for hybrid optimal control problems. However, a quite natural reformulation of the hybrid optimal control problem admits the classical theory for deduction of first-order necessary conditions in the sense of Pontryagin. The charm of this methodology is its comprehensible derivation.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Adams RA, Fournier JJ (2003) Sobolev spaces, vol 140, 2nd edn. Academic Press Adams RA, Fournier JJ (2003) Sobolev spaces, vol 140, 2nd edn. Academic Press
2.
go back to reference Azbelev N, Rakhmatullina L (2007) Introduction to the theory of functional differential equations: methods and applications. Contemporary mathematics and its applications. Hindawi Azbelev N, Rakhmatullina L (2007) Introduction to the theory of functional differential equations: methods and applications. Contemporary mathematics and its applications. Hindawi
3.
go back to reference Bardi M, Capuzzo-Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations. Springer Science & Business Media Bardi M, Capuzzo-Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations. Springer Science & Business Media
4.
go back to reference Bellman RE (2003) Dynamic programming, republication of the edition published by Princeton University Press (1957) edn. Dover Publications Bellman RE (2003) Dynamic programming, republication of the edition published by Princeton University Press (1957) edn. Dover Publications
5.
go back to reference Biswas HA, De Pinho MDR (2011) Necessary conditions for optimal control problems with and without state constraints: a comparative study. WSEAS Trans Syst Control 6(6):217–228 Biswas HA, De Pinho MDR (2011) Necessary conditions for optimal control problems with and without state constraints: a comparative study. WSEAS Trans Syst Control 6(6):217–228
6.
go back to reference Biswas MHA (2013) Necessary conditions for optimal control problems with state constraints: theory and applications. PhD thesis, University of Porto, Nov 2013 Biswas MHA (2013) Necessary conditions for optimal control problems with state constraints: theory and applications. PhD thesis, University of Porto, Nov 2013
7.
go back to reference Boltyanski VG, Poznyak A (2011) The robust maximum principle: theory and applications. Springer Science & Business Media Boltyanski VG, Poznyak A (2011) The robust maximum principle: theory and applications. Springer Science & Business Media
8.
go back to reference Bonnans JF, Hermant A (2009) Revisiting the analysis of optimal control problems with several state constraints. Control Cybernet 38(4A):1021–1052MathSciNetMATH Bonnans JF, Hermant A (2009) Revisiting the analysis of optimal control problems with several state constraints. Control Cybernet 38(4A):1021–1052MathSciNetMATH
9.
go back to reference Bonnans JF, Dupuis X, Pfeiffer L (2014) Second-order necessary conditions in Pontryagin form for optimal control problems. SIAM J Control Optim 52(6):3887–3916MathSciNetCrossRefMATH Bonnans JF, Dupuis X, Pfeiffer L (2014) Second-order necessary conditions in Pontryagin form for optimal control problems. SIAM J Control Optim 52(6):3887–3916MathSciNetCrossRefMATH
10.
go back to reference Bryson A, Ho YC (1975) Applied optimal control-optimization. Estimation and control. Taylor & Francis Inc., New York Bryson A, Ho YC (1975) Applied optimal control-optimization. Estimation and control. Taylor & Francis Inc., New York
11.
go back to reference Capuzzo-Dolcetta I (1998) Hamilton-Jacobi-bellman equations and optimal control. Variational calculus, optimal control and applications. Springer, pp 121–132 Capuzzo-Dolcetta I (1998) Hamilton-Jacobi-bellman equations and optimal control. Variational calculus, optimal control and applications. Springer, pp 121–132
12.
go back to reference Caratheodory C (1935) Variationsrechnung und partielle Differentialgleichungen erster Ordnung. B. G, Teubner, Leipzig Caratheodory C (1935) Variationsrechnung und partielle Differentialgleichungen erster Ordnung. B. G, Teubner, Leipzig
13.
go back to reference Cesari L (2012) Optimization theory and applications: problems with ordinary differential equations, vol 17. Springer Science & Business Media Cesari L (2012) Optimization theory and applications: problems with ordinary differential equations, vol 17. Springer Science & Business Media
14.
go back to reference Clarke F (2013) Functional analysis, calculus of variations and optimal control, vol 264. Springer Science & Business Media Clarke F (2013) Functional analysis, calculus of variations and optimal control, vol 264. Springer Science & Business Media
16.
go back to reference Clarke FH (1990) Optimization and nonsmooth analysis, vol 5. Siam Clarke FH (1990) Optimization and nonsmooth analysis, vol 5. Siam
18.
go back to reference Crandall MG, Evans LC, Lions PL (1984) Some properties of viscosity solutions of Hamilton-Jacobi equations. Trans Am Math Soc 282(2):487–502MathSciNetCrossRefMATH Crandall MG, Evans LC, Lions PL (1984) Some properties of viscosity solutions of Hamilton-Jacobi equations. Trans Am Math Soc 282(2):487–502MathSciNetCrossRefMATH
19.
go back to reference Dmitruk A (1993) Maximum principle for the general optimal control problem with phase and regular mixed constraints. Comput Math Model 4(4):364–377MathSciNetCrossRefMATH Dmitruk A (1993) Maximum principle for the general optimal control problem with phase and regular mixed constraints. Comput Math Model 4(4):364–377MathSciNetCrossRefMATH
20.
go back to reference Dmitruk A, Kaganovich A (2008) The hybrid maximum principle is a consequence of Pontryagin maximum principle. Syst Control Lett 57(11):964–970MathSciNetCrossRefMATH Dmitruk A, Kaganovich A (2008) The hybrid maximum principle is a consequence of Pontryagin maximum principle. Syst Control Lett 57(11):964–970MathSciNetCrossRefMATH
21.
go back to reference Dubovitskii AY, Milyutin A (1965) Constrained extremum problems. Zh v?chisl Mat mat Fiz 5(3):395–453 Dubovitskii AY, Milyutin A (1965) Constrained extremum problems. Zh v?chisl Mat mat Fiz 5(3):395–453
22.
24.
go back to reference Gelfand IM, Silverman RA et al (2000) Calculus of variations. Courier Corporation Gelfand IM, Silverman RA et al (2000) Calculus of variations. Courier Corporation
25.
go back to reference Girsanov IV (1972) Lectures on mathematical theory of extremum problems, vol 67. Springer Girsanov IV (1972) Lectures on mathematical theory of extremum problems, vol 67. Springer
26.
go back to reference Goldstine HH (2012) A history of the calculus of variations from the 17th through the 19th century, vol 5. Springer Science & Business Media Goldstine HH (2012) A history of the calculus of variations from the 17th through the 19th century, vol 5. Springer Science & Business Media
27.
go back to reference Graichen K, Petit N (2009) Incorporating a class of constraints into the dynamics of optimal control problems. Optim Control Appl Methods 30(6):537–561MathSciNetCrossRef Graichen K, Petit N (2009) Incorporating a class of constraints into the dynamics of optimal control problems. Optim Control Appl Methods 30(6):537–561MathSciNetCrossRef
28.
go back to reference Hamilton WR (1834) On a general method in dynamics; by which the study of the motions of all free systems of attracting or repelling points is reduced to the search and differentiation of one central relation, or characteristic function. Philos Trans R Soc Lond 124:247–308CrossRef Hamilton WR (1834) On a general method in dynamics; by which the study of the motions of all free systems of attracting or repelling points is reduced to the search and differentiation of one central relation, or characteristic function. Philos Trans R Soc Lond 124:247–308CrossRef
29.
go back to reference Hamilton WR (1835) Second essay on a general method in dynamics. Philos Trans R Soc Lond 125:95–144CrossRef Hamilton WR (1835) Second essay on a general method in dynamics. Philos Trans R Soc Lond 125:95–144CrossRef
30.
go back to reference Hartl RF, Sethi SP, Vickson RG (1995) A survey of the maximum principles for optimal control problems with state constraints. SIAM Rev 37(2):181–218MathSciNetCrossRefMATH Hartl RF, Sethi SP, Vickson RG (1995) A survey of the maximum principles for optimal control problems with state constraints. SIAM Rev 37(2):181–218MathSciNetCrossRefMATH
31.
go back to reference Hestenes MR (1949) Numerical methods of obtaining solutions of fixed end-point problems in the calculus of variations. Technical report, RAND Corporation Hestenes MR (1949) Numerical methods of obtaining solutions of fixed end-point problems in the calculus of variations. Technical report, RAND Corporation
32.
go back to reference Hestenes MR (1950) A general problem in the calculus of variations with applications to paths of least time. Technical report, Rand Corporation Hestenes MR (1950) A general problem in the calculus of variations with applications to paths of least time. Technical report, Rand Corporation
33.
go back to reference Hestenes MR (1966) Calculus of variations and optimal control theory. Wiley Hestenes MR (1966) Calculus of variations and optimal control theory. Wiley
34.
go back to reference Hoehener D (2013) Second-order necessary optimality conditions in state constrained optimal control. In: Proceedings of the 52nd IEEE conference on decision and control (CDC). IEEE, pp 544–549 Hoehener D (2013) Second-order necessary optimality conditions in state constrained optimal control. In: Proceedings of the 52nd IEEE conference on decision and control (CDC). IEEE, pp 544–549
35.
go back to reference Ioffe AD, Tihomirov VM (1979) Theory of extremal problems. North-Holland Ioffe AD, Tihomirov VM (1979) Theory of extremal problems. North-Holland
36.
go back to reference Jacobson DH, Lele M, Speyer JL (1971) New necessary conditions of optimality for control problems with state-variable inequality constraints. J Math Anal Appl 35(2):255–284MathSciNetCrossRefMATH Jacobson DH, Lele M, Speyer JL (1971) New necessary conditions of optimality for control problems with state-variable inequality constraints. J Math Anal Appl 35(2):255–284MathSciNetCrossRefMATH
37.
go back to reference Kirk D (1970) Optimal control theory: an introduction. Englewood Cliffs, N.J., Prentice-Hall Kirk D (1970) Optimal control theory: an introduction. Englewood Cliffs, N.J., Prentice-Hall
38.
go back to reference Knowles G (1981) An introduction to applied optimal control. Academic Press Knowles G (1981) An introduction to applied optimal control. Academic Press
39.
go back to reference Lee EB, Markus L (1967) Foundations of optimal control theory. Technical report, DTIC Document Lee EB, Markus L (1967) Foundations of optimal control theory. Technical report, DTIC Document
40.
go back to reference Liberzon D (2012) Calculus of variations and optimal control theory: a concise introduction. Princeton University Press Liberzon D (2012) Calculus of variations and optimal control theory: a concise introduction. Princeton University Press
41.
go back to reference Maurer H (1979) On the minimum principle for optimal control problems with state constraints. Rechenzentrum d. Univ. Maurer H (1979) On the minimum principle for optimal control problems with state constraints. Rechenzentrum d. Univ.
43.
go back to reference Passenberg B, Leibold M, Stursberg O, Buss M (2011) The minimum principle for time-varying hybrid systems with state switching and jumps. In: Proceedings of the IEEE conference on decision and control, pp 6723–6729 Passenberg B, Leibold M, Stursberg O, Buss M (2011) The minimum principle for time-varying hybrid systems with state switching and jumps. In: Proceedings of the IEEE conference on decision and control, pp 6723–6729
44.
go back to reference Pesch HJ, Plail M (2009) The maximum principle of optimal control: a history of ingenious ideas and missed opportunities. Control Cybern 38(4A):973–995MathSciNetMATH Pesch HJ, Plail M (2009) The maximum principle of optimal control: a history of ingenious ideas and missed opportunities. Control Cybern 38(4A):973–995MathSciNetMATH
45.
go back to reference Pesch HJ, Plail M (2012) The cold war and the maximum principle of optimal control. Optimization Stories Documenta Mathematica Pesch HJ, Plail M (2012) The cold war and the maximum principle of optimal control. Optimization Stories Documenta Mathematica
46.
go back to reference Pontryagin L, Boltyanskii V, Gamkrelidze R, Mishchenko E (1962) The mathematical theory of optimal processes. Wiley Pontryagin L, Boltyanskii V, Gamkrelidze R, Mishchenko E (1962) The mathematical theory of optimal processes. Wiley
47.
go back to reference Riedinger P, Kratz F (2003) An optimal control approach for hybrid systems. Eur J Control 9:449–458CrossRefMATH Riedinger P, Kratz F (2003) An optimal control approach for hybrid systems. Eur J Control 9:449–458CrossRefMATH
48.
go back to reference Riedinger P, Kratz F, Iung C, Zannes C (1999) Linear quadratic optimization for hybrid systems. In: Proceedings of the 38th IEEE conference on decision and control, pp 3059–3064 Riedinger P, Kratz F, Iung C, Zannes C (1999) Linear quadratic optimization for hybrid systems. In: Proceedings of the 38th IEEE conference on decision and control, pp 3059–3064
49.
go back to reference SA Attia Jr, Azhmyakov V (2007) State jump optimization for a class of hybrid autonomous systems. In: Proceedings of the 16th IEEE international conference on control applications, pp 1408–1413 SA Attia Jr, Azhmyakov V (2007) State jump optimization for a class of hybrid autonomous systems. In: Proceedings of the 16th IEEE international conference on control applications, pp 1408–1413
51.
go back to reference Schori M, Boehme TJ, Jeinsch T, Lampe B (2015) Switching time optimization for discontinuous switched systems. In: Proceedings of the 2015 European control conference (ECC), 15–17 July. Linz, IEEE, pp 1742–1747 Schori M, Boehme TJ, Jeinsch T, Lampe B (2015) Switching time optimization for discontinuous switched systems. In: Proceedings of the 2015 European control conference (ECC), 15–17 July. Linz, IEEE, pp 1742–1747
52.
go back to reference Shaikh MS (2004) Optimal control of hybrid systems: theory and algorithms. PhD thesis, Department of Electrical and Computer Engineering, McGill University, Montreal Shaikh MS (2004) Optimal control of hybrid systems: theory and algorithms. PhD thesis, Department of Electrical and Computer Engineering, McGill University, Montreal
53.
go back to reference Sussmann HJ (1999) A maximum principle for hybrid optimal control problems. In: Proceedings of the 38th IEEE conference on decision and control, Phoenix, vol 1. IEEE, pp 425–430 Sussmann HJ (1999) A maximum principle for hybrid optimal control problems. In: Proceedings of the 38th IEEE conference on decision and control, Phoenix, vol 1. IEEE, pp 425–430
54.
go back to reference Sussmann HJ (2000) Transversality, set separation, and the proof of maximum principle, Part II Sussmann HJ (2000) Transversality, set separation, and the proof of maximum principle, Part II
56.
go back to reference Xu X, Antsaklis P (2003) Optimal control of hybrid autonomous systems with state jumps. In: Proceedings of the American control conference. IEEE, pp 5191–5196 Xu X, Antsaklis P (2003) Optimal control of hybrid autonomous systems with state jumps. In: Proceedings of the American control conference. IEEE, pp 5191–5196
57.
go back to reference Xu X (2001) Analysis and design of switched systems. PhD thesis, University of Notre Dame Xu X (2001) Analysis and design of switched systems. PhD thesis, University of Notre Dame
Metadata
Title
The Minimum Principle and Hamilton–Jacobi–Bellman Equation
Authors
Thomas J. Böhme
Benjamin Frank
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-51317-1_4