Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Buchtitelbild

2018 | OriginalPaper | Buchkapitel

1. Optimal Control

verfasst von : Rushikesh Kamalapurkar, Patrick Walters, Joel Rosenfeld, Warren Dixon

Erschienen in: Reinforcement Learning for Optimal Feedback Control

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This chapter serves as a brief introduction to optimal control. Dynamic programming-based solutions to optimal control problems are derived, and the connections between the methods based on dynamic programming and the methods based on the calculus of variations are discussed. This chapter is by no means a comprehensive treatment of the subject. For more details, the reader is directed to the excellent sources cited in the text. The chapter ends with a brief survey of techniques to solve optimal control problems.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Nächstes Kapitel Approximate Dynamic Programming
Literatur
1.
Carathéodory C (1918) Vorlesungen über reelle Funktionen. Teubner
2.
Coddington EA, Levinson N (1955) Theory of ordinary differential equations. McGraw-Hill
3.
Pontryagin LS, Boltyanskii VG, Gamkrelidze RV, Mishchenko EF (1962) The mathematical theory of optimal processes. Interscience, New York
4.
Bellman R (1954) The theory of dynamic programming. Technical report, DTIC Document
5.
Kirk D (2004) Optimal control theory: an introduction. Dover, Mineola
6.
Bryson AE, Ho Y (1975) Applied optimal control: optimization, estimation, and control. Hemisphere Publishing Corporation
7.
Liberzon D (2012) Calculus of variations and optimal control theory: a concise introduction. Princeton University Press
8.
9.
Beard R, Saridis G, Wen J (1997) Galerkin approximations of the generalized Hamilton–Jacobi–Bellman equation. Automatica 33:2159–2178MathSciNetMATHCrossRef
10.
11.
Bardi M, Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. SpringerMATHCrossRef
12.
Cimen T, Banks SP (2004) Global optimal feedback control for general nonlinear systems with nonquadratic performance criteria. Syst Control Lett 53(5):327–346MathSciNetMATHCrossRef
13.
Cheng T, Lewis FL, Abu-Khalaf M (2007) A neural network solution for fixed-final time optimal control of nonlinear systems. Automatica 43(3):482–490MathSciNetMATHCrossRef
14.
Cheng T, Lewis FL, Abu-Khalaf M (2007) Fixed-final-time-constrained optimal control of nonlinear systems using neural network HJB approach. IEEE Trans Neural Netw 18(6):1725–1737
15.
Kar I, Adhyaru D, Gopal M (2009) Fixed final time optimal control approach for bounded robust controller design using Hamilton–Jacobi–Bellman solution. IET Control Theory Appl 3(9):1183–1195MathSciNetCrossRef
16.
Wang F, Jin N, Liu D, Wei Q (2011) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with epsilon-error bound. IEEE Trans Neural Netw 22:24–36CrossRef
17.
Heydari A, Balakrishnan SN (2012) An optimal tracking approach to formation control of nonlinear multi-agent systems. In: Proceedings of AIAA guidance, navigation and control conference
18.
Wang D, Liu D, Wei Q (2012) Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 78(1):14–22CrossRef
19.
Zhao Q, Xu H, Jagannathan S (2015) Neural network-based finite-horizon optimal control of uncertain affine nonlinear discrete-time systems. IEEE Trans Neural Netw Learn Syst 26(3):486–499MathSciNetCrossRef
20.
Li C, Liu D, Li H (2015) Finite horizon optimal tracking control of partially unknown linear continuous-time systems using policy iteration. IET Control Theory Appl 9(12):1791–1801MathSciNetCrossRef
21.
Ge SS, Zhang J (2003) Neural-network control of nonaffine nonlinear system with zero dynamics by state and output feedback. IEEE Trans Neural Netw 14(4):900–918CrossRef
22.
Wang D, Liu D, Wei Q, Zhao D, Jin N (2012) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8):1825–1832MathSciNetMATHCrossRef
23.
Zhang X, Zhang H, Sun Q, Luo Y (2012) Adaptive dynamic programming-based optimal control of unknown nonaffine nonlinear discrete-time systems with proof of convergence. Neurocomputing 91:48–55CrossRef
24.
Liu D, Huang Y, Wang D, Wei Q (2013) Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming. Int J Control 86(9):1554–1566MathSciNetMATHCrossRef
25.
Bian T, Jiang Y, Jiang ZP (2014) Adaptive dynamic programming and optimal control of nonlinear nonaffine systems. Automatica 50(10):2624–2632MathSciNetMATHCrossRef
26.
Yang X, Liu D, Wei Q, Wang D (2015) Direct adaptive control for a class of discrete-time unknown nonaffine nonlinear systems using neural networks. Int J Robust Nonlinear Control 25(12):1844–1861MathSciNetMATHCrossRef
27.
Kiumarsi B, Kang W, Lewis FL (2016) H-\(\infty \) control of nonaffine aerial systems using off-policy reinforcement learning. Unmanned Syst 4(1):1–10
28.
Song R, Wei Q, Xiao W (2016) Off-policy neuro-optimal control for unknown complex-valued nonlinear systems based on policy iteration. Neural Comput Appl 46(1):85–95
29.
Lyashevskiy S, Meyer AU (1995) Control system analysis and design upon the Lyapunov method. In: Proceedings of the American control conference, vol 5, pp 3219–3223
30.
Fahroo F, Ross IM (2008) Pseudospectral methods for infinite-horizon nonlinear optimal control problems. J Guid Control Dyn 31(4):927–936CrossRef
31.
Pickenhain S (2014) Hilbert space treatment of optimal control problems with infinite horizon. In: Bock GH, Hoang PX, Rannacher R, Schlöder PJ (eds) Modeling, simulation and optimization of complex processes - HPSC 2012: Proceedings of the fifth international conference on high performance scientific computing, 5–9 March 2012, Hanoi, Vietnam. Springer International Publishing, Cham, pp 169–182
32.
Tauchnitz N (2015) The pontryagin maximum principle for nonlinear optimal control problems with infinite horizon. J Optim Theory Appl 167(1):27–48MathSciNetMATHCrossRef
33.
34.
Aseev SM, Kryazhimskii A (2007) The pontryagin maximum principle and optimal economic growth problems. Proc Steklov Inst Math 257(1):1–255MathSciNetMATHCrossRef
35.
Aseev SM, Veliov VM (2015) Maximum principle for infinite-horizon optimal control problems under weak regularity assumptions. Proc Steklov Inst Math 291(1):22–39MathSciNetMATHCrossRef
36.
37.
38.
Hargraves CR, Paris S (1987) Direct trajectory optimization using nonlinear programming and collocation. J Guid Control Dyn 10(4):338–342MATHCrossRef
39.
Huntington GT (2007) Advancement and analysis of a gauss pseudospectral transcription for optimal control. Ph.D thesis, Department of Aeronautics and Astronautics, MIT
40.
Rao AV, Benson DA, Darby CL, Patterson MA, Francolin C, Huntington GT (2010) Algorithm 902: GPOPS, A MATLAB software for solving multiple-phase optimal control problems using the Gauss pseudospectral method. ACM Trans Math Softw 37(2):1–39MATHCrossRef
41.
Darby CL, Hager WW, Rao AV (2011) An hp-adaptive pseudospectral method for solving optimal control problems. Optim Control Appl Methods 32(4):476–502MathSciNetMATHCrossRef
42.
Garg D, Hager WW, Rao AV (2011) Pseudospectral methods for solving infinite-horizon optimal control problems. Automatica 47(4):829–837MathSciNetMATHCrossRef
43.
Freeman R, Kokotovic P (1995) Optimal nonlinear controllers for feedback linearizable systems. In: Proceedings of the American control conference, pp 2722–2726
44.
Lu Q, Sun Y, Xu Z, Mochizuki T (1996) Decentralized nonlinear optimal excitation control. IEEE Trans Power Syst 11(4):1957–1962CrossRef
45.
Nevistic V, Primbs JA (1996) Constrained nonlinear optimal control: a converse HJB approach. Technical report CIT-CDS 96-021, California Institute of Technology, Pasadena, CA 91125
46.
Primbs JA, Nevistic V (1996) Optimality of nonlinear design techniques: A converse HJB approach. Technical report CIT-CDS 96-022, California Institute of Technology, Pasadena, CA 91125
47.
Sekoguchi M, Konishi H, Goto M, Yokoyama A, Lu Q (2002) Nonlinear optimal control applied to STATCOM for power system stabilization. In: Proceedings of the IEEE/PES transmission and distribution conference and exhibition, pp 342–347
48.
Kim Y, Lewis FL (2000) Optimal design of CMAC neural-network controller for robot manipulators. IEEE Trans Syst Man Cybern Part C Appl Rev 30(1):22–31CrossRef
49.
Kim Y, Lewis FL, Dawson D (2000) Intelligent optimal control of robotic manipulator using neural networks. Automatica 36(9):1355–1364MathSciNetMATHCrossRef
50.
Dupree K, Patre P, Wilcox Z, Dixon WE (2008) Optimal control of uncertain nonlinear systems using rise feedback. In: Proceedings of the IEEE conference on decision and control, Cancun, Mexico, pp 2154–2159
51.
Dupree K, Patre PM, Wilcox ZD, Dixon WE (2009) Optimal control of uncertain nonlinear systems using a neural network and rise feedback. In: Proceedings of the American control conference, St. Louis, Missouri, pp 361–366
52.
Freeman RA, Kokotovic PV (1996) Robust nonlinear control design: state-space and lyapunov techniques. Birkhäuser, Boston
53.
Fausz J, Chellaboina VS, Haddad W (1997) Inverse optimal adaptive control for nonlinear uncertain systems with exogenous disturbances. In: Proceedings of the IEEE conference on decision and control, pp 2654–2659
54.
55.
Krstic M, Li ZH (1998) Inverse optimal design of input-to-state stabilizing nonlinear controllers. IEEE Trans Autom Control 43(3):336–350MathSciNetMATHCrossRef
56.
57.
Luo W, Chu YC, Ling KV (2005) Inverse optimal adaptive control for attitude tracking of spacecraft. IEEE Trans Autom Control 50(11):1639–1654MathSciNetMATHCrossRef
58.
Dupree K, Johnson M, Patre PM, Dixon WE (2009) Inverse optimal control of a nonlinear Euler-Lagrange system, part ii: Output feedback. In: Proceedings of the IEEE conference on decision and control, Shanghai, China, pp 327–332
59.
Dupree K, Patre PM, Johnson M, Dixon WE (2009) Inverse optimal adaptive control of a nonlinear Euler-Lagrange system: Part i. In: Proceedings of the IEEE conference on decision and control, Shanghai, China, pp 321–326
60.
Johnson M, Hu G, Dupree K, Dixon WE (2009) Inverse optimal homography-based visual servo control via an uncalibrated camera. In: Proceedings of the IEEE conference on decision and control, Shanghai, China, pp 2408–2413
61.
Wang Q, Sharma N, Johnson M, Gregory CM, Dixon WE (2013) Adaptive inverse optimal neuromuscular electrical stimulation. IEEE Trans Cybern 43:1710–1718CrossRef
62.
Cloutier JR (1997) State-dependent riccati equation techniques: an overview. In: Proceedings of the American control conference, 2:932–936
63.
Çimen T (2008) State-dependent riccati equation (SDRE) control: a survey. In: Proceedings IFAC World Congress, pp 6–11CrossRef
64.
Cimen T (2010) Systematic and effective design of nonlinear feedback controllers via the state-dependent riccati equation (sdre) method. Annu Rev Control 34(1):32–51CrossRef
65.
Yucelen T, Sadahalli AS, Pourboghrat F (2010) Online solution of state dependent riccati equation for nonlinear system stabilization. In: Proceedings of the American control conference, pp 6336–6341
66.
Garcia CE, Prett DM, Morari M (1989) Model predictive control: theory and practice - a survey. Automatica 25(3):335–348MATHCrossRef
67.
68.
Morari M, Lee J (1999) Model predictive control: past, present and future. Comput Chem Eng 23(4–5):667–682CrossRef
69.
Allgöwer F, Zheng A (2000) Nonlinear model predictive control, vol 26. Springer
70.
Mayne D, Rawlings J, Rao C, Scokaert P (2000) Constrained model predictive control: Stability and optimality. Automatica 36:789–814MathSciNetMATHCrossRef
71.
Camacho EF, Bordons C (2004) Model predictive control, vol 2. Springer
72.
73.
Isaacs R (1999) Differential games: a mathematical theory with applications to warfare and pursuit, control and optimization. Dover books on mathematics, Dover Publications
74.
75.
Basar T, Olsder GJ (1999) Dynamic noncooperative game theory, 2nd edn. Classics in applied mathematics, SIAM
76.
77.
Vamvoudakis KG, Lewis FL (2011) Policy iteration algorithm for distributed networks and graphical games. In: Proceedings of the IEEE conference decision control European control conference, pp 128–135
78.
Vamvoudakis KG, Lewis FL, Hudas GR (2012) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611MathSciNetMATHCrossRef
79.
Dolcetta IC (1983) On a discrete approximation of the hamilton-jacobi equation of dynamic programming. Appl Math Optim 10(1):367–377MathSciNetMATHCrossRef
80.
Sethian JA (1999) Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science. Cambridge University Press
81.
Osher S, Fedkiw R (2006) Level set methods and dynamic implicit surfaces, vol 153. Springer Science & Business Media
82.
Doya K (2000) Reinforcement learning in continuous time and space. Neural Comput 12(1):219–245CrossRef
83.
Vrabie D, Lewis FL (2009) Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Netw 22(3):237–246MATHCrossRef
Metadaten
Titel
Optimal Control
verfasst von
Rushikesh Kamalapurkar
Patrick Walters
Joel Rosenfeld
Warren Dixon
Copyright-Jahr
2018
Buch
Reinforcement Learning for Optimal Feedback Control

Print ISBN: 978-3-319-78383-3

Electronic ISBN: 978-3-319-78384-0

Copyright-Jahr: 2018

DOI
https://doi.org/10.1007/978-3-319-78384-0_1

Neuer Inhalt

    Bildnachweise