Top

Decisions in Economics and Finance

Published in:

21-01-2017

An iterative computational scheme for solving the coupled Hamilton–Jacobi–Isaacs equations in nonzero-sum differential games of affine nonlinear systems

Author: M. D. S. Aliyu

Published in: Decisions in Economics and Finance | Issue 1-2/2017

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this paper, we present iterative or successive approximation methods for solving the coupled Hamilton–Jacobi–Isaacs equations (HJIEs) arising in nonzero-sum differential game for affine nonlinear systems. We particularly consider the ones arising in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) control. However, the approach is perfectly general and can be applied to any others including those arising in the N-player case. The convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the utility of the method. The results are also specialized to the coupled algebraic Riccati equations arising typically in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) linear control. In this case, a bound within which the optimal solution lies is established. Finally, based on the iterative approach developed, a local existence result for the solution of the coupled-HJIEs is also established.

next article Reaching nirvana with a defaultable asset?

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Ignoring this assumption only makes the manipulations more lengthy. But the results remain the same.

Abou-Kandil, H., Freiling, G., Jank, G.: Solution and asymptotic behavior of coupled Riccati equations in jump linear systems. IEEE Trans. Autom. Control 39(8), 1631–1636 (1994)CrossRef

Abu-Khalaf, M., Lewis, F.L., Huang, J.: Policy iterations on the Hamilton–Jacobi–Isaacs equation for \({\cal{H}}_{\infty }\) state-feedback control with input saturation. IEEE Trans. Autom. Control 51(12), 1989–1993 (2006)CrossRef

Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. IFAC J. Autom. 41(5), 779–791 (2005)CrossRef

Aliyu, M.D.S.: An approach for solving the Hamilton–Jacobi–Isaac equations arising in nonlinear \(\cal{H}_{\infty }\) control. IFAC J. Autom. 38, 877–884 (2003a)CrossRef

Aliyu, M.D.S.: A transformation approach for solving the Hamilton–Jacobi–Bellman equations in \({\cal{H}}_{2}\) deterministic and stochastic optimal control of affine nonlinear systems. IFAC J. Autom. 39, 1243–1249 (2003b)CrossRef

Aliyu, M.D.S.: Nonlinear \({\cal{H}}_{\infty }\) Control, Hamiltonian Systems and Hamilton–Jacobi Equations. CRC Press, Taylor and Francis, Boca Raton (2011)CrossRef

Aliyu, M.D.S., Smolinsky, L.: A Parameterization approach for solving the Hamilton-Jacobi equation and application to the \({\cal{A}}_{2}\) Toda lattice. Nonlinear Dyn. Syst. Theory 5(4), 323–344 (2005)

Barbu, V., Prata, Da G.: Hamilton–Jacobi Equations in Hilbert Space. Pitman Advanced Publishing Program, London (1983)

Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Systems & Control Foundations, Birhauser (1997)CrossRef

Basar, T., Bernhard, P.: \({\cal{H}}_{\infty }\) Optimal Control and Related Minimax Design. Birkhauser, New York (1991)

Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory. Academic Press, New York (1982)

Beard, R.W., Saridas, G.N., Wen, J.T.: Galerkin approximations of the generalized HJB equation. IFAC J. Autom. 33(12), 2159–2177 (1997)CrossRef

Beard, R.W., Saridas, G.N., Wen, J.T.: Successive Galerkin approximation algorithms for nonlinear optimal and robust control. Int. J. Control 71(5), 717–743 (1998)CrossRef

Benton, S.H.: The Hamilton–Jacobi Equation: A Global Approach. Academic Press, New York (1977)

Chen, B.-S., Chang, Y.-C.: Nonlinear mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control for robust tracking of robotic systems. Int. J. Control 67(6), 837–857 (1998)CrossRef

Coddington, E.A., Levinsone, N.: Ordinary Differential Equations. McGraw Hill, New York (1955)

Cox, D., Little, J., O’Shea, D.: Ideals, Varieties and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3rd edn. Springer, New York (2007)CrossRef

Denman, E.D.: An additional algorithm for a system of coupled algebraic matrix Riccati equations. IEEE Trans. Comput. 26(1), 91–93 (1976)CrossRef

Doyle, J.C., Zhou, K., Glover, K., Bodenheimer, B.: Optimal control with mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) performance objectives-II: optimal control. IEEE Trans. Autom. Control 39(8), 1575–1587 (1994)CrossRef

Evans, L.C.: Partial Differential Equations, Graduate Studies in Mathematics. American Mathmtical Society, Providence (1998)

Fleming, W.H., Soner, M.: Controlled Markov Processes and Viscosity Solutions. Springer, New York (2006)

Freiling, G., Jank, G., AbouKandil, H.: On the global existence of coupled matrix Riccati equations in closed-loop Nash games. IEEE Trans. Autom. Control 41(2), 264–269 (1996)CrossRef

Freiling, G., Lee, S.-R., Jank, G.: Coupled matrix Riccati equations in minimum cost variance control problems. IEEE Trans. Autom. Control 44(3), 556–560 (1999)CrossRef

Glad, S.T.: Robustness of nonlinear state-feedback—a survey. IFAC J. Autom. 23, 425–435 (1987)CrossRef

Isaacs, R.: Differential Games, SIAM Series in Applied Mathematics. Wiley, New York (1965)

Jiang, Y., Jiang, Z.-P.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Control 60(11), 2917–2929 (2015)CrossRef

Johnson, M., Bhasin, S., Dixon, W.E.: Nonlinear two-player zero-sum game approximate solution using a policy iteration algorithm. In: Proceedings 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), pp. 142–147 (2011)

Khalil, H.K.: Nonlinear Systems. Mcmillan Publishers, New York (1992)

Kleinmann, D.L.: On an iterative technique for Riccati equation computations. IEEE Trans. Autom. Control 13, 114–115 (1968)CrossRef

Lee, C.-H.: An improved lower matrix bound for the solution of unified coupled Riccati equation. IEEE Trans. Autom. Control 50(8), 1221–1223 (2005)CrossRef

Li, T.-Y., Gajic, Z.: Lyapunov Iterations for Solving Coupled Algebraic Riccati Equations of Nash Differential games and algebraic Riccati Equations of zero-sum games, New Trends in Dynamic Games and Applications. Birkhuser, Boston (1995)

Limebeer, D.J.N., Anderson, B.D.O., Hendel, B.: A Nash game approach to mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control problem. IEEE Trans. Autom. Control 39(4), 824–839 (1994)

Lin, W.: Mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control for nonlinear systems. Int. J. Control 64(5), 899–922 (1996)CrossRef

Lions, P.L.: Generalized Solutions of Hamilton–Jacobi Equations. Research Notes in Mathematics. Pitman Advanced Publishing Program, London (1982)

Lukes, D.L.: Optimal regulation of nonlinear dynamical systems. SIAM J. Control 7, 75–100 (1969)CrossRef

Meyer, G.G.L., Payne, H.J.: An iterative method of solution of the algebraic Riccati equation. IEEE Trans. Autom. Control 17(6), 550–551 (1972)CrossRef

Ohtsuka, T.: Solutions to the Hamilton–Jacobi equation with algebraic gradients. IEEE Trans. Autom. Control 56(8), 1874–1885 (2011)CrossRef

Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, London (1970)

Papavassilopouos, G.P., Medanic, J.V., Cruz, J.B.: On the existence of Nash strategies and solutions to coupled Riccati equations in linear-quadratic Nash games. J. Optim. Theory Appl. 28(4), 49–76 (1979)CrossRef

Papavassilopouos, G.P., Olsder, G.J.: On linear-quadratic closed-loop no memory Nash game. J. Optim. Theory Appl. 42(4), 551–560 (1984)CrossRef

Salama, A.I.A., Ghourishankar, V.: A Computational algorithm for solving a system of coupled algebraic Riccati equations. IEEE Trans. Comput. 23(1), 100–102 (1974)CrossRef

Starr, A.W., Ho, W.C.: Nonzero-sum differential games. J. Optim. Theory Appl. 3(4), 207–219 (1969a)CrossRef

Starr, A.W., Ho, W.C.: Further properties of nonzero-sum differential games. J. Optim. Theory Appl. 3(3), 184–206 (1969b)CrossRef

Vamvoudakis, K.G., Lewis, F.G.: Nonzero-sum differential games: online learning solution of coupled Hamilton–Jacobi and coupled Riccati equations. In: Proceedings of IEEE International Symposium on Intelligent Control, pp. 171–178, Denver, Colorado, USA (2011a)

Vamvoudakis, K.G., Lewis, F.L.: Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton–Jacobi equations. IFAC J Autom. 47(8), 1556–1569 (2011b)CrossRef

Vamvoudakis, K.G., Lewis, F.L.: Online neural network solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int. J. Robust Nonlinear Control 22(13), 1460–1483 (2012)CrossRef

Vit, K.: Iterative solution of the Riccati equation. IEEE Trans. Autom. Control 17(2), 258–259 (1972)CrossRef

Yong, J., Zhou, X.Y.: Stochastic Controls. Hamiltonian Systems and HJB Equations. Springer, New York (1999)

Zeidler, E.: Nonlinear Functional Analysis and Its Applications: Fixed Point Theorems, vol. 1. Springer, Hiedelberg (1985)CrossRef

Zhou, K., Glover, K., Bodenheimer, B., Doyle, J.C.: Mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) performance objectives-I: robust performance analysis. IEEE Trans. Autom. Control 39(8), 1564–1574 (1994)CrossRef

Title: An iterative computational scheme for solving the coupled Hamilton–Jacobi–Isaacs equations in nonzero-sum differential games of affine nonlinear systems
Author: M. D. S. Aliyu
Publication date: 21-01-2017
Publisher: Springer Milan
Published in: Decisions in Economics and Finance / Issue 1-2/2017
Print ISSN: 1593-8883
Electronic ISSN: 1129-6569
DOI: https://doi.org/10.1007/s10203-017-0184-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1-2/2017

Cyclically monotone equilibrium problems and Ekeland’s principle

Approximating exact expected utility via portfolio efficient frontiers

Generating the efficient frontier of a class of bicriteria generalized fractional programming

A set optimization approach to utility maximization under transaction costs

Robust games: theory and application to a Cournot duopoly model

An axiomatization of continuous quasilinear utility