Top

Published in:

2024 | OriginalPaper | Chapter

Deep-Control of Memory via Stochastic Optimal Control and Deep Learning

Author : Emel Savku

Published in: Mathematical Methods for Engineering Applications

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this survey work, we introduce Stochastic Differential Delay Equations and their impacts on Stochastic Optimal Control problems. We observe time delay in the dynamics of a state process that may correspond to inertia or memory in a financial system. For such systems, we demonstrate two special approaches to handle delayed control problems by applying the Dynamic Programming Principle. Moreover, we clarify the technical challenges rising as a consequence of the conflict between the path-dependent, infinite-dimensional nature of the problem and the necessity of the Markov property. Furthermore, we present two different Deep Learning algorithms to solve targeted delayed control tasks and illustrate the results for a complete memory portfolio optimization problem.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Forms of Assessment in view of Development of Mathematical Competencies

next chapter Exponentiated Weibull Mixture Cure Model to Handle Right-Censored Data Set

https://github.com/frankhan91/RNN-ControlwithDelay.

Bayraktar, E., Keller, C.: Path-dependent Hamilton–Jacobi equations in infinite dimensions. J. Funct. Anal. 275, 2096–2161 (2018)MathSciNetCrossRef

Blechschmidt, J., Ernst, O.: Three ways to solve partial differential equations with neural networks—A review. GAMM-Mitteilungen 44, e202100006 (2021)MathSciNetCrossRef

Castro, J.: The Kolmogorov infinite dimensional equation in a Hilbert space via deep learning methods. Preprint. ArXiv:2206.06451 (2022)

Chang, M., Pang, T., Pemy, M.: Finite difference approximations for stochastic control systems with delay. Stoch. Anal. Appl. 26, 451–470 (2008)MathSciNetCrossRef

Chang, M., Pang, T., Pemy, M.: Optimal control of stochastic functional differential equations with a bounded memory. Stochastics Int. J. Probab. Stoch. Process. 80, 69–96 (2008)MathSciNetCrossRef

Chen, C., Sun, D., Chang, C.: Numerical solution of time-delayed optimal control problems by iterative dynamic programming. Optim. Control Appl. Methods 21, 91–105 (2000)MathSciNetCrossRef

Crandall, M., Lions, P.: Viscosity solutions of Hamilton-Jacobi equations. Trans. Am. Math. Soc. 277, 1–42 (1983)MathSciNetCrossRef

Ekren, I., Touzi, N., Zhang, J.: Viscosity solutions of fully nonlinear parabolic path dependent PDEs: Part I. Ann. Probab. Off. J. Inst. Math. Stat. 44, 1212–1253 (2016)MathSciNet

Elsanosi, I., Larssen, B.: Optimal consumption under partial observations for a stochastic system with delay. Preprint Series. Pure Mathematics. Nb. No/URN: NBN: No-8076 (2001)

10.

Elsanosi, I., Øksendal, B., Sulem, A.: Some solvable stochastic control problems with delay. Stochastics Int. J. Probab. Stoch. Process. 71, 69–89 (2000)MathSciNet

11.

Fabbri, G., Gozzi, F., Swiech, A.: Stochastic Optimal Control in Infinite Dimension. Probability and Stochastic Modelling. Springer (2017)CrossRef

12.

Federico, S.: A stochastic control problem with delay arising in a pension fund model. Finance Stoch. 15, 421–459 (2011)MathSciNetCrossRef

13.

Federico, S., Tankov, P.: Finite-dimensional representations for controlled diffusions with delay. Appl. Math. Optim. 71, 165–194 (2015)MathSciNetCrossRef

14.

Federico, S., Goldys, B., Gozzi, F.: HJB equations for the optimal control of differential equations with delays and state constraints, I: regularity of viscosity solutions. SIAM J. Control Optim. 48, 4910–4937 (2010)

15.

Feo, F., Federico, S., Świech, A.: Optimal control of stochastic delay differential equations and applications to path-dependent financial and economic models. Preprint. ArXiv:2302.08809 (2023)

16.

Fischer, M., Nappo, G.: Time discretisation and rate of convergence for the optimal control of continuous-time stochastic systems with delay. Appl. Math. Optim. 57, 177–206 (2008)MathSciNetCrossRef

17.

Fischer, M., Reiss, M.: Discretisation of stochastic control problems for continuous time dynamics with delay. J. Comput. Appl. Math. 205, 969–981 (2007)MathSciNetCrossRef

18.

Fleming, W., Soner, H.: Controlled Markov Processes and Viscosity Solutions. Springer Science & Business Media (2006)

19.

Fouque, J., Zhang, Z.: Deep learning methods for mean field control problems with delay. Front. Appl. Math. Stat. 6, 11 (2020)CrossRef

20.

Germain, M., Pham, H., Warin, X.: Neural networks-based algorithms for stochastic control and PDEs in finance. Preprint. ArXiv:2101.08068 (2021)

21.

Gozzi, F., Masiero, F.: Stochastic optimal control with delay in the control I: Solving the HJB equation through partial smoothing. SIAM J. Control Optim. 55, 2981–3012 (2017)MathSciNetCrossRef

22.

Gozzi, F., Masiero, F.: Stochastic optimal control with delay in the control II: Verification theorem and optimal feedbacks. SIAM J. Control Optim. 55, 3013–3038 (2017)MathSciNetCrossRef

23.

Gozzi, F., Masiero, F.: Errata: Stochastic optimal control with delay in the control I: Solving the HJB equation through partial smoothing, and stochastic optimal control with delay in the control II: verification theorem and optimal feedbacks. SIAM J. Control Optim. 59, 3096–3101 (2021)MathSciNetCrossRef

24.

Gozzi, F., Roma, S., Marinelli, C.: 13 Stochastic Optimal Control of Delay Equations Arising in Advertising Models. Stochastic Partial Differential Equations and Applications-VII, pp. 133 (2005)

25.

Gozzi, F., Marinelli, C., Savin, S.: On controlled linear diffusions with delay in a model of optimal advertising under uncertainty with memory effects. J. Optim. Theory Appl. 142, 291–321 (2009)MathSciNetCrossRef

26.

Han, J., Hu, R.: Recurrent neural networks for stochastic control problems with delay. Math. Control Signals Syst. 33, 775–795 (2021)MathSciNetCrossRef

27.

Han, J., Jentzen, A., et al.: Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat. 5, 349–380 (2017)MathSciNetCrossRef

28.

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)CrossRef

29.

Ishii, H., Lions, P.: Viscosity solutions of fully nonlinear second-order elliptic partial differential equations. J. Differential Equations 83, 26–78 (1990)MathSciNetCrossRef

30.

Kim, J., Yang, I.: Hamilton-Jacobi-Bellman equations for q-learning in continuous time. In: Learning For Dynamics and Control, pp. 739–748 (2020)

31.

Kushner, H.: Numerical approximations for stochastic systems with delays in the state and control. Stochastics Int. J. Probab. Stoch. Process. 78, 343–376 (2006)MathSciNetCrossRef

32.

Kushner, H.: Numerical Methods for Controlled Stochastic Delay Systems. Springer (2008)

33.

Larssen, B.: Dynamic programming in stochastic control of systems with delay. Stochastics Int. J. Probab. Stoch. Process. 74, 651–673 (2002)MathSciNet

34.

Larssen, B., Risebro, N.: When are HJB-equations in Stochastic Control of Delay Systems Finite Dimensional? Taylor & Francis (2003)

35.

Mohammed, S.: Stochastic Functional Differential Equations. Pitman Advanced Publishing Program (1984)

36.

Mohammed, S.: Stochastic differential systems with memory: theory, examples and applications. In: Stochastic Analysis and Related Topics VI: Proceedings of the Sixth Oslo—Silivri Workshop Geilo 1996, pp. 1–77 (1998)MathSciNet

37.

Nakamura-Zimmerer, T., Gong, Q., Kang, W.: Adaptive deep learning for high-dimensional Hamilton–Jacobi–Bellman equations. SIAM J. Sci. Comput. 43, A1221–A1247 (2021)MathSciNetCrossRef

38.

Øksendal, B., Sulem, A.: Applied Stochastic Control of Jump Diffusions. Springer (2007)

39.

Øksendal, B., Sulem, A., Zhang, T.: Optimal control of stochastic delay equations and time-advanced backward stochastic differential equations. Adv. Appl. Probab. 43, 572–596 (2011)MathSciNetCrossRef

40.

Pang, T., Hussain, A.: A stochastic portfolio optimization model with complete memory. Stoch. Anal. Appl. 35, 742–766 (2017)MathSciNetCrossRef

41.

Peng, S., Yang, Z.: Anticipated backward stochastic differential equations. Ann. Probab. 37, 877–902 (2009)MathSciNetCrossRef

42.

Pham, H.: Continuous-time Stochastic Control and Optimization with Financial Applications. Springer Science & Business Media (2009)

43.

Savku, E.: Memory and Anticipation: Two main theorems for Markov regime-switching stochastic processes. Preprint. ArXiv:2302.13890 (2023)

44.

Savku, E., Weber, G.: A stochastic maximum principle for a Markov regime-switching jump-diffusion model with delay and an application to finance. J. Optim. Theory Appl. 179, 696–721 (2018)MathSciNetCrossRef

45.

Savku, E., Weber, G.: A regime-switching model with applications to finance: Markovian and Non-Markovian cases. In: Dynamic Economic Problems with Regime Switches, pp. 287–309 (2021)

46.

Van Houdt, G., Mosquera, C., Nápoles, G.: A review on the long short-term memory model. Artif. Intell. Rev. 53, 5929–5955 (2020)CrossRef

47.

Yong, J., Zhou, X.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer Science & Business Media (1999)

Title: Deep-Control of Memory via Stochastic Optimal Control and Deep Learning
Author: Emel Savku
Publisher: Springer Nature Switzerland
Book: Mathematical Methods for Engineering Applications
Print ISBN: 978-3-031-49217-4

Electronic ISBN: 978-3-031-49218-1

Copyright Year: 2024
DOI: https://doi.org/10.1007/978-3-031-49218-1_16

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"