Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Journal of Scientific Computing
Erschienen in: Journal of Scientific Computing 2/2023

01.05.2023

Accuracy and Architecture Studies of Residual Neural Network Method for Ordinary Differential Equations

verfasst von: Changxin Qiu, Aaron Bendickson, Joshua Kalyanapu, Jue Yan

Erschienen in: Journal of Scientific Computing | Ausgabe 2/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we investigate residual neural network (ResNet) method to solve ordinary differential equations. We verify the accuracy order of ResNet ODE solver matches the accuracy order of the data. Forward Euler, Runge–Kutta2 and Runge–Kutta4 finite difference schemes are adapted generating three learning data sets, which are applied to train three ResNet ODE solvers independently. The well trained ResNet solvers obtain 2nd, 3rd and 5th orders of one step errors and behave just as its counterpart finite difference method for linear and nonlinear ODEs with regular solutions. In particular, we carry out (1) architecture study in terms of number of hidden layers and neurons per layer to obtain optimal network structure; (2) target study to verify the ResNet solver is as accurate as its finite difference method counterpart; (3) solution trajectory simulations. A sequence of numerical examples are presented to demonstrate the accuracy and capability of ResNet solver.
Vorheriger Artikel A Fast Sine Transform Accelerated High-Order Finite Difference Method for Parabolic Problems over Irregular Domains
Nächster Artikel Analysis and Hermite Spectral Approximation of Diffusive-Viscous Wave Equations in Unbounded Domains Arising in Geophysics

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series, The handbook of brain theory and neural networks (1995)
2.
3.
Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)
4.
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRef
5.
Wang, B., Yuan, B., Shi, Z., Osher, S.J.: EnResNet: ResNets ensemble via the Feynman-Kac formalism for adversarial defense and beyond. SIAM J. Math. Data Sci. 2(3), 559–582 (2020)MathSciNetCrossRefMATH
6.
7.
Chaudhari, P., Oberman, A., Osher, S., Soatto, S., Carlier, G.: Deep relaxation: partial differential equations for optimizing deep neural networks (2017). arXiv:​1704.​04932
8.
9.
Chang, B., Meng, L., Haber, E., Ruthotto, L., Begert, D., Holtham, E.: Reversible architectures for arbitrarily deep residual neural networks, in: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), 2018, AAAI Press, 2018, pp. 2811–2818
10.
Ruthotto, L., Haber, E.: Deep neural networks motivated by partial differential equations. J. Math. Imaging Vis. 62(3), 352–364 (2020)MathSciNetCrossRefMATH
11.
Lu, Y., Zhong, A., Li, Q., Dong, B.: Beyond finite layer neural networks: bridging deep architectures and numerical differential equations, arXiv:​1710.​10121 (2017)
12.
He, J., Xu, J.: MgNet: a unified framework of multigrid and convolutional neural network. Sci. China Math. 62(7), 1331–1354 (2019)MathSciNetCrossRefMATH
13.
14.
Hornik, K., Stinchcombe, M., White, H.: Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Netw. 3(5), 551–560 (1990)CrossRef
15.
Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39(3), 930–945 (1993)MathSciNetCrossRefMATH
16.
17.
Lagaris, I., Likas, A., Fotiadis, D.: Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw. 95, 987–1000 (1998)CrossRef
18.
Rudd, K., Ferrari, S.: A constrained integration (cint) approach to solving partial differential equations using artificial neural networks. Neurocomputing 155, 277–285 (2015)CrossRef
19.
Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)MathSciNetCrossRefMATH
20.
Sirignano, J., Spiliopoulos, K.: DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 1339–1364 (2018)MathSciNetCrossRefMATH
21.
Long, Z., Lu, Y., Dong, B.: PDE-Net 2.0: learning PDEs from data with a numeric-symbolic hybrid deep network. J. Comput. Phys. 399, 108925 (2019)MathSciNetCrossRefMATH
22.
Winovich, N., Ramani, K., Lin, G.: ConvPDE-UQ: convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains. J. Comput. Phys. 394, 263–279 (2019)MathSciNetCrossRefMATH
23.
Beck, C.E.W., Jentzen, A.: Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. J. Nonlinear Sci. 29(4), 1563–1619 (2019)MathSciNetCrossRefMATH
24.
Fan, Y., Lin, L., Ying, L., Zepeda-Núñez, L.: A multiscale neural network based on hierarchical matrices. Multiscale Model. Simul. 17(4), 1189–1213 (2019)MathSciNetCrossRefMATH
25.
Khoo, Y., Lu, J., Ying, L.: Solving parametric pde problems with artificial neural networks, Eur. J. Appl. Math. (2020) 1–15
26.
Li, Y., Lu, J., Mao, A.: Variational training of neural network approximations of solution maps for physical models. J. Comput. Phys. 409, 109338 (2020)MathSciNetCrossRefMATH
27.
Qiu, C., Yan, J.: Cell-average based neural network method for hyperbolic and parabolic partial differential equations, J. Comput. Phys. Under review
28.
Qin, T., Wu, K., Xiu, D.: Data driven governing equations approximation using deep neural networks. J. Comput. Phys. 395, 620–635 (2019)MathSciNetCrossRefMATH
29.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) 770–778
30.
Chen, S., Billings, S.A., Grant, P.M.: Non-linear system identification using neural networks. Int. J. Control 51(6), 1191–1214 (1990)CrossRefMATH
31.
González-García, R., Rico-Martínez, R., Kevrekidis, I.: Identification of distributed parameter systems: A neural net based approach, Computers & Chemical Engineering 22 (1998) S965–S968, european Symposium on Computer Aided Process Engineering-8
32.
Milano, M., Koumoutsakos, P.: Neural network modeling for near wall turbulent flow. J. Comput. Phys. 182(1), 1–26 (2002)CrossRefMATH
33.
Pathak, J., Lu, Z., Hunt, B.R., Girvan, M., Ott, E.: Using machine learning to replicate chaotic attractors and calculate lyapunov exponents from data. Chaos Interdiscip. J. Nonlinear Sci. 27(12), 121102 (2017)MathSciNetCrossRefMATH
34.
Vlachas, P. R., Byeon, W., Wan, Z. Y., Sapsis, T. P., Koumoutsakos, P.: Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 474 (2213) (2018) 20170844
35.
Mardt, A., Pasquali, L., Wu, H., Noé, F.: Vampnets: deep learning of molecular kinetics, Nat. Commun. 9 (5) (2018)
36.
Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for koopman operators of nonlinear dynamical systems. Am. Control Conf. (ACC) 2019, 4832–4839 (2019)
37.
Raissi, M., Perdikaris, P., Karniadakis, G. E.: Multistep neural networks for data-driven discovery of nonlinear dynamical systems (2018). arXiv:​1801.​01236
38.
Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary differential equations 12, 6572–6583 (2018)
39.
Rudy, S.H., Kutz, J.N., Brunton, S.L.: Deep learning of dynamics and signal-noise decomposition with time-stepping constraints. J. Comput. Phys. 396, 483–506 (2019)MathSciNetCrossRefMATH
40.
Sun, Y., Zhang, L., Schaeffer, H.: NeuPDE: neural network based ordinary and partial differential equations for modeling time-dependent data, in: Lu, J., Ward, R. (Eds.), Proceedings of The First Mathematical and Scientific Machine Learning Conference, Vol. 107 of Proceedings of Machine Learning Research, PMLR, Princeton University, Princeton, NJ, USA, 2020, pp. 352–372
41.
42.
Xie, X., Zhang, G., Webster, C.G.: Non-intrusive inference reduced order model for fluids using deep multistep neural network. Mathematics 7(8), 757 (2019)CrossRef
43.
44.
Zagoruyko, S., Komodakis, N.: Wide residual networks, Proceedings of the British Machine Vision Conference (BMVC) (87) (2016) 1–12
45.
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K. Q.: Densely connected convolutional networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 2261–2269
46.
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 2017, 5987–5995 (2017)
47.
Haber, E., Ruthotto, L, Holtham, E.: Learning across scales—A multiscale method for convolution neural networks, arXiv arXiv:​1703.​02009 (2017)
48.
Hornik, K.: Approximation capabilities of multilayer feedforward networks. Neural Netw. 4(2), 251–257 (1991)MathSciNetCrossRef
49.
Leshno, M., Lin, V.Y., Pinkus, A., Schocken, S.: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw. 6(6), 861–867 (1993)CrossRef
50.
Venturi, L., Jelassi, S., Ozuch, T., Bruna, J.: Depth separation beyond radial functions. J. Mach. Learn. Res. 23, 1–56 (2022)
51.
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., Amodei, D.: Scaling laws for neural language models (2020). arXiv:​2001.​08361
52.
53.
Boyce, W. E., DiPrima, R. C.: Elementary differential equations and boundary value problems, John Wiley & Sons, Inc., New York-London-Sydney, 10th Edition
54.
Chartrand, R.: Numerical differentiation of noisy, nonsmooth data. ISRN Appl, Math (2011)
55.
Pulch, R.: Polynomial chaos for semiexplicit differential algebraic equations of index 1. Int. J. Uncertain. Quantif. 3(1), 1–23 (2013)MathSciNetCrossRefMATH
Metadaten
Titel
Accuracy and Architecture Studies of Residual Neural Network Method for Ordinary Differential Equations
verfasst von
Changxin Qiu
Aaron Bendickson
Joshua Kalyanapu
Jue Yan
Publikationsdatum
01.05.2023
Verlag
Springer US
Erschienen in
Journal of Scientific Computing / Ausgabe 2/2023
Print ISSN: 0885-7474
Elektronische ISSN: 1573-7691
DOI
https://doi.org/10.1007/s10915-023-02173-x

Weitere Artikel der Ausgabe 2/2023

Journal of Scientific Computing 2/2023 Zur Ausgabe

OriginalPaper

Analysis of a Fourier–Galerkin Method for the Transmission Eigenvalue Problem based on a Boundary Integral Formulation

OriginalPaper

Solving Nonlinear Elliptic Inverse Source, Coefficient and Conductivity Problems by the Methods with Bases Satisfying the Boundary Conditions Automatically

OriginalPaper

A Note on the Shape Regularity of Worsey–Farin Splits

OriginalPaper

Quasi Non-Negative Quaternion Matrix Factorization with Application to Color Face Recognition

OriginalPaper

Stabilizing Discontinuous Galerkin Methods Using Dafermos’ Entropy Rate Criterion: I—One-Dimensional Conservation Laws

OriginalPaper

Unisolvence of Symmetric Node Patterns for Polynomial Spaces on the Simplex

Premium Partner

    Bildnachweise