Abstract
This paper proposes a new collocation method for estimating parameters of a partial differential equation (PDE), which uses Gaussian process (GP) as a basis function and is termed as Gaussian process for partial differential equation (GPPDE). The conventional method of estimating parameters of a differential equation is to minimize the error between observations and their estimates. The estimates are produced from the forward solution (numerical or analytical) of the differential equation. The conventional approach requires initial and boundary conditions, and discretization of differential equations if the forward solution is obtained numerically. The proposed method requires fitting a GP regression model to the observations of the state variable, then obtaining derivatives of the state variable using the property that derivative of a GP is also a GP, and finally adjusting the PDE parameters so that the GP derived partial derivatives satisfy the PDE. The method does not require initial and boundary conditions, however if these conditions are available (exactly or with measurement errors), they can be easily incorporated. The GPPDE method is evaluated by applying it on the diffusion and the Richards equations. The results suggest that GPPDE can correctly estimate parameters of the two equations. For the Richards equation, GPPDE performs well in the presence of noise. A comparison of GPPDE with HYDRUS-1D software showed that their performances are comparable, though GPPDE has significant advantages in terms of computational time. GPPDE could be an effective alternative to conventional approaches for finding parameters of high-dimensional PDEs where large datasets are available.
Similar content being viewed by others
Abbreviations
- AGM:
-
Adaptive gradient matching
- GM:
-
Gradient matching
- ASMO:
-
Adaptive surrogate modeling based optimization
- ASMO-PODE:
-
ASMO-parameter optimization and distribution estimation
- GP:
-
Gaussian process
- GPODE:
-
Gaussian process ordinary differential equation
- GPPDE:
-
Gaussian process for partial differential equation
- ODE:
-
Ordinary differential equation
- PDE:
-
Partial differential equation
- \(\alpha\) :
-
Air-entry pressure head soil hydraulic parameter
- \(\lambda\) :
-
Diffusivity
- \(\varvec{\lambda }\) :
-
Set of parameters of PDE
- \(\varvec{\phi }\) :
-
\([l, \sigma _s]\)
- \(\varvec{\varPhi }\) :
-
\([l, \sigma _s, \sigma _y]\)
- \(\sigma _s\) :
-
Hyperparameter of GP (scale parameter of kernel function)
- \(\sigma _y\) :
-
Hyperparameter of GP (noise in GP model)
- \(\theta\) :
-
Volumetric soil moisture
- \(\theta _r\) :
-
Residual soil moisture
- \(\theta _s\) :
-
Saturated soil moisture
- \(\Theta\) :
-
Relative saturation; \(\dfrac{\theta-\theta_r}{\theta_s-\theta_r}\)
- \({\mathbf {I}}\) :
-
Identity matrix
- \(I({\varvec{\lambda }})\) :
-
Fisher information matrix
- k :
-
Soil hydraulic conductivity
- \(k_s\) :
-
Saturated soil hydraulic conductivity
- \(k(\cdot )\) :
-
Covariance/kernel function
- \({\mathbf {K}}\) :
-
Covariance matrix
- l :
-
Hyperparameter of GP (length scale of kernel function)
- m :
-
Number of independent variables
- \(m_y\) :
-
Mean of Gaussian process
- n :
-
Pore-size distribution soil hydraulic parameter
- N :
-
Number of observations
- \({\mathbb {R}}\) :
-
Real space
- t :
-
Temporal dimension
- x :
-
Independent variable/a spatial dimension
- \(\mathbf {x}\) :
-
Vector of training independent variables
- \({{\mathbf {x}}'}\) :
-
Vector of testing independent variables
- \(\mathbf {X}\) :
-
Design matrix of dimension \(N \times m\)
- y :
-
State variable
- \({\mathbf {y}}\) :
-
Column vector of state variable
- \({\tilde{y}}\) :
-
Observation
- \({\tilde{{\mathbf {y}}}}\) :
-
Column vector of N observations
- z :
-
A spatial dimension
References
Bilionis I (2016) Probabilistic solvers for partial differential equations. arXiv preprint arXiv:160703526
Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin
Calderhead B, Girolami M, Lawrence ND (2008) Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes. In: Advances in neural information processing systems, pp 217–224
Celia MA, Bouloutas ET, Zarba RL (1990) A general mass-conservative numerical solution for the unsaturated flow equation. Water Resour Res 26(7):1483–1496
Cockayne J, Oates CJ, Sullivan T, Girolami M (2016) Probabilistic meshless methods for partial differential equations and Bayesian inverse problems. arXiv preprint arXiv:160507811
Deb K (2012) Optimization for engineering design: algorithms and examples. PHI Learning Pvt. Ltd., New Delhi
Dondelinger F, Husmeier D, Rogers S, Filippone M (2013) ODE parameter inference using adaptive gradient matching with Gaussian processes. In: Artificial intelligence and statistics, pp 216–228
Farthing MW, Ogden FL (2017) Numerical solution of richards equation: a review of advances and challenges. Soil Sci Soc Am J 81:1257–1269
Fletcher R (2013) Practical methods of optimization. Wiley, Hoboken
Gong W, Duan Q (2017) An adaptive surrogate modeling-based sampling strategy for parameter optimization and distribution estimation (ASMO-PODE). Enviro Model Softw 95:61–75
Graepel T (2003) Solving noisy linear operator equations by Gaussian processes: application to ordinary and partial differential equations. In: ICML, pp 234–241
Hinton GE (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14(8):1771–1800
Holman D, Sridharan M, Gowda P, Porter D, Marek T, Howell T, Moorhead J (2014) Gaussian process models for reference ET estimation from alternative meteorological data sources. J Hydrol 517:28–35
Kool J, Parker J, van Genuchten MT (1985) Determining soil hydraulic properties from one-step outflow experiments by parameter estimation: I. Theory and numerical studies. Soil Sci Soc Am J 49(6):1348–1354
Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J South Afr Inst Min Metall 52(6):119–139
Lagarias JC, Reeds JA, Wright MH, Wright PE (1998) Convergence properties of the nelder-mead simplex method in low dimensions. SIAM J Optim 9(1):112–147
Macdonald B, Husmeier D (2015) Gradient matching methods for computational inference in mechanistic models for systems biology: a review and comparative analysis. Front Bioeng Biotechnol 3:180
Macdonald B, Higham C, Husmeier D (2015) Controversy in mechanistic modelling with Gaussian processes. In: International conference on machine learning, pp 1539–1547
Marrel A, Iooss B, Laurent B, Roustant O (2009) Calculations of sobol indices for the gaussian process metamodel. Reliab Eng Syst Saf 94(3):742–751
Matheron G (1963) Principles of geostatistics. Econ Geol 58(8):1246–1266
Mayraz G, Hinton GE (2001) Recognizing hand-written digits using hierarchical products of experts. In: Advances in neural information processing systems, pp 953–959
McLaughlin D, Townley LR (1996) A reassessment of the groundwater inverse problem. Water Resour Res 32(5):1131–1161
Newton I (1736) The method of fluxions and infinite series. Henry Woodfall, London
Nocedal J, Wright S (1999) Numerical optimization. Springer, New York
Poyton A, Varziri MS, McAuley KB, McLellan P, Ramsay JO (2006) Parameter estimation in continuous-time dynamic models using principal differential analysis. Comput Chem Eng 30(4):698–708
Raissi M, Perdikaris P, Karniadakis GE (2017a) Inferring solutions of differential equations using noisy multi-fidelity data. J Comput Phys 335:736–746
Raissi M, Perdikaris P, Karniadakis GE (2017b) Machine learning of linear differential equations using Gaussian processes. J Comput Phys 348:683–693
Raissi M, Perdikaris P, Karniadakis GE (2018) Numerical Gaussian processes for time-dependent and nonlinear partial differential equations. SIAM J Sci Comput 40(1):A172–A198
Ramsay JO, Hooker G, Campbell D, Cao J (2007) Parameter estimation for differential equations: a generalized smoothing approach. J R Stat Soc Ser B (Stat Methodol) 69(5):741–796
Rao CR (2008) Cramr-Rao bound. Scholarpedia 3(8):6533. https://doi.org/10.4249/scholarpedia.6533
Rao SS (2009) Engineering optimization: theory and practice. Wiley, Hoboken
Rasmussen CE, Williams CK (2006) Gaussian processes for machine learning, vol 1. MIT Press, Cambridge
Reynolds RJ (1987) Diffusivity of glacial-outwash aquifer by the floodwave-response technique. Groundwater 25(3):290–299
Richards LA (1931) Capillary conduction of liquids through porous mediums. Physics 1(5):318–333
Ritter A, Hupet F, Muñoz-Carpena R, Lambot S, Vanclooster M (2003) Using inverse methods for estimating soil hydraulic properties from field data as an alternative to direct methods. Agric Water Manag 59(2):77–96
Rohatgi A (2011) WebPlotDigitizer. http://arohatgi.info/WebPlotDigitizer/app. Accessed 15 Feb 2018
Särkkä S (2011) Linear operators and stochastic partial differential equations in Gaussian process regression. In: International conference on artificial neural networks. Springer, pp 151–158
Simunek J, van Genuchten MT, Sejna M (2005) The HYDRUS-1D software package for simulating the one-dimensional movement of water, heat, and multiple solutes in variably-saturated media. Univ Calif-Riverside Res Rep 3:1–240
Solak E, Murray-Smith R, Leithead WE, Leith DJ, Rasmussen CE (2003) Derivative observations in Gaussian process models of dynamic systems. In: Advances in neural information processing systems, pp 1057–1064
Srivastava R (2006) Aquifer diffusivity estimation from response to stream stage variation. J Hydrol Eng 11(3):273–277
Strang G (2014) Differential equations and linear algebra. Wellesley-Cambridge Press, Wellesley
Sun AY, Wang D, Xu X (2014) Monthly streamflow forecasting using Gaussian process regression. J Hydrol 511:72–81
Swamee PK, Singh SK (2003) Estimation of aquifer diffusivity from stream stage variation. J Hydrol Eng 8(1):20–24
Tripathi S, Govindaraju RS (2007) On selection of kernel parametes in relevance vector machines for hydrologic applications. Stoch Environ Res Risk Assess 21(6):747–764
van Genuchten MT (1980) A closed-form equation for predicting the hydraulic conductivity of unsaturated soils. Soil Sci Soc Am J 44(5):892–898
Varah JM (1982) A spline least squares method for numerical parameter estimation in differential equations. SIAM J Sci Stat Comput 3(1):28–46
Vyshemirsky V, Girolami MA (2007) Bayesian ranking of biochemical system models. Bioinformatics 24(6):833–839
Wang C, Duan Q, Gong W, Ye A, Di Z, Miao C (2014) An evaluation of adaptive surrogate modeling based optimization with two benchmark problems. Environ Model Softw 60:167–179
Wang Y, Barber D (2014) Gaussian processes for Bayesian estimation in ordinary differential equations. In: International conference on machine learning, pp 1485–1493
Wu A, Aoi MC, Pillow JW (2017) Exploiting gradients and hessians in Bayesian optimization and Bayesian quadrature. arXiv preprint arXiv:170400060
Zhang J, Li W, Zeng L, Wu L (2016) An adaptive Gaussian process-based method for efficient Bayesian experimental design in groundwater contaminant source identification problems. Water Resour Res 52(8):5971–5984
Acknowledgements
Pankaj Kumar Rai received scholarship from Ministry of Human Resource Development, Government of India, for his Ph.D. program. Authors are thankful of Mr. Hemanta Medhi and Mr. Pramod Soni for their help in preparing the manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1: Derivative of Gaussian covariance function \(k({{\mathbf {x}}},{\mathbf {x'}})\)
Gaussian covariance function is given by relation
where \(l_i\)’s and \({\sigma _s}\) are the length and the scale parameters, respectively.
1.1 Derivative of covariance function with respect to hyperparameters
The derivatives of \({k_y}\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) ={k}\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) +\sigma _y^2\), where \(\sigma _y\) is the noise term, are given by.
1.2 Derivatives of covariance function with respect to arguments
First derivative
Second derivative
Auto covariance
Appendix 2: Factorization of the Richards equation
This appendix provides the representation of the Richards equation in terms of state variable \((\theta )\) and its partial derivatives \(\left( {\dfrac{{\partial \theta }}{{\partial t}},\dfrac{{\partial \theta }}{{\partial x}}\ {\mathrm {and}} \ \dfrac{{{\partial ^2}\theta }}{{\partial {x^2}}}} \right)\).
The van Genuchten (1980) model is used to relate hydraulic conductivity (k), pressure head (h) and relative saturation \((\Theta )\) as
where \({k_s}\) is saturated hydraulic conductivity, and \({\theta _r}\) and \({\theta _s}\) are residual and saturated soil moisture content, respectively. The relative saturation terms \(\alpha \,\,({\hbox {L}}^{-1})\), m and n represent model parameters where, m and n are related as \(n\left( {1 - m} \right) = 1\). Term \((\Theta )\) is defined as
To write the Richards equation in terms of soil moisture and its partial derivatives, we need to estimate \(\dfrac{\partial {\Theta }}{\partial {\theta }}\), \(\dfrac{\partial {k}}{\partial {\Theta }}\), \(\dfrac{\partial {\Theta }}{\partial {h}}\) and \(\dfrac{\partial {h}}{\partial {\theta }}\). These partial derivatives were estimated as follows. Differentiating \(\Theta\) with respect to \(\theta\)
Differentiating (34) with respect to \(\Theta\)
Rearranging Eq. (34)
Again rearranging (33) and differentiating it with respect to h
Substituting (38) into (41) (to represent \(\dfrac{\partial \Theta }{\partial h}\) as a function of \(\Theta )\)
where \({B_0} = - \dfrac{{\alpha m{\mathop {\mathrm{sign}}\nolimits } \left( h \right) }}{{1 - m}}\)
Using Eq. (42), \(\dfrac{{\partial h}}{{\partial \theta }}\) could be obtained as
The Richards equation can be factorized into three terms as
Term1:
Term1 is obtained by substituting (42) in (47).
Term2:
Deriving \(\dfrac{{{\partial ^2}h}}{{\partial z\partial \theta }}\) by differentiating (43) with respect to z.
Term2 is obtained by substituting (43) and (51) in (48)
Term3:
Using (36), (53) and (37), Eq. (52) is expressed as
Rights and permissions
About this article
Cite this article
Rai, P.K., Tripathi, S. Gaussian process for estimating parameters of partial differential equations and its application to the Richards equation. Stoch Environ Res Risk Assess 33, 1629–1649 (2019). https://doi.org/10.1007/s00477-019-01709-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00477-019-01709-8