Introduction to Gaussian Process Regression in Bayesian Inverse Problems, with New Results on Experimental Design for Weighted Error Measures

Bayesian posterior distributions arising in modern applications are often computationally intractable due to the large computational cost of evaluating the data likelihood. Examples include inverse problems in partial differential equation models arising in climate modeling and in subsurface fluid flow. To alleviate the problem of expensive likelihood evaluation, a natural approach is to use Gaussian process regression to build a surrogate model for the likelihood, resulting in an approximate posterior distribution that is amenable to computations in practice. This paper serves as an introduction to Gaussian process regression, in particular in the context of building surrogate models for inverse problems; we also present new insights into a suitable choice of training points, motivated by the use of Gaussian processes in approximate Bayesian inversion. We show that the error between the true and approximate posterior distribution can be bounded by the error between the true and approximate likelihood, measured in the \(L^2\)-norm weighted by the true posterior; furthermore we show that minimizing the error between the true and approximate likelihood in this norm suggests choosing the training points in the Gaussian process surrogate model based on the true posterior.