Hostname: page-component-7c8c6479df-27gpq Total loading time: 0 Render date: 2024-03-27T08:08:22.648Z Has data issue: false hasContentIssue false

Inverse problems: A Bayesian perspective

Published online by Cambridge University Press:  10 May 2010

A. M. Stuart
Affiliation:
Mathematics Institute, University of Warwick, Coventry CV4 7AL, UK, E-mail: a.m.stuart@warwick.ac.uk

Extract

The subject of inverse problems in differential equations is of enormous practical importance, and has also generated substantial mathematical and computational innovation. Typically some form of regularization is required to ameliorate ill-posed behaviour. In this article we review the Bayesian approach to regularization, developing a function space viewpoint on the subject. This approach allows for a full characterization of all possible solutions, and their relative probabilities, whilst simultaneously forcing significant modelling issues to be addressed in a clear and precise fashion. Although expensive to implement, this approach is starting to lie within the range of the available computational resources in many application areas. It also allows for the quantification of uncertainty and risk, something which is increasingly demanded by these applications. Furthermore, the approach is conceptually important for the understanding of simpler, computationally expedient approaches to inverse problems.

Type
Research Article
Copyright
Copyright © Cambridge University Press 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Adler, R. J. (1990), An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes, Vol. 12 of Institute of Mathematical Statistics Lecture Notes: Monograph Series, Institute of Mathematical Statistics, Hayward, CA.Google Scholar
Akella, S. and Navon, I. (2009), ‘Different approaches to model error formulation in 4D-Var: A study with high resolution advection schemes’, Tellus 61A, 112128.Google Scholar
Alekseev, A. and Navon, I. (2001), ‘The analysis of an ill-posed problem using multiscale resolution and second order adjoint techniques’, Comput. Meth. Appl. Mech. Engrg 190, 19371953.Google Scholar
Antoulas, A., Soresen, D. and Gugerrin, S. (2001), A Survey of Model Reduction Methods for Large Scale Dynamical Systems, AMS.Google Scholar
Apte, A., Hairer, M., Stuart, A. and Voss, J. (2007), ‘Sampling the posterior: An approach to non-Gaussian data assimilation’, Physica D 230, 5064.Google Scholar
Apte, A., Jones, C. and Stuart, A. (2008 a), ‘A Bayesian approach to Lagrangian data assimilation’, Tellus 60, 336347.Google Scholar
Apte, A., Jones, C., Stuart, A. and Voss, J. (2008 b), ‘Data assimilation: Mathematical and statistical perspectives’, Internat. J. Numer. Methods Fluids 56, 10331046.Google Scholar
Archambeau, C., Cornford, D., Opper, M. and Shawe, J.-Taylor (2007), Gaussian process approximations of stochastic differential equations. In JMLR Workshop and Conference Proceedings 1: Gaussian Processes in Practice (Lawrence, N., ed.), The MIT Press, pp. 116.Google Scholar
Archambeau, C., Opper, M., Shen, Y., Cornford, D. and Shawe-Taylor, J. (2008), Variational inference for diffusion processes. In Advances in Neural Information Processing Systems 20 (Platt, J., Koller, D., Singer, Y. and Roweis, S., eds), The MIT Press, Cambridge, MA, pp. 1724.Google Scholar
Backus, G. (1970 a), ‘Inference from inadequate and inaccurate data I’, Proc. Nat. Acad. Sci. 65, 17.Google Scholar
Backus, G. (1970 b), ‘Inference from inadequate and inaccurate data II’, Proc. Nat. Acad. Sci. 65, 281287.Google Scholar
Backus, G. (1970 c), ‘Inference from inadequate and inaccurate data III’, Proc. Nat. Acad. Sci. 67, 282289.Google Scholar
Bain, A. and Crisan, D. (2009), Fundamentals of Stochastic Filtering, Springer.Google Scholar
Bannister, R., Katz, D., Cullen, M., Lawless, A. and Nichols, N. (2008), ‘Modelling of forecast errors in geophysical fluid flows’, Internat. J. Numer. Methods Fluids 56, 11471153.Google Scholar
Beck, J., Blackwell, B. and Clair, C. (2005), Inverse Heat Conduction: Ill-Posed Problems, Wiley.Google Scholar
Bell, M., Martin, M. and Nichols, N. (2004), ‘Assimilation of data into an ocean model with systematic errors near the equator’, Quart. J. Royal Met. Soc. 130, 873894.Google Scholar
Bengtsson, T., Bickel, P. and Li, B. (2008), ‘Curse of dimensionality revisited: The collapse of importance sampling in very large scale systems’, IMS Collections: Probability and Statistics: Essays in Honor of David Freedman 2, 316334.Google Scholar
Bengtsson, T., Snyder, C. and Nychka, D. (2003), ‘Toward a nonlinear ensemble filter for high-dimensional systems’, J. Geophys. Res. 108, 8775.Google Scholar
Bennett, A. (2002), Inverse Modeling of the Ocean and Atmosphere, Cambridge University Press.Google Scholar
Bennett, A. and Budgell, W. (1987), ‘Ocean data assimilation and the Kalman filter: Spatial regularity’, J. Phys. Oceanography 17, 15831601.Google Scholar
Bennett, A. and Chua, B. (1994), ‘Open ocean modelling as an inverse problem’, Monthly Weather Review 122, 13261336.Google Scholar
Bennett, A. and Miller, R. (1990), ‘Weighting initial conditions in variational assimilation schemes’, Monthly Weather Review 119, 10981102.Google Scholar
Bergemann, K. and Reich, S. (2010), ‘A localization technique for ensemble transform Kalman filters’, Quart. J. Royal Met. Soc. To appear.Google Scholar
Berliner, L. (2001), ‘Monte Carlo based ensemble forecasting’, Statist. Comput. 11, 269275.Google Scholar
Bernardo, J. and Smith, A. (1994), Bayesian Theory, Wiley.Google Scholar
Beskos, A. and Stuart, A. (2009), MCMC methods for sampling function space. In Invited Lectures: Sixth International Congress on Industrial and Applied Mathematics, ICIAM07 (Jeltsch, R. and Wanner, G., eds), European Mathematical Society, pp. 337364.Google Scholar
Beskos, A. and Stuart, A. M. (2010), Computational complexity of Metropolis Hastings methods in high dimensions. In Monte Carlo and Quasi-Monte Carlo Methods 2008 (L'Ecuyer, P. and Owen, A. B., eds), Springer, pp. 6172.Google Scholar
Beskos, A., Roberts, G. O. and Stuart, A. M. (2009), ‘Optimal scalings for local Metropolis-Hastings chains on non-product targets in high dimensions’, Ann. Appl. Probab. 19, 863898.Google Scholar
Beskos, A., Roberts, G. O., Stuart, A. M. and Voss, J. (2008), ‘MCMC methods for diffusion bridges’, Stochastic Dynamics 8, 319350.Google Scholar
Bickel, P. and Doksum, K. (2001), Mathematical Statistics, Prentice-Hall.Google Scholar
Bickel, P., Li, B. and Bengtsson, T. (2008), ‘Sharp failure rates for the bootstrap particle filter in high dimensions’, IMS Collections: Pushing the Limits of Contemporary Statistics 3, 318329.Google Scholar
Bogachev, V. (1998), Gaussian Measures, AMS.Google Scholar
Bolhuis, P., Chandler, D., Dellago, D. and Geissler, P. (2002), ‘Transition path sampling: Throwing ropes over rough mountain passes’, Ann. Rev. Phys. Chem. 53, 291318.Google Scholar
Borcea, L. (2002), ‘Electrical impedence tomography’, Inverse Problems 18, R99–R136.Google Scholar
Brasseur, P., Bahurel, P., Bertino, L., Birol, F., Brankart, J.-M., Ferry, N., Losa, S., Remy, E., Schroeter, J., Skachko, S., Testut, C.-E., Tranchat, B., Van Leeuwen, P. and Verron, J. (2005), ‘Data assimilation for marine monitoring and prediction: The Mercator operational assimilation systems and the Mersea developments’, Quart. J. Royal Met. Soc. 131, 35613582.Google Scholar
Breiman, L. (1992), Probability, Vol. 7 of Classics in Applied Mathematics, SIAM, Philadelphia, PA. Corrected reprint of the 1968 original.Google Scholar
Burgers, G., Van Leeuwen, P. and Evensen, G. (1998), ‘On the analysis scheme in the ensemble Kalman filter’, Monthly Weather Review 126, 17191724.Google Scholar
Calvetti, D. (2007), ‘Preconditioned iterative methods for linear discrete ill-posed problems from a Bayesian inversion perspective’, J. Comput. Appl. Math. 198, 378395.Google Scholar
Calvetti, D. and Somersalo, E. (2005 a), ‘Priorconditioners for linear systems’, Inverse Problems 21, 13971418.Google Scholar
Calvetti, D. and Somersalo, E. (2005 b), ‘Statistical elimination of boundary artefacts in image deblurring’, Inverse Problems 21, 16971714.Google Scholar
Calvetti, D. and Somersalo, E. (2006), ‘Large-scale statistical parameter estimation in complex systems with an application to metabolic models’, Multiscale Modeling and Simulation 5, 13331366.Google Scholar
Calvetti, D. and Somersalo, E. (2007 a), ‘Gaussian hypermodel to recover blocky objects’, Inverse Problems 23, 733754.Google Scholar
Calvetti, D. and Somersalo, E. (2007 b), Introduction to Bayesian Scientific Computing, Vol. 2 of Surveys and Tutorials in the Applied Mathematical Sciences, Springer.Google Scholar
Calvetti, D. and Somersalo, E. (2008), ‘Hypermodels in the Bayesian imaging framework’, Inverse Problems 24, #034013.Google Scholar
Calvetti, D., Hakula, H., Pursiainen, S. and Somersalo, E. (2009), ‘Conditionally Gaussian hypermodels for cerebral source location’, SIAM J. Imag. Sci. 2, 879909.Google Scholar
Calvetti, D., Kuceyeski, A. and Somersalo, E. (2008), ‘Sampling based analysis of a spatially distributed model for liver metabolism at steady state’, Multiscale Modeling and Simulation 7, 407431.Google Scholar
Candès, E. and Wakin, M. (2008), ‘An introduction to compressive sampling’, IEEE Signal Processing Magazine, March 2008, 2130.Google Scholar
Chemin, J.-Y. and Lerner, N. (1995), ‘Flot de champs de veceurs non lipschitziens et équations de Navier-Stokes’, J. Diff. Equations 121, 314328.Google Scholar
Chorin, A. and Hald, O. (2006), Stochastic Tools in Mathematics and Science, Vol. 1 of Surveys and Tutorials in the Applied Mathematical Sciences, Springer, New York.Google Scholar
Chorin, A. and Krause, P. (2004), ‘Dimensional reduction for a Bayesian filter’, Proc. Nat. Acad. Sci. 101, 1501315017.Google Scholar
Chorin, A. and Tu, X. (2009), ‘Implicit sampling for particle filters’, Proc. Nat. Acad. Sci. 106, 1724917254.Google Scholar
Chorin, A. and Tu, X. (2010), ‘Interpolation and iteration for nonlinear filters’, Math. Model. Numer. Anal. To appear.Google Scholar
Christie, M. (2010), Solution error modelling and inverse problems. In Simplicity, Complexity and Modelling, Wiley, New York, to appear.Google Scholar
Christie, M., Pickup, G., O'Sullivan, A. and Demyanov, V. (2008), Use of solution error models in history matching. In Proc. European Conference on the Mathematics of Oil Recovery XI, European Association of Geoscientists and Engineers.Google Scholar
Chua, B. and Bennett, A. (2001), ‘An inverse ocean modelling system’, Ocean. Meteor. 3, 137165.Google Scholar
Cohn, S. (1997), ‘An introduction to estimation theory’, J. Met. Soc. Japan 75, 257288.Google Scholar
Cotter, S., Dashti, M., Robinson, J. and Stuart, A. (2009), ‘Bayesian inverse problems for functions and applications to fluid mechanics’, Inverse Problems 25, #115008.Google Scholar
Cotter, S., Dashti, M. and Stuart, A. (2010 a), ‘Approximation of Bayesian inverse problems’, SIAM J. Numer. Anal. To appear.Google Scholar
Cotter, S., Dashti, M., Robinson, J. and Stuart, A. (2010 b). In preparation.Google Scholar
Courtier, P. (1997), ‘Dual formulation of variational assimilation’, Quart. J. Royal Met. Soc. 123, 24492461.Google Scholar
Courtier, P. and Talagrand, O. (1987), ‘Variational assimilation of meteorological observations with the adjoint vorticity equation II: Numerical results’, Quart. J. Royal Met. Soc. 113, 13291347.Google Scholar
Courtier, P., Anderson, E., Heckley, W., Pailleux, J., Vasiljevic, D., Hamrud, M., Hollingworth, A., Rabier, F. and Fisher, M. (1998), ‘The ECMWF implementation of three-dimensional variational assimilation (3D-Var)’, Quart. J. Royal Met. Soc. 124, 17831808.Google Scholar
Cressie, N. (1993), Statistics for Spatial Data, Wiley.Google Scholar
Cui, T., Fox, C., Nicholls, G. and O'Sullivan, M. (2010), ‘Using MCMC sampling to calibrate a computer model of a geothermal field’. Submitted.Google Scholar
Da, G. Prato and Zabczyk, J. (1992), Stochastic Equations in Infinite Dimensions, Vol. 44 of Encyclopedia of Mathematics and its Applications, Cambridge University Press.Google Scholar
Dacarogna, B. (1989), Direct Methods in the Calculus of Variations, Springer, New York.Google Scholar
Dashti, M. and Robinson, J. (2009), ‘Uniqueness of the particle trajectories of the weak solutions of the two-dimensional Navier-Stokes equations’, Nonlinearity 22, 735746.Google Scholar
Dashti, M., Harris, S. and Stuart, A. M. (2010 a), Bayesian approach to an elliptic inverse problem. In preparation.Google Scholar
Dashti, M., Pillai, N. and Stuart, A. (2010 b), Bayesian Inverse Problems in Differential Equations. Lecture notes, available from: http://www.warwick.ac.uk/~masdr/inverse.html.Google Scholar
Derber, J. (1989), ‘A variational continuous assimilation technique’, Monthly Weather Review 117, 24372446.Google Scholar
Deuflhard, P. (2004), Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms, Springer.Google Scholar
DeVolder, B., Glimm, J., Grove, J., Kang, Y., Lee, Y., Pao, K., Sharp, D. and Ye, K. (2002), ‘Uncertainty quantification for multiscale simulations’, J. Fluids Engrg 124, 2942.Google Scholar
Donoho, D. (2006), ‘Compressed sensing’, IEEE Trans. Inform. Theory 52, 1289– 1306.Google Scholar
Dostert, P., Efendiev, Y., Hou, T. and Luo, W. (2006), ‘Coarse-grain Langevin algorithms for dynamic data integration’, J. Comput. Phys. 217, 123142.Google Scholar
Doucet, N., de Frietas, A. and Gordon, N. (2001), Sequential Monte Carlo in Practice, Springer.Google Scholar
Dudley, R. (2002), Real Analysis and Probability, Cambridge University Press, Cambridge.Google Scholar
Dürr, D. and Bach, A. (1978), ‘The Onsager-Machlup function as Lagrangian for the most probable path of a diffusion process’, Comm. Math. Phys. 160, 153170.Google Scholar
Efendiev, Y., Datta-Gupta, A., Ma, X. and Mallick, B. (2009), ‘Efficient sampling techniques for uncertainty quantification in history matching using nonlinear error models and ensemble level upscaling techniques’, Water Resources Res. 45, #W11414.Google Scholar
Eknes, M. and Evensen, G. (1997), ‘Parameter estimation solving a weak constraint variational formulation for an Ekman model’, J. Geophys. Res. 12, 479491.Google Scholar
Ellerbroek, B. and Vogel, C. (2009), ‘Inverse problems in astronomical adaptive optics’, Inverse Problems 25, #063001.Google Scholar
Engl, H., Hanke, M. and Neubauer, A. (1996), Regularization of Inverse Problems, Kluwer.Google Scholar
Engl, H., Hofinger, A. and Kindermann, S. (2005), ‘Convergence rates in the Prokhorov metric for assessing uncertainty in ill-posed problems’, Inverse Problems 21, 399412.Google Scholar
Evensen, G. (2006), Data Assimilation: The Ensemble Kalman Filter, Springer.Google Scholar
Evensen, G. and Van Leeuwen, P. (2000), ‘An ensemble Kalman smoother for nonlinear dynamics’, Monthly Weather Review 128, 18521867.Google Scholar
Fang, F., Pain, C., Navon, I., Piggott, M., Gorman, G., Allison, P. and Goddard, A. (2009 a), ‘Reduced order modelling of an adaptive mesh ocean model’, Internat. J. Numer. Methods Fluids 59, 827851.Google Scholar
Fang, F., Pain, C., Navon, I., Piggott, M., Gorman, G., Farrell, P., Allison, P. and Goddard, A. (2009 b), ‘A POD reduced-order 4D-Var adaptive mesh ocean modelling approach’, Internat. J. Numer. Methods Fluids 60, 709732.Google Scholar
Farmer, C. (2005), Geological modelling and reservoir simulation. In Mathematical Methods and Modeling in Hydrocarbon Exploration and Production (Iske, A. and Randen, T., eds), Springer, Heidelberg, pp. 119212.Google Scholar
Farmer, C. (2007), Bayesian field theory applied to scattered data interpolation and inverse problems. In Algorithms for Approximation (Iske, A. and Levesley, J., eds), Springer, pp. 147166.Google Scholar
Fitzpatrick, B. (1991), ‘Bayesian analysis in inverse problems’, Inverse Problems 7, 675702.Google Scholar
Franklin, J. (1970), ‘Well-posed stochastic extensions of ill-posed linear problems’, J. Math. Anal. Appl. 31, 682716.Google Scholar
Freidlin, M. and Wentzell, A. (1984), Random Perturbations of Dynamical Systems, Springer, New York.Google Scholar
Gelfand, A. and Smith, A. (1990), ‘Sampling-based approaches to calculating marginal densities’, J. Amer. Statist. Soc. 85, 398409.Google Scholar
Gibbs, A. and Su, F. (2002), ‘On choosing and bounding probability metrics’, Internat. Statist. Review 70, 419435.Google Scholar
Gittelson, C. and Schwab, C. (2011), Sparse tensor discretizations of high-dimen-sional PDEs. To appear in Acta Numerica, Vol. 20.Google Scholar
Glimm, J., Hou, S., Lee, Y., Sharp, D. and Ye, K. (2003), ‘Solution error models for uncertainty quantification’, Contemporary Mathematics 327, 115140.Google Scholar
Gratton, S., Lawless, A. and Nichols, N. (2007), ‘Approximate Gauss—Newton methods for nonlinear least squares problems’, SIAM J. Optimization 18, 106132.Google Scholar
Griffith, A. and Nichols, N. (1998), Adjoint methods for treating model error in data assimilation. In Numerical Methods for Fluid Dynamics VI, ICFD, Oxford, pp. 335344.Google Scholar
Griffith, A. and Nichols, N. (2000), ‘Adjoint techniques in data assimilation for treating systematic model error’, J. Flow, Turbulence and Combustion 65, 469488.Google Scholar
Grimmett, G. and Stirzaker, D. (2001), Probability and Random Processes, Oxford University Press, New York.Google Scholar
Gu, C. (2002), Smoothing Spline ANOVA Models, Springer.Google Scholar
Gu, C. (2008), ‘Smoothing noisy data via regularization’, Inverse Problems 24, #034002.Google Scholar
Hagelberg, C., Bennett, A. and Jones, D. (1996), ‘Local existence results for the generalized inverse of the vorticity equation in the plane’, Inverse Problems 12, 437454.Google Scholar
Hairer, E. and Wanner, G. (1996), Solving Ordinary Differential Equations II, Vol. 14 of Springer Series in Computational Mathematics, Springer, Berlin.Google Scholar
Hairer, E., Nørsett, S. P. and Wanner, G. (1993), Solving Ordinary Differential Equations I, Vol. 8 of Springer Series in Computational Mathematics, Springer, Berlin.Google Scholar
Hairer, M. (2009), Introduction to Stochastic PDEs. Lecture notes.Google Scholar
Hairer, M., Stuart, A. M. and Voss, J. (2007), ‘Analysis of SPDEs arising in path sampling II: The nonlinear case’, Ann. Appl. Probab. 17, 16571706.Google Scholar
Hairer, M., Stuart, A. M. and Voss, J. (2009), Sampling conditioned diffusions. In Trends in Stochastic Analysis, Vol. 353 of London Mathematical Society Lecture Notes, Cambridge University Press, pp. 159186.Google Scholar
Hairer, M., Stuart, A. and Voss, J. (2010 a), ‘Sampling conditioned hypoelliptic diffusions’. Submitted.Google Scholar
Hairer, M., Stuart, A. and Voss, J. (2010 b), Signal processing problems on function space: Bayesian formulation, stochastic PDEs and effective MCMC methods. In Oxford Handbook of Nonlinear Filtering (Crisan, D. and Rozovsky, B., eds), Oxford University Press, to appear.Google Scholar
Hairer, M., Stuart, A., Voss, J. and Wiberg, P. (2005), ‘Analysis of SPDEs arising in path sampling I: The Gaussian case’, Comm. Math. Sci. 3, 587603.Google Scholar
Hastings, W. K. (1970), ‘Monte Carlo sampling methods using Markov chains and their applications’, Biometrika 57, 97109.Google Scholar
Hein, T. (2009), ‘On Tikhonov regularization in Banach spaces: Optimal convergence rate results’, Applicable Analysis 88, 653667.Google Scholar
Heino, J., Tunyan, K., Calvetti, D. and Somersalo, E. (2007), ‘Bayesian flux balance analysis applied to a skeletal muscle metabolic model’, J. Theor. Biol. 248, 91110.Google Scholar
Herbei, R. and McKeague, I. (2009), ‘Geometric ergodicity of hybrid samplers for ill-posed inverse problems’, Scand. J. Statist. 36, 839853.Google Scholar
Herbei, R., McKeague, I. and Speer, K. (2008), ‘Gyres and jets: Inversion of tracer data for ocean circulation structure’, J. Phys. Oceanography 38, 11801202.Google Scholar
Hofinger, A. and Pikkarainen, H. (2007), ‘Convergence rates for the Bayesian approach to linear inverse problems’, Inverse Problems 23, 24692484.Google Scholar
Hofinger, A. and Pikkarainen, H. (2009), ‘Convergence rates for linear inverse problems in the presence of an additive normal noise’, Stoch. Anal. Appl. 27, 240257.Google Scholar
Huddleston, M., Bell, M., Martin, M. and Nichols, N. (2004), ‘Assessment of wind stress errors using bias corrected ocean data assimilation’, Quart. J. Royal Met. Soc. 130, 853872.Google Scholar
Hurzeler, M. and Kunsch, H. (2001), Approximating and maximizing the likelihood for a general state space model. In Sequential Monte Carlo Methods in Practice (Doucet, A., de Freitas, N. and Gordon, N., eds), Springer, pp. 159175.Google Scholar
Huttunen, J. and Pikkarainen, H. (2007), ‘Discretization error in dynamical inverse problems: One-dimensional model case’, J. Inverse and Ill-posed Problems 15, 365386.Google Scholar
Ide, K. and Jones, C. (2007), ‘Data assimilation’, Physica D 230, vii–viii.Google Scholar
Ide, K., Kuznetsov, L. and Jones, C. (2002), ‘Lagrangian data assimilation for pointvortex system’, J. Turbulence 3, 53.Google Scholar
Ikeda, N. and Watanabe, S. (1989), Stochastic Differential Equations and Diffusion Processes, second edn, North-Holland, Amsterdam.Google Scholar
Jardak, M., Navon, I. and Zupanski, M. (2010), ‘Comparison of sequential data assimilation methods for the Kuramoto—Sivashinsky equation’, Internat. J. Numer. Methods Fluids 62, 374402.Google Scholar
Johnson, C., Hoskins, B. and Nichols, N. (2005), ‘A singular vector perspective of 4DVAR: Filtering and interpolation’, Quart. J. Royal Met. Soc. 131, 120.Google Scholar
Johnson, C., Hoskins, B., Nichols, N. and Ballard, S. (2006), ‘A singular vector perspective of 4DVAR: The spatial structure and evolution of baroclinic weather systems’, Monthly Weather Review 134, 34363455.Google Scholar
Kaipio, J. and Somersalo, E. (2000), ‘Statistical inversion and Monte Carlo sampling methods in electrical impedance tomography’, Inverse Problems 16, 14871522.Google Scholar
Kaipio, J. and Somersalo, E. (2005), Statistical and Computational Inverse problems, Vol. 160 of Applied Mathematical Sciences, Springer.Google Scholar
Kaipio, J. and Somersalo, E. (2007 a), ‘Approximation errors in nonstationary inverse problems’, Inverse Problems and Imaging 1, 7793.Google Scholar
Kaipio, J. and Somersalo, E. (2007 b), ‘Statistical inverse problems: Discretization, model reduction and inverse crimes’, J. Comput. Appl. Math. 198, 493504.Google Scholar
Kalnay, E. (2003), Atmospheric Modeling, Data Assimilation and Predictability, Cambridge University Press.Google Scholar
Kalnay, E., Li, H., Miyoshi, S., Yang, S. and Ballabrera-Poy, J. (2007), ‘4D-Var or ensemble Kalman filter?’, Tellus 59, 758773.Google Scholar
Kaltenbacher, B., Schöpfer, F. and Schuster, T. (2009), ‘Iterative methods for non-linear ill-posed problems in Banach spaces: Convergence and applications to parameter identification problems’, Inverse Problems 25, #065003.Google Scholar
Kennedy, M. and O'Hagan, A. (2001), ‘Bayesian calibration of computer models’, J. Royal Statist. Soc. 63B, 425464.Google Scholar
Kinderlehrer, D. and Stampacchia, G. (1980), An Introduction to Variational In-equalities and their Applications, SIAM.Google Scholar
Kolda, T. and Bader, B. (2009), ‘Tensor decompositions and applications’, SIAM Review 51, 455500.Google Scholar
Kuznetsov, L., Ide, K. and Jones, C. (2003), ‘A method for assimilation of Lagrangian data’, Monthly Weather Review 131, 22472260.Google Scholar
Lassas, M. and Siltanen, S. (2004), ‘Can one use total variation prior for edge-preserving Bayesian inversion?’, Inverse Problems 20, 15371563.Google Scholar
Lassas, M., Saksman, E. and Siltanen, S. (2009), ‘Discretization-invariant Bayesian inversion and Besov space priors’, Inverse Problems and Imaging 3, 87122.Google Scholar
Lawless, A. and Nichols, N. (2006), ‘Inner loop stopping criteria for incremental four-dimensional variational data assimilation’, Monthly Weather Review 134, 34253435.Google Scholar
Lawless, A., Gratton, S. and Nichols, N. (2005 a), ‘Approximate iterative methods for variational data assimilation’, Internat. J. Numer. Methods Fluids 47, 11291135.Google Scholar
Lawless, A., Gratton, S. and Nichols, N. (2005 b), ‘An investigation of incremental 4D-Var using non-tangent linear models’, Quart. J. Royal Met. Soc. 131, 459476.Google Scholar
Lawless, A., Nichols, N., Boess, C. and Bunse-Gerstner, A. (2008 a), ‘Approximate Gauss—Newton methods for optimal state estimation using reduced order models’, Internat. J. Numer. Methods Fluids 56, 13671373.Google Scholar
Lawless, A., Nichols, N., Boess, C. and Bunse-Gerstner, A. (2008 b), ‘Using model reduction methods within incremental four-dimensional variational data assimilation’, Monthly Weather Review 136, 15111522.Google Scholar
Lehtinen, M., Paivarinta, L. and Somersalo, E. (1989), ‘Linear inverse problems for generalized random variables’, Inverse Problems 5, 599612.Google Scholar
Lifshits, M. (1995), Gaussian Random Functions, Vol. 322 of Mathematics and its Applications, Kluwer, Dordrecht.Google Scholar
Livings, D., Dance, S. and Nichols, N. (2008), ‘Unbiased ensemble square root filters’, Physica D: Nonlinear Phenomena 237, 10211028.Google Scholar
Lo, M.éve (1977), Probability Theory I, fourth edn, Vol. 45 of Graduate Texts in Mathematics, Springer, New York.Google Scholar
Loéve, M. (1978), Probability Theory II, fourth edn, Vol. 46 of Graduate Texts in Mathematics, Springer, New York.Google Scholar
Lorenc, A. (1986), ‘Analysis methods for numerical weather prediction’, Quart. J. Royal Met. Soc. 112, 11771194.Google Scholar
Lubich, C. (2008), From Quantum to Classical Molecular Dynamics: Reduced Models and Numerical Analysis, European Mathematical Society.Google Scholar
Ma, X., Al-Harbi, M., Datta-Gupta, A. and Efendiev, Y. (2008), ‘Multistage sampling approach to quantifying uncertainty during history matching geological models’, Soc. Petr. Engrg J. 13, 7787.Google Scholar
Majda, A. and Gershgorin, B. (2008), ‘A nonlinear test model for filtering slow-fast systems’, Comm. Math. Sci. 6, 611649.Google Scholar
Majda, A. and Grote, M. (2007), ‘Explicit off-line criteria for stable accurate filtering of strongly unstable spatially extended systems’, Proc. Nat. Acad. Sci. 104, 11241129.Google Scholar
Majda, A. and Harlim, J. (2010), ‘Catastrophic filter divergence in filtering nonlinear dissipative systems’, Comm. Math. Sci. 8, 2743.Google Scholar
Majda, A., Harlim, J. and Gershgorin, B. (2010), ‘Mathematical strategies for filtering turbulent dynamical systems’, Disc. Cont. Dyn. Sys. To appear.Google Scholar
Mandelbaum, A. (1984), ‘Linear estimators and measurable linear transformations on a Hilbert space’, Probab. Theory Rel. Fields 65, 385397.Google Scholar
Martin, M., Bell, M. and Nichols, N. (2002), ‘Estimation of systematic error in an equatorial ocean model using data assimilation’, Internat. J. Numer. Methods Fluids 40, 435444.Google Scholar
McKeague, I., Nicholls, G., Speer, K. and Herbei, R. (2005), ‘Statistical inversion of south Atlantic circulation in an abyssal neutral density layer’, J. Marine Res. 63, 683704.Google Scholar
McLaughlin, D. and Townley, L. (1996), ‘A reassessment of the groundwater inverse problem’, Water Resources Res. 32, 11311161.Google Scholar
Metropolis, N., Rosenbluth, R., Teller, M. and Teller, E. (1953), ‘Equations of state calculations by fast computing machines’, J. Chem. Phys. 21, 10871092.Google Scholar
Meyn, S. P. and Tweedie, R. L. (1993), Markov Chains and Stochastic Stability, Communications and Control Engineering Series, Springer, London.Google Scholar
Michalak, A. and Kitanidis, P. (2003), ‘A method for enforcing parameter nonnegativity in Bayesian inverse problems with an application to contaminant source identification’, Water Resources Res. 39, 1033.Google Scholar
Mitchell, T., Buchanan, B., DeJong, G., Dietterich, T., Rosenbloom, P. and Waibel, A. (1990), ‘Machine learning’, Annual Review of Computer Science 4, 417433.Google Scholar
Mohamed, L., Christie, M. and Demyanov, V. (2010), ‘Comparison of stochastic sampling algorithms for uncertainty quantification’, Soc. Petr. Engrg J. To appear. http://dx.doi.org/10.2118/119139-PAGoogle Scholar
Mosegaard, K. and Tarantola, A. (1995), ‘Monte Carlo sampling of solutions to inverse problems’, J. Geophys. Research 100, 431447.Google Scholar
Neubauer, A. (2009), ‘On enhanced convergence rates for Tikhonov regularization of nonlinear ill-posed problems in Banach spaces’, Inverse Problems 25, #065009.Google Scholar
Neubauer, A. and Pikkarainen, H. (2008), ‘Convergence results for the Bayesian inversion theory’, J. Inverse and Ill-Posed Problems 16, 601613.Google Scholar
Nichols, N. (2003 a), Data assimilation: Aims and basic concepts. In Data Assimilation for the Earth System (Swinbank, R., Shutyaev, V. and Lahoz, W. A., eds), Kluwer Academic, pp. 920.Google Scholar
Nichols, N. (2003 b), Treating model error in 3-D and 4-D data assimilation. In Data Assimilation for the Earth System (Swinbank, R., Shutyaev, V. and Lahoz, W. A., eds), Kluwer Academic, pp. 127135.Google Scholar
Nodet, M. (2005), Mathematical modeling and assimilation of Lagrangian data in oceanography. PhD thesis, University of Nice.Google Scholar
Nodet, M. (2006), ‘Variational assimilation of Lagrangian data in oceanography’, Inverse Problems 22, 245263.Google Scholar
Oksendal, B. (2003), Stochastic Differential Equations: An Introduction with Applications, sixth edn, Universitext, Springer.Google Scholar
Orrell, D., Smith, L., Barkmeijer, J. and Palmer, T. (2001), ‘Model error in weather forecasting’, Non. Proc. in Geo. 8, 357371.Google Scholar
O'Sullivan, A. and Christie, M. (2006 a), ‘Error models for reducing history match bias’, Comput. Geosci. 10, 405–405.Google Scholar
O'Sullivan, A. and Christie, M. (2006 b), ‘Simulation error models for improved reservoir prediction’, Reliability Engineering and System Safety 91, 13821389.Google Scholar
Ott, E., Hunt, B., Szunyogh, I., Zimin, A., Kostelich, E., Corazza, M., Kalnay, E., Patil, D. and Yorke, J. (2004), ‘A local ensemble Kalman filter for atmospheric data assimilation’, Tellus A 56, 273277.Google Scholar
Palmer, T., Doblas-Reyes, F., Weisheimer, A., Shutts, G., Berner, J. and Murphy, J. (2009), ‘Towards the probabilistic earth-system model’, J. Climate 70, 419435.Google Scholar
Pikkarainen, H. (2006), ‘State estimation approach to nonstationary inverse problems: Discretization error and filtering problem’, Inverse Problems 22, 365379.Google Scholar
Pimentel, S., Haines, K. and Nichols, N. (2008 a), ‘The assimilation of satellite derived sea surface temperatures into a diurnal cycle model’, J. Geophys. Research: Oceans 113, #C09013.Google Scholar
Pimentel, S., Haines, K. and Nichols, N. (2008 b), ‘Modelling the diurnal variability of sea surface temperatures’, J. Geophys. Research: Oceans 113, #C11004.Google Scholar
Ramsay, J. and Silverman, B. (2005), Functional Data Analysis, Springer.Google Scholar
Reznikoff, M. and Vanden Eijnden, E. (2005), ‘Invariant measures of SPDEs and conditioned diffusions’, CR Acad. Sci. Paris 340, 305308.Google Scholar
Richtmyer, D. and Morton, K. (1967), Difference Methods for Initial Value Problems, Wiley.Google Scholar
Roberts, G. and Rosenthal, J. (1998), ‘Optimal scaling of discrete approximations to Langevin diffusions’, J. Royal Statist. Soc. B 60, 255268.Google Scholar
Roberts, G. and Rosenthal, J. (2001), ‘Optimal scaling for various Metropolis—Hastings algorithms’, Statistical Science 16, 351367.Google Scholar
Roberts, G. and Tweedie, R. (1996), ‘Exponential convergence of Langevin distributions and their discrete approximations’, Bernoulli 2, 341363.Google Scholar
Roberts, G., Gelman, A. and Gilks, W. (1997), ‘Weak convergence and optimal scaling of random walk Metropolis algorithms’, Ann. Appl. Probab. 7, 110120.Google Scholar
Rudin, L., Osher, S. and Fatemi, E. (1992), ‘Nonlinear total variation based noise removal algorithms’, Physica D 60, 259268.Google Scholar
Rue, H. and Held, L. (2005), Gaussian Markov Random Fields: Theory and Applications, Chapman & Hall.Google Scholar
Salman, H., Ide, K. and Jones, C. (2008), ‘Using flow geometry for drifter deployment in Lagrangian data assimilation’, Tellus 60, 321335.Google Scholar
Salman, H., Kuznetsov, L., Jones, C. and Ide, K. (2006), ‘A method for assimilating Lagrangian data into a shallow-water equation ocean model’, Monthly Weather Review 134, 10811101.Google Scholar
Sanz-Serna, J. M. and Palencia, C. (1985), ‘A general equivalence theorem in the theory of discretization methods’, Math. Comp. 45, 143152.Google Scholar
Scherzer, O., Grasmair, M., Grossauer, H., Haltmeier, M. and Lenzen, F. (2009), Variational Methods in Imaging, Springer.Google Scholar
Schwab, C. and Todor, R. (2006), ‘Karhunen-Loeve approximation of random fields in domains by generalized fast multipole methods’, J. Comput. Phys. 217, 100122.Google Scholar
Shen, Y., Archambeau, C., Cornford, D. and Opper, M. (2008 a), Variational Markov chain Monte Carlo for inference in partially observed nonlinear diffusions. In Proceedings of the Workshop on Inference and Estimation in Probabilistic Time-Series Models (Barber, D., Cemgil, A. T. and Chiappa, S., eds), Isaac Newton Institute for Mathematical Sciences, Cambridge, pp. 6778.Google Scholar
Shen, Y., Archambeau, C., Cornford, D., Opper, M., Shawe-Taylor, J. and Barillec, R. (2008 b), ‘A comparison of variational and Markov chain Monte Carlo methods for inference in partially observed stochastic dynamic systems’, J. Signal Processing Systems. In press (published online).Google Scholar
Shen, Y., Cornford, D., Archambeau, C. and Opper, M. (2010), ‘Variational Markov chain Monte Carlo for Bayesian inference in partially observed non-linear diffusions’, Comput. Statist. Submitted.Google Scholar
Smith, A. and Roberts, G. (1993), ‘Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods’, J. Royal Statist. Soc. B 55, 323.Google Scholar
Snyder, T., Bengtsson, T., Bickel, P. and Anderson, J. (2008), ‘Obstacles to high-dimensional particle filtering’, Monthly Weather Review 136, 46294640.Google Scholar
Spanos, P. and Ghanem, R. (1989), ‘Stochastic finite element expansion for random media’, J. Engrg Mech. 115, 10351053.Google Scholar
Spanos, P. and Ghanem, R. (2003), Stochastic Finite Elements: A Spectral Approach, Dover.Google Scholar
Spiller, E., Budhiraja, A., Ide, K. and Jones, C. (2008), ‘Modified particle filter methods for assimilating Lagrangian data into a point-vortex model’, Physica D 237, 14981506.Google Scholar
Stanton, L., Lawless, A., Nichols, N. and Roulstone, I. (2005), ‘Variational data assimilation for Hamiltonian problems’, Internat. J. Numer. Methods Fluids 47, 13611367.Google Scholar
Stuart, A., Voss, J. and Wiberg, P. (2004), ‘Conditional path sampling of SDEs and the Langevin MCMC method’, Comm. Math. Sci 2, 685697.Google Scholar
Talagrand, P. and Courtier, O. (1987), ‘Variational assimilation of meteorological observations with the adjoint vorticity equation I: Theory’, Quart. J. Royal Met. Soc. 113, 13111328.Google Scholar
Tarantola, A. (2005), Inverse Problem Theory, SIAM.Google Scholar
Tierney, L. (1998), ‘A note on Metropolis—Hastings kernels for general state spaces’, Ann. Appl. Probab. 8, 19.Google Scholar
Todor, R. and Schwab, C. (2007), ‘Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients’, IMA J. Numer. Anal. 27, 232261.Google Scholar
Uhlmann, G. (2009), Visibility and invisibility. In Invited Lectures, Sixth International Congress on Industrial and Applied Mathematics, ICIAM07 (Jeltsch, R. and Wanner, G., eds), European Mathematical Society, pp. 381408.Google Scholar
Van Leeuwen, P. (2001), ‘An ensemble smoother with error estimates’, Monthly Weather Review 129, 709728.Google Scholar
Van Leeuwen, P. (2003), ‘A variance minimizing filter for large-scale applications’, Monthly Weather Review 131, 20712084.Google Scholar
Van Leeuwen, P. (2009), ‘Particle filtering in geophysical systems’, Monthly Weather Review 137, 40894114.Google Scholar
Vernieres, G., Ide, K. and Jones, C. (2010), ‘Lagrangian data assimilation, an application to the Gulf of Mexico’, Physica D. Submitted.Google Scholar
Vogel, C. (2002), Computational Methods for Inverse Problems, SIAM.Google Scholar
Vossepoel, F. and Van Leeuwen, P. (2007), ‘Parameter estimation using a particle method: Inferring mixing coefficients from sea-level observations’, Monthly Weather Review 135, 10061020.Google Scholar
Vrettas, M., Cornford, D. and Shen, Y. (2009), A variational basis function approximation for diffusion processes. In Proceedings of the 17th European Symposium on Artificial Neural Networks, D-side publications, Evere, Belgium, pp. 497502.Google Scholar
Wahba, G. (1990), Spline Models for Observational Data, SIAM.Google Scholar
Watkinson, L., Lawless, A., Nichols, N. and Roulstone, I. (2007), ‘Weak constraints in four dimensional variational data assimilation’, Meteorologische Zeitschrift 16, 767776.Google Scholar
White, L. (1993), ‘A study of uniqueness for the initialization problem for Burgers' equation’, J. Math. Anal. Appl. 172, 412431.Google Scholar
Williams, D. (1991), Probability with Martingales, Cambridge University Press, Cambridge.Google Scholar
Wlasak, M. and Nichols, N. (1998), Application of variational data assimilation to the Lorenz equations using the adjoint method. In Numerical Methods for Fluid Dynamics VI, ICFD, Oxford, pp. 555562.Google Scholar
Wlasak, M., Nichols, N. and Roulstone, I. (2006), ‘Use of potential vorticity for incremental data assimilation’, Quart. J. Royal Met. Soc. 132, 28672886.Google Scholar
Yu, L. and O'Brien, J. (1991), ‘Variational estimation of the wind stress drag coefficient and the oceanic eddy viscosity profile’, J. Phys. Ocean. 21, 13611364.Google Scholar
Zeitouni, O. and Dembo, A. (1987), ‘A maximum a posteriori estimator for trajectories of diffusion processes’, Stochastics 20, 221246.Google Scholar
Zimmerman, D., de Marsily, G., Gotway, C., Marietta, M., Axness, C., Beauheim, R., Bras, R., Carrera, J., Dagan, G., Davies, P., Gallegos, D., Galli, A., Gomez-Hernandez, J., Grindrod, P., Gutjahr, A., Kitanidis, P., Lavenue, A., McLaughlin, D., Neuman, S., RamaRao, B., Ravenne, C. and Rubin, Y. (1998), ‘A comparison of seven geostatistically based inverse approaches to estimate transmissivities for modeling advective transport by groundwater flow’, Water Resources Res. 6, 13731413.Google Scholar
Zuazua, E. (2005), ‘Propagation, observation, control and numerical approximation of waves approximated by finite difference method’, SIAM Review 47, 197243.Google Scholar
Zupanski, D. (1997), ‘A general weak constraint applicable to operational 4DVAR data assimilation systems’, Monthly Weather Review 125, 22742292.Google Scholar
Zupanski, M., Navon, I. and Zupanski, D. (2008), ‘The maximum likelihood ensemble filter as a non-differentiable minimization algorithm’, Quart. J. Royal Met. Soc. 134, 10391050.Google Scholar