Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2019 | OriginalPaper | Chapter

Fundamentals of Data Assimilation and Theoretical Advances

Authors: Hamid Moradkhani, Grey S. Nearing, Peyman Abbaszadeh, Sahani Pathiraja

Published in: Handbook of Hydrometeorological Ensemble Forecasting

Publisher: Springer Berlin Heidelberg

share
SHARE

Abstract

Hydrometeorological predictions are not perfect as models often suffer either from inadequate conceptualization of underlying physics or non-uniqueness of model parameters or inaccurate initialization. During the past two decades, Data Assimilation (DA) has received increased prominence among researchers and practitioners as an effective and reliable method to integrate the hydrometeorological observations from in situ measure and remotely-sensed sensors into predictive models for enhancing the forecast skills while taking into account all sources of uncertainties. The successful application of DA in different disciplines has resulted in an ever-increasing publications. This chapter provides a progressive essay covering fundamental and theoretical underpinnings of DA techniques and their applications in a variety of scientific fields. More detailed examples of applications are presented in following chapters in this section.
Literature
go back to reference K.M. Andreadis, D.P. Lettenmaier, Assimilating remotely sensed snow observations into a macroscale hydrology model. Adv. Water Resour. 29, 872–886 (2006) CrossRef K.M. Andreadis, D.P. Lettenmaier, Assimilating remotely sensed snow observations into a macroscale hydrology model. Adv. Water Resour. 29, 872–886 (2006) CrossRef
go back to reference J.D. Annan, J.C. Hargreaves, N.R. Edwards, R. Marsh, Parameter estimation in an intermediate complexity Earth system model using an ensemble Kalman filter. Ocean Model. 8(1), 135–154 (2005) CrossRef J.D. Annan, J.C. Hargreaves, N.R. Edwards, R. Marsh, Parameter estimation in an intermediate complexity Earth system model using an ensemble Kalman filter. Ocean Model. 8(1), 135–154 (2005) CrossRef
go back to reference M.S. Arulampalam, S. Maskell, N. Gordon, T. Clapp, A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002) CrossRef M.S. Arulampalam, S. Maskell, N. Gordon, T. Clapp, A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002) CrossRef
go back to reference D.M. Barker, W. Huang, Y.-R. Guo, A.J. Bourgeois, Q.N. Xiao, A three-dimensional variational data assimilation system for MM5: implementation and initial results. Mon. Weather Rev. 132(4), 897–914 (2004) CrossRef D.M. Barker, W. Huang, Y.-R. Guo, A.J. Bourgeois, Q.N. Xiao, A three-dimensional variational data assimilation system for MM5: implementation and initial results. Mon. Weather Rev. 132(4), 897–914 (2004) CrossRef
go back to reference T. Bengtsson, P. Bickel, B. Li, Curse of dimensionality revisited: the collapse of importance sampling in very large scale systems, in IMS Collections: Probability and Statistics: Essays in Honor of David A. Freedman,, vol. 2, ed. by D. Nolan, T. Speed (Institute of Mathematical Statistics, Beachwood), pp. 316–334 (2008) T. Bengtsson, P. Bickel, B. Li, Curse of dimensionality revisited: the collapse of importance sampling in very large scale systems, in IMS Collections: Probability and Statistics: Essays in Honor of David A. Freedman,, vol. 2, ed. by D. Nolan, T. Speed (Institute of Mathematical Statistics, Beachwood), pp. 316–334 (2008)
go back to reference M.P. Clark, D.E. Rupp, R.A. Woods, X. Zheng, R.P. Ibbitt, A.G. Slater, J. Schmidt, M.J. Uddstrom, Hydrological data assimilation with the ensemble Kalman filter: use of streamflow observations to update states in a distributed hydrological model. Adv. Water Resour. 31, 1309 (2008) CrossRef M.P. Clark, D.E. Rupp, R.A. Woods, X. Zheng, R.P. Ibbitt, A.G. Slater, J. Schmidt, M.J. Uddstrom, Hydrological data assimilation with the ensemble Kalman filter: use of streamflow observations to update states in a distributed hydrological model. Adv. Water Resour. 31, 1309 (2008) CrossRef
go back to reference W.T. Crow, E.F. Wood, The assimilation of remotely sensed soil brightness temperature imagery into a land surface model using ensemble Kalman filtering: a case study based on ESTAR measurements during SGP97. Adv. Water Resour. 26(2), 137–149 (2003) CrossRef W.T. Crow, E.F. Wood, The assimilation of remotely sensed soil brightness temperature imagery into a land surface model using ensemble Kalman filtering: a case study based on ESTAR measurements during SGP97. Adv. Water Resour. 26(2), 137–149 (2003) CrossRef
go back to reference G.J.M. De Lannoy, R.H. Reichle, K.R. Arsenault, P.R. Houser, S. Kumar, N.E.C. Verhoest, V. Pauwels, Multiscale assimilation of advanced microwave scanning radiometer–EOS snow water equivalent and moderate resolution imaging spectroradiometer snow cover fraction observations in northern Colorado. Water Resour. Res. 48, W01522 (2012). https://​doi.​org/​10.​1029/​2011WR010588 G.J.M. De Lannoy, R.H. Reichle, K.R. Arsenault, P.R. Houser, S. Kumar, N.E.C. Verhoest, V. Pauwels, Multiscale assimilation of advanced microwave scanning radiometer–EOS snow water equivalent and moderate resolution imaging spectroradiometer snow cover fraction observations in northern Colorado. Water Resour. Res. 48, W01522 (2012). https://​doi.​org/​10.​1029/​2011WR010588
go back to reference C. DeChant, H. Moradkhani, Radiance data assimilation for operational snow and streamflow forecasting. Adv. Water Resour. 34(3), 351–364 (2011b) CrossRef C. DeChant, H. Moradkhani, Radiance data assimilation for operational snow and streamflow forecasting. Adv. Water Resour. 34(3), 351–364 (2011b) CrossRef
go back to reference C.M. DeChant, H. Moradkhani, Examining the effectiveness and robustness of sequential data assimilation methods for quantification of uncertainty in hydrologic forecasting. Water Resour. Res. 48(4), W04518 (2012) CrossRef C.M. DeChant, H. Moradkhani, Examining the effectiveness and robustness of sequential data assimilation methods for quantification of uncertainty in hydrologic forecasting. Water Resour. Res. 48(4), W04518 (2012) CrossRef
go back to reference C.M. DeChant, H. Moradkhani, Hydrologic prediction and uncertainty quantification, in Handbook of Engineering Hydrology, Modeling, Climate Change and Variability (CRC Press, Taylor & Francis Group, Boca Raton, 2014b), pp. 387–414 CrossRef C.M. DeChant, H. Moradkhani, Hydrologic prediction and uncertainty quantification, in Handbook of Engineering Hydrology, Modeling, Climate Change and Variability (CRC Press, Taylor & Francis Group, Boca Raton, 2014b), pp. 387–414 CrossRef
go back to reference D.P. Dee et al., The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137(656), 553–597 (2011) CrossRef D.P. Dee et al., The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137(656), 553–597 (2011) CrossRef
go back to reference R. Douc, O. Cappe, Comparison of resampling schemes for particle filtering, paper presented at image and signal processing and analysis, 2005. ISPA 2005, in Proceedings of the 4th International Symposium on, 15–17 Sept 2005 (2005) R. Douc, O. Cappe, Comparison of resampling schemes for particle filtering, paper presented at image and signal processing and analysis, 2005. ISPA 2005, in Proceedings of the 4th International Symposium on, 15–17 Sept 2005 (2005)
go back to reference M. Durand, S.A. Margulis, Effects of uncertainty magnitude and accuracy on assimilation of multiscale measurements for snowpack characterization. J. Geophys. Res. 113(D2), D02105 (2008) M. Durand, S.A. Margulis, Effects of uncertainty magnitude and accuracy on assimilation of multiscale measurements for snowpack characterization. J. Geophys. Res. 113(D2), D02105 (2008)
go back to reference R.M. Errico, What is an adjoint model? Bull. Am. Meteorol. Soc. 78(11), 2577–2591 (1997) CrossRef R.M. Errico, What is an adjoint model? Bull. Am. Meteorol. Soc. 78(11), 2577–2591 (1997) CrossRef
go back to reference G. Evensen, The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn. 53(4), 343–367 (2003) CrossRef G. Evensen, The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn. 53(4), 343–367 (2003) CrossRef
go back to reference Z. Ghahramani, S.T. Roweis, Learning nonlinear dynamical systems using an EM algorithm. Adv. Neural Inf. Process. Syst. 11, 431–437 (1999) Z. Ghahramani, S.T. Roweis, Learning nonlinear dynamical systems using an EM algorithm. Adv. Neural Inf. Process. Syst. 11, 431–437 (1999)
go back to reference M.E. Gharamti, J. Tjiputra, I. Bethke, A. Samuelsen, I. Skjelvan, M. Bentsen, L. Bertino, Ensemble data assimilation for ocean biogeochemical state and parameter estimation at different sites. Ocean Model. 112, 65–89 (2017) CrossRef M.E. Gharamti, J. Tjiputra, I. Bethke, A. Samuelsen, I. Skjelvan, M. Bentsen, L. Bertino, Ensemble data assimilation for ocean biogeochemical state and parameter estimation at different sites. Ocean Model. 112, 65–89 (2017) CrossRef
go back to reference R. Giering, Tangent Linear and Adjoint Model Compiler, Users Manual (Center for Global Change Sciences, Department of Earth, Atmospheric, and Planetary Science. MIT, Cambridge, 1997) R. Giering, Tangent Linear and Adjoint Model Compiler, Users Manual (Center for Global Change Sciences, Department of Earth, Atmospheric, and Planetary Science. MIT, Cambridge, 1997)
go back to reference N. Gordon, D. Salmond, A. Smith, Novel approach to nonlinear/non-Gaussian Bayesian state estimation. Proc. Inst. Elect. Eng. F. 140(2), 107–113 (1993) CrossRef N. Gordon, D. Salmond, A. Smith, Novel approach to nonlinear/non-Gaussian Bayesian state estimation. Proc. Inst. Elect. Eng. F. 140(2), 107–113 (1993) CrossRef
go back to reference P. Guingla, D. Antonio, R. De Keyser, G. De Lannoy, L. Giustarini, P. Matgen, V. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012) CrossRef P. Guingla, D. Antonio, R. De Keyser, G. De Lannoy, L. Giustarini, P. Matgen, V. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012) CrossRef
go back to reference C.M. Hoppe, H. Elbern, J. Schwinger, A variational data assimilation system for soil–atmosphere flux estimates for the Community Land Model (CLM3. 5). Geosci. Model Dev. 7(3), 1025–1036 (2014) CrossRef C.M. Hoppe, H. Elbern, J. Schwinger, A variational data assimilation system for soil–atmosphere flux estimates for the Community Land Model (CLM3. 5). Geosci. Model Dev. 7(3), 1025–1036 (2014) CrossRef
go back to reference T. Hou, F. Kong, X. Chen, H. Lei, Impact of 3DVAR data assimilation on the prediction of heavy rainfall over Southern China. Adv. Meteorol. 2013, 1 (2013) CrossRef T. Hou, F. Kong, X. Chen, H. Lei, Impact of 3DVAR data assimilation on the prediction of heavy rainfall over Southern China. Adv. Meteorol. 2013, 1 (2013) CrossRef
go back to reference P.L. Houtekamer, H.L. Mitchell, Data assimilation using an ensemble Kalman filter technique. Mon. Weather Rev. 126(3), 796–811 (1998) CrossRef P.L. Houtekamer, H.L. Mitchell, Data assimilation using an ensemble Kalman filter technique. Mon. Weather Rev. 126(3), 796–811 (1998) CrossRef
go back to reference H. Lee, D.J. Seo, Y. Liu, V. Koren, P. McKee, R. Corby, Variational assimilation of streamflow into operational distributed hydrologic models: effect of spatiotemporal scale of adjustment. Hydrol. Earth Syst. Sci. 16(7), 2233–2251 (2012) CrossRef H. Lee, D.J. Seo, Y. Liu, V. Koren, P. McKee, R. Corby, Variational assimilation of streamflow into operational distributed hydrologic models: effect of spatiotemporal scale of adjustment. Hydrol. Earth Syst. Sci. 16(7), 2233–2251 (2012) CrossRef
go back to reference M. Leisenring, H. Moradkhani, Snow water equivalent prediction using Bayesian data assimilation methods. Stoch. Environ. Res. Risk Assess. 25(2), 253–270 (2011) CrossRef M. Leisenring, H. Moradkhani, Snow water equivalent prediction using Bayesian data assimilation methods. Stoch. Environ. Res. Risk Assess. 25(2), 253–270 (2011) CrossRef
go back to reference Y. Liu, A.H. Weerts, M. Clark, H.J. Hendricks Franssen, S. Kumar, H. Moradkhani, D.J. Seo, D. Schwanenberg, P. Smith, A.I.J.M. van Dijk, N. van Velzen, M. He, H. Lee, S.J. Noh, O. Rakovec, P. Restrepo, Toward advancing data assimilation in operational hydrologic forecasting and water resources management: current status, challenges, and emerging opportunities. Hydrol. Earth Syst. Sci. 16, 3863–3887 (2012) CrossRef Y. Liu, A.H. Weerts, M. Clark, H.J. Hendricks Franssen, S. Kumar, H. Moradkhani, D.J. Seo, D. Schwanenberg, P. Smith, A.I.J.M. van Dijk, N. van Velzen, M. He, H. Lee, S.J. Noh, O. Rakovec, P. Restrepo, Toward advancing data assimilation in operational hydrologic forecasting and water resources management: current status, challenges, and emerging opportunities. Hydrol. Earth Syst. Sci. 16, 3863–3887 (2012) CrossRef
go back to reference A.C. Lorenc, The potential of the ensemble Kalman filter for NWP – a comparison with 4D-Var. Q. J. R. Meteorol. Soc. 129(595), 3183–3203 (2003) CrossRef A.C. Lorenc, The potential of the ensemble Kalman filter for NWP – a comparison with 4D-Var. Q. J. R. Meteorol. Soc. 129(595), 3183–3203 (2003) CrossRef
go back to reference H. Moradkhani, S. Sorooshian, H.V. Gupta, P.R. Houser, Dual state–parameter estimation of hydrological models using ensemble Kalman filter. Adv. Water Resour. 28(2), 135–147 (2005a) CrossRef H. Moradkhani, S. Sorooshian, H.V. Gupta, P.R. Houser, Dual state–parameter estimation of hydrological models using ensemble Kalman filter. Adv. Water Resour. 28(2), 135–147 (2005a) CrossRef
go back to reference H. Moradkhani, K.L. Hsu, H. Gupta, S. Sorooshian, Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter. Water Resour. Res. 41, W05012 (2005b) CrossRef H. Moradkhani, K.L. Hsu, H. Gupta, S. Sorooshian, Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter. Water Resour. Res. 41, W05012 (2005b) CrossRef
go back to reference G.S. Nearing, H.V. Gupta, The quantity and quality of information in hydrologic models. Water Resour. Res. 51(1), 524–538 (2015) CrossRef G.S. Nearing, H.V. Gupta, The quantity and quality of information in hydrologic models. Water Resour. Res. 51(1), 524–538 (2015) CrossRef
go back to reference G.S. Nearing, H.V. Gupta, W.T. Crow, Information loss in approximately bayesian estimation techniques: a comparison of generative and discriminative approaches to estimating agricultural productivity. J. Hydrol. 507, 163–173 (2013) CrossRef G.S. Nearing, H.V. Gupta, W.T. Crow, Information loss in approximately bayesian estimation techniques: a comparison of generative and discriminative approaches to estimating agricultural productivity. J. Hydrol. 507, 163–173 (2013) CrossRef
go back to reference S.J. Noh, Y. Tachikawa, M. Shiiba, S. Kim, Applying sequential Monte Carlo methods into a distributed hydrologic model: lagged particle filtering approach with regularization. Hydrol. Earth Syst. Sci. 15(10), 3237 (2011) CrossRef S.J. Noh, Y. Tachikawa, M. Shiiba, S. Kim, Applying sequential Monte Carlo methods into a distributed hydrologic model: lagged particle filtering approach with regularization. Hydrol. Earth Syst. Sci. 15(10), 3237 (2011) CrossRef
go back to reference S. Park, J.P. Hwang, E. Kim, H. Kang, A new evolutionary particle filter for the prevention of sample impoverishment. IEEE Trans. Signal Process. 13(4), 801–809 (2009) S. Park, J.P. Hwang, E. Kim, H. Kang, A new evolutionary particle filter for the prevention of sample impoverishment. IEEE Trans. Signal Process. 13(4), 801–809 (2009)
go back to reference D.A. Plaza, R. De Keyser, G.J.M. De Lannoy, L. Giustarini, P. Matgen, V.R.N. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012) CrossRef D.A. Plaza, R. De Keyser, G.J.M. De Lannoy, L. Giustarini, P. Matgen, V.R.N. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012) CrossRef
go back to reference R.H. Reichle, D. Entekhabi, D.B. McLaughlin, Downscaling of radio brightness measurements for soil moisture estimation: a four-dimensional variational data assimilation approach. Water Resour. Res. 37(9), 2353–2364 (2001) CrossRef R.H. Reichle, D. Entekhabi, D.B. McLaughlin, Downscaling of radio brightness measurements for soil moisture estimation: a four-dimensional variational data assimilation approach. Water Resour. Res. 37(9), 2353–2364 (2001) CrossRef
go back to reference R.H. Reichle, D.B. McLaughlin, D. Entekhabi, Hydrologic data assimilation with the ensemble Kalman filter. Mon. Weather Rev. 130(1), 103–114 (2002) CrossRef R.H. Reichle, D.B. McLaughlin, D. Entekhabi, Hydrologic data assimilation with the ensemble Kalman filter. Mon. Weather Rev. 130(1), 103–114 (2002) CrossRef
go back to reference J. Ruiz, M. Pulido, Parameter estimation using ensemble-based data assimilation in the presence of model error. Mon. Weather Rev. 143(5), 1568–1582 (2015) CrossRef J. Ruiz, M. Pulido, Parameter estimation using ensemble-based data assimilation in the presence of model error. Mon. Weather Rev. 143(5), 1568–1582 (2015) CrossRef
go back to reference P. Salamon, L. Feyen, Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter. J. Hydrol. 376(3), 428–442 (2009) CrossRef P. Salamon, L. Feyen, Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter. J. Hydrol. 376(3), 428–442 (2009) CrossRef
go back to reference D.-J. Seo, V. Koren, N. Cajina, Real-time variational assimilation of hydrologic and hydrometeorological data into operational hydrologic forecasting. J. Hydrometeorol. 4(3), 627–641 (2003) CrossRef D.-J. Seo, V. Koren, N. Cajina, Real-time variational assimilation of hydrologic and hydrometeorological data into operational hydrologic forecasting. J. Hydrometeorol. 4(3), 627–641 (2003) CrossRef
go back to reference A.G. Slater, M.P. Clark, Snow data assimilation via an ensemble Kalman filter. J. Hydrometeorol. 7, 478 (2005) CrossRef A.G. Slater, M.P. Clark, Snow data assimilation via an ensemble Kalman filter. J. Hydrometeorol. 7, 478 (2005) CrossRef
go back to reference P.J. Smith, G.D. Thornhill, S.L. Dance, A.S. Lawless, D.C. Mason, N.K. Nichols, Data assimilation for state and parameter estimation: application to morphodynamic modelling. Q. J. R. Meteorol. Soc. 139(671), 314–327 (2013) CrossRef P.J. Smith, G.D. Thornhill, S.L. Dance, A.S. Lawless, D.C. Mason, N.K. Nichols, Data assimilation for state and parameter estimation: application to morphodynamic modelling. Q. J. R. Meteorol. Soc. 139(671), 314–327 (2013) CrossRef
go back to reference C. Snyder, T. Bengtsson, P. Bickel, J. Anderson, Obstacles to high-dimensional particle filtering. Mon. Weather Rev. 136(12), 4629 (2008) CrossRef C. Snyder, T. Bengtsson, P. Bickel, J. Anderson, Obstacles to high-dimensional particle filtering. Mon. Weather Rev. 136(12), 4629 (2008) CrossRef
go back to reference R.D. Wilkinson, M. Vrettas, D. Cornford, J.E. Oakley, Quantifying simulator discrepancy in discrete-time dynamical simulators. J. Agric. Biol. Environ. Stat. 16(4), 554–570 (2011) CrossRef R.D. Wilkinson, M. Vrettas, D. Cornford, J.E. Oakley, Quantifying simulator discrepancy in discrete-time dynamical simulators. J. Agric. Biol. Environ. Stat. 16(4), 554–570 (2011) CrossRef
go back to reference S. Yin, X. Zhu, Intelligent particle filter and its application to fault detection of nonlinear systems. IEEE Trans. Ind. Electron. 62(6), 3852–3861 (2015) S. Yin, X. Zhu, Intelligent particle filter and its application to fault detection of nonlinear systems. IEEE Trans. Ind. Electron. 62(6), 3852–3861 (2015)
go back to reference D.a. Županski, F. Mesinger, Four-dimensional variational assimilation of precipitation data. Mon. Weather Rev. 123(4), 1112–1127 (1995) CrossRef D.a. Županski, F. Mesinger, Four-dimensional variational assimilation of precipitation data. Mon. Weather Rev. 123(4), 1112–1127 (1995) CrossRef
Metadata
Title
Fundamentals of Data Assimilation and Theoretical Advances
Authors
Hamid Moradkhani
Grey S. Nearing
Peyman Abbaszadeh
Sahani Pathiraja
Copyright Year
2019
Publisher
Springer Berlin Heidelberg
DOI
https://doi.org/10.1007/978-3-642-39925-1_30