nach oben

Erschienen in:

2019 | OriginalPaper | Buchkapitel

Verification Metrics for Hydrological Ensemble Forecasts

verfasst von : François Anctil, Maria-Helena Ramos

Erschienen in: Handbook of Hydrometeorological Ensemble Forecasting

Verlag: Springer Berlin Heidelberg

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This chapter reviews the most commonly used verification metrics for measuring the performance of hydrological ensemble forecasts. It links metrics to the different attributes of forecast quality and discusses the links between verification variables, metrics, and applications in a broad perspective. It provides an overview of the use of these metrics in forecast evaluation studies and general insights into what forecasters, practitioners, and end-users should consider when applying verification measures in the practice of hydrological ensemble forecasting.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Attributes of Forecast Quality

Nächstes Kapitel Verification of Meteorological Forecasts for Hydrological Applications

M. Abaza, F. Anctil, V. Fortin, R. Turcotte, A comparison of the Canadian global and regional meteorological ensemble prediction systems for short-term hydrological forecasting. Mon. Weather Rev. 141, 3462–3472 (2013). Corrigendum. Mon. Weather Rev. 142, 2561–2562CrossRef

M. Abaza, F. Anctil, V. Fortin, R. Turcotte, Sequential streamflow assimilation for short-term hydrological ensemble forecasting. J. Hydrol. 519, 2692–2706 (2014)CrossRef

L. Alfieri, F. Pappenberger, F. Wetterhall, T. Haiden, D. Richardson, P. Salamon, Evaluation of ensemble streamflow predictions in Europe. J. Hydrol. 517, 913–922 (2014)CrossRef

C. Alvarez-Garreton, D. Ryu, A.W. Western, C.-H. Su, W.T. Crow, D.E. Robertson, C. Leahy, Improving operational flood ensemble prediction by the assimilation of satellite soil moisture: comparison between lumped and semi-distributed schemes. Hydrol. Earth Syst. Sci. 19, 1659–1676 (2015)CrossRef

F. Anctil, C. Michel, C. Perrin, V. Andréassian, A soil moisture index as an auxiliary ANN input for stream flow forecasting. J. Hydrol. 286, 155–167 (2004)CrossRef

J.L. Anderson, A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Clim. 9, 1518–1530 (1996)CrossRef

D. Anghileri, N. Voisin, A.F. Castelletti, F. Pianosi, B. Nijssen, D.P. Lettenmaier, Value of long-term streamflow forecast to reservoir operations for water supply in snow-dominated catchments. Water Resour. Res. 52(6), 4209–4225 (2016). https://doi.org/10.1002/2015WR017864CrossRef

F. Atger, Verification of intense precipitation forecasts from single models and ensemble prediction systems. Nonlinear Process. Geophys. 8, 401–417 (2001)CrossRef

L. Baringhaus, C. Franz, On a new multivariate two-sample test. J. Multivar. Anal. 88(1), 190–206 (2004)CrossRef

J.C. Bartholmes, J. Thielen, M.H. Ramos, S. Gentilini, The European Flood Alert System EFAS – part 2: statistical skill assessment of probabilistic and deterministic operational forecasts. Hydrol. Earth Syst. Sci. 13(2), 141–153 (2009)CrossRef

M.A. Boucher, J.P. Laliberté, F. Anctil, An experiment on the evolution of an ensemble of neural networks for streamflow forecasting. Hydrol. Earth Syst. Sci. 14, 603–612 (2010)CrossRef

M.-A. Boucher, D. Tremblay, L. Delorme, L. Perreault, F. Anctil, Hydro-economic assessment of hydrological forecasting systems. J. Hydrol. 416, 133–144 (2012). https://doi.org/10.1016/j.jhydrol.2011.11.042CrossRef

F. Bourgin, M.-H. Ramos, G. Thirel, V. Andréassian, Investigating the interactions between data assimilation and post-processing in hydrological ensemble forecasting. J. Hydrol. 519, 2775–2784 (2014)CrossRef

A.A. Bradley, T. Hashino, S.S. Schwartz, Distributions-oriented verification of probability forecasts for small data samples. Weather Forecast. 18, 903–917 (2003)CrossRef

A.A. Bradley, J. Demargne, J.J. Franz, Attributes of forecast quality, in Handbook of Hydrometeorological Ensemble Forecasting, ed. by Q. Duan, F. Pappenberger, J. Thielen, A. Wood, H. Cloke, J. Schaake (Springer, Berlin/Heidelberg, 2016), 46p. https://doi.org/10.1007/978-3-642-40457-3_2-1

D. Brochero, F. Anctil, C. Gagné, Simplifying a hydrological ensemble prediction system with a backward greedy selection of members, part I: optimization criteria. Hydrol. Earth Syst. Sci. 15, 3307–3325 (2011)CrossRef

J.D. Brown, J. Demargne, D.J. Seo, Y. Liu, The Ensemble Verification System (EVS): a software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations. Environ. Model. Softw. 25(7), 854–872 (2010). https://doi.org/10.1016/j.envsoft.2010.01.009CrossRef

J.D. Brown, M. He, S. Regonda, L. Wu, H. Lee, D.-J. Seo, Verification of temperature, precipitation, and streamflow forecasts from the NOAA/NWS Hydrologic Ensemble Forecast Service (HEFS): 2. Streamflow verification. J. Hydrol. 519, 2847–2868 (2014)CrossRef

T.M. Carpenter, K.P. Georgakakos, Impacts of parametric and radar rainfall uncertainty on the ensemble streamflow simulations of a distributed hydrologic model. J. Hydrol. 298, 202–221 (2004)CrossRef

B. Casati, L.J. Wilson, D.B. Stephenson, Forecast verification: current status and future directions. Meteorol. Appl. 15(1), 3–18 (2008)CrossRef

M.P. Clark, A.G. Slater, Probabilistic quantitative precipitation estimation in complex terrain. J. Hydrometeorol. 7, 3–22 (2006)CrossRef

H. Cloke, F. Pappenberger, Evaluating forecasts of extreme events for hydrological applications: an approach for screening unfamiliar performance measures. Meteorol. Appl. 15, 181–197 (2008)CrossRef

C. Corradini, F. Melone, L. Ubertini, A semi-distributed adaptive model for real-time flood forecasting. Water Resour. Bull. 22, 1031–1038 (1986)CrossRef

L. Crochemore, M.-H. Ramos, F. Pappenberger, Bias correcting precipitation forecasts to improve the skill of seasonal streamflow forecasts. Hydrol. Earth Syst. Sci. 20, 3601–3618 (2016). https://doi.org/10.5194/hess-20-3601-2016CrossRef

J. Demargne, J.D. Brown, Y. Liu, D.-J. Seo, L. Wu, Z. Toth, Y. Zhu, Diagnostic verification of hydrometeorological and hydrologic ensembles. Atmos. Sci. Lett. 11(2), 114–122 (2010)CrossRef

D. Demeritt, S. Nobert, H.L. Cloke, F. Pappenberger, The European Flood Alert System and the communication, perception, and use of ensemble predictions for operational flood risk management. Hydrol. Process. 27, 147–157 (2013). https://doi.org/10.1002/hyp.9419CrossRef

K. Engeland, I. Steinsland, Probabilistic postprocessing models for flow forecasts for a system of catchments and several lead times. Water Resour. Res. 50, 182–197 (2014). https://doi.org/10.1002/2012WR012757CrossRef

F.M. Fan, W. Collischonn, A. Meller, L.C.M. Botelho, Ensemble streamflow forecasting experiments in a tropical basin: the São Francisco River case study. J. Hydrol. 519, 2906–2919 (2014)CrossRef

V. Fortin, M. Abaza, A. Anctil, R. Turcotte, Why should ensemble spread match the RMSE of the ensemble mean? J. Hydrometeorol. 15, 1708–1713 (2014)CrossRef

K.J. Franz, T.S. Hogue, M. Barik, Assessment of SWE data assimilation for ensemble streamflow predictions. J. Hydrol. 519(Part D), 2737–2746 (2014)CrossRef

T. Gneiting, A.E. Raftery, A.H. Westveld, T. Goldman, Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Weather Rev. 133, 1098–1118 (2005). https://doi.org/10.1175/MWR2904.1CrossRef

T. Gneiting, A.E. Raftery, Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102, 359–378 (2007)CrossRef

T. Gneiting, F. Balabdaoui, A.E. Raftery, Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. Ser. B Stat. Methodol. 69, 243–268 (2007)CrossRef

I.J. Good, Rational decisions. J. R. Stat. Soc. 14, 107–114 (1952)

T. Hamill, Interpretation of rank histograms for verifying ensemble forecasts. Mon. Weather Rev. 129, 550–560 (2001)CrossRef

T.M. Hamill, S.J. Colucci, Verification of Eta RSM short-range ensemble forecasts. Mon. Weather Rev. 125, 1312–1327 (1997)CrossRef

T. Hashino, A.A. Bradley, S.S. Schwartz, Evaluation of bias-correction methods for ensemble streamflow volume forecasts. Hydrol. Earth Syst. Sci. 11, 939–950 (2007)CrossRef

H. Hersbach, Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather Forecast. 15, 559–570 (2000)CrossRef

H. Holt, J. Pullen, C. Bishop, Urban and ocean ensembles for improved meteorological and dispersion modeling of the coastal zone. Tellus 61A, 232–249 (2009)CrossRef

I.T. Jolliffe, D.B. Stephenson, Forecast Verification: A practitioner’s Guide in Atmospheric Science, 2nd edn. (Wiley, New York, 2012). https://doi.org/10.1002/9781119960003CrossRef

Y.-O. Kim, H. Eum, E.G. Lee, I.H. Ko, Optimizing operational policies of a Korean multireservoir system using sampling stochastic dynamic programming with ensemble streamflow prediction. J. Water Resour. Plan. Manag. 133, 4–14 (2007). https://doi.org/10.1061/(ASCE)0733-9496(2007)133:1(4)CrossRef

P.K. Kitanidis, R.L. Bras, Real-time forecasting with a conceptual hydrologic model. 2. Applications and results. Water Resour. Res. 16(6), 1034–1044 (1980)CrossRef

F. Laio, S. Tamea, Verification tools for probabilistic forecasts of continuous hydrological variables. Hydrol. Earth Syst. Sci. 11(4), 1267–1277 (2007)CrossRef

K. Liechti, M. Zappa, F. Fundel, U. Germann, The potential of radar-based ensemble forecasts for flash-flood early warning in the southern Swiss Alps. Hydrol. Earth Syst. Sci. 17, 3853–3869 (2013)CrossRef

Y. Liu, J.D. Brown, J. Demargne, D.-J. Seo, A wavelet-based approach to assessing timing errors in hydrologic predictions. J. Hydrol. 397(3–4), 210–224 (2011)CrossRef

S.J. Mason, A model for assessment of weather forecast. Aust. Meteorol. Mag. 30, 291–303 (1982)

S. Matte, M.-A. Boucher, V. Boucher, T.-C. Fortier Filion, Moving beyond the cost–loss ratio: economic assessment of streamflow forecasts for a risk-averse decision maker. Hydrol. Earth Syst. Sci. 21, 2967–2986 (2017)CrossRef

D.N. Moriasi, J.G. Arnold, M.W. Van Liew, R.L. Bingner, R.D. Harmel, T.L. Veith, Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans. ASABE 50, 885–900 (2007)CrossRef

A.H. Murphy, A new vector partition of the probability score. J. Appl. Meteorol. 12(4), 595–600 (1973)CrossRef

A. Murphy, What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather Forecast. 8(2), 281–293 (1993)CrossRef

T. Palmer, R. Buizza, R. Hagedorn, A. Lawrence, M. Leutbecher, L. Smith, Ensemble prediction: a pedagogical perspective. ECMWF Newsl. 106, 10–17 (2005). ECMWF, Reading

F. Pappenberger, K. Scipal, R. Buizza, Hydrological aspects of meteorological verification. Atmos. Sci. Lett. 9, 43–52 (2008)CrossRef

F. Pappenberger, M.H. Ramos, H.L. Cloke, F. Wetterhall, L. Alfieri, K. Bogner, A. Mueller, P. Salamon, How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction. J. Hydrol. 522, 697–713 (2015)CrossRef

W.W. Peterson, T.G. Birdsall, W.C. Fox, The theory of signal detectability. Trans. IRE Prof. Group Inf. Theory 2–4, 171–212 (1954)CrossRef

M.H. Ramos, J. Bartholmes, J. Thielen-del Pozo, Development of decision support products based on ensemble forecasts in the European Flood Alert System. Atmos. Sci. Lett. 8, 113–119 (2007). https://doi.org/10.1002/asl.161CrossRef

M.H. Ramos, T. Mathevet, J. Thielen, F. Pappenberger, Communicating uncertainty in hydro-meteorological forecasts: mission impossible? Meteorol. Appl. 17, 223–235 (2010)CrossRef

A. Randrianasolo, M.H. Ramos, G. Thirel, V. Andreassian, E. Martin, Comparing the scores of hydrological ensemble forecasts issued by two different hydrological models. Atmos. Sci. Lett. 11(2), 100–107 (2010)CrossRef

B. Renard, D. Kavetski, G. Kuczera, M. Thyer, S.W. Franks, Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour. Res. 46, W05521 (2010)CrossRef

D.S. Richardson, Skill and relative economic value of ECMWF ensemble prediction system. Q. J. R. Meteorol. Soc. 126, 649–667 (2000)CrossRef

E. Roulin, Skill and relative economic value of medium-range hydrological ensemble predictions. Hydrol. Earth Syst. Sci. 11, 725–737 (2007)CrossRef

E. Roulin, S. Vannitsem, Post-processing of medium-range probabilistic hydrological forecasting: impact of forcing, initial conditions and model errors. Hydrol. Process. 29(6), 1434–1449 (2015)CrossRef

M.S. Roulston, L.A. Smith, Evaluating probabilistic forecasts using information theory. Mon. Weather Rev. 130, 1653–1660 (2002)CrossRef

M.S. Roulston, L.A. Smith, Combining dynamical and statistical ensembles. Tellus A 55, 16–30 (2003). https://doi.org/10.1034/j.1600-0870.2003.201378.xCrossRef

D.L. Shrestha, D.E. Robertson, J.C. Bennett, Q.J. Wang, Improving precipitation forecasts by generating ensembles through postprocessing. Mon. Weather Rev. 143, 3642–3663 (2015)CrossRef

G. Székely, M. Rizzo, A new test for multivariate normality. J. Multivar. Anal. 1(93), 58–80 (2005)CrossRef

O. Talagrand, R. Vautard, B. Strauss, Evaluation of probabilistic prediction systems, in Workshop on Predictability, ed. by for Medium-Range Weather Forecasts, E. C., Shinfield Park, Reading (1997), pp. 1–25

A. Thiboult, F. Anctil, M.-A. Boucher, Accounting for three sources of uncertainty in ensemble hydrological forecasting. Hydrol. Earth Syst. Sci. 20, 1809–1825 (2016)CrossRef

A. Thiboult, F. Anctil, M.H. Ramos, How does the quantification of uncertainties affect the quality and value of flood early warning systems? J. Hydrol. 551, 365–373 (2017). https://doi.org/10.1016/j.jhydrol.2017.05.014CrossRef

P. Trambauer, M. Werner, H.C. Winsemius, S. Maskey, E. Dutra, S. Uhlenbrook, Hydrological drought forecasting and skill assessment for the Limpopo River basin, southern Africa. Hydrol. Earth Syst. Sci. 19, 1695–1711 (2015). https://doi.org/10.5194/hess-19-1695-2015CrossRef

J. Van den Bergh, E. Roulin, Hydrological ensemble prediction and verification for the Meuse and Scheldt basins. Atmos. Sci. Lett. 11, 64–71 (2010). https://doi.org/10.1002/asl.250CrossRef

J.-A. Velázquez, F. Anctil, C. Perrin, Performance and reliability of multimodel hydrological ensemble simulations based on seventeen lumped models and a thousand catchments. Hydrol. Earth Syst. Sci. 14, 2303–2317 (2010)CrossRef

J.S. Verkade, M.G.F. Werner, Estimating the benefits of single value and probability forecasting for flood warning. Hydrol. Earth Syst. Sci. 15, 3751–3765 (2011)CrossRef

J.S. Verkade, J.D. Brown, P. Reggiani, A.H. Weerts, Post-processing ECMWF precipitation and temperature ensemble reforecasts for operational hydrologic forecasting at various spatial scales. J. Hydrol. 501, 73–91 (2013)CrossRef

N. Voisin, F. Pappenberger, D.P. Lettenmaier, R. Buizza, J.C. Schaake, Application of a medium-range global hydrologic probabilistic forecast scheme to the Ohio River Basin. Weather Forecast. 26, 425–446 (2011)CrossRef

A.S. Weigend, S. Shi, Predicting daily probability distributions of S&P500 returns. J. Forecast. 19, 375–392 (2000)CrossRef

S.V. Weijs, R. van Nooijen, N. van de Giesen, Kullback–Leibler divergence as a forecast skill score with classic reliability–resolution–uncertainty decomposition. Mon. Weather Rev. 138, 3387–3399 (2010)CrossRef

K. Werner, J.S. Verkade, T.C. Pagano, Application of hydrological forecast verification information, in Handbook of Hydrometeorological Ensemble Forecasting, ed. by Q. Duan, F. Pappenberger, J. Thielen, A. Wood, H. Cloke, J. Schaake (Springer, Berlin/Heidelberg, 2016), 22p. https://doi.org/10.1007/978-3-642-40457-3_7-1

D.S. Wilks, Statistical Methods in the Atmospheric Sciences: An Introduction (Academic, 2011), Amsterdam, 676p

A.W. Wood, A. Kumar, D.P. Lettenmaier, A retrospective assessment of National Centers for Environmental Prediction climate model–based ensemble hydrologic forecasting in the western United States. J. Geophys. Res. Atmos. 110, D04105 (2005). https://doi.org/10.1029/2004JD004508CrossRef

X. Yuan, J. Roundy, E. Wood, J. Sheffield, Seasonal forecasting of global hydrologic extremes: system development and evaluation over GEWEX basins. Bull. Am. Meteorol. Soc., 1895–1912 (2015). https://doi.org/10.1175/BAMS-D-14-00003.1CrossRef

I. Zalachori, M.H. Ramos, R. Garçon, T. Mathevet, J. Gailhard, Statistical processing of forecasts for hydrological ensemble prediction: a comparative study of different bias correction strategies. Adv. Sci. Res. 8, 135–141 (2012)CrossRef

M. Zappa, F. Fundel, S. Jaun, A ‘Peak-Box’ approach for supporting interpretation and verification of operational ensemble peak-flow forecasts. Hydrol. Process. 27(1), 117–131 (2013). https://doi.org/10.1002/hyp.9521CrossRef

Titel: Verification Metrics for Hydrological Ensemble Forecasts
verfasst von: François Anctil
Maria-Helena Ramos
Verlag: Springer Berlin Heidelberg
Buch: Handbook of Hydrometeorological Ensemble Forecasting
Print ISBN: 978-3-642-39924-4

Electronic ISBN: 978-3-642-39925-1

Copyright-Jahr: 2019
DOI: https://doi.org/10.1007/978-3-642-39925-1_3