Skip to main content
SpringerLink
Log in

On the problem of model validation for predictive exposure assessments

  • Originals
  • Published:
Stochastic Hydrology and Hydraulics Aims and scope Submit manuscript

Abstract

The development and use of models for predicting exposures are increasingly common and are essential for many risk assessments of the United States Environmental Protection Agency (EPA). Exposure assessments conducted by the EPA to assist regulatory or policy decisions are often challenged to demonstrate their “scientific validity”. Model validation has thus inevitably become a major concern of both EPA officials and the regulated community, sufficiently so that the EPA's Risk Assessment Forum is considering guidance for model validation. The present paper seeks to codify the issues and extensive foregoing discussion of validation with special reference to the development and use of models for predicting the impact of novel chemicals on the environment. Its preparation has been part of the process in formulating a White Paper for the EPA's Risk Assessment Forum. Its subject matter has been drawn from a variety of fields, including ecosystem analysis, surface water quality management, the contamination of groundwaters from high-level nuclear waste, and the control of air quality. The philosophical and conceptual bases of model validation are reviewed, from which it is apparent that validation should be understood as a task of product (or tool) design, for which some form of protocol for quality assurance will ultimately be needed. The commonly used procedures and methods of model validation are also reviewed, including the analysis of uncertainty. Following a survey of past attempts at resolving the issue of model validation, we close by introducing the notion of a model having maximum relevance to the performance of a specific task, such as, for example, a predictive exposure assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ababou, R.; Sagar, B.; Wittmeyer, G. 1992: Testing procedures for spatially distributed flow models, Adv. Water Resour. 15(3), 181–198

    Article  Google Scholar 

  • Akaike, H. 1974: A new look at statistical model identification, IEEE Trans on Automatic Control AC-19, 716–722

    Article  Google Scholar 

  • American Society for Testing and Materials. 1984: Standard practice for evaluating environmental fate models of chemicals, Standard E 978-84, American Society for Testing and Materials, Philadelphia, Pennsylvania

    Google Scholar 

  • Beck, M.B. 1987: Water quality modeling: a review of the analysis of uncertainty, Water Resour. Res. 23(8), 1393–1442

    Article  CAS  Google Scholar 

  • Beck, M.B. 1991: Forecasting environmental change, J. Forecasting 10(1/2), 3–19

    Google Scholar 

  • Beck, M.B. 1994: Understanding uncertain environmental systems, in Predictability and Nonlinear Modelling in Natural Sciences and Economics (J. Grasman and G. van Straten, eds), Kluwer, Dordrecht, pp 294–311

    Google Scholar 

  • Beck, M.B.; Halfon E. 1991: Uncertainty, identifiability and the propagation of prediction errors: a case study of Lake Ontario, J. Forecasting 10(1/2), 135–161

    Google Scholar 

  • Beck, M.B.; Chen, J. 1977: Process models: validation and structural change, in Assessment of Environmental Impact (Proceedings of a Workshop), International Centre for Mathematical Sciences, Edinburgh, UK (in preparation)

    Google Scholar 

  • Burns, L.A. 1983: Validation of exposure models: the role of conceptual verification, sensitivity analysis, and alternative hypotheses, in Proceedings 6th Symposium, Aquatic Toxicology and Hazard Assessment (W E Bishop, R D Cardwell, and B B Heidolph, eds), American Society for Testing and Materials, Philadelphia, ASTM. Special Technical Publication 802, pp 255–281

    Google Scholar 

  • Burns, L.A.; Barber, M.C.; Bird, S.L.; Mayer, F.L., Suárez, A. 1990: PIRANHA: Pesticide and Industrial Chemical Risk Analysis and Hazard Assessment, Internal Report, Environmental Research Laboratory, United States Environmental Protection Agency, Athens, Georgia

    Google Scholar 

  • Cale, W.G. Jr.; O'Neill, R.V.; Shugart, H.H. 1983: Development and application of desirable ecological models, Ecological Modelling 18, 171–186

    Article  Google Scholar 

  • Caswell, H. 1976: The validation problem, in Systems Analysis and Simulation in Ecology (B C Patten, ed), Academic, New York, Vol IV, pp 313–325

    Google Scholar 

  • Chen, J.; Beck, M.B. 1977: Discriminating power of the EPA's Multi-Media Model in performing a screening task, Technical Report, United States Environmental Protection Agency, Athens, Georgia (in review)

    Google Scholar 

  • Costanza, R.; Sklar, F.H. 1975: Mathematical models of freshwater wetlands and shallow water ecosystems: an articulated review, in Proceedings SCOPE International Conference on Freshwater Wetlands and Shallow Water Bodies

  • Cox, D.C.; Baybutt, P. 1981: Methods of uncertainty analysis: a comparative survey, Risk Analysis 1(4), 251–258

    Article  Google Scholar 

  • Davis, P.A.; Price, L.L.; Wahi, K.K.; Goodrich, M.T.; Gallegos, D.P.; Bonano E.J.; Guzowksi, R.V. 1990: Components of an overall performance assessment methodology, Report NUREG/CR5256, SAND88-3020, Sandia National Laboratories, Albuquerque, New Mexico

    Google Scholar 

  • Donigian A.S.; Rao, P.S.C. 1990: Selection, application, and validation of environmental models, in Proceedings International Symposium on Water Quality Modeling of Agricultural Non-Point Sources, (D G DeCoursey, ed), Report ARS-81, Agricultural Research Service, United States Department of Agriculture, pp 577–600

  • Environmental Protection Agency. 1991: Guidelines for Exposure Assessment, Science Advisory Board Draft Final, August

  • Funtowicz, S.O.; Ravetz, J. 1990: Uncertainty and Quality in Science for Policy, Kluwer, Dordrecht

    Google Scholar 

  • Hassanizadeh, S.M.; Carrera, J. 1992: Editorial: special issue on validation of geo-hydrological models, Adv. Water Resour. 15(1), 1–3

    Article  Google Scholar 

  • Hermann, C.F. 1967: Validation problems in games and simulations with special reference to models of international politics, Behavioral Science 12, 216–231

    CAS  Google Scholar 

  • Hornberger, G.M.; Spear, R.C. 1980: Eutrophication in Peel Inlet, I. Problem-defining behaviour and a mathematical model for the phosphorus scenario, Water Res. 14, 29–42

    Article  CAS  Google Scholar 

  • Iman, R.L.; Helton, J.C. 1988: An investigation of uncertainty and sensitivity analysis techniques for computer models, Risk Analysis 8(1), 71–90

    Article  Google Scholar 

  • Konikow, L.F.; Bredehoeft, J.D. 1992: Ground-water models cannot be validated, Adv. Water Resour. 15(1), 75–83

    Article  Google Scholar 

  • Luis, S.J.; McLaughlin, D.B. 1992: A stochastic approach to model validation, Adv. Water Resour. 15(1), 15–32

    Article  Google Scholar 

  • MacFarlane, A.G.J. 1990: Interactive computing: a revolutionary medium for teaching and design, Computing and Control Engineering Journal 1(4), 149–158

    Article  Google Scholar 

  • Mankin, J.B.; O'Neill, R.V.; Shugart, H.H.; Rust, B.W. 1977: The importance of validation in ecosystem analysis, in New Directions in the Analysis of Ecological Systems, (G S Innis, ed), Simulation Council, La Jolla, California, Proceedings Series 5(1), pp 63–72

    Google Scholar 

  • Mihram, G.A. 1973: Some practical aspects of the verification and validation of simulation models, Operations Research Quarterly 23, 17–29

    Article  Google Scholar 

  • Miller, D.R.; Butler, G.; Bramall, C. 1976: Validation of ecological system models, J. Environmental Management 4, 383–401

    Google Scholar 

  • Morton, A. 1993: Mathematical models: questions of trustworthiness, British J. Phil of Science 44, 659–674

    Google Scholar 

  • National Acid Precipitation Assessment Program (NAPAP). 1990: Evaluation of regional acidic deposition models and selected applications of RADM, Acidic Deposition: State of Science and Technology, Vol I, Report 5, The National Acid Precipitation Assessment Program, Washington, DC

    Google Scholar 

  • National Research Council. 1990: Ground Water Models: Scientific and Regulatory Applications, Water Science and Technology Board, United States National Research Council, National Academy Press, Washington DC

    Google Scholar 

  • Oreskes, N.; Shrader-Frechette, K.; Belitz, K. 1994: Verification, validation, and confirmation of numerical models in the earth sciences, Science 263, 641–646

    Google Scholar 

  • Parrish, R.S.; Smith, C.N. 1990: A method for testing whether model predictions fall within a prescribed factor of true values, with an application to pesticide leaching, Ecological Modelling 51, 59–72

    Article  CAS  Google Scholar 

  • Popper, K.R. 1959: The Logic of Scientific Discovery, Harper, New York

    Google Scholar 

  • Popper, K.R. 1963: Conjectures and Refutations: The Growth of Scientific Knowledge, Harer, New York

    Google Scholar 

  • Reckhow, K.H.; Chapra, S.C. 1983: Confirmation of water quality models, Ecological Modelling 20, 113–133

    Article  Google Scholar 

  • Reckhow, K.H.; Clements, J.T.; Dodd, R.C. 1990: Statistical evaluation of mechanistic water-quality models, J. Environmental Engineering, Proc American Society of Civil Engineers, 116(2), 250–268.

    CAS  Google Scholar 

  • Scavia D. 1980: An ecological model of Lake Ontario, Ecological Modelling 8, 49–78

    Article  CAS  Google Scholar 

  • Spear, R.C.; Hornberger, G.M. 1980: Eutrophication in Peel Inlet, II. Identification of critical uncertainties via generalised sensitivity analysis, Water Res. 14, 43–49

    Article  CAS  Google Scholar 

  • United States Environmental Protection Agency. 1989: Resolution on Use of Mathematical Models by EPA for Regulatory Assessment and Decision-Making, Report, Environmental Engineering Committee, Science Advisory Board, United States Environmental Protection Agency, Washington DC

  • Usunoff, E.; Carrera, J.; Mousavi, S.F. 1992: An approach to the design of experiments for discriminating among alternative conceptual models, Adv. Water Resour. 15(3), 199–214

    Article  Google Scholar 

  • Van Straten, G.; Keesman, K.J. 1991: Uncertainty propagation and speculation in projective forecasts of environmental change: a lake-eutrophication example, J. Forecasting 10(1/2), 163–190

    Google Scholar 

  • Varis, O. 1995: Belief networks for modelling and assessment of environmental change, Environmetrics 6, 439–444

    Google Scholar 

  • Versar Inc. 1988: Current and suggested practices in the validation of exposure assessment models, Report, Office of Health and Environmental Assessment, United States Environmental Protection Agency, Washington DC

    Google Scholar 

  • Young, P.C. 1978: General theory of modelling badly defined systems, in Modelling, Identification and Control in Environmental Systems, (G.C. Vansteenkiste, ed), North-Holland, Amsterdam, pp 103–135

    Google Scholar 

  • Young, P.C.; Hornberger, G.M.; Spear, R.C. 1978: Modelling badly defined systems — Some further thoughts, in Proceedings SIMSIG Simulation Conference, Australian National University, Canberra, 24–32

    Google Scholar 

  • Zimmerman, D.A.; Wahi, K.K.; Gutjahr, A.L.; Davis, P.A. 1990: A review of techniques for propagating data and parameter uncertainties in high-level radioactive waste repository performance assessment models, Report NUREG/CR-5393, SAND89-1432, Sandia National Laboratories, Albuquerque, New Mexico

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Warnell School of Forest Resources, University of Georgia, 30602-2152, Athens, GA, USA

    M. B. Beck

  2. The Research Methods Consultancy Ltd., 196 Clarence Gate Gardens, Glentworth Street, NW1 6AU, London, UK

    J. R. Ravetz

  3. Environmental Research Laboratory, United States Environmental Protection Agency, College Station Road, 30605-2720, Athens, GA, USA

    L. A. Mulkey & T. O. Barnwell

Authors
  1. M. B. Beck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. R. Ravetz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. L. A. Mulkey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. T. O. Barnwell
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Beck, M.B., Ravetz, J.R., Mulkey, L.A. et al. On the problem of model validation for predictive exposure assessments. Stochastic Hydrol Hydraul 11, 229–254 (1997). https://doi.org/10.1007/BF02427917

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02427917

Key words

Search

Navigation