Daily reservoir inflow forecasting using artificial neural networks with stopped training approach

https://doi.org/10.1016/S0022-1694(00)00214-6Get rights and content

Abstract

In this paper, an early stopped training approach (STA) is introduced to train multi-layer feed-forward neural networks (FNN) for real-time reservoir inflow forecasting. The proposed method takes advantage of both Levenberg–Marquardt Backpropagation (LMBP) and cross-validation technique to avoid underfitting or overfitting on FNN training and enhances generalization performance. The methodology is assessed using multivariate hydrological time series from Chute-du-Diable hydrosystem in northern Quebec (Canada). The performance of the model is compared to benchmarks from a statistical model and an operational conceptual model. Since the ultimate goal concerns the real-time forecast accuracy, overall the results show that the proposed method is effective for improving prediction accuracy. Moreover it offers an alternative when dynamic adaptive forecasting is desired.

Introduction

Accurate real-time forecasts of natural inflows to hydropower reservoirs are of particular interest for operation and scheduling. A variety of methods have been proposed for this purpose including conceptual (physical) and empirical (statistical) models (WMO, 1994) but none of them can be considered as a single superior model (Shamseldin, 1997). In a large scale hydrosystem context, owing to the complexity of hydrological processes, there are many situations where accurate site-specific predictions remain a difficult task using the linear recurrence relations or physically based watershed models. The former do not attempt to take into account the nonlinear dynamic of hydrological processes, and the latter generally ignore the stochastic behavior underlying any hydrosystem. Nonlinear statistical methods have been suggested and discussed (Jacoby, 1966, Amorocho and Brandstetter, 1971, Tong, 1990). Owing to difficulties of formulating reasonable nonlinear watershed models, recent attempts have resort to artificial neural network (ANN) approach for complex hydrologic modeling (Saad et al., 1996, Clair and Ehrman, 1998, Jain et al., 1999, Coulibaly et al., 2000).

ANNs are data dependent. They do not impose functional relationship between the independent and dependent variables. Instead, the functional relationship is determined by the data in the training (or calibration) process. The advantage of such an approach is that a network with sufficient hidden units is able to approximate any continuous function to any degree of accuracy, if efficient training is performed (Cybenko, 1989, Hornik et al., 1989). In the hydrological forecasting context, recent experiments have reported that ANNs may offer a promising alternative for rainfall–runoff modeling (Zhu and Fujita, 1994, Smith and Eli, 1995, Hsu et al., 1995, Shamseldin, 1997, Sajikumar and Thandaveswara, 1999, Tokar and Johnson, 1999), streamflow prediction (Kang et al., 1993, Karunanithi et al., 1994, Thirumalaiah and Deo, 1998; Clair and Ehrman, 1998, Zealand et al., 1999, Campolo et al., 1999), and reservoir inflow forecasting (Saad et al., 1996, Coulibaly et al., 1998, Jain et al., 1999). Recently, Coulibaly et al. (1999) reviewed the ANN-based modeling in hydrology over the last years, and reported that about 90% of the experiments extensively make use of the multi-layer feed-forward neural networks (FNN) trained by the standard backpropagation (BP) algorithm (Rumelhart et al., 1986). Although BP training has proved to be efficient in some applications, its convergence tends to be very slow, and it often yields sub-optimal solutions (Baldi and Hornik, 1989, Mülenbein, 1990, Sima, 1996). This may not be suitable for dynamic adaptive accurate forecasting purpose. FNN using BP algorithm simply fails to find a solution to even rather simple pattern classification problems (Mülenbein, 1990). Recent attempts to use BP training for real-time flood prediction has lead to relatively poor performance particularly for peak flows (Khondker et al., 1998). A major objective of training an ANN for prediction is to generalize, i.e. to have the outputs of the network approximate target values given inputs that were not in the training set. An efficient backpropagtion training requires at least some heuristic modifications (Jacobs, 1988, Tollenaere, 1990, Weigend et al., 1991, Yu and Chen, 1997) or some of the numerical optimization techniques fully described by Hagan et al. (1996, p. 12.8). The most well-known BP variant named cascade-correlation (Fahlman and Lebiere, 1990) has shown promising river flow prediction skill (Karunanithi et al., 1994, Thirumalaiah and Deo, 1998), however Smieja (1993) argued that it may generalize better when learning about logical rather than natural problems.

This paper investigates the use of early stopped training approach (STA) to improve multi-layer FNN training for real-time reservoir inflow forecasting. The idea of “early stopping” or “stopped training” before network convergence has first been introduced by Nelson and Illingworth (1991, p.165) to avoid the problem of overfitting in large FNN. Unfortunately, the stopping criterion used was not a good estimate of the generalization error (Sarle, 1995). Recently, Prechelt (1998) proposed an empirical early stopping criterion to improve the network generalization ability. In practice, this method requires a rather long training time. Here we use a generalization loss criterion to perform the early stopping training with Levenberg–Marquardt Backpropagation (LMBP).

This study aims to find an acceptable trade-off between network training time and valid generalization ability in order to enhance predictions accuracy. In Section 2, the proposed method is presented. The selected forecast models are described in Section 3. Results from the forecast experiment are reported in Section 4, and finally some conclusions are drawn in Section 5.

Section snippets

The FNN architecture

In general, the architecture of multi-layer FNN can have many layers where a layer represents a set of parallel processing units (or nodes). The three-layer FNN (Fig. 1) used in this study contains only one intermediate (hidden) layer. Multi-layer FNN can have more than one hidden layer, however theoretical works have shown that a single hidden layer is sufficient for ANNs to approximate any complex nonlinear function (Cybenko, 1989, Hornik et al., 1989). Indeed many experimental results seem

The conceptual model (PREVIS)

A conceptual model (PREVIS) maintained by the Energy Division of Aluminum Company of Canada (Alcan) for real-time reservoir operation is selected. The PREVIS model is an extended version of the global conceptual model proposed by Kite (1978) for the Canadian watersheds. The model structure incorporates interconnected conceptual storage systems that are found to have significant contribution to the generation of streamflow. The inputs to the PREVIS model are total precipitation (rainfall,

Basic dataset

Data for the experiment were taken from Chute-du-Diable watershed, in northern Quebec. The watershed is 9700 km2 in size and contains a large hydropower reservoir. The original data consist of 32 years (1964–1995) of daily natural inflows, precipitation (rain and snow) and estimated snowmelt, daily maximum, minimum and mean temperature. We used 29 years (1964–1992) of daily records for the models calibration and the last 3 years (1993–1995) for the prediction. When we use the FNN with STA, the

Conclusions

In this paper, the FNN is trained using the early stopped training approach (STA) with the second-order optimization method (LMBP) for real-time hydrologic forecasting. From the results obtained, the proposed method appears to be an effective tool for daily real-time reservoir inflow forecasting. An important advantage of the STA over the existing ANN pruning methods, is that it is faster than training to complete convergence followed by pruning. Moreover, the use of the STA secures the network

Acknowledgements

Financial support has been granted through a grant from the Natural Sciences and Engineering Research Council of Canada (NSERC) to the second author. This support is gratefully acknowledged. We gratefully acknowledge Professor Jean Rousselle (Civil Engineering Department, Ecole Polytechnique of Montreal) and Aluminum Company of Canada (Alcan) for providing the experiment data. The authors would like to thank two anonymous reviewers for their valuable comments and suggestions.

References (47)

  • G. Zhang et al.

    Forecasting with artificial neural networks: the state of the art

    Int. J. Forecasting

    (1998)
  • J. Amorocho et al.

    Determination of nonlinear function response functions in rainfall–runoff processes

    Water Resour. Res.

    (1971)
  • D.P. Bertsekas et al.

    Neuro-Dynamic Programming

    (1996)
  • G.E.P. Box et al.

    Time Series Analysis: Forecasting and Control

    (1976)
  • M. Campolo et al.

    River flood forecasting with a neural network model

    Water Resour. Res.

    (1999)
  • T.A. Clair et al.

    Using neural networks to assess the influence of changing seasonal climates in modifying discharge, dissolved organic carbon, and nitrogen export in eastern Canadian rivers

    Water Resour. Res.

    (1998)
  • Coulibaly, P., Anctil, F., Bobée, B., 1998. Real time neural network-based forecasting system for hydropower...
  • P. Coulibaly et al.

    Prévision hydrologique par réseaux de neurones artificiels: état de l'art

    Can. J. Civil Engng

    (1999)
  • Coulibaly, P., Anctil, F., Rasmussen, P., Bobée, B., 2000. A recurrent neural networks approach using indices of...
  • G. Cybenko

    Approximation by superposition of a sigmoidal function

    Math. Control Signals Syst.

    (1989)
  • S.E. Fahlman et al.

    The cascade-correlation learning architecture

  • M.T. Hagan et al.

    Training feedforward networks with the Marquardt algorithm

    IEEE Trans. Neural Networks

    (1994)
  • M.T. Hagan et al.

    Neural Network Design

    (1996)
  • Cited by (0)

    View full text