Daily reservoir inflow forecasting using artificial neural networks with stopped training approach
Introduction
Accurate real-time forecasts of natural inflows to hydropower reservoirs are of particular interest for operation and scheduling. A variety of methods have been proposed for this purpose including conceptual (physical) and empirical (statistical) models (WMO, 1994) but none of them can be considered as a single superior model (Shamseldin, 1997). In a large scale hydrosystem context, owing to the complexity of hydrological processes, there are many situations where accurate site-specific predictions remain a difficult task using the linear recurrence relations or physically based watershed models. The former do not attempt to take into account the nonlinear dynamic of hydrological processes, and the latter generally ignore the stochastic behavior underlying any hydrosystem. Nonlinear statistical methods have been suggested and discussed (Jacoby, 1966, Amorocho and Brandstetter, 1971, Tong, 1990). Owing to difficulties of formulating reasonable nonlinear watershed models, recent attempts have resort to artificial neural network (ANN) approach for complex hydrologic modeling (Saad et al., 1996, Clair and Ehrman, 1998, Jain et al., 1999, Coulibaly et al., 2000).
ANNs are data dependent. They do not impose functional relationship between the independent and dependent variables. Instead, the functional relationship is determined by the data in the training (or calibration) process. The advantage of such an approach is that a network with sufficient hidden units is able to approximate any continuous function to any degree of accuracy, if efficient training is performed (Cybenko, 1989, Hornik et al., 1989). In the hydrological forecasting context, recent experiments have reported that ANNs may offer a promising alternative for rainfall–runoff modeling (Zhu and Fujita, 1994, Smith and Eli, 1995, Hsu et al., 1995, Shamseldin, 1997, Sajikumar and Thandaveswara, 1999, Tokar and Johnson, 1999), streamflow prediction (Kang et al., 1993, Karunanithi et al., 1994, Thirumalaiah and Deo, 1998; Clair and Ehrman, 1998, Zealand et al., 1999, Campolo et al., 1999), and reservoir inflow forecasting (Saad et al., 1996, Coulibaly et al., 1998, Jain et al., 1999). Recently, Coulibaly et al. (1999) reviewed the ANN-based modeling in hydrology over the last years, and reported that about 90% of the experiments extensively make use of the multi-layer feed-forward neural networks (FNN) trained by the standard backpropagation (BP) algorithm (Rumelhart et al., 1986). Although BP training has proved to be efficient in some applications, its convergence tends to be very slow, and it often yields sub-optimal solutions (Baldi and Hornik, 1989, Mülenbein, 1990, Sima, 1996). This may not be suitable for dynamic adaptive accurate forecasting purpose. FNN using BP algorithm simply fails to find a solution to even rather simple pattern classification problems (Mülenbein, 1990). Recent attempts to use BP training for real-time flood prediction has lead to relatively poor performance particularly for peak flows (Khondker et al., 1998). A major objective of training an ANN for prediction is to generalize, i.e. to have the outputs of the network approximate target values given inputs that were not in the training set. An efficient backpropagtion training requires at least some heuristic modifications (Jacobs, 1988, Tollenaere, 1990, Weigend et al., 1991, Yu and Chen, 1997) or some of the numerical optimization techniques fully described by Hagan et al. (1996, p. 12.8). The most well-known BP variant named cascade-correlation (Fahlman and Lebiere, 1990) has shown promising river flow prediction skill (Karunanithi et al., 1994, Thirumalaiah and Deo, 1998), however Smieja (1993) argued that it may generalize better when learning about logical rather than natural problems.
This paper investigates the use of early stopped training approach (STA) to improve multi-layer FNN training for real-time reservoir inflow forecasting. The idea of “early stopping” or “stopped training” before network convergence has first been introduced by Nelson and Illingworth (1991, p.165) to avoid the problem of overfitting in large FNN. Unfortunately, the stopping criterion used was not a good estimate of the generalization error (Sarle, 1995). Recently, Prechelt (1998) proposed an empirical early stopping criterion to improve the network generalization ability. In practice, this method requires a rather long training time. Here we use a generalization loss criterion to perform the early stopping training with Levenberg–Marquardt Backpropagation (LMBP).
This study aims to find an acceptable trade-off between network training time and valid generalization ability in order to enhance predictions accuracy. In Section 2, the proposed method is presented. The selected forecast models are described in Section 3. Results from the forecast experiment are reported in Section 4, and finally some conclusions are drawn in Section 5.
Section snippets
The FNN architecture
In general, the architecture of multi-layer FNN can have many layers where a layer represents a set of parallel processing units (or nodes). The three-layer FNN (Fig. 1) used in this study contains only one intermediate (hidden) layer. Multi-layer FNN can have more than one hidden layer, however theoretical works have shown that a single hidden layer is sufficient for ANNs to approximate any complex nonlinear function (Cybenko, 1989, Hornik et al., 1989). Indeed many experimental results seem
The conceptual model (PREVIS)
A conceptual model (PREVIS) maintained by the Energy Division of Aluminum Company of Canada (Alcan) for real-time reservoir operation is selected. The PREVIS model is an extended version of the global conceptual model proposed by Kite (1978) for the Canadian watersheds. The model structure incorporates interconnected conceptual storage systems that are found to have significant contribution to the generation of streamflow. The inputs to the PREVIS model are total precipitation (rainfall,
Basic dataset
Data for the experiment were taken from Chute-du-Diable watershed, in northern Quebec. The watershed is 9700 km2 in size and contains a large hydropower reservoir. The original data consist of 32 years (1964–1995) of daily natural inflows, precipitation (rain and snow) and estimated snowmelt, daily maximum, minimum and mean temperature. We used 29 years (1964–1992) of daily records for the models calibration and the last 3 years (1993–1995) for the prediction. When we use the FNN with STA, the
Conclusions
In this paper, the FNN is trained using the early stopped training approach (STA) with the second-order optimization method (LMBP) for real-time hydrologic forecasting. From the results obtained, the proposed method appears to be an effective tool for daily real-time reservoir inflow forecasting. An important advantage of the STA over the existing ANN pruning methods, is that it is faster than training to complete convergence followed by pruning. Moreover, the use of the STA secures the network
Acknowledgements
Financial support has been granted through a grant from the Natural Sciences and Engineering Research Council of Canada (NSERC) to the second author. This support is gratefully acknowledged. We gratefully acknowledge Professor Jean Rousselle (Civil Engineering Department, Ecole Polytechnique of Montreal) and Aluminum Company of Canada (Alcan) for providing the experiment data. The authors would like to thank two anonymous reviewers for their valuable comments and suggestions.
References (47)
- et al.
Neural network and principal component analysis: learning from examples without local minima
Neural Networks
(1989) - et al.
Improving model selection by nonconvergent methods
Neural Networks
(1993) - et al.
Multilayer feedforward networks are universal approximators
Neural Networks
(1989) - et al.
River flow forecasting through conceptual models. Part I—a discussion of principles
J. Hydrol.
(1970) Automatic early stopping using cross validation: quantifying the criteria
Neural Networks
(1998)- et al.
A non-linear rainfall–runoff model using an artificial neural network
J. Hydrol.
(1999) Application of a neural network technique to rainfall–runoff modelling
J. Hydrol.
(1997)SuperSAB: Fast adaptive backpropagation with good scalling properties
Neural Networks
(1990)- et al.
Efficient backpropagation learning using optimal learning rate and momentum
Neural Networks
(1997) - et al.
Short term streamflow forecasting using artificial neural networks
J. Hydrol.
(1999)