Comparison of direct and iterative artificial neural network forecast approaches in multi-periodic time series forecasting
Introduction
There are a few methods commonly used for time series forecasting. The Box–Jenkins method, which gives quite good results with linear time series, is one of them. The Box–Jenkins technique is considerably effective especially with linear and stationary data sets and with series that are non-stationary but turned into stationary via transformations. However, series extracted from the real world are not generally linear. Therefore, modeling nonlinear time series requires different techniques. For the last 25 years, a number of nonlinear time series models have been developed; such as the Bilinear Model, threshold autoregressive (TAR) model, and autoregressive conditional heteroscedastic (ARCH) model. These models are applicable, when information concerning the correlation between data is available and clear (Zhang, Patuwo, & Hu, 1998). Moreover, none of these models is effective in elucidating a whole nonlinear structure hidden in the data set (Zhang, Patuwo, & Hu, 2001). Although these models are better than the linear models, their applications are difficult, problem specific and not prone to generalization (Ghiassi, Saidane, & Zimbra, 2004).
One of the techniques that has been used for time series forecasting since the late 1980s is artificial neural networks (ANN). ANN, an approach to linear and nonlinear time series, is widely used to forecast the future. ANN provides linear and nonlinear modeling without the necessity of preliminary information and assumptions as to the relation between input and output variables. Therefore, ANN is more flexible and applicable than other methods (Zhang et al., 1998).
There have been many studies aimed at the utilization of ANN for time series forecasting. Tang, Almeida, & Fishwick, 1991 studied ANN and Box–Jenkins forecasting comparatively in time series. They remarked that ANN structure and training parameters such as learning rate and momentum coefficient affect ANN performance. Hill, O’Connor, and Remus (1996) discussed the data period (annual, quarterly, monthly), functional form of the series (linear, nonlinear), forecasting horizon, and number of observation as factors that affect the ANN performance. Faraway and Chatfield (1998) studied with a seasonal time series which is called airline data. Faraway and Chatfield’s (1998) study showed the dependence of different ANN structure on time series forecasting performance. Detailed literature studies were presented by Zhang et al. (1998), and Adya and Collopy (1998). Nelson, Hill, Remus, and O’Connor (1999) focused on seasonality in time series structure and discussed the necessity of deseasonalizing the data. Chu and Zhang (2003) compared the accuracy of various linear and nonlinear models for forecasting aggregate retail sales.
Time series forecasting can be performed for both single and multiple periods. Two different approaches can be used for forecasting multiple periods. One of them is a single period iterative forecast, as in the Box–Jenkins models. The other one is a direct method in which multiple periods are estimated simultaneously.
Using the autoregressive methods, Findley (1985) showed that the direct forecasting approach is better than the iterative forecasting approach for a covariance stationary autoregressive process that is not degenerate to a finite order. The results founded by Bhansali, 1996, Bhansali, 1997 support Findley’s (1985) conclusion. Kang (2003) studied monthly data on various US economic time series. The results found by Kang (2003) show that the direct method may or may not improve forecast accuracy upon the iterative method. Kang stated that the forecast performance of the direct method relative to the iterative method appears to depend on, optimal order selection criteria, forecast periods, forecast horizons and the time series to be forecasted.
On the other hand, there are contrary findings when forecasting the time series with ANN. Weigend, Huberman, and Rumelhart (1992) showed that the iterative forecast yields better results than the direct forecast method in their sunspot data analysis. This conclusion is supported by Hill, Marquez, O’Connor, and Remus (1994), in their 111 M-competition time series study. According to Zhang (1994), the direct method gives better results. Lachtermacher and Fuller (1995) emphasize that the direct method requires a larger time series than the iterative method in order to avoid generalization. Kline (2004) used three methods (iterative, independent and joint method) for multi-step forecasting. In Kline’s study (2004), the independent method is based on the separated networks forecasting for each period, and the joint method is the same as the direct method used in this study. According to Kline (2004), the independent method is better than joint method but training sample size and forecast horizon might affect this superiority.
The above studies show that it is not clear which method gives better results. It is thought that the direct method gives better results when compared with the iterative method because it is based on past data. In this paper, a comparison of the iterative and direct forecasting methods is presented. They are considered to be influential on multi-periodic forecast performance of ANN. The organization of rest of the paper is as follows: the second section presents time series. Important parameters of the ANN models are given in the third section. The fourth section discusses an example application. To assess and compare the performance of the methods in the application part, grey relational analysis (GRA) is utilized. The last section gives the main conclusions.
Section snippets
Time series
A time series can be defined as a set of values of a variable during consecutive time increments. Time increments vary from series to series. That is to say, time series can be generated according to hourly, daily, weekly, monthly, quarterly, and annual data or according to another time scale. In a time series, observed data at any given time t is represented by Yt.
The most comprehensive of all popular and widely known statistical methods used for time series forecasting are Box–Jenkins Models.
Artificial neural networks
ANN, imitating the functioning of human brain, is a tool of great importance in classification, pattern recognition and forecasting. A typical ANN model is a combination of layers made of neurons. The most widely used type of ANN for forecasting is the multi layer perceptron (MLP).
In a MLP designed for time series forecasting, determining the variables, such as the number of input, hidden and output neurons, is highly important. However, these parameters depend on the problem. In other words,
An application of the methods
In this study, the series employed by Hansen, Mcdonald, and Nelson (1999) are utilized to forecast through an application aimed to handle real life time series. They worked on six different series (Series A, B, C, D, E, F) that Box and Jenkins (1976) used previously and which are shown in Fig. 1.
Hansen et al. (1999) compared their results based on two ANN models with the techniques suggested by McDonald and Xu (1994). Partially adaptive estimation techniques described by McDonald and Xu (1994)
Conclusions
In this paper, comparisons of the iterative and direct forecasting methods, which are considered to be influential on multi-periodic forecast performance of ANN, were presented. Also, in this study, performances of these approaches were compared with each other by using standard ARIMA models and partial adaptive estimation techniques. GRA was utilized to compare the performance of the investigated methods over different time series. Based on the performed comparison, superiority of the direct
References (34)
- et al.
A comparative study of linear and nonlinear models for aggregate retail sales forecasting
International Journal of Production Economics
(2003) - et al.
Artificial neural networks for forecasting and decision making
International Journal of Forecasting
(1994) - et al.
Multilayer feed-forward networks are universal approximators
Neural Networks
(1989) - et al.
Designing a neural network for forecasting financial and econometric time series
Neurocomputing
(1996) Multi-period forecasting using different models for different horizons: an application to U.S. economic time series data
International Journal of Forecasting
(2003)- et al.
Some forecasting applications of partially adaptive estimators of ARIMA models
Economics Letters
(1994) Time series forecasting using backpropagation neural networks
Neurocomputing
(1991)- et al.
Forecasting with artificial neural networks: the state of the art
International Journal of Forecasting
(1998) - et al.
A simulation study of artificial neural networks for nonlinear time-series forecasting
Computers and Operations Research
(2001) - et al.
How effective are neural networks at forecasting and prediction? A review and evaluation
Journal of Forecasting
(1998)
Developing neural network applications
AI Expert
Asymptotically efficient autoregressive model selection for multistep prediction
Annals of the Institute of Statistical Mathematics
Direct autoregressive predictors for multi step prediction: order selection and performance relative to the plug in predictors
Statistica Sinica
Time series analysis forecasting and control
Approximation by superposition of a sigmoidal function
Mathematics of Control Signals and Systems
Introduction to grey system
Journal of Grey System
Time series forecasting with neural networks: a comparative study using the airline data
Applied Statistics
Cited by (184)
Deep learning based prediction of traffic peaks in mobile networks
2024, Computer NetworksSA–EMD–LSTM: A novel hybrid method for long-term prediction of classroom PM<inf>2.5</inf> concentration
2023, Expert Systems with ApplicationsA dynamic-inner LSTM prediction method for key alarm variables forecasting in chemical process
2023, Chinese Journal of Chemical EngineeringNonlinearity in forecasting energy commodity prices: Evidence from a focused time-delayed neural network
2023, Research in International Business and FinanceCitation Excerpt :The use of nonlinear transfer functions in hidden layers allows the ANN to learn a nonlinear and linear relationship, while the linear function in the output layer is enough to produce accurate forecasts. For our purpose, this kind of function sequence will be used in accordance with the main studies and practices in forecasting (Zhang et al., 1998; HamzaCebi et al., 2009). The momentum term causes the method to converge fast and minimizes oscillations in the weight space.