A self adaptive differential harmony search based optimized extreme learning machine for financial time series prediction

Highlights

• An optimized extreme learning machine is presented for financial time series prediction.

• SADHS is a variant of harmony search technique that uses the current to best mutation scheme of DE.

• Two SLFNs like the RBF and low complexity FLANN are trained by an evolutionary ELM algorithm.

• Several learning algorithms are used for comparison to validate the superior performance of SADHS.

Abstract

This paper proposes a hybrid learning framework called Self Adaptive Differential Harmony Search Based Optimized Extreme Learning Machine (SADHS-OELM) for single hidden layer feed forward neural network (SLFN). The new learning paradigm seeks to take advantage of the generalization ability of extreme learning machines (ELM) along with the global learning capability of a self adaptive differential harmony search technique in order to optimize the fitting performance of SLFNs. SADHS is a variant of harmony search technique that uses the current to best mutation scheme of DE in the pitch adjustment operation for harmony improvisation process. SADHS has been used for optimal selection of the hidden layer parameters, the bias of neurons of the hidden-layer, and the regularization factor of robust least squares, whereas ELM has been applied to obtain the output weights analytically using a robust least squares solution. The proposed learning algorithm is applied on two SLFNs i.e. RBF and a low complexity Functional link Artificial Neural Networks (CEFLANN) for prediction of closing price and volatility of five different stock indices. The proposed learning scheme is also compared with other learning schemes like ELM, DE-OELM, DE, SADHS and two other variants of harmony search algorithm. Performance comparison of CEFLANN and RBF with different learning schemes clearly reveals that CEFLANN model trained with SADHS-OELM outperforms other learning methods and also the RBF model for both stock index and volatility prediction.

Introduction

Accurate forecasting of future behavior of the financial time series data with respect to its tremendous sudden variation and complex non-linear dimensionality is a big challenge for most of the investors and professional analysts. Indeed financial time series is highly volatile across time and is prone to fluctuation not only for economic factors but also for non economic factors like political conditions, investor׳s expectations based on actual and future economic etc. However, the benefits involved in accurate prediction have motivated researchers to develop newer and advanced tools and models. Forecasting financial time series is primarily focused on estimation of future stock price index and accurate forecasting of its volatility. The models used for financial time series forecasting fall into two categories. The first category involves models based on statistical theories, e.g. autoregressive moving average (ARMA), autoregressive integrated moving average (ARIMA), autoregressive conditional heteroscedasticity (ARCH), and generalized autoregressive conditional heteroskedasticity (GARCH) models. All these models assume the linearity of previous and current variables. Generally the financial time series data, being chaotic and noisy in nature, do not necessarily follow a fixed pattern or linearity and thus the statistical approaches do not perform very well in predicting stock market indices accurately. The second category includes models based on artificial intelligence, like ANN, Fuzzy set theory, Support Vector Machine, Rough Set theory etc. Due to the inherent capabilities to identify complex nonlinear relationship present in the time series data based on historical data and to approximate any nonlinear function to a high degree of accuracy, the application of ANN in modeling economic conditions is expanding rapidly. A survey of literature indicates that among different types of ANNs, i.e. Multi Layer Perception Network (MLP) [1], [2], [3], [4], [5], [6], [7], [8], [9], Radial Basis Function Neural Network (RBF) [10], [11] and Functional Link Artificial Neural Network (FLANN) [12], [13] are the most popular ANN tool used for predictions of financial time series.

The traditional back propagation algorithm with gradient descent method is the commonly used learning technique for ANNs. But it suffers from the issues of imprecise learning rate, local minimal and slow rate of convergence. To avoid the common drawbacks of back propagation algorithm and to increase the accuracy some scholars proposed several improved measures, including additional momentum method, self-adaptive learning rate adjustment method, Recursive Least square method and various search algorithms like GA, PSO DE, HS algorithms [10], [14], [15], [16], [17], [18], in the training step of the neural network to optimize the parameters of the network like the network weights and the number of hidden units in the hidden layer etc. To increase forecasting speed and accuracy, researchers have also tried to combine and optimize different algorithms, and build hybrid models. A hybridization of PSO and BP algorithm for training of ANN has been proposed in [19], [20]. A hybrid evolutionary PSO and adaptive recursive least-squares (RLS) learning algorithms called RPSO has been considered together for efficiently acquiring available parameters value of the RBFNs prediction systems in [11]. In [36] a novel fuzzy time series forecasting model with entropy discretization and a Fast Fourier Transform algorithm has also been proposed for stock price forecasting. The model uses an FFT to deal with historical training data for forecasting stock prices. The approach provides a more accurate forecasting result compared to traditional ARCH, GARCH and other fuzzy time series models. Another high-order fuzzy time series model based on entropy-based partitioning and adaptive expectation model for time series forecasting has been proposed in [37]. The empirical results of the model show that entropy based discretization partitioning could be more suitable than the conventional time series model. Recently a new batch learning algorithm called Extreme Learning Machine (ELM) has been proposed in [21] for training of single hidden layer feed forward neural network. Mainly the ELM algorithm randomly initializes parameters of hidden nodes and analytically determines output weights of SLFNs. The main advantage of ELM is that the hidden layer of SLFNs need not be tuned. In fact, for the randomly chosen input weights and hidden layer biases, ELM will lead to a least squares solution of a system of linear equations for the unknown output weights having the smallest norm property. ELM has shown good generalization performances for many real applications with an extremely fast learning speed [22], [23], [24], [25], [26]. However, like other similar approaches based on feed forward neural networks, some issues with the practical applications of the ELM still arise, most importantly, how to choose the optimal number of hidden nodes for a given problem which is usually done by trial and error method. The ELM tends to require more hidden neurons than conventional tuning based learning algorithms (based on error back propagation or other learning methods where the output weights are not obtained by the least squares method) in some applications, which can negatively affect SLFN performance. Again the stochastic nature of the hidden layer output matrix may also lower the learning accuracy of ELM. Hence in order to further boost the generalization ability of extreme learning machines and to cope with the local minima problem a global optimization method called self adaptive differential harmony search (SADHS) hybridized with ELM has been proposed in this paper for learning of single-hidden layer feed forward neural networks.

The hybrid learning framework named Self Adaptive Differential Harmony Search Based Optimized Extreme Learning Machine (SADHS-OELM) uses the same concept of the ELM where the output weights are obtained using a robust least squares solution but, in order to optimize the fitting performance, the selection of the weights of connections between the input layer and the hidden layer, the bias of neurons of the hidden-layer, and the regularization factor of robust least square are done using SADHS. SADHS is a variant of harmony search technique that uses the current to best mutation scheme of DE in the pitch adjustment operation for harmony improvisation process. The proposed learning algorithm has been applied on two SLFN i.e. RBF and a low complexity Functional link Artificial Neural Network (CEFLANN) for prediction of stock index and volatility of five different stock indices. The proposed learning scheme has been also compared with other learning schemes like ELM, DE-OELM, DE, SADHS and two other variants of harmony search.