Rule induction for forecasting method selection: Meta-learning the characteristics of univariate time series

Abstract

For univariate forecasting, there are various statistical models and computational algorithms available. In real-world exercises, too many choices can create difficulties in selecting the most appropriate technique, especially for users lacking sufficient knowledge of forecasting. This study focuses on rule induction for forecasting method selection by understanding the nature of historical forecasting data. A novel approach for selecting a forecasting method for univariate time series based on measurable data characteristics is presented that combines elements of data mining, meta-learning, clustering, classification and statistical measurement. We conducted a large-scale empirical study of over 300 time series using four of the most popular forecasting methods. To provide a rich portrait of the global characteristics of univariate time series, we extracted measures from a comprehensive set of features such as trend, seasonality, periodicity, serial correlation, skewness, kurtosis, nonlinearity, self-similarity, and chaos. Both supervised and unsupervised learning methods are used to learn the relationship between the characteristics of the time series and the forecasting method suitability, providing both recommendation rules, as well as visualizations in the feature space. A derived weighting schema based on the rule induction is also used to improve forecasting accuracy based on combined forecasting models.

Introduction

Time series forecasting has been a traditional research area for decades, and various statistical models and advanced computational algorithms have been developed to improve forecasting accuracy. With the continuous emergence of more methods, forecasters have been given more choices. However, more options could also create potential problems in practice especially when forecasts are based on a trial-and-error procedure with little understanding of the conditions under which certain forecasting methods perform well. Certainly the ‘no free lunch theorem’ [1] informs us that there is never likely to be a single method that fits all situations. In the forecasting context, therefore, recommendation rules on how to select a suitable forecasting method for a given type of time series have attracted attention.

From more than a decade ago, the research on forecasting methods selection attracted many attempts to find recommendations and rules [2], [3], [4], [5] mostly based on expert systems approaches. Certainly there are obvious limitations for systems based on human judgment, with the strongest concern being that expert system based rules are not dynamic and therefore require significant rework and validation prior to updating. This is not a trivial problem especially when the forecasting domain or situation changes. In this study, we aim to develop an automated rule induction system that couples forecasting methods performance with time series data characteristics. Metrics to characterize a time series are developed to provide a rich portrait of the time series including its trend, seasonality, serial correlation, nonlinearity, skewness, kurtosis, self-similarity, chaos, and periodicity. Self-organizing maps (SOMs) and decision trees (DTs) are used to induce rules explaining the relationships between these characteristics and forecasting method performance. The induced rules from such a system are envisaged to provide recommendations to forecasters on how to select forecasting methods. In the proposed system, a data-driven approach based on a meta-learning framework and machine learning algorithms are employed, reducing the dependence on expert knowledge. Such an automated system is more flexible, adaptive and efficient when situations change and rules are required to be revised.

After outlining related work in forecasting method selection, as well as relevant cross-disciplinary work in Section 2, we explain the detailed components and procedures of the proposed meta-learning based system in Section 3. Then the background knowledge on four forecasting methods which were used as candidates in our empirical study is provided in Section 4. Section 5 then follows in which each identified characteristic for univariate time series data in our study are introduced including algorithms used to extract descriptive metric for each characteristic. Three machine learning techniques used for learning the relationship between time series characteristics and forecasting method performance are discussed in Section 6. Our empirical study and experimental results including induced rules are demonstrated in Section 7. Future research directions are discussed and conclusions drawn in Section 8.