1 Introduction
2 Methods
2.1 Data
2.2 Time-series features
2.3 Performance-based selection
scikit-learn
and code to reproduce all of our analyses is accessible on GitHub (https://github.com/chlubba/op_importance).
2.4 Quantifying feature performance
2.5 Statistical prefiltering
2.6 Selecting a canonical set of features
2.7 Overall classification performance
2.8 Execution times and scaling
2.9 Selecting the two most informative features from a small subset
3 Results
3.1 Performance diversity across classification tasks
3.2 Features with performance consistent with chance
3.3 Top-performing features
3.4 A canonical feature set, catch22
hctsa feature name | Description |
---|---|
Distribution | |
DN_HistogramMode_5 | Mode of z-scored distribution (5-bin histogram) |
DN_HistogramMode_10 | Mode of z-scored distribution (10-bin histogram) |
Simple temporal statistics | |
SB_BinaryStats_mean_longstretch1 | Longest period of consecutive values above the mean |
DN_OutlierInclude_p_001_mdrmd | Time intervals between successive extreme events above the mean |
DN_OutlierInclude_n_001_mdrmd | Time intervals between successive extreme events below the mean |
Linear autocorrelation | |
CO_f1ecac | First 1 / e crossing of autocorrelation function |
CO_FirstMin_ac | First minimum of autocorrelation function |
SP_Summaries_welch_rect_area_5_1 | Total power in lowest fifth of frequencies in the Fourier power spectrum |
SP_Summaries_welch_rect_centroid | Centroid of the Fourier power spectrum |
FC_LocalSimple_mean3_stderr | Mean error from a rolling 3-sample mean forecasting |
Nonlinear autocorrelation | |
CO_trev_1_num | Time-reversibility statistic, \(\langle (x_{t+1}-x_t)^3\rangle _t\) |
CO_HistogramAMI_even_2_5 | Automutual information, \(m=2, \tau =5\) |
IN_AutoMutualInfoStats_40_gaussian_fmmi | First minimum of the automutual information function |
Successive differences | |
MD_hrv_classic_pnn40 | Proportion of successive differences exceeding \(0.04\sigma \) (Mietus 2002) |
SB_BinaryStats_diff_longstretch0 | Longest period of successive incremental decreases |
SB_MotifThree_quantile_hh | Shannon entropy of two successive letters in equiprobable 3-letter symbolization |
FC_LocalSimple_mean1_tauresrat | Change in correlation length after iterative differencing |
CO_Embed2_Dist_tau_d_expfit_meandiff | Exponential fit to successive distances in 2-d embedding space |
Fluctuation Analysis | |
SC_FluctAnal_2_dfa_50_1_2_logi_prop_r1 | Proportion of slower timescale fluctuations that scale with DFA (50% sampling) |
SC_FluctAnal_2_rsrangefit_50_1_logi_prop_r1 | Proportion of slower timescale fluctuations that scale with linearly rescaled range fits |
Others | |
SB_TransitionMatrix_3ac_sumdiagcov | Trace of covariance of transition matrix between symbols in 3-letter alphabet |
PD_PeriodicityWang_th0_01 | Periodicity measure of (Wang et al. 2007) |
tsfeatures
package (Hyndman et al. 2019) of which certain features were used for forecasting (Bandara et al. 2017), anomaly detection (Hyndman et al. 2016), and clustering (Williams 2014). While not being explicitly optimized for classification and clustering, its widespread adoption demonstrates its versatility in characterizing time series and makes it an interesting candidate to compare with catch22. We classified all datasets based on the 16 default features of tsfeatures
(version 1.0.0) listed in “Appendix” Sect. 5.4. Reassuringly, the class-balanced accuracies of both feature sets were very similar across the generic UEA/UCR datasets, with a Pearson correlation coefficient \(r = 0.93\) (Fig. 5). The mean accuracy across tasks and folds, \(a^\text {tot}\), was slightly higher for catch22 (71.7%) than tsfeatures
(69.4%). Our pipeline is general, and can select informative subsets of features for any collection of problems; e.g., for a more complex set of time-series classification tasks, our pipeline may yield estimators of more distinctive and complex dynamics.CO_FirstMin_ac
’ which finds the first minimum in the autocorrelation function. In some datasets, high performance can be attained using just a single feature, e.g., in ‘ChlorineConcentration’ (‘SB_motifThree_quantile.hh
’, 52.3% versus 67.5% class-balanced mean accuracy over folds a for catch22 vs. all features) and ‘TwoPatterns’ (‘CO_trev_1.num
’, 73.4% vs. 88.1%).
3.5 Computation time and complexity
DN_HistogramMode_10
(\(<0.1\) ms for our 10,000-sample reference series) to PD_PeriodicityWang_th0_01
(79 ms), with the latter representing approximately one third of the total computation time for catch22. A further acceleration by a factor of 3 could be achieved through parallelization, limited by the slowest feature PD_PeriodicityWang_th0_01
which takes up one third of the overall computation time.3.6 Performance comparison
3.7 Characteristics of datasets that favor feature- or shape-based representations
3.8 Informative features provide understanding
SB_BinaryStats_diff_longstretch0
, clearly distinguishes the two classes. This simple measure quantifies the length of the longest continued descending increments in the data which enables a perfect separation of the two classes because time series of the ‘triangle’ class vary on a slower timescale than ‘noise’ time series.FC_LocalSimple_mean3_stderr
captures variability in residuals for local 3-sample mean predictions of the next datapoint applied to through time, while the second feature, SP_Summaries_welch_rect_area_5_1
, captures the proportion of low-frequency power in the time series. We discover, e.g., that time series of ‘F-14 wings open’ are less predictable from a 3-sample running mean than other planes, and that time series of ‘Harrier’ planes exhibit a greater proportion of low-frequency power than other types of planes. Thus, in cases when both shape-based and feature-based methods exhibit comparable performance (unbalanced accuracies \(a^\text {ub}\) on given split: 89.5% by catch22 vs. 99.1% mean over other classifiers), the ability to understand class differences can be a major advantage of the feature-based approach.