1 Introduction
2 Related literature
-
Engelberg et al. (2022): “Financial researchers have examined the predictability of stock returns for over a century (e.g., Gibson 1906) and a large literature has documented evidence of predictability in the cross-section of stock returns. A separate literature has examined the predictability of the equity risk-premium using time-series predictive variables. To date, these two literatures have evolved relatively independently.”
-
Dong et al. (2022): “The first examines whether firm characteristics can predict the cross-sectional dispersion in stock returns. These studies identify numerous equity market anomalies (e.g., Fama and French 2015; Harvey et al. 2016; McLean and Pontiff 2016; Hou et al. 2018). The second line of research investigates the time-series predictability of the aggregate market excess return based on a variety of economic and financial variables, such as valuation ratios, interest rates, and inflation (Nelson 1976; Campbell 1987; Fama and French 1988, 1989; Pástor and Stambaugh 2009). Studies in this vein attempt to shed light on the variables that affect the equity risk premium.”
-
Gu et al. (2020): “The first strand models differences in expected returns across stocks as a function of stock-level characteristics, and is exemplified by Fama and French (2008) and Lewellen (2015). The typical approach in this literature runs cross-sectional regressions of future stock returns on a few lagged stock characteristics. The second strand forecasts the time-series of returns and is surveyed by Welch and Goyal (2008), Koijen and van Nieuwerburgh (2011), and Rapach and Zhou (2013).”
-
The time-series predictability approach considers the following time-series multiple regression model for an individual assetfor \(t = 1, . . . , T\), where r is the return on a broad stock market index in excess of the risk-free return, T is the number of time-series observations, \(x_{j,t-1}\) are the lagged predictor variables, and \(\epsilon _{t}\) is a zero-mean disturbance term.$$\begin{aligned} r_{t}=\alpha +\sum _{j=1}^{J}\beta _{j}x_{j,t-1}+\epsilon _{t}, \end{aligned}$$(1)
-
The cross-sectional predictability approach considers the following cross-sectional multiple regression model for a certain monthfor \(i = 1, . . . , N\), where \(z_{i,j}\) is the jth lagged characteristic of firm i and N is the number of available firms.7$$\begin{aligned} r_{i}=\alpha + \sum _{j=1}^{J}\beta _{j}z_{i,j}+\epsilon _{i}, \end{aligned}$$(2)
3 Methodology
3.1 Simple and penalized linear regression
3.2 Support vector machine
3.3 Tree-based methods: random forest and gradient boosted regression trees
3.4 Neural networks
3.5 Estimation, tuning and testing
4 Data
Panel: A | Panel: B | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Avg | Std | N | 1 | 2 | 3 | 4 | 5 | 5-1 | pVal (5-1) | |
ret | 0.0088 | 0.0886 | 1755 | |||||||
mom | 0.1324 | 0.3685 | 1755 | 0.27 | 0.69 | 0.90 | 1.18 | 1.63 | 1.37 | 0.00 |
beta | 0.5609 | 0.4154 | 1755 | 1.02 | 1.08 | 1.06 | 0.87 | 0.64 | − 0.37 | 18.24 |
size | 5.8364 | 1.9083 | 1755 | 0.91 | 0.93 | 0.96 | 0.96 | 0.92 | 0.01 | 95.61 |
be2me | 0.7408 | 0.6483 | 1755 | 0.68 | 0.85 | 0.88 | 1.04 | 1.24 | 0.56 | 0.16 |
op | 0.0002 | 0.0003 | 1755 | 0.50 | 0.94 | 1.05 | 1.10 | 1.09 | 0.59 | 0.00 |
inv | 18.7620 | 47.0595 | 1755 | 1.02 | 1.08 | 1.07 | 0.96 | 0.56 | − 0.46 | 0.07 |
5 Results
5.1 Predictive accuracy
Size > q | OLS | Lasso | Ridge | Enet | SVR | RF | GBRT | NN |
---|---|---|---|---|---|---|---|---|
0.00 | 0.43 | 0.40 | 0.42 | 0.37 | 0.14 | 0.58 | 0.60 | 0.67 |
0.10 | 0.36 | 0.33 | 0.34 | 0.28 | 0.15 | 0.36 | 0.44 | 0.63 |
0.20 | 0.36 | 0.32 | 0.34 | 0.27 | 0.19 | 0.36 | 0.45 | 0.66 |
0.30 | 0.36 | 0.30 | 0.34 | 0.27 | 0.20 | 0.35 | 0.46 | 0.68 |
0.40 | 0.35 | 0.29 | 0.33 | 0.26 | 0.20 | 0.34 | 0.47 | 0.70 |
0.50 | 0.31 | 0.24 | 0.30 | 0.23 | 0.17 | 0.27 | 0.46 | 0.70 |
0.60 | 0.26 | 0.18 | 0.25 | 0.19 | 0.12 | 0.20 | 0.45 | 0.67 |
0.70 | 0.17 | 0.08 | 0.17 | 0.11 | -0.01 | 0.09 | 0.43 | 0.63 |
0.80 | − 0.03 | − 0.10 | − 0.01 | − 0.06 | − 0.26 | − 0.17 | 0.32 | 0.50 |
0.90 | − 0.32 | − 0.39 | − 0.29 | − 0.35 | − 0.58 | − 0.45 | 0.14 | 0.24 |
OLS | Lasso | Ridge | Enet | SVR | RF | GBRT | NN | |
---|---|---|---|---|---|---|---|---|
Panel A: Full sample | ||||||||
OLS | – | |||||||
Lasso | 0.84 | – | ||||||
Ridge | 0.58 | 0.19 | – | |||||
Enet | 0.95 | 0.92 | 1.00 | – | ||||
SVR | 0.98 | 0.96 | 0.96 | 0.93 | – | |||
RF | 0.00 | 0.00 | 0.01 | 0.00 | 0.00 | – | ||
GBRT | 0.02 | 0.02 | 0.02 | 0.01 | 0.01 | 0.41 | – | |
NN | 0.02 | 0.01 | 0.01 | 0.00 | 0.01 | 0.24 | 0.17 | – |
Panel B: Size > q90 | ||||||||
OLS | – | |||||||
Lasso | 0.65 | – | ||||||
Ridge | 0.27 | 0.20 | – | |||||
Enet | 0.49 | 0.27 | 0.66 | – | ||||
SVR | 0.85 | 0.77 | 0.95 | 0.90 | – | |||
RF | 0.73 | 0.60 | 0.90 | 0.78 | 0.22 | – | ||
GBRT | 0.04 | 0.04 | 0.03 | 0.03 | 0.00 | 0.01 | – | |
NN | 0.06 | 0.05 | 0.05 | 0.04 | 0.01 | 0.02 | 0.25 | – |
5.2 Economic gains
OLS | Lasso | Ridge | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Avg | Std | Shp | Pred | Avg | Std | Shp | Pred | Avg | Std | Shp | Pred | |
L | − 0.29 | 6.54 | − 0.04 | − 0.22 | − 0.21 | 6.51 | − 0.03 | 0.06 | − 0.27 | 6.64 | − 0.04 | − 0.05 |
2 | 0.16 | 5.25 | 0.03 | 0.17 | 0.01 | 5.34 | 0.00 | 0.32 | 0.13 | 5.29 | 0.02 | 0.27 |
3 | 0.34 | 4.40 | 0.08 | 0.42 | 0.24 | 4.50 | 0.05 | 0.52 | 0.35 | 4.39 | 0.08 | 0.48 |
4 | 0.53 | 4.10 | 0.13 | 0.62 | 0.45 | 4.17 | 0.11 | 0.69 | 0.53 | 4.09 | 0.13 | 0.65 |
5 | 0.65 | 3.83 | 0.17 | 0.79 | 0.62 | 3.87 | 0.16 | 0.84 | 0.66 | 3.80 | 0.17 | 0.80 |
6 | 0.88 | 3.61 | 0.24 | 0.95 | 0.84 | 3.62 | 0.23 | 0.97 | 0.89 | 3.60 | 0.25 | 0.94 |
7 | 1.03 | 3.45 | 0.30 | 1.12 | 0.90 | 3.55 | 0.25 | 1.11 | 1.02 | 3.46 | 0.30 | 1.08 |
8 | 1.21 | 3.35 | 0.36 | 1.30 | 1.10 | 3.47 | 0.32 | 1.26 | 1.20 | 3.32 | 0.36 | 1.23 |
9 | 1.25 | 3.40 | 0.37 | 1.52 | 1.12 | 3.57 | 0.31 | 1.44 | 1.26 | 3.41 | 0.37 | 1.42 |
H | 1.45 | 3.58 | 0.40 | 1.85 | 1.38 | 3.83 | 0.36 | 1.70 | 1.46 | 3.56 | 0.41 | 1.70 |
H-L | 1.74 | 5.09 | 0.34 | 2.07 | 1.79 | 5.36 | 0.33 | 1.70 | 1.73 | 5.21 | 0.33 | 1.75 |
pVal(H-L) | 0.04 | 0.08 | 0.04 | |||||||||
pVal(MR) | 0.00 | 51.10 | 0.20 | 0.40 | 57.70 | 0.50 | 0.00 | 39.30 | 0.10 |
Enet | SVR | RF | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Avg | Std | Shp | Pred | Avg | Std | Shp | Pred | Avg | Std | Shp | Pred | |
L | − 0.10 | 6.54 | − 0.01 | 0.14 | − 0.35 | 6.68 | − 0.05 | − 0.57 | − 0.40 | 6.83 | − 0.06 | − 0.22 |
2 | 0.04 | 5.53 | 0.01 | 0.31 | 0.18 | 5.12 | 0.03 | − 0.04 | 0.09 | 5.06 | 0.02 | 0.29 |
3 | 0.28 | 4.68 | 0.06 | 0.51 | 0.45 | 4.30 | 0.10 | 0.23 | 0.39 | 4.40 | 0.09 | 0.51 |
4 | 0.47 | 4.15 | 0.11 | 0.68 | 0.61 | 3.87 | 0.16 | 0.42 | 0.53 | 4.03 | 0.13 | 0.67 |
5 | 0.65 | 3.99 | 0.16 | 0.83 | 0.75 | 3.78 | 0.20 | 0.57 | 0.67 | 3.81 | 0.18 | 0.83 |
6 | 0.84 | 3.63 | 0.23 | 0.97 | 0.90 | 3.65 | 0.25 | 0.70 | 0.87 | 3.46 | 0.25 | 0.97 |
7 | 0.97 | 3.58 | 0.27 | 1.11 | 0.97 | 3.61 | 0.27 | 0.83 | 0.89 | 3.30 | 0.27 | 1.08 |
8 | 1.10 | 3.46 | 0.32 | 1.26 | 1.10 | 3.53 | 0.31 | 0.97 | 1.02 | 3.34 | 0.31 | 1.18 |
9 | 1.13 | 3.51 | 0.32 | 1.45 | 1.20 | 3.45 | 0.35 | 1.14 | 1.20 | 3.57 | 0.34 | 1.32 |
H | 1.41 | 3.74 | 0.38 | 1.71 | 1.42 | 3.38 | 0.42 | 1.40 | 1.97 | 4.00 | 0.49 | 1.76 |
H-L | 1.78 | 5.60 | 0.32 | 1.72 | 1.77 | 4.96 | 0.36 | 1.97 | 2.38 | 4.90 | 0.48 | 1.98 |
pVal(H-L) | 0.24 | 0.03 | 0.00 | |||||||||
pVal(MR) | 0.10 | 51.50 | 0.30 | 0.00 | 0.40 | 0.00 | 0.10 | 71.60 | 0.00 |
GBRT | NN | |||||||
---|---|---|---|---|---|---|---|---|
Avg | Std | Shp | Pred | Avg | Std | Shp | Pred | |
L | − 0.36 | 6.35 | − 0.06 | − 0.22 | − 0.27 | 6.28 | − 0.04 | 0.03 |
2 | 0.10 | 4.91 | 0.02 | 0.19 | 0.11 | 5.13 | 0.02 | 0.36 |
3 | 0.23 | 4.79 | 0.05 | 0.36 | 0.36 | 4.47 | 0.08 | 0.55 |
4 | 0.46 | 3.99 | 0.12 | 0.51 | 0.46 | 4.08 | 0.11 | 0.70 |
5 | 0.70 | 3.67 | 0.19 | 0.67 | 0.66 | 3.91 | 0.17 | 0.83 |
6 | 0.70 | 3.52 | 0.20 | 0.78 | 0.86 | 3.58 | 0.24 | 0.96 |
7 | 0.84 | 3.38 | 0.25 | 0.89 | 0.98 | 3.50 | 0.28 | 1.08 |
8 | 1.08 | 3.35 | 0.32 | 1.01 | 1.15 | 3.44 | 0.34 | 1.22 |
9 | 1.24 | 3.65 | 0.34 | 1.15 | 1.33 | 3.28 | 0.41 | 1.39 |
H | 1.97 | 3.67 | 0.54 | 1.44 | 1.56 | 3.39 | 0.46 | 1.64 |
H-L | 2.33 | 4.16 | 0.56 | 1.66 | 1.83 | 4.38 | 0.42 | 1.61 |
pVal(H-L) | 0.00 | 0.01 | ||||||
pVal(MR) | 0.00 | 70.20 | 0.00 | 0.00 | 38.50 | 0.00 |
Size > q | OLS | Lasso | Ridge | Enet | SVR | RF | GBRT | NN |
---|---|---|---|---|---|---|---|---|
0.00 | 1.74 | 1.79 | 1.73 | 1.78 | 1.77 | 2.38 | 2.33 | 1.83 |
0.10 | 1.79 | 1.89 | 1.79 | 1.84 | 1.95 | 1.79 | 1.83 | 1.87 |
0.20 | 1.84 | 1.92 | 1.84 | 1.89 | 2.00 | 1.85 | 1.84 | 1.93 |
0.30 | 1.76 | 1.81 | 1.77 | 1.82 | 1.93 | 1.76 | 1.79 | 1.87 |
0.40 | 1.67 | 1.70 | 1.66 | 1.73 | 1.80 | 1.66 | 1.70 | 1.78 |
0.50 | 1.51 | 1.53 | 1.48 | 1.51 | 1.64 | 1.46 | 1.53 | 1.65 |
0.60 | 1.32 | 1.32 | 1.31 | 1.32 | 1.44 | 1.33 | 1.40 | 1.51 |
0.70 | 1.13 | 1.10 | 1.15 | 1.15 | 1.27 | 1.10 | 1.19 | 1.24 |
0.80 | 0.77 | 0.74 | 0.76 | 0.75 | 0.93 | 0.67 | 0.91 | 0.91 |
0.90 | 0.53 | 0.48 | 0.55 | 0.58 | 0.73 | 0.46 | 0.59 | 0.70 |
6 Additional results
6.1 Transaction costs
Size > q | OLS | Lasso | Ridge | Enet | SVR | RF | GBRT | NN |
---|---|---|---|---|---|---|---|---|
0.00 | 125 | 124 | 124 | 125 | 133 | 154 | 144 | 132 |
0.10 | 125 | 125 | 126 | 126 | 131 | 152 | 143 | 131 |
0.20 | 124 | 126 | 123 | 126 | 131 | 152 | 141 | 133 |
0.30 | 125 | 126 | 124 | 125 | 131 | 153 | 140 | 131 |
0.40 | 124 | 126 | 124 | 124 | 129 | 154 | 140 | 133 |
0.50 | 125 | 128 | 125 | 126 | 129 | 151 | 140 | 133 |
0.60 | 127 | 129 | 128 | 129 | 127 | 150 | 141 | 131 |
0.70 | 128 | 129 | 128 | 129 | 127 | 147 | 140 | 127 |
0.80 | 127 | 129 | 128 | 129 | 127 | 144 | 138 | 127 |
0.90 | 126 | 129 | 127 | 130 | 129 | 142 | 140 | 127 |
6.2 Performance within subperiods
Size > q | OLS | Lasso | Ridge | Enet | SVR | RF | GBRT | NN |
---|---|---|---|---|---|---|---|---|
Panel A: Bull | ||||||||
0.00 | 1.22 | 1.18 | 1.22 | 1.16 | 0.74 | 1.33 | 1.19 | 1.29 |
0.10 | 1.29 | 1.24 | 1.29 | 1.22 | 0.83 | 1.23 | 1.12 | 1.37 |
0.20 | 1.35 | 1.29 | 1.35 | 1.27 | 0.91 | 1.28 | 1.19 | 1.44 |
0.30 | 1.38 | 1.31 | 1.38 | 1.29 | 0.97 | 1.30 | 1.22 | 1.46 |
0.40 | 1.42 | 1.35 | 1.42 | 1.35 | 1.05 | 1.37 | 1.29 | 1.52 |
0.50 | 1.45 | 1.37 | 1.46 | 1.38 | 1.13 | 1.39 | 1.32 | 1.55 |
0.60 | 1.44 | 1.37 | 1.45 | 1.38 | 1.21 | 1.40 | 1.34 | 1.50 |
0.70 | 1.40 | 1.32 | 1.42 | 1.35 | 1.22 | 1.36 | 1.36 | 1.48 |
0.80 | 1.22 | 1.18 | 1.26 | 1.21 | 1.15 | 1.21 | 1.35 | 1.36 |
0.90 | 0.99 | 0.95 | 1.06 | 1.00 | 1.10 | 1.18 | 1.36 | 1.23 |
Panel B: Bear | ||||||||
0.00 | − 0.40 | − 0.41 | − 0.42 | − 0.46 | − 0.48 | − 0.20 | − 0.01 | 0.03 |
0.10 | − 0.54 | − 0.55 | − 0.57 | − 0.61 | − 0.51 | − 0.47 | − 0.21 | − 0.08 |
0.20 | − 0.57 | − 0.60 | − 0.61 | − 0.66 | − 0.49 | − 0.51 | − 0.24 | − 0.08 |
0.30 | − 0.58 | − 0.63 | − 0.62 | − 0.68 | − 0.52 | − 0.53 | − 0.24 | − 0.04 |
0.40 | − 0.60 | − 0.65 | − 0.64 | − 0.70 | − 0.55 | − 0.58 | − 0.25 | − 0.01 |
0.50 | − 0.67 | − 0.73 | − 0.70 | − 0.75 | − 0.66 | − 0.69 | − 0.28 | − 0.02 |
0.60 | − 0.73 | − 0.81 | − 0.75 | − 0.81 | − 0.80 | − 0.81 | − 0.29 | − 0.02 |
0.70 | − 0.87 | − 0.97 | − 0.89 | − 0.94 | − 1.03 | − 0.99 | − 0.35 | − 0.10 |
0.80 | − 1.03 | − 1.13 | − 1.02 | − 1.07 | − 1.38 | − 1.27 | − 0.50 | − 0.18 |
0.90 | − 1.27 | − 1.37 | − 1.27 | − 1.33 | − 1.80 | − 1.63 | − 0.75 | − 0.48 |
Size > q | OLS | Lasso | Ridge | Enet | SVM | RF | GBRT | NN |
---|---|---|---|---|---|---|---|---|
Panel A: Bull | ||||||||
0.00 | 1.77 | 1.73 | 1.75 | 1.68 | 1.41 | 2.33 | 2.21 | 1.76 |
0.10 | 1.84 | 1.80 | 1.82 | 1.73 | 1.60 | 1.85 | 1.83 | 1.84 |
0.20 | 1.85 | 1.80 | 1.83 | 1.74 | 1.66 | 1.91 | 1.84 | 1.90 |
0.30 | 1.74 | 1.72 | 1.78 | 1.73 | 1.67 | 1.76 | 1.77 | 1.82 |
0.40 | 1.66 | 1.60 | 1.65 | 1.65 | 1.53 | 1.75 | 1.74 | 1.75 |
0.50 | 1.56 | 1.53 | 1.54 | 1.52 | 1.47 | 1.59 | 1.59 | 1.70 |
0.60 | 1.31 | 1.33 | 1.36 | 1.37 | 1.33 | 1.46 | 1.42 | 1.45 |
0.70 | 1.12 | 1.04 | 1.18 | 1.12 | 1.16 | 1.09 | 1.14 | 1.17 |
0.80 | 0.77 | 0.72 | 0.78 | 0.78 | 0.90 | 0.74 | 0.93 | 0.91 |
0.90 | 0.66 | 0.57 | 0.66 | 0.72 | 0.69 | 0.47 | 0.66 | 0.68 |
Panel B: Bear | ||||||||
0.00 | 1.69 | 1.86 | 1.71 | 1.89 | 2.23 | 2.43 | 2.48 | 1.91 |
0.10 | 1.74 | 2.01 | 1.74 | 1.96 | 2.40 | 1.71 | 1.83 | 1.91 |
0.20 | 1.81 | 2.07 | 1.86 | 2.05 | 2.42 | 1.76 | 1.83 | 1.97 |
0.30 | 1.78 | 1.93 | 1.76 | 1.93 | 2.26 | 1.75 | 1.81 | 1.93 |
0.40 | 1.68 | 1.84 | 1.67 | 1.82 | 2.14 | 1.55 | 1.66 | 1.83 |
0.50 | 1.44 | 1.54 | 1.40 | 1.49 | 1.86 | 1.29 | 1.45 | 1.58 |
0.60 | 1.32 | 1.30 | 1.24 | 1.27 | 1.59 | 1.15 | 1.37 | 1.59 |
0.70 | 1.14 | 1.17 | 1.11 | 1.19 | 1.41 | 1.11 | 1.26 | 1.32 |
0.80 | 0.77 | 0.76 | 0.72 | 0.71 | 0.97 | 0.58 | 0.88 | 0.92 |
0.90 | 0.37 | 0.37 | 0.40 | 0.43 | 0.77 | 0.44 | 0.51 | 0.73 |
6.3 Forecast combinations
Size > q | Including linear methods | Excluding linear methods | ||||
---|---|---|---|---|---|---|
EW | IRANK | IMSE | EW | IRANK | IMSE | |
0.00 | 0.55 | 0.60 | 0.55 | 0.64 | 0.67 | 0.64 |
0.10 | 0.45 | 0.47 | 0.45 | 0.51 | 0.51 | 0.51 |
0.20 | 0.46 | 0.48 | 0.46 | 0.53 | 0.52 | 0.53 |
0.30 | 0.46 | 0.47 | 0.46 | 0.54 | 0.52 | 0.54 |
0.40 | 0.45 | 0.47 | 0.45 | 0.54 | 0.53 | 0.54 |
0.50 | 0.42 | 0.43 | 0.42 | 0.52 | 0.49 | 0.52 |
0.60 | 0.37 | 0.37 | 0.37 | 0.47 | 0.44 | 0.47 |
0.70 | 0.29 | 0.28 | 0.29 | 0.39 | 0.36 | 0.39 |
0.80 | 0.11 | 0.08 | 0.11 | 0.21 | 0.16 | 0.21 |
0.90 | -0.16 | -0.19 | -0.16 | -0.05 | -0.10 | -0.05 |
Size > q | Including linear methods | Excluding linear methods | ||||
---|---|---|---|---|---|---|
EW | IRANK | IMSE | EW | IRANK | IMSE | |
0.00 | 2.27 | 2.32 | 2.27 | 2.46 | 2.47 | 2.46 |
0.10 | 1.97 | 1.94 | 1.98 | 2.05 | 2.02 | 2.05 |
0.20 | 2.02 | 1.97 | 2.02 | 2.09 | 2.05 | 2.09 |
0.30 | 1.94 | 1.85 | 1.94 | 1.99 | 1.93 | 2.00 |
0.40 | 1.87 | 1.78 | 1.87 | 1.90 | 1.83 | 1.90 |
0.50 | 1.67 | 1.63 | 1.67 | 1.74 | 1.68 | 1.74 |
0.60 | 1.48 | 1.41 | 1.48 | 1.51 | 1.46 | 1.51 |
0.70 | 1.24 | 1.21 | 1.24 | 1.33 | 1.23 | 1.34 |
0.80 | 0.83 | 0.78 | 0.83 | 0.89 | 0.81 | 0.89 |
0.90 | 0.62 | 0.61 | 0.62 | 0.67 | 0.66 | 0.66 |