On the Directional Accuracy of United States Housing Starts Forecasts: Evidence from Survey Data

I refer to forecasters as semi-professional if they do not consider forecasting a major part of their main job. See “Data” for a description of the surveys’ panelists.

Technically, the presence of noise is also a type of signal that the operator observes. For ease of understanding, I will subsequently use the term nonsignal as a synonym for noise. See also “Directional Forecast Accuracy using ROC Analysis” for the definition used in this paper.

An exceptionally high current year forecast from 1986:Q2 stands out. It remains unclear whether the specific individual forecast was actually on that level or if it is due to a type error in the data sheet.

Test results for differences in the AUC statistics of the three forecast types based on Eq. 9 are not presented here. The AUC statistics of individual forecasts are smaller than those for mean and median forecasts at all forecast horizons. However, in all cases (except for the current-quarter-ahead forecasts, where the difference is significant at the 5% level) the differences are not statistically significant. The same result holds for the differences in mean and median forecasts. Hence, while the AUC statistics for individual forecasts have the smallest values at all forecast horizons, the differences are not sufficiently large enough to prefer a particular type of forecast when working with the data. All results are available from the author upon request.

Test results for differences in the AUC statistics of the three forecast types are not presented because none of the results are statistically significant. All results are available from the author upon request.

Rolling window estimates are robust to perturbations of the length of the estimation window. Estimates for fifteen and twenty year rolling windows are not presented but available from the author upon request.

Results of the forecaster subgroups AUC comparison are not presented here because 34 out of 36 differences are largely insignificant. Hence, when using forecasts from the LS there is relatively little discrepancy in using aggregate data or subgroups because the directional forecast accuracy is relatively homogeneous across the subgroups of forecasters. The results are available from the author upon request.

In general, the results of the Efficiency index look similar to those of the Youden Index with a relatively higher volatility. They are not presented here but available from the author upon request.