Corporate bankruptcy prediction: a high dimensional analysis

A weak classifier is one that predicts a little better than a random guess.

Gradient boosting has conceptual similarities with forward stepwise regression. This method starts with an intercept term and sequentially adds variables based on statistical criteria such as model-fit improvement. However, stepwise has many limitations not shared by gradient boosting. For instance, parameter estimates, R-squares and t-values can be severely over-stated and lack interpretability (see Harrell, 2001).

In contrast to conventional models, gradient boosting estimation is less affected by outliers, variable scaling/transformations, inclusion of irrelevant inputs, missing observations, database errors, and even data snooping—this type of researcher intervention can introduce significant bias to the modeling exercise (Freedman 2010).

The averages discussed above ignore lower impacting variables with RVIs less than 10.

Hillegeist et al. (2004) argue that asset volatility is important because it indicates the likelihood that the value of the firm’s assets will decline to such a degree that the firm will be unable to repay its debts. In addition to providing direct measures of volatility, Beaver et al. (2005) argue that market price variables are appealing because they reflect a rich and comprehensive mix of information, which includes financial statement data as a subset. Furthermore, market prices can also be measured with a finer partition of time. Whereas most bankruptcy studies use annual data, market prices can exploit the availability of daily prices (Frino et al. 2007).

According to Beaver et al. (2005), this slight loss in predictive power appears to be due to increased discretion in financial reporting or the increase in intangible assets not being offset by improvements due to additional FASB standards.

However, the international evidence is somewhat mixed on this issue (e.g., Frino et al. 2014).

Insider ownership is also likely to play a similar role. If the directors and CEO are also stockholders, their interests are expected to be more closely aligned with the longer-term financial performance of the firm. Insiders with higher levels of stock ownership will have stronger incentives to limit the organization’s exposure to bankruptcy risks, which inevitably leads to significant destruction in stockholder value.

Macroeconomic variables are extracted from a commercial source TradingEconomics​.​com. Real GDP and real GDP growth are key measures of overall economic health and prosperity. A strong and growing economy is expected to lead to lower default risk and lower overall probability of corporate bankruptcy. Higher interest rates are often associated with general tightness in the economy and increased likelihood of liquidity and debt-servicing pressures on firms. Inflation is a widely cited economic indicator, and the general perception is that high inflation is unhealthy for the economy, although its economic impacts can be ambiguous (Figlewski et al. 2012; Jones et al. 2015). There is also a common perception that a high ratio of public debt to GDP and high unemployment is a sign of broad economic weakness and vulnerability. Hence I expect these indicators to be positively associated with corporate failure. The NBER recession indicator, the Michigan sentiment index, and the leading index provide more direct measures of current economic sentiment and conditions. The Moody’s AAA and BBB seasoned bond yields are broad measures of credit risk. For instance, rising bond yields indicate more risk in the economy including default risk.

This is based on their review of more than 150 empirical studies in the field.

Boosting is a general method for improving the performance of learning algorithms (the idea of combining and weighting many weak classifiers into a powerful ‘voting’ committee of classifiers). While most applications of boosting use decision trees as the base classifier, some studies have examined how well the boosting technique works when neural networks or logit are used as the base classifier (examples of such studies are included in Appendix Table 12).

This study offers a number of improvements over this literature which is reflected in the generally stronger empirical results. For instance, this study uses a much larger sample size; a much wider range of predictive inputs (and related interaction effects); and applies a consistent definition of corporate failure (Chapter 11 filings). I also use a prize winning commercial gradient boosting package known as TreeNet®, which is well known for its predictive power. I discuss these issues in more detail in Section 6.

An RVI above zero indicates that the variable adds something to the predictive success of the overall model.

The antecedents of modern boosting models come from the classification and regression trees (CART™) technique. However, despite the early popularity of CART™ (particularly in health diagnostics), the technique became associated with a number of limitations, most notably high variance. (It does not generalize well.) More sophisticated techniques, starting with bagging (Breiman 1996), began to develop, which significantly improved on the performance of CART™. Bagging can dramatically reduce the variance of unstable procedures (like trees), leading to improved prediction outcomes. While bagging is a major improvement on CART™, more sophisticated boosting methodologies, such as random forests, adaptive boosting (AdaBoost), and gradient boosting, began to develop. Random forests are essentially a refined form of bagging. The technique improves on bagging by “de-correlating” the trees, which maximizes the reduction in variance. (See Hastie et al. 2009 for details.) There is now an extensive literature devoted to the gradient boosting framework (including related approaches such as AdaBoost), which highlight the many advantages of this approach and, more particularly, the superior forecasting accuracy of the model. Recent applications include biological sciences (such as DNA research), text and speech recognition and processing, satellite imaging analysis (such as oil spill detection), cyber security, geological mapping, and credit risk. Specific applications of boosting to bankruptcy research are provided in Appendix Table 12.

Both AdaBoost and gradient boosting are conceptually similar techniques. Both approaches boost the performance of a base classifier by iteratively focusing attention on observations that are difficult to predict. AdaBoost achieves this by increasing the weight on observations that were incorrectly classified in the previous round. With gradient boosting, difficult observations are identified by large residuals computed in the previous iterations. The idea behind gradient boosting is to build the new base classifiers to be maximally correlated with the negative gradient of the loss function, across the whole tree ensemble (Friedman 2001).

The squared relative importance of variable X _ℓ is the sum of such squared improvements over all internal nodes for which it was chosen as the splitting variable (Hastie et al. 2009).

As shown by Hastie et al. (2009, p.368), it is straightforward to adapt these expressions to classification problems.

At the end of the process, the entire family of dependence curves is averaged and centered to produce the final dependence plot on X ₁ . (More details are provided in “Introduction to TreeNet” Salford Systems, San Diego, 2015.)

If there are two perfectly correlated predictors in the dataset, the RVI would be 100% for the first one found by gradient boosting and zero for the other (i.e., the second variable is redundant). If two variables are strongly but not perfectly correlated, the gradient boosting algorithm simply partitions the correlated variables, looking for the greatest space between them that can enhance predictive power. A predictive variable will gain a higher RVI, while the other high correlated predictor gets a low RVI, because of redundancy in signal. (It may still have some signal because the correlation is not perfect.)

This approach does involve look-ahead bias. The impact of look-ahead bias on empirical results is examined further in Section 6. Studies by Shumway (2001), Hillegeist et al., (2004) and Beaver et al., (2005) use hazard/duration models to predict corporate bankruptcy. For this type of modeling, it is clearly more appropriate to code a firm as bankrupt only in the year of bankruptcy as hazard models are designed to predict time-to-event using a survival function.

A random guess describes a horizontal curve through the unit interval and has an AUC of exactly 0.5. As a minimum, classifiers are expected to perform >.5 (ie., better than random guessing), whereas an AUC score of 1 represents perfect classification accuracy (zero Type I and Type II errors).

Note that the RVIs reported in Table 3 incorporate all important interaction effects.

I re-estimated a gradient boosting model using all the variables in Table 3 but removing all financial variables with RVIs less than 10 (around 30+ variables were dropped as a result). However, I found that the RVIs of the remaining variables did not change significantly.

Based on the same 70/30 random allocation to learn and test samples used for the gradient boosting model in Table 3.

The parameter estimate is an “average” effect and has the same value across all observations.

Other market based variables evidence some interesting nonlinear effects. Consistent with previous literature, my analysis of the marginal effects indicates that stock price volatility also exhibits a strong positive relationship with the failure outcome (high stock price volatility increases the probability of failure). Higher levels of short interest are also increase with the failure outcome, but again the impacts are nonlinear and vary over different levels of short interest.

Several other accounting based variables exhibit similar nonlinear relationships with the bankruptcy outcome. For instance, the EBIT to total assets variable displays a similar pattern to cash flow returns.

The commercial version of gradient boosting, TreeNet®, used for this study, allows for automatic detection of all interaction effects. According to Salford Systems, interaction effects are based on comparisons with a genuine bivariate plot (where variables are allowed to interact) and an additive combination of the two corresponding univariate plots. By determining the differences between the two response surfaces, the strength of interaction effect can be measured for a given pair of variables. The core idea behind interaction testing in TreeNet® is that variables can only interact within individual trees (see “Introduction to TreeNet,” Salford Systems, San Diego, 2015).

Measure 1 also shows the effect of the interacting pair normalized to the overall response surface (total variation of predicted response).

The two-way interaction effects suggest that higher levels of ownership concentration and institutional ownership reduce the impact of excess returns on the failure outcome.

According to Salford Systems, TreeNet®‘s high predictive accuracy comes from the algorithm’s capacity to generate thousands of small decision trees built in a “sequential error–correcting process that converges to a highly accurate model.” Other reasons include the power of TreeNet®‘s interaction detection engine. TreeNet® establishes whether higher order interactions are required in a predictive model. The interaction detection feature not only helps improve model performance (sometimes dramatically) but also assists in the discovery of valuable new variables and patterns. This software has automatic features for detecting higher order interaction effects across any number of variables (not available in most packages or older methods). TreeNet® also has optimization features to help find the analyst find the best tree depth which maximizes predictive performance.

The predictive power of interaction effects can readily be isolated by TreeNet® by estimating a model with and without interaction effects and comparing the AUCs accordingly.

A basic issue in any machine learning model is the learn curve—this shows how well the model predicts as the training sample is increased. The smaller the sample size, the higher the expected bias in the model. Model variance is also expected to diminish as the sample size is monotonically increased. A model also needs to be tested out of sample, and the more sample available, the more confidence we can have in the generalization error of the model. It goes without saying that the efficient estimation of more complex models involving many feature vectors and higher order interaction effects requires larger sample sizes, particularly when comparing predicting performance across alternative models.

Jones et al. (2015) reported that gradient boosting and AdaBoost strongly outperformed when decision trees are used as the base classifier. For instance, they examined boosting models where both logit and probit were used as the base classifier, but they did not perform as strongly.

This is double TreeNet®‘s battery petition default setting of 10-repeat models.

Bankruptcy researchers often want to predict the success of their models a number of years prior to failure, not just in the year of failure. For instance, testing the predictive power of a model at t = −3 (three years from failure) requires knowledge of which companies failed in t = 0 (year of failure). Hence look-ahead bias is unavoidable in this situation.