Another look at GAAP versus the Street: an empirical assessment of measurement error bias

While most studies use the actual earnings reported by commercial forecast data providers such as I/B/E/S to proxy for Street earnings (e.g., Bhattacharya, Black, Christensen, & Larson, 2003; Bradshaw & Sloan, 2002; Brown & Sivakumar, 2003; Doyle, Lundholm, & Soliman, 2003), others use pro-forma earnings figures collected from press releases (e.g., Bhattacharya et al., 2003; Johnson & Schwartz, 2005; Lougee & Marquardt, 2004).

This error is discussed in Abarbanell and Lehavy (2006, 2002), Bradshaw and Sloan (2002), and Bradshaw (2003).

Following Abarbanell and Lehavy (2006), we refer to the procedural and definitional changes as the I/B/E/S “regime shift” throughout the paper.

Bradshaw (2003) is among the first to discuss this classic “errors in variables” problem; see also Bhattacharya et al. (2003) and Berger (2005).

Note that while the estimated GAAP and Street earnings surprises are the same in the pre-regime shift period (i.e., when I/B/E/S reports a GAAP figure as its actual earnings, both surprise measures are equal to the difference between realized GAAP earnings and I/B/E/S forecasted earnings), the true GAAP and Street earnings surprises are different. Therefore, the two measurement errors (I/B/E/S Adjustment Error and GAAP Expectation Error) are not the same. See Sect. 1 for a more detailed discussion.

For instance, the findings of some recent studies that use pre-1992 I/B/E/S data to construct forecast errors (e.g., Collins et al., 2005; Doyle, Lundholm, & Soliman, 2006; Doyle & Soliman, 2004) may be affected because of the measurement error in earnings surprises that arises from the I/B/E/S Adjustment Error.

Throughout the paper, we assume that the true unexpected earnings is uncorrelated with the measurement error, that is, \({\hbox{var}(F\hat{E})=\hbox{var}(FE)+\hbox{var}(\eta).}\)

Note that EPS _Diff is the difference between the true realized GAAP and Street EPS and not the difference between the estimated GAAP and Street EPS. Hence, while the empirical proxies for GAAP and Street EPS are the same for the pre-regime shift period (i.e., both are equal to EPS _GAAP ) and hence the difference between the estimated GAAP and Street EPS is equal to zero in our model, the true EPS _Diff is not equal to zero.

Note that because E(EPS _Diff ) is uncorrelated with the unexpected return (UR) by definition, the following two covariance terms are equivalent to each other: cov(FE _Diff , UR) =  cov(EPS _Diff , UR), where FE _Diff is defined as FE _GAAP  − FE _Street . Therefore, we can rewrite the estimated Street ERC expression as follows: \({\hat{E}RC_{Street}=\frac{\hbox{cov}(\mathop{FE}\nolimits_{Street} ,UR)+\hbox{cov}(\mathop{FE}\nolimits_{Diff} ,UR)}{\hbox{var}(\mathop{FE}\nolimits_{Street})+\hbox{var}(\mathop{EPS}\nolimits_{Diff})}}\). It is then easy to show that \({\hat{E}RC_{Street}}\) is biased downward whenever the following inequality holds: \({\frac{\hbox{cov}(\mathop{FE}\nolimits_{Street},UR)} {\hbox{var}(\mathop{FE}\nolimits_{Street})} > \frac{\hbox{cov}(\mathop{FE}\nolimits_{Diff},UR)} {\hbox{var}(E(\mathop{EPS}\nolimits_{Diff})) +\hbox{var}(\mathop{FE}\nolimits_{Diff})}}\). This inequality is expected to hold at all times given that the expression on the left-hand side is the true ERC for Street earnings, while the expression on the right-hand side is a downward-biased ERC for excluded items.

Our empirical results (presented in Sects. 3.2 and 3.3) are robust to using operating earnings (as in Bhattacharya et al., 2003) instead of earnings before extraordinary items. Total operating earnings is computed as GAAP basic earnings per share from operations (Compustat quarterly data item #177) multiplied by the number of basic shares outstanding (Compustat quarterly data item #15).

I/B/E/S’ EPS value is adjusted for dilution if the corresponding GAAP EPS is calculated on a diluted basis.

Consistent with our definition of EPS _GAAP , I/B/E/S’ definition of earnings per share also excludes extraordinary items and earnings from discontinued operations (The I/B/E/S Glossary, 2001).

None of our results are sensitive to winsorization of returns or other variables.

If several earnings forecasts were issued on that particular date, we use the median value as an estimate. We choose to employ the most recent forecast of a single individual analyst to improve the accuracy and timeliness of the forecast (O’Brien, 1988). Our results are robust to using the most recent consensus forecasts from I/B/E/S’ summary files.

Bradshaw and Sloan (2002) use long-window stock returns in their analyses, which they define as the buy-and-hold returns from the two days after the last quarterly announcement through the day after the current-period earnings announcement. As a sensitivity check, we also compute long-window stock returns and report these results in Sect. 4.2.

Bradshaw and Sloan’s (2002) sample spans 1985–1997. Abarbanell and Lehavy’s (2006) sample spans 1985–1998.

Briefly, the expression for the reverse ERC is derived from: \({Rev\hat{E}RC=\hbox{cov}(FE,UR)/\hbox{var}(UR)=\hbox{cov}(FE,UR)/\left( {ERC^2*\hbox{var}(FE)+\hbox{var}(\varepsilon)} \right)=\hbox{cov}(FE,UR)/\left( {ERC*\hbox{cov}(FE,UR)+\hbox{var}(\varepsilon)}\right)}\).

From Figs. 2 and 4, we note that while the magnitudes of the reciprocal reverse ERCs (Fig. 4, Panel B) are substantially larger than their original counterparts (Fig. 2), they are in line with the numbers reported in prior studies (e.g., Basu, 1997). As noted earlier, the reciprocal reverse ERCs are the upper bound of the true ERCs, where the exact magnitude of the bias depends upon the R-squared of the returns-earnings regression. In other words, when a large extent of the returns variance is unexplained by earnings surprises (i.e., low R-squared), the reciprocal reverse ERCs would be substantially different from the original ERCs.

Changes in accounting standards in the early 1990s are a potential alternative explanation for the reverse regression results. Specifically, if the new accounting standards move towards fair-value accounting and thereby create more transitory items in GAAP earnings, estimating reverse earnings-returns regressions would remove not only the measurement error in forecast errors, but also the variance of the transitory component embedded in GAAP earnings. However, examining the accounting standards that became effective in the early 1990s, the first standard that may have a significant impact on GAAP earnings, SFAS 121, became effective in 1995, which is after the regime shift period. Moreover, such accounting changes would increase the variance of GAAP earnings without necessarily affecting the variance of Street earnings, an opposite pattern to the one observed in our ERC decomposition results.

Specifically, Abarbanell and Lehavy first sort all observations into three groups: zero, positive and negative earnings differences; they then rank the earnings differences within the positive group and within the negative group, separately, into quintiles. We would like to note that we find a lower incidence of zero differences in our sample than that reported in Abarbanell and Lehavy, probably because of differences in our measures of GAAP EPS. In particular, while we calculate GAAP EPS by dividing income before extraordinary item by the weighted average number of shares outstanding (see footnote 10), they use the Data 9 or Data 19 EPS figures from Compustat, which are generally accurate to the second decimal place. When we follow their approach to calculate GAAP EPS, we find very similar results. We therefore construct the negative earnings difference portfolio (i.e., the Portfolio 1 reported in Table 2 of Abarbanell and Lehavy) using their measure of GAAP EPS. In addition, we perform two alternative sensitivity tests by removing the extreme 5% or 10% observations on the left tail of the entire earnings difference distribution. Results from the three sensitivity tests are qualitatively and statistically similar.

Also, by excluding observations with the largest differences between observed Street and GAAP forecast errors, Abarbanell and Lehavy may omit firms that have valid reasons for excluding certain (less persistent) items, introducing potential sample selection bias into their research design.

Bradshaw & Sloan (2002) do not report observations in their study by year, and hence we cannot directly compare our sample size by year. We note that the number of firm-quarter observations in our sample from 1985 to 1997 (the sample period in their study) is 112,962, which is reasonably similar to that in Bradshaw and Sloan (98,647 firm-quarter observations).

Requiring a constant sample composition for the entire sample period (i.e., 1985–2003) results in a prohibitively small number of observations, and we therefore conduct the robustness test using a shorter sample period (i.e., 1989–1998). We also construct an alternative sample by retaining firms that are present in 16 (64) out of 19 years (76 quarters) in the 1985–2003 period and we find qualitatively and statistically similar results.

For ease of exposition, we do not distinguish between error terms for the sub-periods before and after I/B/E/S’ regime shift in Table A1. However, Fig. A1 and subsequent discussion reflect a possibility that the error terms may change over time.

We make the restrictive assumption that FE _Diff is uncorrelated with FE _Street .

The latter inequality holds following the chain of arguments below: \({X=\frac{ERC_{Street} +A2}{1+B(ERC_{Street} +A2)}-ERC_{Street} \Rightarrow \left[ {X+ERC_{Street} } \right] = \frac{ERC_{Street} +A2}{1+B(ERC_{Street} +A2)} < ERC_{Street} +A2 \Rightarrow X < A2}\).