The effect of enforcement transparency: Evidence from SEC comment-letter reviews

In the aftermath of the financial crisis, for instance, regulators opted to publicly disclose stress tests to better inform investors on the risks taken by banks (e.g., Goldstein and Sapra 2013). More recently, the Reed–Grassley bill sought to publicly disseminate the Public Company Accounting Oversight Board’s (PCAOB) inspection reports of auditors (PCAOB Enforcement Transparency Act of 2017).

For instance, some politicians have raised concerns about “the utility of devoting hundreds of professional staff to a process that is not designed to detect fraudulent conduct.” (e.g., Paredes 2009; Katz 2010, 2011).

Several additional considerations suggest that CLs do matter. First, they allow the SEC to obtain answers to questions that are frequently dodged, dismissed, or ignored when asked by investors or analysts (Hollander et al. 2010). Second, the large backlog of Freedom of Information Act (FOIA) requests preceding the policy change suggests a vivid public interest in these letters (OIG 2007). Third, the SEC believes that these letters prompt firms to change their reporting practices (OIG 2008b). Finally, short-sellers use CLs (Dechow et al. 2016).

We focus on earnings announcements, rather than SEC filings, because prior work shows that these announcements are important disclosure events in terms of impact on security prices (e.g., Kothari 2001; Basu et al. 2013). We provide more insights on this choice in Section 3.3.

On its website, the SEC (2015) describes the objective of CL reviews as follows: “Much of the Division’s review involves evaluating the disclosure from a potential investor’s perspective and asking questions that an investor might ask when reading the document. When the staff identifies instances when it believes a company can improve its disclosure or enhance its compliance with the applicable disclosure requirements, it provides the company with comments.”

There could be economic reasons for firms to adjust or drop disclosures triggered by CLs a certain time after the review. For example, changes in a firm’s economics, modifications in the accounting standards, or changes in the materiality of a disclosure could lead firms to adjust or drop disclosures over time. The following quote by Wayne Carnall (the former chief accountant of the Division of Corporation Finance) illustrates this idea and suggests that the SEC does not necessarily expect a permanent effect from its CLs: “As with all disclosures, you provide what is right and meaningful and material. If it is not material, not relevant, companies do not have to continue to provide that disclosure” (PwC 2016b).

These cross-sectional results do not allow to unambiguously attribute our findings to one of the two mechanisms. For example, the presence of dedicated investors potentially increases reputation costs for the SEC, inducing the SEC to exert more effort in its reviews. Thus part of the evidence could also be consistent with supervisory discipline.

Dechow et al. (2010) report that many studies use discretionary accruals and restatements as measures of reporting quality. Consistent with this, Blackburne (2014) and Cunningham et al. (2017) use discretionary accruals and restatements as measures of reporting quality in the context of SEC CLs. The evidence in the work of Bozanic et al. (2017) suggests that the length of narratives is associated with the quality of firms’ disclosures.

For example, Beaver et al. (2018) show that the ERC increased following SOX, and Cohen et al. (2008) document a decline in accruals-based earnings management after SOX. More broadly, Coates (2007) cautions researchers to be careful in drawing strong inferences around times of multifaceted regulatory change.

Informed by our study, the European Securities and Markets Authority (ESMA), which coordinates the capital-market supervision of the national regulators in the European Union, recommends the public disclosure of oversight activities there (ESMA 2017).

The notion that firms start adjusting their reports, while under review, is supported by the inner-workings of the CL process revealed in the OIG’s audit of Bear Stearns’ CL review (OIG 2008b). John W. White noted that the SEC sent a CL to Bear Stearns related to the company’s fiscal year 2006 10-K on September 27, 2007. That letter included a focus on subprime mortgage matters. Soon after receiving this letter—and well before the public release of the CL and Bear Stearns’ collapse in March 2008—Bear Stearns improved its disclosures about subprime mortgage securities in its form 10-Q filed on October 10, 2007. (Specifically, details on net inventory markdowns related to losses in residential mortgages and leveraged finance areas were added.) John W. White also emphasized that the CL review of Bear Stearns was not unique and explained more generally how CLs improve disclosure. He said: “The goal of disclosure of material information to investors is achieved by seeking improvements to a company’s public disclosures in its periodic and current reports. Those reports are readily available to all investors. … Our experience is that, similar to the Bear Stearns review described above, a company may respond to staff comments in its public disclosure documents.” For confirmation purposes, we also exploit an alternative research design using the dissemination date as the beginning of the 360-day period (untabulated). We find consistent results.

We estimate equation (1) using weighted-least-squares (“robust”) regressions that place less weight on estimates with large absolute residuals. We perform robust regressions using Stata’s “rreg” procedure, which eliminates any observations with a Cook’s distance greater than one and weighs the remaining observations based on the absolute residuals. As explained in Section 5.6., our results are robust to alternative approaches to deal with extreme UE observations.

Our results are robust to using firm fixed effects instead of review fixed effects.

Table 2, Panel A, does not report the main effects of Public_Review and SEC_Budget, because these variables are subsumed by the fixed effects.

We also report results from tests including alternative fixed effect structures and additional interactions for our pooled sample in Table 9. The results hold.

Disclosure of CLs may also affect firms’ reporting credibility. As described in Section 5.4., we rerun our main specification and define the period between the first and last letter from the SEC as the treatment period. Under this alternative research design, the public is still unaware of the CL during the treatment period. We find results similar to those in Table 2 (untabulated), suggesting that changes in reporting credibility alone are unlikely to explain our findings.

While the results presented in Table 2, Panel B, columns 3–4 show that the ERC effect is significant in the public period, this is less clear for our pooled sample (Table 2, Panel A, column 2). To examine this, we test whether the difference 0.820 (0.803–0.037 – 0.034 + 0.088) – 0.769 (0.803–0.034) = 0.051 is statistically significant. We find that the difference is statistically significant at the p < 0.02 level, confirming a statistically significant increase in ERC following a CL in the public period.

The magnitude of our increase in ERC is modest in comparison to prior literature. For instance, Chen et al. (2014), who examine capital-market responses to unexpected earnings releases following material restatements, find a decrease in ERC of approximately 56% in the 11 quarters following the restatement. However, material restatements are much more severe and less frequent than CLs.

Prior research shows that many firms provide substantial qualitative disclosures and detailed GAAP financial statement information in their EAs (Chen et al. 2002; Baber et al. 2006; Wasley and Wu 2006; D’Souza et al. 2010). Regulators and practitioners also strongly encourage companies to include more detailed disclosures in the EAs. For example, the SEC Committee on Improvements in Financial Reports (CIFiR) urges companies to include in their EAs a balance sheet and cash flow statement, in addition to the income statement (SEC 2008).

The information provided in the earnings announcements is subject to the anti-fraud provisions of Section 10(b) and Rule 10b-5 of the Securities Exchange Act of 1934 (Steinberg 2009). Consistent with this, PwC (2016a) emphasizes the importance of providing information in the earnings announcements consistent with the information included in the subsequent filing, especially as the SEC staff reviews public information for consistency.

The proper identification of the mechanism requires exogenous (or independent) variation in SEC reputation costs and market monitoring. As such, our cross-sectional results do not allow to unambiguously attribute our findings to one specific mechanism.

Data on investor classification are retrieved from http://​acct3.​wharton.​upenn.​edu/​faculty/​bushee/​IIclass.​html.

We thank Terrence Blackburne for sharing this data with us.

Differences documented in Table 5 may be driven by SOX rather than by the 2004 SEC disclosure policy change. We examine this possibility by comparing the characteristics of private reviews conducted before and after SOX (untabulated). The only significant differences between these two types of private reviews are that, after SOX, the Time_from_Filing_Date is significantly shorter, and the number of core accounting topics is significantly smaller. Therefore it is unlikely that the pattern in Table 5 is driven by SOX.

Our results also hold when we use the standard deviation of the residuals as the dependent variable.

We cannot include the variable Big4_Audit (which equals 1 if the company has been audited by a Big Four accounting firm and 0 otherwise) and Public_Review, because they are subsumed by review fixed effects.

Bozanic et al. (2017) construct an index from various measures, such as length, readability, and tone, and find that the correlation between this index and the length of the narratives is very high (0.985).

We obtain this measure from Bill McDonald’s website (http://​www3.​nd.​edu/​~mcdonald/​Word_​Lists.​html).

Using Audit Analytics’ restatement data, we observe that the announcement of a restatement usually occurs within 365 days after the filing date of the restated report.

The means of Accruals, Text_Length, and Restatements are 0.03, 25.5, and 0.11, respectively. Note that abnormal accruals increase following CLs conducted before the policy change. In other words, prior to the policy change, abnormal accruals increased following CLs, and after the policy change abnormal, accruals did not increase following CLs, explaining the negative and significant coefficient on Treatment*Public_Review.

We also analyze whether earnings announcements during the review period are more likely to be followed by price reversals. A price reversal would suggest that investors overreact to the accounting information (perhaps because they consider this information more credible) and subsequently adjust their views about the informativeness of these announcements (e.g., Ball and Brown 1968). In particular, we re-estimate equation (1) and modify the measurement of the dependent variable, CAR, by compounding returns over the (+2, +90) window after the earnings announcement and subtract the market return compounded over the same horizon. In untabulated tests, we find a pattern qualitatively similar to that in Table 2, but the coefficient on UE*Treatment_Period*Public_Review is insignificant in all specifications. This suggests that our main results are unlikely to be driven by a market overreaction to earnings announcements during the review period.

Before the SEC rule “Additional Form 8-K Disclosure Requirements and Acceleration of Filing Date” issued in 2004, the classification of events reported in firms’ 8-Ks is not reliable.

The FOIA Action Plan (2008) states: “There are primarily two entities filing thousands of requests per year to obtain access to these letters for commercial use, thereby creating the overwhelming backlog. … The FY 03 review was completed in May 2006. … The review of FY 04 requests is on-going.” (The report was written in 2008.) See also OIG report (2007).

Similarly, in the comment to the policy change, John Gavin wrote in 2004: “SEC Insight currently has over 6,000 pending FOIA requests for comment and response letters relating to completed reviews.”

In untabulated tests, we also conduct propensity-score matching tests, following the procedure used by Johnston and Petacchi (2017). We find consistent results.

Using this alternative control group addresses the concern that the population of firms not receiving a CL could contain cases where a firm uses other firms’ comment letters to anticipate and fix priority issues before it is reviewed, thereby negating a minor review that might have otherwise elicited a comment letter (e.g., Brown et al. 2018). In this case, it is less clear that disclosure of CLs would improve reporting.

In row (4), as the dates are counterfactually shifted from the true dates, the coefficient on β₁ becomes smaller. (As expected, the coefficients remain significant because there is considerable overlap with the true dates.)

The SEC announced the disclosure policy change on June 24, 2004. While firms could have filed their reports before August 1, 2004, to avoid the public disclosure of a potential CL review, the time from the announcement to the start of the new disclosure policy was very short, thus limiting this possibility. We do not observe any unusual filing patterns between June 24 and August 1 of that year.

While the original compliance date of SOX 404 for accelerated filer was June 15, 2004 (SEC Release No. 33–8238, published on June 5, 2003), SEC Release No. 33–8392 published on February 24, 2004, extended the compliance date to November 15, 2004. Other SOX provisions, such as Regulation G, became effective as of March 28, 2003 (https://​www.​sec.​gov/​rules/​final/​33-8176.​htm). In a similar manner, Section 408 of SOX, which requires the SEC to review each company at least once every three years, became effective with the passage of SOX in 2002. Similarly, firms had to be in compliance with Section 301 (https://​www.​sec.​gov/​rules/​final/​33-8220.​htm) and Section 208(a) (https://​www.​sec.​gov/​rules/​final/​33-8183.​htm) by the start of 2004.

For non-accelerated filers, Section 404 became effective for fiscal years ending on or after July 15, 2007 (SEC Release No. 33–8618, published on September 22, 2005).

Compared to the results presented in Table 2, Panel A, the magnitude of the coefficient on UE*Treatment_Period*Public_Review is larger. One potential explanation for the larger magnitude is that the policy change was unexpected, and therefore initially firms made more substantial changes around publicly disclosed CL, resulting in a stronger initial ERC effect. Another possibility is that firms have improved their reporting over time, and thus there is a decreasing need for CL reviews (Beaver et al. 2018).

While the SEC already aimed to review each firm at least once every three years before the Sarbanes-Oxley Act of 2002 (OIG 2000), Section 408 of SOX made the three-year frequency a requirement. SOX also mentions the following specific factors the SEC should consider when deciding which firms to review more frequently: “(1) issuers that have issued material restatements of financial results; (2) issuers that experience significant volatility in their stock price as compared to other issuers; (3) issuers with the largest market capitalization; (4) emerging companies with disparities in price to earnings ratios; (5) issuers whose operations significantly affect any material sector of the economy; and (6) any other factors that the Commission may consider relevant.”

The ROC curve plots the probability of detecting a true signal (sensitivity) and a false signal (1 – specificity) for the entire range of possible cutoff points (Kim and Skinner 2012). The area under the ROC curve (AUC), which ranges from 0 to 1, provides a measure of the model’s ability to discriminate. A value of 0.5 indicates no ability to discriminate, while a value of 1 indicates perfect ability to discriminate. A greater area indicates a better performance of the model. The usual convention is that a model with an area of less than 0.7 is considered to have no discrimination ability, a model with an area between 0.7 and 0.8 is considered to have acceptable discrimination ability, and a model with an area between 0.8 and 0.9 is considered to have excellent discrimination ability.