Inter-Institutional Dynamics: Assessments of the Supreme Court in a Separation of Powers Context

We note that not all scholars share McCloskey’s view of the Court’s role. For alternative perspectives see, e.g., Carolene Products (1938), Footnote 4 (304 U.S. 144) and Friedman (2009). In the interest of clarity, we focus on McCloskey’s characterization as a first step in testing for an empirical connection between the separation of powers context and individuals’ assessments of the Court’s legitimacy.

See Mondak (1992) for a discussion of the link between policy legitimation and institutional support.

“Court legitimacy” has been variously referred to as several different terms throughout the literature. For the purposes of this article, we will interchangeably use the terms “court legitimacy,” “institutional support,” “diffuse support,” and “assessments of the Court.”

Empirically, scholars have drawn dramatically different conclusions about the nature of public response to Court opinions depending upon whether diffuse or specific support is being measured (e.g., Murphy, Tanenhaus, and Kastner 1973; Caldeira and Gibson 1992; Durr et al. 2000). Further, specific support has been shown to be one factor which may contribute to institutional support (Gibson et al. 1998).

In a somewhat related vein, Clark (2009) also posits that Court legitimacy is connected to the interplay between the Court and Congress. Specifically, he examines whether Court behavior in terms of the number of laws struck down each year is responsive to signals about its legitimacy via the interaction between Court-curbing attempts and public support for the Court.

See Hoekstra (2003) for the importance of case salience on the public impact of a case.

In this instance, the relationship between knowledge and institutional support holds in bivariate analysis but not when included in a multiple regression. However, this appears to be due to multicollinearity with independent variables measuring adherence to democratic norms and level of education (p. 59).

To be clear, Gibson (2007) notes that higher levels of Court-specific knowledge are positively related to measures of the Court’s institutional support (as do Gibson and Caldiera 2009a and Caldeira and Gibson 1995). Here, we rely on general political knowledge questions as the basis for a distinction between political sophisticates and non-sophisticates.

Others appear to share the sense that those with low knowledge of courts will respond differently to stimuli about the Courts. Gibson et al. (1998), for instance, focus their analyses of institutional support on individuals who have at least a moderate level of awareness of the Court, assuming that this group will be distinctive from those with low or no knowledge of the Court.

We use the situation where Congress is at fault for high levels of disagreement as our comparison group because we do not have theoretical reasons to believe that respondents would be more disappointed with the Court when it fails to uphold its delicate balance in one way more than the other. In other words, we do not necessarily expect legitimacy to be harmed more in the situation where the Court is at fault for high levels of disagreement than in the situation where the Court and Congress agree most of the time.

While any differences between the demographics of our sample and the general population limit the generalizability of specific levels of institutional support for the Court, the power of random assignment gives us confidence in the comparisons between experimental conditions, and thus any causal relationships detected (see footnote 14).

Scholars have extensively studied whether those who are sophisticated about politics—and, therefore, have more information about politics and issues—will be more or less susceptible to framing effects. Theoretical arguments lay out compelling reasons for either directional assumption. For instance, the politically unsophisticated may be more susceptible to persuasion by frames because they have fewer established opinions about political issues—these individuals, then, should be more likely to accept a frame’s perspective at face value and not counter-argue the frame’s assertion (e.g., Kinder and Sanders 1990; Haider-Markel and Joslyn 2001). Alternatively, others argue that sophisticates have the capacity to meaningfully incorporate a new frame into their existing knowledge and beliefs about political matters (e.g., Nelson et al. 1997; Miller and Krosnick 2000; Druckman and Nelson 2003). The bulk of the empirical evidence is coalescing around the latter reasoning: that political sophisticates will respond to frames more so than political non-sophisticates.

Because the subjects were randomly assigned, we can be confident that any differences between conditions can be attributed to the experimental treatments (or lack thereof) (Druckman et al. 2011). As such, it is unnecessary to administer pre- and post-treatment questionnaires to the subjects. We also address the slim possibility of a break-down in the random assignment procedure by further analyzing the data with multiple regression controls. For further explanation of experimental methods and the trade-offs between these two types of designs, see Druckman et al. (2011).

We are uncertain about how the Control group will compare to the other conditions because any expectations would theoretically depend on respondents' beliefs about the actual level of policy agreement between the Court and Congress. If, for example, a respondent thinks the Court and Congress do actually agree a large percentage of the time, her responses may mimic those in the Agreement condition. However, we could not easily ask a question about this without the risk that doing so would prime the separation of powers context, thus compromising the idea that the Control group is not subjected to an experimental treatment. Instead, we included the Control condition to explore the extent to which respondents may possess a shared baseline perception of these inter-branch dynamics that would be revealed by a comparison with the treatment groups. Our primary focus, though, is to test for differences between the alternative characterizations of the separation of powers context.

Respondents are well-distributed across conditions. There are 91, 91, 81, and 90 respondents, respectively, in the Agreement, Congress Fault, Court Fault, and Control groups.

In constructing the framing manipulation, a key decision was whether to present the (dis)agreement to respondents as relating to one specific case, or to an aggregation of cases. Ultimately, we chose to focus on an aggregation of cases. We contend that doing so more accurately reflects how individuals assess the Supreme Court in a real world setting. Infrequently, a case is salient enough to draw the attention of the broad public (e.g., Bush v. Gore or Roe v. Wade). Past work, however, has shown that individual cases may temporarily shift institutional support for the Court, but that it relatively quickly returns to pre-decision equilibrium levels (e.g., Durr et al. 2000; Yates and Whitford 2002). This suggests that there is a strong, established basis for opinions about the Supreme Court which are only temporarily influenced by major cases. It is this foundation of support that we hope to tap and test in a separation of powers context.

In addition, we argue that by aggregating cases, we minimize the potential distraction of ideology amongst our respondents. Our goal is to test McCloskey’s framework by examining whether institutional support for the Court is influenced by a consideration of how the Court's actions relate to actions taken by Congress. It would have been possible to present respondents with an aggregation of cases in the frame which also persistently pointed in one ideological direction—varying, for instance, not just the levels of (dis)agreement, but also the ideological direction of that disagreement. This would be an interesting venue for further investigation but is beyond the scope of this initial experiment. McCloskey’s framework focuses on the separation between fundamental law and popular will, which does not directly correspond to an ideological dimension. What is more, over the course of most respondents’ lifetimes, they will have seen both “liberal” and “conservative” courts, balancing out the differences when it comes to aggregate assessments of institutional legitimacy. Furthermore, we control for respondents’ ideological predispositions in the models to guard against any potential influence along these lines.

Finally, actual Supreme Court news stories suggest this aggregate focus is externally valid. For instance, in a New York Times report during the confirmation of Chief Justice John Roberts, Greenhouse (2005) notes that, “In a series of decisions…the Court declared unconstitutional acts of Congress that had been passed by broad bipartisan majorities.” Later, Greenhouse continues outlining how the Court's decisions relate to actions taken by Congress in the aggregate, writing, “From 1995 to 2003, the Supreme Court overturned all or parts of 33 federal statutes…Until then, the modern court gave Congress a wide berth to define its own role.” These excerpts suggest the treatment articles used here, which also discuss Court’s actions as they relate to Congress in the aggregate, are appropriately similar to real news stories.

In the debate of whether political sophisticates will be more or less susceptible to framing effects, Druckman and Nelson (2003) find NE to be a key control variable. This construct measures a subject’s propensity to thoroughly examine a new argument or, perhaps, a new frame. Respondents who record high levels of NE tend to have many prior opinions and are chronic evaluators (Jarvis and Petty 1996). Once the authors control for NE, sophisticates are more susceptible to the media frames, while high-NE individuals are typically less susceptible to the frames (Druckman and Nelson 2003, p. 740). We include the measure here to control for NE’s potential to be conflated with sophistication.

We generally follow Gibson et al.'s (2003) scale with only minor adjustments. Our Cronbach’s α indicates it is appropriate to scale these items (α = 0.80). See Appendix for the questions included.

Political sophistication is relatively well-distributed across the range of the variable. Specifically, it ranges from 0 to 10 with a mean of 6.96 and a standard deviation of 2.67. Additionally, there are respondents at each of the 11 levels of the scale.

We include those who scored at the median with sophisticates since we believe answering 7 out of 10 political knowledge questions correctly (the median value), including some fill in the blank questions, signals substantial awareness and understanding of politics. Additionally, we explored other possible operationalizations of sophistication including a three-way split (i.e., low, mid, and high sophistication based on one standard deviation above and below the median) and a continuous measure of sophistication and found substantively similar or slightly weaker results, which is consistent with Druckman (2004) and Miller and Krosnick (2000). See Druckman (2004) for a discussion of the advantages of a median split.

Subsequent to pre-testing for this survey, Gibson and Caldeira (2009b) noted the positive bivariate relationship between courts-based knowledge and institutional support. While our measure relies on broader political knowledge questions, we include three Court-specific knowledge questions. The correlation between the number of correct answers on these three questions and the more general political knowledge scale is high: 0.85. What is more, when we run our final model, basing the sophistication split on the Court-only questions, the results largely correspond to the final model used here.

The questionnaire included three need to evaluate questions (see Appendix). However the Cronbach’s α for the three-item scale is 0.64 but increases to 0.74 without the third item. Thus, we proceed using the two-item measure, which asks whether the respondent tends to have an opinion about a few things or many things and whether the respondent has fewer or more opinions than others. It ranges from 0 to 1, has a mean of 0.61 and a standard deviation of 0.21. For more on NE, see footnote 19.

Predicted values are calculated with all independent variables at their median or modal values unless otherwise noted.

This lack of statistical significance could again be related to the relatively small number of non-sophisticate respondents.