Accuracy and Bias in Perceptions of Political Knowledge

A direct answer to this question is beyond the scope of this paper. Obviously, perceived expertise serves as a better proxy for actual expertise if there is less bias in people’s perceptions.

For an explanation as to why two different questions were asked see Huckfeldt et al. (1998). An analysis of three data sets—including the ISL—shows that both questions result in networks with similar characteristics (Klofstad et al. 2009). I assume, therefore, that all dyads discuss politics regardless of the particular network generator used to elicit the name of the discussant.

Alternative measures of agreement could have been used (e.g., the average distance between main respondent and discussant attitudes on a series of issues). However, whether individuals are on the same sides of an issue better predicts whether the individuals believe they agree than the actual distance between the pair on a policy scale (Kenny and Jenner 2008).

Besides the measures of objective knowledge, the two differences in ISL and NES measures are education and income. In the NES, education is measured using the respondent’s highest degree attained. Income is measured using six ordered categories as in the ISL, but the cut points between the categories are different.

The questions that were used to construct the ISL and NES objective knowledge scales are listed in the Appendix.

The post-election interviewer rating is preferable for three reasons. First, the pre-election interviewer rating is skewed with more interviewers saying respondents are well informed. Second, the post-election interviewer asked the general knowledge battery. This interviewer, therefore, should have a better idea of the actual level of respondent knowledge. Third, 80% of respondents had the same interviewer for the pre- and post-election survey. So, they should have learned more about the individual over the course of both interviews. As a result, the post-election measure should provide a more stringent test of factors biasing perceptions of knowledge.

None of the models in Table 1 perfectly replicate the models in Huckfeldt (2001, p. 429). The first model in Table 5 in the appendix replicates Huckfeldt’s model. The second model adds the variables of interest for this paper. This analysis supports the conclusions from Table 1. Both a likelihood-ratio test and the AIC support the new model even though the AIC punishes less parsimonious models. A likelihood-ratio test cannot be performed unless both models use the same cases. As a result, the n in the Huckfeldt replication is lower than in the original because of missing data in the additional variables.

Predicted values are calculated by varying the value of the variable of interest while holding all other variables at their means. Predicted values are based on coefficient estimates from the first model in Table 1.

About 13% of the dyads are married.

Discussants at the ninetieth percentile for engagement have their mean political discussion variable set at 3.667 out of four and they are very interested in the campaign and have participated in three activities. Discussants at the ninetieth percentile for actual knowledge answered all three objective knowledge questions correctly and spent 18 years in school. Discussants at the tenth percentile have their mean political discussion variable is set at 1.5. They are not at all interested in the campaign, but did participate in one activity. They answer one objective knowledge question correctly and had 12 years of schooling.

The proper analytic solution to this problem would be to model the selection process first and then use results from that model as part of the perceived knowledge estimation (Achen 1986). I cannot do this, however, because I do not have data on those individuals who were not named as discussants. Since expertise (Ahn et al. 2010) and sex (Huckfeldt and Sprague 1995; Klofstad et al. 2006; Mendez and Osborn 2005) are important factors in the selection process, their effects may be underestimated. The other factors that bias perceptions of political knowledge may also impact the probability of selection and those effects may also be underestimated, but there is no way of knowing with the available data.

One of the major difficulties in measuring objective political knowledge is that some groups are more likely to guess than others. If a respondent says, “I don’t know” in response to a knowledge question, this individual is typically coded as having given an incorrect answer. If a respondent guesses, there is some probability that the person will guess correctly. Thus, those groups more likely to guess will have higher levels of political knowledge than they should. This should be less of a problem in the 2000 NES as interviewers discouraged “don’t know” responses. To ensure the results were not driven by the remaining “don’t knows”, I reran the models using two different measures of political knowledge. First, I measured political knowledge as the number of correct answers over the number questions answered—the denominator varies by person based on how often they said “don’t know”. Second, I randomly assigned answers to those questions to which the respondent answered “don’t know” (Mondak and Anderson 2004). In both cases, the coefficients for the engagement and demographic variables were larger than those reported in Table 2. This suggests that the model in Table 2 may underestimate the amount of bias in the interview assessments of the respondent’s political knowledge. I present these results, however, because they provide the most conservative estimates of the amount of bias.

The 2000 NES was conducted by the Center for Political Studies of the Institute for Social Research. Interviewers were trained at regional conferences prior to the study implementation. To ensure the quality of their work, the interviewers’ work was reviewed by supervisors throughout the interview period. Interviewers are not given specific instructions as how to make a judgment about a respondent’s expertise. They are, however, aware that they will have to make such a judgment and have experience doing so. More information on the design and implementation of the NES is available at http://​www.​electionstudies.​org/​overview/​overview.​htm.

This is, of course, an imperfect measure of influence because it is unclear if the discussant influenced the main respondent or if the main respondent influenced the discussant. It is, also, possible that an outside factor influenced both individuals. This is, however, a standard measure of discussion partner influence in cross-sectional studies (e.g., Huckfeldt and Sprague 1991).