Opinion Shift and Stability: The Information Environment and Long-Lasting Opposition to Trump’s Muslim Ban

For an exception, see Christenson and Glick (2015b)’s use of a multi-wave MTurk panel design.

While many studies have relied on online platforms such as MTurk for various experiments, Christenson and Glick (2013) have shown that MTurk, primarily due to its speed and flexibility, also offers clear advantages for panel studies, especially those that require immediate implementation.

https://​morningconsult.​com/​wp-content/​uploads/​2017/​04/​170404-crosstabs-Brands-v3-AG.​pdf.

Please see "Appendix 3" for a more detailed methodology. We relied on three trained coders, with an inter-coder reliability of 0.92 on 25% of the sample.

https://​www.​mccain.​senate.​gov/​public/​index.​cfm/​press-releases?​ID=​587F2A2D-8A47-48F7-9045-CF30F0A77889.

After matching respondents based on MTurk IDs and conducting data quality checks, the final dataset consists of \(n=422\) Wave 1, \(n=280\) Wave 2, and \(n=161\) Wave 3 respondents.

Variable summary statistics are provided in Appendix 2 Tables 6, 7, and 8 .

As an additional robustness check, we imputed the data using a \(m=5\) chained dataset, based on education, age, income, party identification, race, gender, and Trump approval. That is, we imputed the full (three wave) dataset for all wave 1 respondents and then again for just wave 2 and wave 3 respondents. In both imputed datasets, presented in Appendix 2 Table 13, mean ban attitudes shifted from wave 1 to wave 2, but not from wave 2 to wave 3. These results are substantively similar to the main findings detailed below.

https://​dataverse.​harvard.​edu/​dataset.​xhtml?​persistentId=​doi%3A10.​7910/​DVN/​GDF6Z0.

https://​www.​census.​gov/​cps/​data/​cpstablecreator.​html.

We are also sensitive to the fact that MTurk respondents may be particularly aware of the news and more likely to be exposed to the Muslim ban backlash compared to the general population. If overly attentive respondents bias our results, those who reportedly watched the protests in wave 2 possibly gave different ban attitude responses than those who reported not watching the protests (presumably less attentive people). We tested this by conducting a \(\chi ^{2}\) test between wave 2 ban attitude and “watched demonstrations” (1 = yes, 0 = no). We find no evidence of a statistically significant difference in wave 2 ban attitudes between the two groups (\(\chi ^{2}\) =1.8, df = 4, p = 0.771), or even between the two variables when we subset to just respondents who completed the wave 2 survey (\(\chi ^{2}\) = 2.6, df = 4, p = 0.617). This provides some evidence that an overly news attentive sample is not biasing our findings. We also located a 2016 probability-based representative Pew Research Center survey that asked questions about local news and national news consumption. We downloaded the data and calculated the percentage of respondents who reported watching television news. About 81% of respondents said they watched news, which is very similar to the percentage of our wave 2 respondents who reported watching local or national television news (78%). Thus, external evidence suggests our sample is not biased in the form of greater news consumption relative to the adult U.S. population. We provide greater discussion of the Pew sample in ″Appendix 1″.

Due to the administration’s executive order ban changes, the wave 3 ban question read: “President Trump’s executive order restricting immigration from Syria, Iran, Libya, Yemen, Somalia, and Chad—do you strongly agree (5), somewhat agree (4), neither agree nor disagree (3), somewhat disagree (2) or strongly disagree with this order (1)?” These changes are minor, and given citizens’ low state of political knowledge, are unlikely to produce attitudinal effects due to question wording alone (Schuman and Presser 1996).

Question wording of all the variables used in the analyses is presented in ″Appendix 1″.

Krosnick et al. (2002) examine whether no-opinion options (e.g. “don’t know”) may discourage some respondents from cognitively engaging in work necessary to report the true opinions they do have. They find that the inclusion of no-opinion options may not enhance data quality and instead may preclude measurement of some meaningful opinions. This is particularly applicable in a context where online survey respondents may be motivated by speed and might not deeply engage with the survey questions when “don’t know” options are present. Based on this line of research, we precluded no-opinion options.

https://​www.​washingtonpost.​com/​blogs/​plum-line/​wp/​2017/​02/​07/​a-new-poll-shows-a-surprisingly-big-public-swing-against-trumps-muslim-ban/​?​utm_​term=​.​9d1ee0984a47.

We excluded a dummy variable for Trump vote from the three-wave regression analysis due to a high number of missing cases. However, we do include a Trump favorability variable, which is highly correlated with Trump vote at 0.831.

For more details refer to ″Appendix 1″.

A concern is that our respondents vary in how much attention they pay to the survey, and that this might impact the reliability of our results. One way to test this possibility is to examine whether answers to factual questions about the political environment correlate with how long it takes the respondent to complete the survey. Respondents who take the survey more quickly may be giving random responses and thus may be more likely to get the factual questions incorrect. We test this possibility by correlating political knowledge with survey time completion. First, political knowledge is completely uncorrelated with length of survey completion. Second, respondents who took longer to answer the survey (above the mean length) were no different in political knowledge than those who did not take as long (below the mean length), as measured by a chi-square test. We find similar results between reported Wave 1 Muslim ban attitudes and survey length (χ² = 5.66, p = 0.22); Wave 2 Muslim ban attitudes and survey length (χ² = 4.373, p = 0.348); Wave 3 Muslim ban attitudes and survey length (χ² = 3.09, p = 0.541).

Support for the border wall appears to have dropped somewhat, but the difference is not statistically significant. That said, to the extent the drop may be realized with a larger dataset, the change is sensible given the continued discussion and criticism of the border wall throughout the year.

We are cognizant that self-reported media consumption may be subject to social desirability. We note here again that our share of respondents reporting they watched local or national news is about 78%, whereas the comparable number from Pew is 81%. This provides added confidence that our respondents are answering truthfully.

One may observe that the Republican coefficient slightly increases from wave 1 to wave 2, but this is due to a comparison of a slightly larger drop in ban support among independents (the comparison group).

https://​www.​vanityfair.​com/​news/​2017/​01/​republicans-trump-muslim-ban.

Our wave 2, but not wave 1, survey included measures of attention to news. We asked respondents if they “watched local or national television news” (1 = No, I have not done that, 2 = Yes, once or twice, 3 = Yes, several times), and whether they “read print or digital news stories” (1 = No, I have not done that, 2 = Yes, once or twice, 3 = Yes, several times). We combined answers to the two questions, then divided them into high attention (5 or 6 on the scale) and lower attention on the scale (2–4). It is possible that the protests stimulated citizens’ interest in the news and so this is not such a great baseline political attention measure. To attempt to compensate for this, we proxy for attention with education measured in wave 1.

Search terms: (1) democrat, democrats; (2) republican, republicans; (3) trump; (4) protest, protesters, protests, airport, airports; (5) graham, mccain; (6) schumer, pelosi; (7) American, unamerican, un-american, core values, religious freedom, religious test, liberty, violation, nation of immigrants.