Ideology and Vote Choice in U.S. Mayoral Elections: Evidence from Facebook Surveys

For instance, Hajnal and Trounstine (2014) assemble perhaps the most comprehensive data set on local voter behavior by merging exit poll data from five cities, noting that their sample is simultaneously as “broad as possible” and yet not representative of “the entire urban arena” (69).

These figures are cited in Samuels and Zucco (2013, p. 5), and I have updated them by visiting http://​www.​socialbakers.​com/​statistics/​facebook/​ (figures current as of May 4, 2016).

In Brazil, in contrast, Samuels and Zucco (2013) report 40 million users out of a population of 200 million, or 20%.

See http://​www.​qualtrics.​com.

For the Nashville ad, subjects could intuit the institutional sponsor of the survey by reading the accompanying link.

Advertisers have the choice of paying Facebook per ad view (called impressions), or per click. In either case, Facebook presents advertisers with a range of possible prices (called bids) that might actually be paid, depending on the demand for ads. Following Samuels and Zucco (2013), I chose to pay per click, as the goal of my ad campaign was to have users click on the survey link. Actual costs per click for my surveys varied from $0.30 in Memphis, to $0.38 in Illinois, to $0.98 in Nashville. Actual costs per survey completion were $5.48 in Memphis, $4.45 in Nashville, and $1.31 in Illinois. The variation in cost per completion may be due to the vastly greater number of Facebook respondents in the suburban Illinois pool (5.9 million) versus in the Tennessee cities (about 300,000 each). I show screenshots of the ads used in Sect. 2 of the Online Appendix.

In Memphis, I restricted age to 18 and older. In Nashville, I restricted the population to 23 and older to avoid sampling college students. On the consent screen of each survey, respondents affirmed that they were eligible to vote in their respective mayoral election. For comparison, the Census estimates of the 18+ population for Memphis and the 23+ population in Nashville are 482,000 and 472,000, respectively.

I limited the audience to small cities by targeting based on zip codes. Details on the generation of small-city zip codes are provided in Sect. 5 of the Online Appendix.

I provide a tabular summary of the recruitment process in Sect. 1 of the Online Appendix. Survey break-off rates [partial responses/partial & complete responses; Callegaro and DiSogra 2008) were 50% in Memphis, 35% in Nashville, 32% in Illinois. These rates were not that out of line with median break-off rates found in meta-analyses of online surveys (Peytchev 2009, p. 75)], and were only slightly higher than the median break-off rate (27%) in my own analysis of eight studies I previously fielded via Survey Sampling International (detailed results available on request). However, break-off rates are significantly higher than surveys fielded by GFK (formerly Knowledge Networks); see, for example, Hainmueller and Hiscox (2010, p. 67) who report a break-off rate of 4.5%. Additionally, Peytchev (2009) reports break-off rates of 16 and 9% in two SSI samples fielded in 2003 and 2004. Section 3 of the Online Appendix tests whether break-off and attrition are correlated with any observable characteristics, or any particular moment in the survey. The major determinant of break-off appears to be fatigue, with most break-off occurring before around the tenth question. Those who start but fail to complete a single wave are slightly less likely to be Barry voters in Nashville, but there are no consistent predictors in the other two surveys. Those who fail to continue to the second wave in Nashville are slightly less likely to be Barry voters and slightly less likely to be white; while significant, these differences are substantively small.

Samuels and Zucco (2013) also note that Facebook surveys are amenable to quota sampling, and attempt to over-sample certain subpopulations in their second study. See Zhang et al. (2017) for an implementation of quota sampling in the United States.

The age densities from the Census are computed as the number of persons in each age group, divided by the total number of persons 18 and older (22 and older in Nashville).

I show full question wordings in Sect. 10 of the Online Appendix.

There are 98 respondents in the Memphis data in this figure, and 203 in the Nashville data. I use the responses from the full sample, including non-responses, later in the analysis.

Howard Gentry, an African American and Clerk of the City Criminal Court, received 11.6% of the vote in the general election. Dropping the 40 Gentry voters from the sample has no substantive impact on the estimates.

Respondents were offered the chance to win an additional $100 Amazon.com gift card for their participation in the second wave.

I also fielded a third wave in September, after the runoff election. Because there was little learning between the second and third wave, I omit this sample from the analysis. Results are substantively similar if I compare the first and third wave instead of the first and second wave. I provide details on the third wave, including sample size, in Sect. 1 of the Online Appendix. Sect. 6 of the Online Appendix presents an analysis of election-related press coverage in the 2015 election.

Among the full sample of wave 1 participants who answered the relevant items, wave 1 knowledge was 50%.

While a direct replication of the prior studies would be valuable, it would be logistically more difficult as small IL cities vary in election timing, and sampling from a particular small city would likely yield too few respondents.

In Illinois, small municipalities are known as cities or villages. Generally, the chief executive of a city is referred to as a mayor, while in a village she is referred to as the president. I show a screenshot of the conjoint experiment in Sect. 8 of the Online Appendix.

To be precise, each respondent was actually shown five pairs of candidates. Due to a programming error, only the attributes of the first two pairs were recorded in the survey data, and so only the responses from the first two pairs can be analyzed.

Formally, the measure of policy distance iswhere i indexes respondents, c indexes candidates, and \(x_{i}\) and \(x_{c}\) are the respondent’s and candidate’s views on raising property taxes, where \(x_{i}\in \{0=\text {supports cuts},.5=\text {supports maintaining current levels},1=\text {supports increases}\}\).

Section 9 of the Online Appendix replicates the analysis presented here using a measure of distance based on voter’s seven point ideology (making the assumption that those less than four on the scale are the same as voters who wish to raise taxes, those that are equal to four wish to maintain taxes, and those greater than four wish to cut taxes). The results are substantively very similar.

As in Hainmueller et al. (2014), this figure is constructed by estimating four regressions (one for each set of attributes) where vote choice is regressed on indicator variables for each attribute value. For instance, the first four estimates are from a regression of the form:where i indexes respondents and c indexes candidate pairs. The three \(\beta\) coefficient estimates and their 95% confidence intervals are then plotted in Fig. 5. As the attributes are independent of one another, there is no need to include attributes in a single regression; doing so, however, yields substantively similar results.