Skip to main content

01.03.2011 | Original Paper

Considering Mixed Mode Surveys for Questions in Political Behavior: Using the Internet and Mail to Get Quality Data at Reasonable Costs

verfasst von: Lonna Rae Atkeson, Alex N. Adams, Lisa A. Bryant, Luciana Zilberman, Kyle L. Saunders

Erschienen in: Political Behavior | Ausgabe 1/2011

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Telephone surveys have been a principle means of learning about the attitudes and behaviors of citizens and voters. The single mode telephone survey, however, is increasingly threatened by rising costs, the declining use of landline telephones, and declining participation rates. One solution to these problems has been the introduction of mixed-mode surveys. However, such designs are relatively new and questions about their representativeness and the intricacies of the methodology remain. We report on the representativeness of a post election mixed-mode (Internet and mail) survey design of 2006 general election voters. We compare sample respondent means to sample frame means on key demographic characteristics and examine how mail and Internet respondents differed in terms of attitudes, behaviors and demographics. We find that overall the Internet respondents were representative of the population and that respondent choice of mode did not influence item response. We conclude that mixed-mode designs may allow researchers to ask important questions about political behavior from their desktops.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
Examples such as the American National Election Studies (ANES) and the General Social Survey (GSS) to the contrary.
 
3
Also see: http://​web.​mit.​edu/​polisci/​portl/​cces/​sampledesign.​html for a description of Polimetrix’s sampling method.
 
5
We thank New Mexico Secretary of State Rebecca Vigil-Giron and Colorado Secretary of State Gigi Dennis who provided us with the voter registration files for free.
 
6
A frequency report of the Election Administration Survey can be found at: www.​unm.​edu/​~atkeson.
 
7
We strongly recommend a FAQ because the phone calls about this issue stopped after we included it on the website.
 
8
Seventeen mail surveys were returned, all of them voters. In NM1 10% (n = 5) and in COCD1 about 5% (n = 3) of sample letters were returned, even though we had received no returned post cards from these addresses in previous contacts. Thus, lower mail delivery quality may be an important concern in terms of sample recruitment and in evaluating the quality of the sample. Therefore, when implementing this type of survey researchers should carefully monitor the returned mail and use first class mail to ensure return delivery.
 
9
Because of page constraints one question had to be moved elsewhere.
 
10
Many unreachable were inactive voters who are sent an official mailing from the County Clerk or the Secretary of State that is returned as undeliverable. This suggests that removing inactive registered voters from the population frame would be a prudent and cost effective research design measure in any future study iteration.
 
11
A partial survey is less than 50% of substantive questions completed with something other than don’t know or refused.
 
12
This is the minimum response rate (RR2) as defined by the AAPOR (AAPOR 2008).
 
13
The cooperation rate is the minimum cooperation rate (COOP2) by AAPOR (AAPOR 2008).
 
14
Our higher voter validation compared to the ANES may be due to the fact that the population frame is registered voters while the ANES population frame is citizens.
 
15
We note there may be some error in our matching. There is also likely some error in the voter registration files (Cassel 2004; Presser et al. 1990). For example, the official 2006 canvass for Bernalillo County reported 198,611 voters, but the voter record file we have indicated that only 194,582 voters cast ballots, a difference of 4,029 votes. We also know that when voters move their vote history follow them to their new voting location, but the voting location for those votes does not. Therefore, some errors are created in the way in which the file is administered. Voter files are not well characterized yet (but see McDonald 2007).
 
16
Because Colorado had very few Hispanic voters, we matched Hispanic surname with the New Mexico sample only (See Word and Perkins 1996).
 
17
Spanish language contact letters, postcards and survey were not included in our study. Theoretically this choice could hurt our response rate among Spanish-speaking sample members. In a 2008 phone study, we provided this option and 1.8% of sample respondents chose to take the survey in Spanish suggesting the problem is relatively small. In addition, 80% of Hispanics speak English at home or describe themselves as fluent in English in Bernalillo County (U.S. Census Bureau 2008). According to the U.S. Census Bureau, New Mexico Hispanics have the highest English fluency across the states. Thus, any impact is likely very small.
 
18
House campaign activities include: attending a rally, fundraising, canvassing, convincing others, writing a letter to a magazine/newspaper or Internet site, advertising (e.g. sign, bumper sticker, button), and contributing money. We also counted the number of ways respondents learned about the US House candidates in their district including: met him/her personally, attended a meeting or rally, contacted in-person by candidate, party or interest group, received an e-mail from candidate, party or interest groups, contacted by phone by candidate, party or interest group, visited candidate, party or interest group web site, or read online or local newspaper.
 
19
We substituted the income mean for system-missing codes because of the large number of missing data.
 
20
These include: No One Encouraged Respondent to Vote, Did Not Have to Show ID to Vote, Never Had a Problem with Voting and the Ideology of Democratic House Candidate, Trust in Government, Satisfaction with the 2006 Fall Election, and Political Discussion.
 
21
The quality of voter registration files varies by states and localities and this may influence the ability to adequately pursue this type of design.
 
Literatur
Zurück zum Zitat Atkeson, L. R., & Adams, A. N. (2010). A comparison of data quality in post election mixed mode and phone surveys. Presented at the Western Political Science Association’s Annual Meeting, April 1–3, San Francisco, California. Atkeson, L. R., & Adams, A. N. (2010). A comparison of data quality in post election mixed mode and phone surveys. Presented at the Western Political Science Association’s Annual Meeting, April 1–3, San Francisco, California.
Zurück zum Zitat Atkeson, L. R., & Tafoya, L. (2008a). Close, but not close enough: Democrats lose again by the slimmest of margins in New Mexico’s first congressional district. In D. Magleby & K. Patterson (Eds.), War games: Issues and resources in the battle for control of congress. Boulder: Paradigm Publishers. Atkeson, L. R., & Tafoya, L. (2008a). Close, but not close enough: Democrats lose again by the slimmest of margins in New Mexico’s first congressional district. In D. Magleby & K. Patterson (Eds.), War games: Issues and resources in the battle for control of congress. Boulder: Paradigm Publishers.
Zurück zum Zitat Atkeson, L. R., & Tafoya, L. (2008b). Surveying political activists: An examination of the effectiveness of a mixed-mode (Internet and mail) survey design. Journal of Elections Public Opinion and Parties, 18(4), 367–386.CrossRef Atkeson, L. R., & Tafoya, L. (2008b). Surveying political activists: An examination of the effectiveness of a mixed-mode (Internet and mail) survey design. Journal of Elections Public Opinion and Parties, 18(4), 367–386.CrossRef
Zurück zum Zitat Barreto, M. A., Streb, M. J., Marks, M., & Guerra, F. (2006). Do absentee voters differ from polling place voters? Public Opinion Quarterly, 70(2), 224–234.CrossRef Barreto, M. A., Streb, M. J., Marks, M., & Guerra, F. (2006). Do absentee voters differ from polling place voters? Public Opinion Quarterly, 70(2), 224–234.CrossRef
Zurück zum Zitat Belli, R. F., Traugott, M. W., & Beckmann, M. N. (2001). What leads to voting overreports? Contrasts of overreporters and admitted non voters in the American National Election Studies. Journal of Official Statistics, 17(4), 479–498. Belli, R. F., Traugott, M. W., & Beckmann, M. N. (2001). What leads to voting overreports? Contrasts of overreporters and admitted non voters in the American National Election Studies. Journal of Official Statistics, 17(4), 479–498.
Zurück zum Zitat Bullock III, C. S., Hood III, M. V., & Gonzalez, J. (2007). Examining methods for identifying Latino voters. Election Law Journal, 6(2), 202–208.CrossRef Bullock III, C. S., Hood III, M. V., & Gonzalez, J. (2007). Examining methods for identifying Latino voters. Election Law Journal, 6(2), 202–208.CrossRef
Zurück zum Zitat Cassel, C. A. (2004). Voting records and validated voting studies. Public Opinion Quarterly, 68(1), 102–108.CrossRef Cassel, C. A. (2004). Voting records and validated voting studies. Public Opinion Quarterly, 68(1), 102–108.CrossRef
Zurück zum Zitat Chang, L. C., & Krosnick, J. (2010). National Surveys via RDD telephone interviewing versus the Internet: Comparing sample representativeness and response quality. Public Opinion Quarterly, 73(4), 641–678.CrossRef Chang, L. C., & Krosnick, J. (2010). National Surveys via RDD telephone interviewing versus the Internet: Comparing sample representativeness and response quality. Public Opinion Quarterly, 73(4), 641–678.CrossRef
Zurück zum Zitat Christian, L. M., & Dillman, D. A. (2004). The influence of graphical and symbolic language manipulations on responses to self-administered questions. Public Opinion Quarterly, 68(1), 57–80.CrossRef Christian, L. M., & Dillman, D. A. (2004). The influence of graphical and symbolic language manipulations on responses to self-administered questions. Public Opinion Quarterly, 68(1), 57–80.CrossRef
Zurück zum Zitat Cobanoglu, C., Warde, B., & Morec, P. J. (2001). A comparison of mail, fax, and web survey methods. International Journal of Market Research, 43, 441–452. Cobanoglu, C., Warde, B., & Morec, P. J. (2001). A comparison of mail, fax, and web survey methods. International Journal of Market Research, 43, 441–452.
Zurück zum Zitat Curtin, R., Presser, S., & Singer, E. (2000). The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly, 64, 413–428.CrossRef Curtin, R., Presser, S., & Singer, E. (2000). The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly, 64, 413–428.CrossRef
Zurück zum Zitat De Leeuw, E. (2005). To mix or not to mix: Data collection modes in surveys. Journal of Official Statistics, 21, 233–255. De Leeuw, E. (2005). To mix or not to mix: Data collection modes in surveys. Journal of Official Statistics, 21, 233–255.
Zurück zum Zitat De Leeuw, E., & Van der Zouwen J. (1988). Data quality in telephone and face to face surveys: A comparative meta-analysis. In R. Groves, P. P. Bimer, L. Lyberg, I. T. Massey, W. L. Nicholls, & J. Waksberg. Telephone survey methodology. New York: Wiley. De Leeuw, E., & Van der Zouwen J. (1988). Data quality in telephone and face to face surveys: A comparative meta-analysis. In R. Groves, P. P. Bimer, L. Lyberg, I. T. Massey, W. L. Nicholls, & J. Waksberg. Telephone survey methodology. New York: Wiley.
Zurück zum Zitat Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method (2nd ed.). New York: Wiley. Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method (2nd ed.). New York: Wiley.
Zurück zum Zitat Dillman, D. A., & Christian, L. M. (2005). Survey mode as a source of instability in responses across surveys. Field Methods, 17(1), 30–52.CrossRef Dillman, D. A., & Christian, L. M. (2005). Survey mode as a source of instability in responses across surveys. Field Methods, 17(1), 30–52.CrossRef
Zurück zum Zitat Dillman, D. A., R. Sangster, J. Tarnai, & T. Rockwood (1996). In M. T. Braverman, J. K. Slater (Eds.), Current issues in survey research: New directions for evaluation, #70 (Chap. 4, pp. 45–62). San Francisco: Jossey-Bass. Dillman, D. A., R. Sangster, J. Tarnai, & T. Rockwood (1996). In M. T. Braverman, J. K. Slater (Eds.), Current issues in survey research: New directions for evaluation, #70 (Chap. 4, pp. 45–62). San Francisco: Jossey-Bass.
Zurück zum Zitat Fowler, F. J., Roman, A. M., Jr., & Di, Z. X. (1998). Mode effects in a survey of medicare prostate surgery patients. Public Opinion Quarterly, 62(1), 29–46.CrossRef Fowler, F. J., Roman, A. M., Jr., & Di, Z. X. (1998). Mode effects in a survey of medicare prostate surgery patients. Public Opinion Quarterly, 62(1), 29–46.CrossRef
Zurück zum Zitat Groves, R. M., & Couper, M. P. (1998). Non-response in Household Interview Surveys. New York: Wiley. Groves, R. M., & Couper, M. P. (1998). Non-response in Household Interview Surveys. New York: Wiley.
Zurück zum Zitat Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68(1), 2–31.CrossRef Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68(1), 2–31.CrossRef
Zurück zum Zitat Keeter, S., Miller, C., Kohut, A., Groves, R. M., & Presser, S. (2000). Consequences of reducing non-response in a national telephone survey. Public Opinion Quarterly, 64, 125–148.CrossRef Keeter, S., Miller, C., Kohut, A., Groves, R. M., & Presser, S. (2000). Consequences of reducing non-response in a national telephone survey. Public Opinion Quarterly, 64, 125–148.CrossRef
Zurück zum Zitat Lensvelt-Mulders, G. J. L. M., P. Lugtig, & M. Hubregtse (2009). Separating selection bias and non-coverage in Internet panels using propensity matching. Survey Practice. http://surveypractice.org/. Lensvelt-Mulders, G. J. L. M., P. Lugtig, & M. Hubregtse (2009). Separating selection bias and non-coverage in Internet panels using propensity matching. Survey Practice. http://​surveypractice.​org/​.
Zurück zum Zitat Malhotra, N., & Krosnick, J. A. (2007). The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to Internet surveys with nonprobability samples. Political Analysis, 15, 286–323.CrossRef Malhotra, N., & Krosnick, J. A. (2007). The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to Internet surveys with nonprobability samples. Political Analysis, 15, 286–323.CrossRef
Zurück zum Zitat McDermott, M. L. (1998). Race and gender cues in low-information elections. Political Research Quarterly, 51(4), 895–918. McDermott, M. L. (1998). Race and gender cues in low-information elections. Political Research Quarterly, 51(4), 895–918.
Zurück zum Zitat McDonald, M. P. (2007). The true electorate: A cross validation of voter registration files and election survey demographics. Public Opinion Quarterly, 71(4), 588–602.CrossRef McDonald, M. P. (2007). The true electorate: A cross validation of voter registration files and election survey demographics. Public Opinion Quarterly, 71(4), 588–602.CrossRef
Zurück zum Zitat Merkle, D. M., & Edelman M. (2002). Non-response in exit polls: A comprehensive analysis. In R. M. Groves, D. A. Dillman, J. L. Eltinge, R. J. A. Little (Eds.), Survey non-response (pp. 243–258). New York: Wiley. Merkle, D. M., & Edelman M. (2002). Non-response in exit polls: A comprehensive analysis. In R. M. Groves, D. A. Dillman, J. L. Eltinge, R. J. A. Little (Eds.), Survey non-response (pp. 243–258). New York: Wiley.
Zurück zum Zitat Presser, S., Traugott, M. W., & Traugott, S. (1990). Vote ‘Over’ reporting in surveys: The records or the respondents. Technical Report No. 39. Ann Arbor, MI: National Election Studies. Presser, S., Traugott, M. W., & Traugott, S. (1990). Vote ‘Over’ reporting in surveys: The records or the respondents. Technical Report No. 39. Ann Arbor, MI: National Election Studies.
Zurück zum Zitat Rosen, R., & Gomes, T. (2004). Converting CES reporters from TDE to web data collection. In Paper presented at the Joint Statistical Meetings, Toronto, Canada. Rosen, R., & Gomes, T. (2004). Converting CES reporters from TDE to web data collection. In Paper presented at the Joint Statistical Meetings, Toronto, Canada.
Zurück zum Zitat Sanders, D., Clarke, H. D., Stewart, M. C., & Whitely, P. (2007). Does mode matter for modeling political choice? Evidence from the 2005 British Election Study. Political Analysis, 15, 257–286.CrossRef Sanders, D., Clarke, H. D., Stewart, M. C., & Whitely, P. (2007). Does mode matter for modeling political choice? Evidence from the 2005 British Election Study. Political Analysis, 15, 257–286.CrossRef
Zurück zum Zitat Schuman, H. (1992). Context effects: State of the past/state of the art. In N. Schwarz, S. Sudman (Eds.), Context effects in social and psychological research (pp. 5–20). New York: Springer-Verlag. Schuman, H. (1992). Context effects: State of the past/state of the art. In N. Schwarz, S. Sudman (Eds.), Context effects in social and psychological research (pp. 5–20). New York: Springer-Verlag.
Zurück zum Zitat Schuman, H., & Presser, S. (1981). Questions and answers in attitude survey: Experiments on question form, wording and context. New York: Academic Press. Schuman, H., & Presser, S. (1981). Questions and answers in attitude survey: Experiments on question form, wording and context. New York: Academic Press.
Zurück zum Zitat Smyth, J. D., Dillman, D., Christian, L. M., & Stern, M. J. (2006). Effects of using visual design principles to group response options in web survey. International Journal of Internet Science, 1, 6–16. Smyth, J. D., Dillman, D., Christian, L. M., & Stern, M. J. (2006). Effects of using visual design principles to group response options in web survey. International Journal of Internet Science, 1, 6–16.
Zurück zum Zitat Sudman, S., Bradburn, N. M., & Schwarz, Norbert. (1996). Thinking about answers. San Francisco: Josey-Bass. Sudman, S., Bradburn, N. M., & Schwarz, Norbert. (1996). Thinking about answers. San Francisco: Josey-Bass.
Zurück zum Zitat Tourangeau, R., Couper, M., & Conrad, F. (2004). Spacing, position and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68(3), 368–393.CrossRef Tourangeau, R., Couper, M., & Conrad, F. (2004). Spacing, position and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68(3), 368–393.CrossRef
Zurück zum Zitat Traugott, S. (1989). Validating self-reported vote: 1964–1988. ANES technical report series, No. nes010152. Ann Arbor Michigan: National Election Studies. Traugott, S. (1989). Validating self-reported vote: 1964–1988. ANES technical report series, No. nes010152. Ann Arbor Michigan: National Election Studies.
Zurück zum Zitat U.S. Census Bureau. (2008). American Community Survey 2006–2008 three year estimates. Data profile: New Mexico: Bernalillo County: Hispanics (All). Accessed 15 Jan 2010. U.S. Census Bureau. (2008). American Community Survey 20062008 three year estimates. Data profile: New Mexico: Bernalillo County: Hispanics (All). Accessed 15 Jan 2010.
Zurück zum Zitat Visser, P. S., Krosnick, J. A., Marquette, J., & Curtin, M. (1996). Mail surveys for election forecasting? An evaluation of the Columbus dispatch poll. Public Opinion Quarterly, 60(2), 181–227.CrossRef Visser, P. S., Krosnick, J. A., Marquette, J., & Curtin, M. (1996). Mail surveys for election forecasting? An evaluation of the Columbus dispatch poll. Public Opinion Quarterly, 60(2), 181–227.CrossRef
Metadaten
Titel
Considering Mixed Mode Surveys for Questions in Political Behavior: Using the Internet and Mail to Get Quality Data at Reasonable Costs
verfasst von
Lonna Rae Atkeson
Alex N. Adams
Lisa A. Bryant
Luciana Zilberman
Kyle L. Saunders
Publikationsdatum
01.03.2011
Verlag
Springer US
Erschienen in
Political Behavior / Ausgabe 1/2011
Print ISSN: 0190-9320
Elektronische ISSN: 1573-6687
DOI
https://doi.org/10.1007/s11109-010-9121-1

Premium Partner