The Effects of Questionnaire Length and Behavioral Consequences on Careless Responding
Abstract
Abstract. The current paper reports the results of two randomized experiments designed to test the effects of questionnaire length on careless responding (CR). Both experiments also examined whether the presence of a behavioral consequence (i.e., a reward or a punishment) designed to encourage careful responding buffers the effects of questionnaire length on CR. Collectively, our two studies found (a) some support for the main effect of questionnaire length, (b) consistent support for the main effect of the consequence manipulations, and (c) very limited support for the buffering effect of the consequence manipulations. Because the advancement of many subfields of psychology rests on the availability of high-quality self-report data, further research should examine the causes and prevention of CR.
References
2019). Is reliability compromised towards the end of long personality inventories? European Journal of Psychological Assessment, 35, 14–23. https://doi.org/10.1027/1015-5759/a000363
(2015). RStudio (Version 0.98.1103) [Function in RStudio]. Wright State University: Tyler Barnes.
(2008). MMPI-2-RF, Minnesota Multiphasic Personality Inventory-2 Restructured Form: Manual for administration, scoring and interpretation. Minneapolis, MN: University of Minnesota Press.
(1992). MMPI-2 random responding indices: Validation using a self-report methodology. Psychological Assessment, 4, 340–345. https://doi.org/10.1037/1040-3590.4.3.340
(2016). Who cares and who is careless? Insufficient effort responding as a reflection of respondent personality. Journal of Personality and Social Psychology, 111, 218–229. https://doi.org/10.1037/pspp0000085
(1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources.
(2010). Random responding as a threat to the validity of effect size estimates in correlational research. Educational and Psychological Measurement, 70, 596–612.
(2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19.
(2015). Best practice recommendations for data screening. Journal of Organizational Behavior, 36, 171–181.
(2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.
(1993). Boredom at work: A neglected concept. Human Relations, 46, 395–417.
(2006). Dropouts on the web: Effects of interest and burden experienced during an online survey. Journal of Official Statistics, 22, 313–328.
(2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73, 349–360.
(1999).
(A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models . In I. MervieldeI. DearyF. De FruytF. OstendorfEds., Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg, The Netherlands: Tilburg University Press.1996). California Psychological Inventory: Administrator’s guide (3rd ed.). Palo Alto, CA: Consulting Psychologists Press.
(2010). Ego depletion and the strength model of self-control: A meta-analysis. Psychological Bulletin, 136, 495–525.
(2005). Designing a formative measure for corporate reputation. Corporate Reputation Review, 8, 95–109.
(1981). Effects of questionnaire length on response quality. Public Opinion Quarterly, 45, 549–559.
(1985). Towards a model of boredom. British Journal of Psychology, 76, 235–240.
(1983).
(Fatigue . In G. R. J. HockeyEd., Stress and fatigue in human performance. Chichester, UK: Wiley.2015). Detecting insufficient effort responding with an infrequency scale: Evaluating validity and participant reactions. Journal of Business and Psychology, 30, 299–311.
(2012). Detecting and deterring insufficient effort respond to surveys. Journal of Business and Psychology, 27, 99–114.
(2015). Insufficient effort responding: Examining an insidious confound in survey data. Journal of Applied Psychology, 100, 828–845.
(1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213–236.
(2002). The impact of “no opinion” response options on data quality: Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66, 371–403.
(2014). Caring about carelessness: Participant inattention and its effects on research. Journal of Research in Personality, 48, 61–83. https://doi.org/10.1016/j.jrp.2013.09.008
(2012). Identifying careless responses in survey data. Psychological Methods, 17, 437–455. https://doi.org/10.1037/a0028085
(2007). Using multivariate statistics. Boston, MA: Allyn & Bacon.
(1999). Intrinsic and extrinsic motivation in Internet usage. Omega, 27, 25–37.
(2012). Distinguishing ordinal and disordinal interactions. Psychological Methods, 17, 615–622. https://doi.org/10.1037/a0030003
(