Is extreme response style domain specific? Findings from two studies in four countries

In terms of the terminology used, a cautionary note is warranted. In the context of their study, Weijters et al. (2008) use the term “response style indicator” to refer to an individual-level summary statistic denoting the number of times a focal response style occurs in a randomly drawn set of survey items (in their study: drawn from a broad variety of domains). However, as mentioned below, in our study every single survey item included in the random set (in this study: drawn from a more narrowly defined domain) is referred to as a response style indicator. As a consequence, in Weijters et al. (2008), a response style indicator represents a count variable (in our study a count variable would qualify as a “summary statistic”), whereas in our study a response style indicator can only take on a value of zero (i.e., response style is not present) or a value of one (i.e., response style is present), depending on what answer the respondent provided to the survey item under consideration.

For studies involving an assessment of response styles, Weijters et al. (2008) proposed a criterion of using 10–14 response style indicators to quantify summary statistics for response styles. Closer inspection of the empirical results from a sensitivity analysis on which this criterion has been proposed, clearly shows that this criterion applies primarily to response styles other than ERS (i.e., acquiescence, disacquiescence, and midpoint response style). The sensitivity analysis conducted by Weijters et al. (2008), showed that ERS is—by far—the most stable and reliable response style, and ERS measures tend to demonstrate strong construct reliability and reliable variance estimates with as few as five survey items used to compute an ERS summary statistic. Hence, our current use of eight survey items per summary statistic is consistent with response style measurement guidelines.

We used the random number generator in MS Excel to obtain 400 different ways of splitting the 16 ERS indicators into two halves forming the two ERS summary statistics; e.g., in one dataset, the first CB ERS summary statistic might consist of CB survey items 1, 4, 5, 7, 10, 11, 12, 16 while the other ERS summary statistic consists of CB survey items 2, 3, 6, 8, 9, 13, 14, 15.