Do politicians use the representativeness heuristic when making judgements, that is, when they appraise the likelihood or frequency of an outcome that is unknown or unknowable? Heuristics are cognitive shortcuts that facilitate judgements and decision making. Oftentimes, heuristics are useful, but they may also lead to systematic biases that can be detrimental for decision making in a representative democracy. Thus far, we lack experimental evidence on whether politicians use the representativeness heuristic. To contribute to and extend the existing literature, we develop and conduct a survey experiment with as main participants Dutch elected local politicians from the larger municipalities (n = 211). This survey experiment examines whether politician participants display two decision-making biases related to the representativeness heuristic: the conjunction error and scope neglect. We also run the experiment with a student sample (n = 260), mainly to validate the experimental design. Our findings show that politician participants neglect scope in one scenario and that they display the conjunction error in two of three scenarios. These results suggest that politician participants use the representativeness heuristic. Conversely, our third conjunction error scenario does not find evidence for politician participants displaying this bias. As we discuss in the article, the latter may be an artifact of our experimental design. Overall, our findings contribute fundamentally to our understanding of how politicians process information and how this influences their judgements and decision making.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
Politicians—such as members of parliament, ministers, or presidents—often have to make judgements, that is ‘appraisals about the likelihood, probability or frequency of an outcome which is unknown or unknowable’ (McDermott 2001: 7). Judgements typically take place prior to a decision, although they do not necessarily lead to one. They are assessments about external events and usually occur under conditions of uncertainty (McDermott 2001: 7). As Hammond (1996: 5) states, ‘(h)uman judgment is the hidden, mysterious link in the process that forms the policies and plans that directly affect, if not control, the nature of our society, as well as its interaction with other societies’. There is a large literature that shows that in making judgements, people regularly apply heuristics: cognitive shortcuts that facilitate judgement and decision making (see e.g., Gilovich et al. 2002). Using heuristics may, however, also lead to systematic biases, that is deviations from comprehensive rationality or normative rational choice theory (Gilovich and Griffin 2002).
In this paper, we concentrate on a heuristic that is particularly relevant for political judgement and decision making: representativeness. The representativeness heuristic is one of the so-called general purpose heuristics from Kahneman and Tversky’s heuristics and biases tradition (for an overview, see Gilovich et al. 2002; for a discussion of this tradition and the so-called fast & frugal one, see Kelman 2011). The representativeness heuristic is at work when people judge probabilities ‘by the degree to which A is representative of B, that is, by the degree to which A resembles B’ (Tversky and Kahneman 1974: 1124). Oftentimes, representative events will be frequent events, but ‘they may better be thought of as ones that fit a stereotype or as diagnostic of group membership’ (Kelman 2011: 23). The second general purpose heuristic—that we do not focus on here, but that we also examined in our survey experiment—is availability (Tversky and Kahneman 1974). People employ the availability heuristic when they assess how likely it is that something occurs by focusing on the ease with which they can think of instances or occurrences of it. The availability heuristic can lead to biases because salient and attention-grabbing events will more readily come to mind (such as political scandals) or because a dramatic event (like a plane crash) temporarily increases the availability of its category (Kahneman 2011). Both representativeness and availability thus answer the difficult question of probability with the answer to the simpler question of resemblance (Kahneman and Frederick 2002: 50). The difference between the two is that representativeness works through resemblance to for instance a prototype or stereotype, whereas availability works through resemblance to specific instances.
Advertisement
What are the effects of the representativeness heuristic on the functioning of democracy? Drawing on similarity between the instance at hand and a general category—which is what people do when using the representativeness heuristic—facilitates judgements and decision making. This makes it easier for politicians to do their job, which may be good for the functioning of democracy. However, drawing on such similarity may also lead to biases (Tversky and Kahneman 1974). Such biases occur, for example, when politicians judge a proposal’s likelihood of success by thinking only of successful proposals, thereby overestimating the proposal’s likely success. What is more, politicians who think negatively about whole groups of people—say, Muslims—because of the activities of some of them rely on the representativeness heuristic (Vis 2019). In this latter case, bias occurs because politicians may take policy measures that (negatively) affect the whole group, while only a few of them are in fact problematic. Since politicians’ judgements and subsequent decisions have far-reaching consequences for the lives of many people, it is important to investigate how politicians arrive at them. In this article, we therefore ask: Do politicians use the representativeness heuristic when making judgements and taking decisions? There are several observational studies suggesting that they do (Böhmelt et al. 2016; Vis 2019; Weyland 2007, 2014, see below), but these studies do not offer a systematic experimental test. What is more, the incentives for politicians to “get it right” are high, indicating that they may process information systematically instead of heuristically. There is evidence suggesting that experienced elites revise their heuristics more efficiently (Alevy et al. 2007; Chi 1978; Feltovich et al. 2006). To date, existing studies do not offer conclusive evidence on these largely conflicting expectations.
We fill this lacuna by developing and conducting a survey experiment with as main participants Dutch local elected politicians from the larger municipalities (between 100 and 230 k inhabitants). We also run the experiment on a student sample,1 primarily to validate our experimental design; as we discuss in more detail below, we do not expect any substantive differences between the politician and student participants. Like their national counterparts, politicians from the larger municipalities are selected competitively and make decisions that influence the lives of large numbers of citizens. Similar to many of their national counterparts, many local politicians operate under (severe) time pressure (Pommer and Boelhouwer 2017: 397). The Netherlands is a relevant context because powers have increasingly shifted from the national to the local level. For example, child protection, labor market policies, chronic health care and care for the elderly are decided upon by local politicians since 2015. The annual budget of a typical municipality included in our sample frame, Apeldoorn, amounts to about 600 million euros (Apeldoorn 2016), indicating that these politicians preside over significant public funds.
By studying the potential biases that result from the representativeness heuristic, we contribute to the growing literature on the role of biases in political decision making. Keller and Yang (2016), for example, show that initial problem representation shapes foreign policy decision making processes. Since the representativeness heuristic yields the associations constituting such initial representations and given that option generation is also important in many other policy domains, Keller and Yang’s findings suggest how the representativeness heuristic may influence political decision making. Moreover, Baekgaard et al. (2019) show that politicians are just as likely as non-politicians to rely on motivated reasoning when facing new information. And Pedersen (2017) demonstrates how ratio bias—that is, people’s tendency to focus on numerators and pay insufficient attention to denominators—may influence citizens’ interpretations of numbers in political issues.
We complement and extend the existing literature by examining whether politicians display the biases related to the representativeness heuristic by adjusting existing scenarios to a contemporary and novel context. The representativeness heuristic has wide applicability to many domains of decision making, extending beyond the specific biases we investigate here (see e.g. Kahneman 2011; Kelman 2011). In addition, the representativeness heuristic’s foundation offers a concrete theoretical starting point for how to address these biases and potentially improve political decision making.2 Overall, our findings thus contribute fundamentally to our understanding of how politicians process information and how this influences their judgements and decision making.
Advertisement
How Do Politicians Process Information?
Do politicians use the representativeness heuristic when processing information? The relatively scant existing empirical evidence typically suggests that yes, they do (Weyland 2007, 2014; see Vis 2019). Weyland (2007), for example, demonstrated that key policy makers’ use of the representativeness heuristic explains why so many countries in Latin America followed Chile’s bold pension reform; they saw the Chilean success as representative of a larger set of reform successes (see also Weyland 2008, 2014). Conversely, in their quantitative analysis of how party policy positions diffuse across countries, Böhmelt et al. (2016) did not find support for the representativeness heuristic.3 Notwithstanding the latter finding, existing studies typically reveal that if politicians differ from ordinary citizens in relying on heuristics, this is mainly a difference in degree not in kind. Also politicians are “normal mortals” (Weyland 2014: 58, see p. 54) who resort to heuristic-processing when the environment is highly uncertain and complex (see also Jacobs 2011: 41–42). And uncertainty and complexity is precisely what characterizes many political judgements and decisions (Bursens et al. 2017). Therefore, we do not expect any substantive differences between the politician and student participants.
Nonetheless, there are indications that politicians might process information differently, contra our expectation. Politicians are typically more experienced than are ordinary citizens, and experience usually results in expertise (Hafner-Burton et al. 2013). This could make politicians—including local ones—less susceptible to the biases related to the representativeness heuristic. However, experience in one specific domain does not automatically improve decision making in another domain, even if it is closely related (Hafner-Burton et al. 2013). Since the range of judgements and decisions that politicians typically have to make is wide, this might limit the effect of experience. Most politicians in our sample are no expert in the specific domains of our survey. Tetlock (2017) furthermore demonstrated that people who are experts in making political judgement generally do not better—and oftentimes worse—than do non-experts. This means that it is unclear whether experience will lead politicians to rely less on the representativeness heuristic.
Another potential reason why politicians might differ in their reliance on heuristics is that the amount of information politicians receive is much larger than the amount of information ordinary citizens receive (Walgrave et al. 2013). The former typically have more resources available to process this information, for instance in the form of support staff. Much of the information politicians receive is also pre-structured by the institutional structures (Baumgartner and Jones 2015). Moreover, politicians’ information processing has consequences for themselves (e.g., their political career) and for citizens, which may make them more motivated to use the information in the best way possible (Anderson and Harbridge 2014; Jacobs 2011; Jones 2001; Sheffer et al. 2018; Weyland 2014). The upshot of this is that, at least under some conditions and contra our main expectation, politicians may not use the representativeness heuristic in their judgement and decision making. We extend the existing literature’s findings by assessing these–partly–conflicting expectations directly and experimentally.
Experimental Design and Hypotheses
To assess whether politicians use the representativeness heuristic, we follow most extant work and Tversky and Kahneman’s own approach and infer heuristics ‘by measuring the biases that their use is supposed to evoke’ (Bellur and Sundar 2014: 121).4 In the case of the representativeness heuristic, these biases are the conjunction error and scope neglect (discussed in more detail below). To this end, we adjust four existing tests so that they fit the contemporary Dutch context (see Online Appendix 1 for a description of our four-stage procedure, which included pre-registration of the larger project from which the data for this article is drawn5; see Online Appendix 6 for an overview of the full set of hypotheses included in the pre-registration plan; see Online Appendix 3 for the text of the survey experiment). We invited the politician participants in their role as municipal council member (see Online Appendix 2a for the text of the invitation email).6 Most of our scenarios are closer to the judgements and decisions local politicians truly make than really abstract scenarios would be. Still, also these scenarios remain a simplification of reality and are not intended to be truly mundane (Iyengar 2011).
We also include a number of questions in our survey to capture the participants’ background characteristics, particularly gender, level of education, expertise (measured by being a spokesperson on a for this survey experiment relevant area, like safety or public management), experience (measured by among other questions how long someone has been a local council member), self-reported math skills, and self-placement on a left–right scale. The relevant questions can be found in Online Appendix 3.7
Conjunction Error I: The Linda/Vera Problem
A first bias related to the representativeness heuristic is the conjunction error. People make a conjunction error when they consider the conjunction A ⋀ B—with the “˄” indicating logical-AND—more likely than, for instance, A. The latter is logically speaking at least as large, and probably larger, than the conjunction A ⋀ B. To test whether the politician participants make a conjunction error, we adapt the classic test for this bias, the so-called Linda problem (Tversky and Kahneman 1983), to the contemporary Dutch context. In the original test, Linda is described as a strong-willed woman who was stereotypically related to someone active in the feminist movement. Based on this description, 85 per cent of Tversky and Kahneman’s (1983) respondents judged it more likely that Linda was both a bank-teller and active in the feminist movement than only a bank-teller.8 As Lu (2016: 510) states, ‘the accumulated evidence (…) has suggested that the violation [i.e. the conjunction error] is highly robust to variations in response modes and is very easy to replicate in a variety of contexts’.
While the Linda problem differs from the type of judgements local politicians truly make, we include it as a benchmark for the other, more mundane, scenarios. If we find that politician participants display the conjunction error in this scenario, in which they are arguably least likely to respond in their role as politicians, this means that the sample is not a priori different from samples that have been used previously. What is more, since this test is often replicated, it also provides a benchmark for the student sample. If we would not find the conjunction error in this latter sample, this would call into question the student sample’s quality. In addition to this more methodological justification for including this scenario, the Linda-problem is also substantively relevant for politics. When making a conjunction error, people make a judgement that is logically incorrect by (over-)relying on a stereotype (for more on the relation between the representativeness heuristic and stereotypes, see Gilovich and Griffin 2002). This means that the resulting decision may be sub-optimal from the standpoint of means–goal efficiency. A hypothetical, yet realistic, example is when politicians (or voters) consider it more likely that people who cause a nuisance in a municipality (such as drugs dealers, or some addicts or homeless people)9 have a low socioeconomic status and a migrant background (A ⋀ B) than that they have a low socioeconomic status only (A). Politicians may then spend more resources to address A ⋀ B (or that voters ask of them to spend more resources on this), whereas A truly is the larger problem.10 Indeed, this example reveals that stereotypes are often at the forefront of many political issues (for instance ethnicity or gender). When such stereotypes trigger biases, this might lower the quality of the subsequent political decision, that is, this decision might not be in line with the politician’s stated goals and/or values (for examples on how stereotypes influence political decision making by voters, see Arceneaux 2008; Herrmann and Tepe 2017). To avoid any potential confusion with a popular Dutch TV-host named Linda, we used the name Vera. In line with earlier findings on ordinary citizens (see above), we expect that also politician participants make the conjunction error, indicating that they use the representativeness heuristic:
H1
Politician participants will generally judge the conjunction (“Vera is a bank employee and is active in the feminist movement”) more likely than one or both of its parts (“Vera is a bank employee” or “Vera is active in the feminist movement”).11
Conjunction Error II: Making the Headlines Scenario
A second scenario to assess whether the politician participants display biases related to the representativeness heuristic is the causal conjunction test (Tversky and Kahneman 1983). In the original scenario, different groups of participants were asked to rate the likelihood of a suspension of diplomatic relations between the USA and the USSR sometime in 1983, compared to the likelihood of that same suspension but then due to a Russian invasion of Poland. If people make a causal conjunction error, the adding of a probable cause (Russia’s invasion of Poland) to an unlikely event (the suspension of diplomatic relations) makes this event appear more likely. Note that adding a specific cause decreases the logical odds, since other causes are excluded. But a combination of cause and event is more representative of something that might happen than the event by itself.
Our adjusted scenario is closer to home for the local politician participants than diplomatic relations would be. One group of participants rates the likelihood of the event that their municipality makes the headlines of all major newspapers next year. The other group rates the same likelihood but will be giving an exact cause for this event: “due to a terrorist attack on King’s day”—a public holiday in the Netherlands at which all municipalities have activities in the public space. Adding this cause makes the “making the headlines” event more representative of something that might happen. In 2009, a terrorist attack happened in Apeldoorn on Queen’s day,12 the then host of the Queen and her family. A terrorist drove into the crowd toward an open bus carrying the royal family. Eight people were killed because of the attack, putting a dark mark on this public holiday. This incident was covered by many major international news agencies. This was such a salient event, that most people who were of age in 2009 still remember it. For most of our local politicians, the scenario is thus—unfortunately—representative. It could also be representative for the (much) younger students, because every year around King’s Day, the media discusses safety, sometimes referring to Apeldoorn 2009. Both politicians and students may also think about the attacks at Christmas markets in Berlin, Germany in 2016 and Straatsburg, France in 2018—both of which are relatively close to the Netherlands and have been widely covered in Dutch media. This leads to the causal conjunction error hypothesis (H2):
H2
Politician participants will judge it more likely that a terrorist attack will lead to their municipality making the headlines of all major newspapers, compared to making those headlines in general.13
The causal conjunction error has important implications for politics. Most importantly, if at work, it privileges outcomes of conjunctive scenarios in which the causal arguments are apparent—such as chemical or biological warfare—over outcomes that emerge from causally less clear sequences of events—such as accidents or mistakes (McDermott 2001, pp. 10–12; see also Norman and Delfin 2012).
Conjunction Error III: The Earthquake Scenario
Our third scenario also tests a variant of a conjunction error, in this case related to scope neglect, i.e. people’s tendency to neglect a representative event’s scope. The scenario is based on Tversky and Kahneman’s (1983) California earthquake scenario in which one group of participants assesses the likelihood of an earthquake hitting California next year and causing a massive flood, while the other group assesses the likelihood of a massive flood somewhere in North America next year. Participants judged the, representative, image of the earthquake and flood in California as more likely than a flood somewhere in North America, even though California is part of North America, and floods can originate from different causes.
We adjusted this scenario into the earthquake scenario. The damage caused by earthquakes in Groningen (a province in the North of the Netherlands) is currently an important theme in Dutch politics. At the same time, natural disasters like earthquakes are generally rare in the Netherlands. We therefore expect that Dutch citizens—politicians and students alike—will view Groningen more representative of earthquakes than is the eastern half of the Netherlands as a whole of natural disasters. However, Groningen is geographically much smaller than is the much larger eastern half of the Netherlands. Comparing participants’ assessment of the likelihood on an earthquake versus a natural disaster thus constitutes a test similar to the California earthquake scenario. If our participants neglect scope, which we expect them to, the following hypothesis (H3) holds:
H3
Politician participants will judge the likelihood of an earthquake in Groningen to be higher than of a natural disaster in the eastern half of the Netherlands.14
Also scope neglect has important political implications. If politicians consider a specific event (like an earthquake hitting California and causing a massive flood) more likely than they would a more general event (flood in North America), it means that they concentrate on the more specific event and will probably be willing to spend more resources to avoid this specific event than they would to avoid the more general event. This could result in policy underreaction to the more general problem (Maor 2014).
Scope Neglect I & II: The Nuisance Scenario
Our fourth and final scenario also assesses whether politician participants neglect scope. Scope neglect may result from a conjunction error, but also from other applications of representative prototypes, as is the case in this scenario (see Kahneman and Frederick 2002). Scope neglect is thus slightly different from, but strongly related to the use of the representativeness heuristic. Politicians deal with scope both in their estimates of the likelihood of events (as in the previous scenario) as well as in their decisions on, for instance, budget allocation. Scope neglect may thus influence how much of the budget politicians are willing to spend on a problem. If politicians would neglect scope, this means that they either under- or overestimate the severity of problem. This, in turn, may result in policy underreaction (Maor 2014) or policy overreaction (Maor 2013). We adjust a scenario from a study that found that Toronto residents were willing to pay the same amount to clean up polluted lakes in a small region of Ontario as they were to clean up lakes throughout—the larger region of—Ontario (Kahneman and Knetsch 1992; see Desvousges et al. 2012; Kahneman and Frederick 2002). This finding indicates that the problem’s scope did not relate to their willingness to pay.
To examine whether politicians are sensitive to this type of scope neglect, we let one group of participants judge the importance of the problem of 23 people who cause a nuisance in their municipality, while the other group judges the importance of 53 people who cause a nuisance. To obtain the numbers of people causing a nuisance that in this context would be qualitatively different, we consulted an expert on policing and Dutch municipalities who advised using 23 and 53 because these represent categorically different sizes (an average number of people who cause a nuisance for a municipality this size versus record numbers of people who cause a nuisance for a municipality this size). We restricted the politician sample to municipalities of a comparable size (see below). If participants—politicians and students alike—appraise the problem via representative prototypes, they think of how important in general they find it to deal with people who cause a nuisance, so whether they are asked about 23 or 53 such people should not matter. This yields the following hypothesis (H4):
H4
Politician participants will generally consider it equally important to deal with 23 people who cause a nuisance as with 53 ones.15
Moreover, and more directly comparable to Kahneman and Knetsch’s (1992) original scenario, if participants neglect scope, they would allocate the same budget to a group of 23 and to a group of 53 people who cause a nuisance.
H5
Politician participants will generally allocate an equal budget to deal with 23 people who cause a nuisance as they do to deal with 53 ones.16
Data
Our sample frame of elected politicians17 from the 27 larger Dutch municipalities is 1063 and excludes the four largest municipalities (≈ 838 k, ≈ 631 k, ≈ 520 k and ≈ 340 k inhabitants). Note that we do not expect any differences across these largest four municipalities’ politicians and the ones we focus on here in how they make judgements. However, politicians in the largest four municipalities probably face qualitatively different problems, which may influence their responses. We send out invitation emails first through each municipality’s official channel (the “griffier”), followed by two reminders to each politician’s official e-mail address. This resulted in 211 completed surveys (from 264 registered (partial) responses, ≈ 20% response rate). Item non-response is low, as we show in Online Appendix 8.
Table S4 in Online Appendix 7 displays the politician sample’s descriptives. This sample is representative of the full population of Dutch elected council members (all municipalities, n = 8931) in terms of age—average 52.7 years—and gender—28 per cent female. Our sample is somewhat more highly educated; in our sample, almost 90 per cent self-report holding an applied higher college or university degree compared to 67 per cent on average for the full population (Ministerie van Binnenlandse Zaken en Koninkrijksrelaties 2016). Also in terms of party membership, our sample is largely representative of the full population of council members (see Table S3 in Online Appendix 7). Only council members from local parties are somewhat underrepresented in our sample (18.5 per cent versus 27.8 per cent in the population). Randomization tests indicate that gender, age, municipality, party and education level were not significantly different between the different conditions (see Table S9 in Online Appendix 12).
Online Appendix 5 presents information on recruitment, response rates and characteristics of the student sample. Also for the student sample, item non-response is low (see Online Appendix 8). Many student participants live in one of the larger municipalities that we exclude in our sample of municipalities. To make the student participants’ results as comparable as possible to the politician participants’, we ask the student participants in the nuisance scenario to imagine being a municipal council member of Apeldoorn. In the making the headlines scenario, we also ask the student participants to think of Apeldoorn, but not as a municipal council member (see Online Appendix 3 for the question wording in the survey).
Results
Conjunction Error I: The Linda/Vera Problem
Recall that we asked the participants to judge the likelihood that Vera was a bank employee, active in the feminist movement, or both. In line with the conjunction error hypothesis (H1), we found that 134 politicians (72%, n = 186)18 committed the conjunction error by ranking the likelihood of Vera being both active in the feminist movement and a bank employee to be higher than one of its constituents, see Fig. 1. The results for the student sample were very similar to those of the politician sample (see Table S8 and Figure S1 in Online Appendix 11).
Fig. 1
Frequency of politician participants committing the conjunction error (n = 186)
×
Table 1 shows that 56 politician participants (30 per cent) choose order (b)—the stereotypical ordering of Vera being active in the feminist movement as most likely, followed by a bank employee who is active in the feminist movement, and her being a bank employee as least likely. The reverse ordering—order (f)—was chosen by 41 politician participants (22 per cent).19 The two orderings that do not imply the conjunction error—orderings c and e—, were chosen by 28 politician participants (15 per cent) and 24 (13 per cent), respectively. These results confirm H1 that politician participants make the conjunction error. Still, the findings also show that they did not implement stereotypical/representativeness reasoning in a 1:1 fashion: its reverse—which is still a conjunction error—was picked by almost a quarter of all participants.
Table 1
Politician participants selection of each ordering for the Vera-problem
Ordering
Frequency of politician participants choosing this ordering (%)
a. Most likely bank employee who is active in the feminist movement, than someone who is active in the feminist movement, than bank employee
23 (12%)
b. Most likely someone who is active in the feminist movement, than bank employee who is active in the feminist movement, than bank employee
56 (30%)
c. Most likely bank employee, than someone who is active in the feminist movement, than bank employee who is active in the feminist movement
24 (13%)
d. Most likely bank employee who is active in the feminist movement, than bank employee, than someone who is active in the feminist movement
14 (8%)
e. Most likely someone who is active in the feminist movement, than bank employee, than bank employee who is active in the feminist movement
28 (15%)
f. Most likely bank employee, than bank employee who is active in the feminist movement, than someone who is active in the feminist movement
41 (22%)
Total
186
Conjunction Error II: Making the Headlines Scenario
Interestingly, the results of the second conjunction error test—the making the headlines scenario—are quite different from those of the Linda/Vera problem. If politician participants make the conjunction error, we should find support for H2 that they will judge it more likely that a terrorist attack will lead to their municipality making the headlines of all major newspapers, compared to making those headlines in general. This is not what we find. Politician participants’ likelihood estimates of the headlines (either due to a terrorist attack or in general) averaged to about 16 per cent (standard deviation ± 28%), spanning the full range of the scale from 0 to 100 per cent. The distribution of judged likelihood estimates is quite skewed to the left and contains quite some outliers (i.e. estimations over two standard deviations above the mean). In line with existing work (Lodge and Taber 2005; Wolsiefer et al. 2017), including willing-to-pay studies like ours (Tubeuf et al. 2015), we adjust for the skewed distribution by applying a log-transformation (see Online Appendix 14 for a more extensive discussion). After doing so, we find evidence to reject H2. As Fig. 2 shows, politicians judged the likelihood of making the headlines in general on average considerably higher than making them due to a terrorist attack on King’s day (t = 12.53; p < 0.001 single tailed, N(total) = 195: n(general) = 107, n(terrorism) = 88).20 The results of this test obtained in the student sample are in the same direction as those on the politician sample (Figure S2, Online Appendix 10) suggesting that this might not be a valid causal conjunction error test.
Fig. 2
Estimated log-likelihood of a council member’s municipality making the headlines in 2018 in general (on the left), versus due to a terrorist attack (on the right) (n = 195 politician participants)
×
Conjunction Error III: Earthquake Scenario
In our third conjunction error test, we examined if participants neglect scope. If they do, we should find support for H3 that participants judge the likelihood of an earthquake in Groningen to be higher than that of a natural disaster in the Netherlands’ eastern half. This is indeed what we find. The participants’ estimates were quite skewed to the left, with a significant number of outliers at the higher end of the scale (n = 201, M = 5.30, SD = 10.22, range [0; 89]). Therefore, we again apply a log-transformation (see Online Appendix 14 for a more extensive discussion). As Fig. 3 shows, the politician participants judged the earthquake’s likelihood to be higher than that of the natural disaster (N(total) = 201: n(earthquake) = 103, n(natural disaster) = 98, t = 1.69, p = 0.05 single tailed). On average, the politician participants thus neglected the larger scope of the East of the Netherlands compared to the province of Groningen (which it contains) as well as the larger class of events of natural disasters versus earthquakes (which it again contains). This result suggests that the politician participants display this conjunction error. The results based on the student sample match those presented here (see Figure S3 in Online Appendix 10), supporting the validity of this test.
Fig. 3
Estimated log-likelihood of dozens of people getting injured in 2018 due to a natural disaster in the eastern half of the Netherlands (on the left), versus due to an earthquake in Groningen (on the right) (n = 201 politician participants)
×
Scope Neglect I & II: Nuisance Scenario
Turning to the nuisance scenario, we find support for H4 that politician participants will generally consider it equally important to deal with 23 people who cause a nuisance (4.88, n = 105) as they do to deal with 53 ones (4.75, n = 106) (t = − 1.15, p = 0.87 single tailed in the direction that 53 people who cause a nuisance are more important than 23 ones) (see Fig. 4). This indicates that the politician participants neglect scope, confirming H4. Note that given that the scale runs from [1–6], this corresponds to in-between “important” and “very important”.
Fig. 4
Importance of dealing with 23 people who cause a nuisance (on the left) versus 53 ones (on the right) in a council member’s municipality on a scale of 1 “not important at all” to 6 “utmost important” (n = 211 politician participants)
×
When looking at the budgets allocated in each condition, we find further support for politicians neglecting scope. Figure 5 shows that there was no difference between the budgets that politician participants allocated to 23 who cause a nuisance and 53 ones when adjusting for the disproportional influence of outliers through a log-transformation (N(total) = 210: n(23 people who cause a nuisance) = 105, n(53) = 105, t = 0.02, p = 0.49 single tailed in the direction that 53 people who cause a nuisance receive a larger budget than 23 ones). This confirms H5 that politician participants will generally allocate an equal budget to deal with 23 people who cause a nuisance than they do to deal with 53 ones. Interestingly, the difference in importance (H4) and budget allocated (H5) is significant in the student sample (see Table S7 & Figure S4, Online Appendix 10). A possible explanation for the difference between the politician participants and the student participants is that the former were asked about their own municipality whereas the latter were asked about Apeldoorn.21 However, the difference in findings across the two samples is substantively not that large, since the difference between the student sample and the politician sample itself is not significant.
Fig. 5
Budget allocated to deal with 23 people who cause a nuisance (on the left) versus 53 ones (on the right) in a council member’s municipality (n = 210 politician participants)
×
Additional and Robustness Analyses
We conducted several additional and robustness analyses. First, as an additional analysis, we examined if politician participants thought that the difference in scope (23 versus 53 people who cause a nuisance) should be reflected in the allocated budget (Online Appendix 9). We find that they thought it should, which contradicts the between-subjects’ findings we report above. We return to the implications of this finding in the discussion.
As an exploratory analysis, we examined whether left wing politicians will rank the conjunction (“A ⋀ B”—see above) as more probable for Vera compared to her being active in the feminist movement (“B”), while right wing politicians will rank the conjunction (“A ⋀ B”) as more probable for Vera compared to her being a bank teller (“A”). This hypothesis was part of our pre-registration plan (see Online Appendices 1 & 6). The results of this analysis, which Online Appendix 11 reports, are contradictory. Specifically, right wing politicians were more likely to choose the reverse stereotypical ordering, but left wing politicians were less likely to make the conjunction error. This finding is difficult to interpret and suggests that further research is needed.
We also conducted several robustness analyses. To examine whether politicians might be less prone to display the biases related to the representativeness heuristic because of specific traits, we first ran a regression analysis with the following control variables added: gender, education level, political experience, math skill, right-left self-placement.22 The results in Table S10 in Online Appendix 12 are similar to the results presented in the main text. Furthermore, we ran four additional analyses to explore whether politicians with different levels of experience (Table S11 in Online Appendix 12), expertise (Table S12), education level (Table S13) and self-reported math skill (Table S14) are sensitive to displaying the biases related to the representativeness heuristic. For none of these variables did we find a significant interaction effect, suggesting that politician participants who differ on these traits were not more or less sensitive to the difference between the conditions in the various scenarios.23
Discussion and Conclusion
Are politicians’ judgements, which shape their decisions, influenced by the representativeness heuristic? In this article, we presented a new survey questionnaire to answer this question. Our results demonstrate that in most scenarios politicians—in this case: elected local politicians from the larger Dutch municipalities—displayed the biases related to the representativeness heuristic. Specifically, the politician participants made the conjunction error in two scenarios and neglected scope in one scenario. Conversely, we did not find support for the conjunction error in a third scenario. The latter finding conflicts with existing findings on a similar scenario. We also failed to find support for the conjunction error in this scenario in the student sample (see Online Appendix 10), which suggests that there might be an issue with the validity of this item. This could be a relevant avenue for further research.
Our results are likely generalizable to “real” political decisions, where deliberation and counter-arguing is possible. In our scenarios, our politician participants made an individual judgement, while in practice deliberation with advisors and other politicians—that is, group decision making—may help to dissipate the biases before a final decision is made (Finucane et al. 2000; Kaplan and Miller 1978). Still, the positive effect of deliberation requires the absence of strong time pressure (Bang and Frith 2017: 6), whereas politicians—including local ones—typically operate under severe time pressure (Pommer and Boelhouwer 2017; Goetz 2014). Moreover, even without time pressure, these biases will not disappear when a group decision is made by people who are similar. Given that most individuals display biases, they appear similar in this respect. We confirmed the presence of these biases for most of our politician participants, not just for specific individuals. Furthermore, if a group polarizes—which regularly happens with group decision making—, the bias at the individual level may even been amplified at the group level (Bang and Frith 2017: 10). Group decision making thus does not necessarily resolve biases (Kerr et al. 1996). It would be an interesting avenue for future research to examine the extent to and conditions under which political deliberation and polarized debate may contribute to or hinder bias reduction.
The elected local council members who were the main participants in our survey experiment are part time rather than full time politicians: on average, they spend about two or three days a week on their work as a council member. This means that they are less “elite” than politicians at the national level are. Still, our results are probably generalizable to the national level, as well as to politicians from other countries. For one, we replicated Tversky and Kahneman’s findings (1983) which were based on a US sample in the 1980s in another country (the Netherlands) in another period (2017), illustrating their generic character. In addition, our survey experiment included a scenario on framing and risk preferences that was fielded previously among a sample of Dutch national-level politicians (members of parliamentarian, MPs), thereby allowing us to compare local and national politicians directly.24 We found similar results in both cases (Linde and Vis 2017), suggesting that our results generalize to the national level. Moreover, our finding for Dutch MPs on this item was similar to those of Sheffer et al. (2018) for Belgian and Canadian MPs, indicating that our results based on samples from the Netherlands are likely generalizable to these other systems.
We conducted various additional analyses to better understand what drives our results. Together, our results support the conclusion that politicians display the biases related to the representativeness heuristic, but they do not necessarily imply that all politicians are equally likely to display the biases under all circumstances. Further research might explore whether there are certain politicians who are in general more likely to base their judgements on heuristics like representativeness and, if so, which characteristics increase or decrease that probability. Are there different “types” of politicians, with some more likely to use System 1 and others more likely to use System 2 (Kahneman 2011)? While the sample size does not allow us to draw strong conclusions here, we conducted a preliminary analysis to answer this question. Specifically, we examined whether we could predict a politician participant’s answer in a scenario based on whether s/he made the conjunction error in the Vera scenario. If we could, this would offer preliminary evidence that these politician participants systematically used System 1. Our results—see Table S15 in Online Appendix 13—show, conversely, that this is not the case; there is no systematic relationship between making the conjunction error in the Vera scenario and biases in other scenarios.
Another line of research might explore the contextual circumstances that increase or decrease the probability of politicians to use heuristic reasoning. In our additional analyses, reported in the results section in the main text, we found that politician participants indicated themselves that the difference in scope (23 versus 53 people who cause a nuisance) should be reflected in the allocated budget (see also Online Appendix 9). Our main findings, however, showed that when asked between subjects, the difference in scope did not influence the allocated budget. Since both findings are based on the same set of politicians, it appears that the context under which an issue is evaluated plays an important role. This finding suggests that politicians in general would like to use System 2, while in practice they oftentimes use System 1 instead. Learning more about the conditions under which the preference for using System 2 is stronger than the inclination to go with System 1 would also be a worthy topic to explore.
Summing up, we presented the results of a newly developed survey experiment to evaluate the influence of the representativeness heuristic on political judgement, and thereby to better understand how politicians process information. Our finding that politicians display the conjunction error and neglect scope indicate that they use the representativeness heuristic, which has important implications for political judgement and decision making. For one, the responses to our Vera-question illustrate how politicians bias their likelihood assessment when confronted with a stereotype. Note that they did so despite being aware of the stereotypical nature of the question.25 So even though stereotypes are often at the forefront of political debates (e.g. discrimination, gender) and politicians can thus be expected to deal with them carefully, they were still influenced by such stereotypes in non-obvious ways. Biased likelihood assessments related to stereotypes might support political decision making, whether pro or counter, which re-affirms that stereotype by giving stereotypical groups more (policy) attention than warranted (cf. Forbes and Schmader 2010).
Acknowledgements
The research project on which this article is based was funded by a VIDI grant awarded to Barbara Vis from the Netherlands Organization for Scientific Research (NWO, Grant No. 452-11-005). Earlier drafts of this article have been presented at the 2017 NIG Work Conference, Maastricht, the Netherlands; the 2017 General Conference in Oslo, Norway; at the 2017 ECPR Joint Sessions of Workshops in Nottingham, the UK; the 2017 IMEBESS conference in Barcelona, Spain; the 2017 Politicologenetmaal conference in Leiden, the Netherlands; and the 2017 ISPP annual meeting, Edinburgh, Scotland. We thank all participants of these events for their valuable comments and suggestions. We also thank the four anonymous reviewers and the editor of Political Behavior for their useful comments.
Compliance with Ethical Standards
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the research committee of the Faculty of Social Sciences of the Vrije Universiteit Amsterdam and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Our student sample can be seen as drawn from the population of non-politicians. Since we are interested in the basic processes that may be true for all humans, using such a student sample makes sense (Druckman and Kam 2011; Morton and Williams 2010: chap 9).
For instance Morewedge et al. (2015) show that a single “debiasing” training intervention—a video or a (serious) game—reduced bias from over-relying on representativeness immediately and three months later.
Note that Böhmelt et al. (2016) do not discuss what their empirical findings imply regarding the representativeness heuristic. Because the representativeness heuristic is included in their theorizing, this is–at a minimum–a missed opportunity. Against this backdrop, we consider it reasonable to interpret what the findings imply regarding the representativeness heuristic.
Bellur and Sundar (2014) present an approach to establish directly whether a heuristic has been used. An approach to quantify heuristic bias in clinical decision making is proposed by Richie and Josephson (2017).
We preregistered the design and our expectations for the full project at AsPredicted.org on April 4, 2017 (https://aspredicted.org/5ye4c.pdf). We uploaded a so-called add-on to our pre-registration at AsPredicted.org on May 9, 2017 (https://aspredicted.org/pi28u.pdf). More information is available in Online Appendix 1.
Even though this percentage dropped, a majority still made a conjunction error when the test was made more transparent. Despite criticisms on the Linda problem (see Kahneman 2011: chap. 15), the evidence in favor of the conjunction error is still strong (Kahneman and Frederick 2002: 66, see also pp. 65–68 for a discussion of the critique).
For Dutch municipalities of the size we focus on (see "Data" section), these are the type of people who cause a nuisance that Dutch local politicians face. Note that this, for instance, does not mean that all homeless people or addicts cause a nuisance.
We chose not to use this specific scenario to (a) stay close to Tversky and Kahneman’s (1983) original formulation of the problem and to (b) avoid a strong political connotation.
This hypothesis differs somewhat from the hypothesis as included in the pre-registration plan from May 2017 (see Online Appendix 1). Recall that Online Appendix 6 presents an overview of all hypotheses from this pre-registration and includes information on whether and if so why we diverted (somewhat) from a specific hypothesis. In this appendix, we also list those hypotheses that are part of the larger project and that are or will be discussed elsewhere.
The number of respondents is lower than the full sample due to item non-response. Table S5 in Online Appendix 8 displays the non-response per item, which is low for all the substantively relevant items.
Consequently, slightly less than half of all participants chose another ordering (45 per cent, NB: numbers do not add to 100 per cent due to rounding).
Adding control variables like gender, political experience, education, left–right self-placement, math-skill, or number of statistics courses followed has no influence on this result (see Table S9 in Online Appendix 13).
The choice of city—Apeldoorn, which witnessed a terrorist attack in 2009—could have influenced the results of the nuisance scenario. However, this was not the case. If student participants would have been primed with danger, this would have increased the importance of safety, which should have made them willing to spend more on people who cause a nuisance. However, we find that they are willing to spend less to address this problem. Also in the making the headlines scenario, the choice of city could have influenced our findings. If the King’s day attack in Apeldoorn would have been “available” to the student participants, the availability heuristic would predict them to find this attack to be more likely, thus bolstering our expectation from H2. Since we do not find that students believed this to be more likely (see Table S7 in Online Appendix 10), there is little reason to suspect that our choice of city influenced the results we obtained in this scenario.
As we explain in Online Appendix 13 in more detail, while the sample size for experience is too small to draw firm conclusions, the results still provide first evidence that experience has no significant effect.
Note that we did not discuss this item elsewhere in this article, because, although it is related, it concerned the availability heuristic and not the representativeness one that we focus on here.
