Confirmation bias: methodological causes and a palliative response

The metric used is from Harzing’s Publish or Perish, which uses Google Scholar, and reported (25/3/2015) 1749 citations. Second was Mynatt et al. 1977 with only 391.

Thus Caballero (2010) treats such tendencies as an example of ‘pareidolia’, the familiar situation where the mind sees patterns (such as faces in clouds) that do not actually exist.

In this paper I use the term ignorance to mean unknowable; alternative meanings are that it means not knowing that which can be, and perhaps should be, known. For me, then, the basic duality of ignorance/knowledge is that between knowability/unknowability.

Here I avoid using the term ‘inductive trap’ as that may suggest that I am referring to the wide range of discussions, often associated with Hume, about whether solely inductive reasoning may or may not lead to truth. It should be obvious from my approach here, which views epistemological matters as inseparable from social practices, that I mean something quite different. By ‘inductive box’ I mean that first stage of the quite orthodox idea of the “double, inductive-deductive procedure” where, before theory is used deductively to produce testable statements, theory is developed, in some relation to its ‘other’ – a reality, perhaps – which is to do with ‘induction’, and whilst it is being developed has to be kept isolated, and I call this being ‘in a box’ – the ‘inductive box’.

What is meant here by ‘tested deductively’ follows the sense of the Aristotelian procedure of movement between induction and deduction, modified by Grosseteste to require that deductions be confronted with empirical testing. If chosen logical rules are followed, then the deductions from a given theory are true as logical deductions, premised upon the truth of whatever they rest on – they are relatively true; what is meant here, though, is that such deductions can be also be considered as untrue and their plausibility to be assessed empirically.

Ignorance then means not knowing something that is knowable, which is not the general meaning I use in this paper generally.

It would be easy but is not really necessary to present historical examples of application of allegedly predictively known economic relationships to policy, with poor results; recent ones would be the Asian Financial Crisis, the Global Financial Crisis and austerity policies.

This field is a very useful case study, suggesting that confirmation bias is deeply-rooted in mainstream governance practices [Fforde 2015] that assume a relative power over other poorer regions that is arguably absent [Fforde and Seidel 2015]. It can be argued that economists are particularly at risk, as their theoretical priors are particularly strong, perhaps compared with other disciplines such as anthropologists and sociologists. See Fourcade et al. (2015) for a discussion. In the terms used in this paper and in Fforde (2016) forthcoming, economists are more inclined to be trapped in an ‘inductive box’, failing to return to disbelief in theory and adopt a skepticism as deductions from it are confronted with data.

I am generalising, yet Yonay (1998) offers an intriguing historical account of what happened within the discipline in the 1930 s when data became available to confront theory. He argues that ‘those who won’ tended to continue to believe that data was not very important to their belief in theory. Economics textbooks often assert the value of theory to students without much engagement with data (Fforde 2013).