Abstract
Experimenter demand effects refer to changes in behavior by experimental subjects due to cues about what constitutes appropriate behavior. We argue that they can either be social or purely cognitive, and that, when they may exist, it crucially matters how they relate to the true experimental objectives. They are usually a potential problem only when they are positively correlated with the true experimental objectives’ predictions, and we identify techniques such as non-deceptive obfuscation to minimize this correlation. We discuss the persuasiveness or otherwise of defenses that can be used against demand effects criticisms when such correlation remains an issue.
Similar content being viewed by others
References
Abbink, K., & Hennig-Schmidt, H. (2006). Neutral versus loaded instructions in a bribery experiment. Experimental Economics, 9, 103–121.
Ackert, L. F., Charupat, N., Church, B. K., & Deaves, R. (2006). An experimental examination of the house money effect in a multi-period setting. Experimental Economics, 9, 5–16.
Alm, J., McClelland, G. H., & Schultze, W. D. (1992). Why do people pay taxes? Journal of Public Economics, 48, 21–38.
Andreoni, J., & Miller, J. (2002). Giving according to GARP: An experimental test of the consistency of preferences for altruism. Econometrica, 70, 737–753.
Ariely, D., Loewenstein, G., & Prelec, D. (2003). Coherent arbitrariness: Stable demand curves without stable preferences. Quarterly Journal of Economics, 118, 73–105.
Asch, S. E. (1956). Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychology Monographs, 70, 1–70.
Bacharach, M., Guerra, G., & Zizzo, D. J. (2007). The self-fulfilling property of trust: An experimental study. Theory and Decision, 63, 349–388.
Baldry, J. C. (1986). Tax evasion is not a gamble. Economics Letters, 22, 333–335.
Ball, S., Eckel, C., Grossman, P. J., & Zane, W. (2001). Status in markets. Quarterly Journal of Economics, 116, 161–188.
Bardsley, N. (2005). Experimental economics and the artificiality of alteration. Journal of Economic Methodology, 12, 239–251.
Bardsley, N. (2008). Dictator game giving: Altruism or artefact? Experimental Economics, 11, 122–133.
Binmore, K., Shaked, A., & Sutton, J. (1985). Testing noncooperative bargaining theory: a preliminary study. American Economic Review, 75, 1178–1180.
Blass, T. (1999). The Milgram paradigm after 35 years: Some things we now know about obedience to authority. Journal of Applied Social Psychology, 29, 955–978.
Bosman, R., & van Winden, F. (2002). Emotional hazard in a power-to-take experiment. Economic Journal, 112, 147–169.
Branas-Garza, P. (2006). Poverty in dictator games: Awakening solidarity. Journal of Economic Behavior and Organization, 60, 306–320.
Branas-Garza, P. (2007). Promoting helping behavior with framing in dictator games. Journal of Economic Psychology, 28, 477–486.
Burnham, T. C. (2003). Engineering altruism: a theoretical and experimental investigation of anonymity and gift giving. Journal of Economic Behavior and Organization, 50, 133–144.
Camerer, C. F. (2003). Behavioral game theory: experiments in strategic interaction. Princeton: Princeton University Press.
Casari, M., & Cason, T. N. (2009). The strategy method lowers measured trustworthy behavior. Economics Letters, 103, 157–159.
Cason, T. N., & Sharma, T. (2007). Recommended play and correlated equilibria: an experimental study. Economic Theory, 33, 11–27.
Chou, E., McConnell, M., Nagel, R., & Plott, C. R. (2009). The control of game form recognition in experiments: Understanding dominant strategy failures in a simple two person “guessing” game. Experimental Economics, 12, 159–179.
Crowne, D. P., & Marlowe, D. (1964). The approval motive. New York: Wiley.
Davis, D. D., & Holt, C. A. (1993). Experimental economics. Princeton: Princeton University Press.
Eckel, C. C., & Grossman, P. J. (2000). Volunteers and pseudo-volunteers: the effect of recruitment method in dictator experiments. Experimental Economics, 3, 101–120.
Eckel, C., & Holt, C. A. (1989). Strategic voting in agenda-controlled committee experiments. American Economic Review, 79, 763–773.
Ellingsen, T., & Johannesson, M. (2008). Pride and prejudice: the human side of incentive theory. American Economic Review, 98, 990–1008.
Fehr, E., Kirchsteiger, G., & Riedl, A. (1993). Does fairness prevent market clearing? An experimental investigation. Quarterly Journal of Economics, 108, 437–459.
Festinger, L. (1957). A theory of cognitive dissonance. Evanston: Peterson Row.
Fischbacher, U., Gachter, S., & Fehr, E. (2001). Are people conditionally cooperative? Evidence from a public goods experiment. Economics Letters, 71, 397–404.
Fleming, P., Townsend, E., Lowe, K. C., & Ferguson, E. (2007). Social desirability effects on biotechnology across the dimensions of risk, ethicality and naturalness. Journal of Risk Research, 10, 989–1003.
Forsythe, R., Rietz, T., Myerson, R., & Weber, R. (1996). An experimental study of voting rules and polls in three-candidate elections. International Journal of Game Theory, 25, 335–383.
Frank, B. L. (1998). Good news for the experimenters: Subjects do not care about your welfare. Economics Letters, 61, 171–174.
French, J. R. P., Jr., & Raven, B. (1959). The bases of social power. In D. Cartwright (Ed.), Studies in social power (150–167). Ann Arbor: Research Center for Group Dynamics University of Michigan.
Gale, J., Binmore, K. G., & Samuelson, L. (1995). Learning to be imperfect: the ultimatum game. Games and Economic Behavior, 8, 856–890.
Golding, S. L., & Lichtenstein, E. (1970). Confession of awareness and prior knowledge of deception as a function of interview set and approval motivation. Journal of Personality and Social Psychology, 14, 213–223.
Guerra, G., & Zizzo, D. J. (2004). Trust responsiveness and beliefs. Journal of Economic Behavior and Organization, 55, 25–30.
Harrison, G. W. (2007). House money effects in public good experiments: Comment. Experimental Economics, 10, 429–437.
Harrison, G. W., & Johnson, L. T. (2006). Identifying altruism in the laboratory. In D. Davis & R. M. Isaac (Eds.), Research in experimental economics : Vol. 11. Experiments investigating fundraising and charitable contributors (pp. 177–223). Amsterdam: Elsevier.
Heinemann, F., Nagel, R., & Ockenfels, P. (2004). The theory of global games on test: Experimental analysis of coordination games with public and private information. Econometrica, 72, 1583–1599.
Hoffman, E., McCabe, K., Shachat, K., & Smith, V. (1994). Preferences, property rights, and anonymity in bargaining games. Games and Economic Behavior, 7, 346–380.
Huck, S., Normann, H.-T., & Oechssler, J. (2004). Two are few and four are many: number effects in experimental oligopolies. Journal of Economic Behavior and Organization, 53, 435–446.
Jones, S. R. G. (1984). The economics of conformism. Oxford: Blackwell.
Jones, S. R. G. (1992). Was there a Hawthorne effect? American Journal of Sociology, 98, 451–468.
Kirkpatrick, L. A., & Ellis, B. J. (2004). An evolutionary-psychological approach to self-esteem: multiple domains and multiple functions. In M. B. Brewer & M. Hewstone (Eds.), Self and social identity (pp. 52–77). Oxford: Blackwell.
Lei, V., Noussair, C. N., & Plott, C. R. (2001). Nonspeculative bubbles in experimental asset markets: Lack of common knowledge of rationality vs. actual irrationality. Econometrica, 69, 831–859.
Levitt, S. D., & List, J. A. (2007). What do laboratory experiments measuring social preferences reveal about the real world? Journal of Economic Perspectives, 21, 153–174.
List, J. A. (2007). On the interpretation of giving in dictator games. Journal of Political Economy, 115, 482–493.
Loomes, G. (1999). Some lessons from past experiments and some challenges for the future. Economic Journal, 109, F35–F45.
Markman, A. B., & Moreau, C. P. (2001). Analogy and analogical comparison in choice. In D. Gentner, K. J. Holyoak & B. Kokinov (Eds.), The analogical mind: perspectives from cognitive science (pp. 363–399). Cambridge: MIT Press.
Mayo, D. G. (1996). Error and the growth of experimental knowledge. Chicago: Chicago University Press.
Mayo, E. (1933). The human problems of an industrial civilization. New York: MacMillan.
Milgram, S. (1974). Obedience to authority: an experimental view. New York: Harper and Row.
Noussair, C. M., & Powell, O. (2009). Peaks and valleys: experimental asset markets with non-monotonic fundamentals. Journal of Economic Studies (forthcoming).
Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776–783.
Orne, M. T. (1973). Communication by the total experimental situation: why it is important, how it is evaluated, and its significance for the ecological validity of findings. In P. Pliner, L. Krames, & T. M. Alloway (Eds.), Communication and affect (pp. 157–191). San Diego: Academic Press.
Ortmann, A., & Hertwig, A. (2002). The costs of deception: evidence from psychology. Experimental Economics, 5, 111–131.
Plott, C. R., & Levine, M. E. (1978). A model of agenda influence on committee decisions. American Economic Review, 68, 146–160.
Rosnow, R. L., & Rosenthal, R. (1997). People studying people: artifacts and ethics in behavioral research. New York: Freeman.
Sarin, R. K., & Weber, M. (1993). Effects of ambiguity in market experiments. Management Science, 39, 602–615.
Schotter, A., & Sopher, B. (2003). Social learning and coordination conventions in intergenerational games: an experimental study. Journal of Political Economy, 111, 498–529.
Shang, J., & Croson, R. (2009). Field experiments in charitable contribution: the impact of social information on the voluntary provision of public goods. Economic Journal, 119, 1422–1439.
Shanks, D. R., & St. John, M. F. (1994). Characteristics of dissociable human learning systems. Behavioral and Brain Sciences, 17, 367–447.
Sitzia, S., & Zizzo, D. J. (2009). In search of product complexity effects in experimental retail markets. Theory and Decision (forthcoming).
Smith, V. L. (1982). Microeconomic systems as an experimental science. American Economic Review, 72, 923–955.
Smith, V. L. (1994). Economics in the laboratory. Journal of Economic Perspectives, 8, 113–131.
Solnick, S. (2007). Cash and alternate methods of accounting in an experimental game. Journal of Economic Behavior and Organization, 62, 316–321.
Stewart, N., Chater, N., Stott, H. P., & Reimers, S. (2003). Prospect relativity: how choice options influence decision under risk. Journal of Experimental Psychology: General, 132, 23–46.
Stocco, A., & Fum, D. (2008). Implicit emotional biases in decision making: the case of the Iowa gambling task. Brain and Cognition, 66, 253–259.
Tan, J. H. W. (2006). Religion and social preferences: an experimental study. Economics Letters, 90, 60–67.
van Huyck, J., Gillette, A. B., & Battalio, R. (1992). Credible assignments in coordination games. Games and Economic Behavior, 4, 606–626.
Zizzo, D. J. (2003). Verbal and behavioral learning in a probability compounding task. Theory and Decision, 54, 287–314.
Zizzo, D. J. (2008). Experimenter demand effects in economic experiments. Social Science Research Network discussion paper, July.
Author information
Authors and Affiliations
Corresponding author
Additional information
I wish to thank participants to a presentation at Bologna Forli’ and Piers Fleming, Nick Bardsley, Dan Benjamin, Pablo Branas-Garza, Tim Cason, Gary Charness, Andrew Colman, John Duffy, Dirk Engelmann, Paul Ferraro, Shaun Hargreaves Heap, Alexander Koch, Charlie Plott, Jen Shang and two anonymous referees for useful feedback and references on the topic. Partial financial support from the Nuffield Foundation (SGS/35859) is gratefully acknowledged. The usual disclaimer applies.
Rights and permissions
About this article
Cite this article
Zizzo, D.J. Experimenter demand effects in economic experiments. Exp Econ 13, 75–98 (2010). https://doi.org/10.1007/s10683-009-9230-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10683-009-9230-z
Keywords
- Experimenter demand effects
- Experimental design
- Experimental instructions
- Social desirability
- Social pressure
- Framing
- Methodology