Skip to main content
SpringerLink
Log in

Session-effects in the laboratory

  • Published:
Experimental Economics Aims and scope Submit manuscript

Abstract

In experimental economics, where subjects participate in different sessions, observations across subjects of a given session might exhibit more correlation than observations across subjects in different sessions. The main goal of this paper is to clarify what are session-effects: what can cause them, what forms they can take, and what are the potential problems. It will be shown that standard solutions are at times inadequate, and that their properties are sometimes misunderstood.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. See for instance Davis and Holt (1993, pp. 527–528) and Friedman and Sunder (1994, pp. 98–99).

  2. If the variables that need to be controlled for only take a small set of values, the sample can simply be separated into subgroups, but when controlling for continuous variables this is not an option.

  3. Clearly “relevant” here excludes the source of the session-effects which although relevant, are either unobserved or unknown to matter to the experimenter.

  4. We assume that there is only one treatment for simplicity of exposition, but nothing in this analysis depends on this assumption.

  5. This specification is more restrictive than required, but this has no bearing on what follows. For instance one can easily accommodate sequential games where an agent’s decisions depend on prior decisions.

    Note also that (1) allows for correlation of the observations across the decisions of a subject. Although this is not addressed in the current paper, if this is relevant for a given application, it would need to be taken into account in the estimation. An example where this is discussed explicitly in an experimental context is Fréchette (2009).

  6. This could include the decisions of others in the session, the realization of a random variable, the past outcome of some decision, etc.

  7. Note that to use session averages as the unit of observation with auction data, one would typically have to transform the variable of interest. For instance, one could use the bid minus the value or the bid divided by the value (depending on the bid function).

  8. Menstrual Synchrony has been observed in many species, including humans, for females living together (see Stern and McClintock 1998 and the references therein).

  9. Examples of this sort can be constructed with any hormone which varies over a period of time in a similar way for all (or a subgroup of) subjects. For instance, there is evidence that testosterone increases aggressiveness (see Beatty 1992 for evidence in rodents and Lehrer et al. 2004 for a brief summary of the evidence in humans) and it is known that testosterone decreases during the day (Dabbs 1990). Hence, any task where aggressiveness matters could result in sessions conducted at a similar time in the day being correlated. Similarly, Schipper (2011) finds that risk aversion correlates with testosterone and cortisol in men.

    Note that I am not giving examples because I think they are likely, or important in magnitude, but rather they are provided as an illustration of something that is possible.

  10. In the particular example of Table 1 there is no correlation between the point estimate of the treatment effect. If instead the tall RA had conducted sessions 5 through 8 and the short RA sessions 1 through 4, then the session-effect would also be (perfectly) correlated to the treatment, which would lead to additional problems.

  11. For example both Seinen and Schram (2006) and Ivanova-Stenzel and Salmon (2008) refer to a pairwise Mann-Whitney test on session averages as conservative.

  12. In the turnpike protocol, subjects cannot influence the decisions of future subjects they will be paired with through the decisions they take in the current match. To understand this procedure, line up the subjects in two equal rows, each subject interacts with the person in front of them, then all the subjects in one of the row moves one seat, and interact with the new person in front of them (the person at the extremity moves back to the beginning of the line). This can be repeated until the players would end up in their original position, at which point it must stop. Note that this is different from what is know as the absolute stranger design which only requires that no subject is matched with someone else twice.

  13. This structure is in the spirit of fictitious play (see for instance Fudenberg and Levine 1999). However, for simplicity, the example, unlike in a typical model of fictitious play, assumes that all decisions after period 1 do not depend on the prior subjects hold in period 1.

  14. Cameron et al. (2008) show in specific environments, through the use of Monte Carlo simulations, that when there is a small number of clusters, other corrections (different from the one typically employed) have better performance. However, the results as to what corrections perform best may be sensitive to the specifics of the data generating process.

  15. See Chamberlain (1980) and Heckman (1981), or Wooldridge (2002) for a more recent discussion (under “initial conditions problem” and also “Chamberlain’s random effects probit models”).

  16. See for instance Hsiao (2003). See also Arellano (2003) for a reference on dynamic panel data analysis.

References

  • Altman, D. G., & Bland, J. M. (1997). Statistical notes: units of analysis. British Medical Journal, 314, 1874.

    Article  Google Scholar 

  • Aoyagi, M., & Fréchette, G. R. (2009). Collusion as public monitoring becomes noisy: experimental evidence. Journal of Economic Theory, 144(3), 1135–1165.

    Article  Google Scholar 

  • Apesteguia, J., Huck, S., & Oechssler, J. (2007). Imitation-theory and experimental evidence. Journal of Economic Theory, 136, 217–235.

    Article  Google Scholar 

  • Arellano, M. (2003). Panel data econometrics, OUP Catalogue. London: Oxford University Press.

    Book  Google Scholar 

  • Armantier, O. (2004). Does observation influence learning? Games and Economic Behavior, 46(2), 221–239.

    Article  Google Scholar 

  • Beatty, W. W. (1992). Gonadal hormones and sex differences in nonreproductive behaviors. In A. A. Gerall, H. Moltz, & I. L. Ward (Eds.), Handbook of behavioral neurology: Vol. 2. New York: Plenum Press.

    Google Scholar 

  • Blume, A., Duffy, J., & Franco, A. M. (2009). Decentralized organizational learning: an experimental investigation. American Economic Review, 99(4), 1178–1205.

    Article  Google Scholar 

  • Camerer, C. F., Ho, T.-H., & Chong, J.-K. (2002). Sophisticated EWA learning and strategic teaching in repeated games. Journal of Economic Theory, 104, 137–188.

    Article  Google Scholar 

  • Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008). Bootstrap-based improvements for inference with clustered errors. Review of Economics and Statistics, 90, 414–427.

    Article  Google Scholar 

  • Chamberlain, G. (1980). Analysis of covariance with qualitative data. Review of Economic Studies, 47, 225–238.

    Article  Google Scholar 

  • Charness, G., Fréchette, G. R., & Kagel, J. H. (2004). How robust is laboratory gift exchange? Experimental Economics, 7(2), 189–205.

    Article  Google Scholar 

  • Charness, G., Fréchette, G. R., & Qin, C.-Z. (2007). Endogenous transfers in the prisoner’s dilemma game: an experimental test of cooperation and coordination. Games and Economic Behavior, 60(2), 287–306.

    Article  Google Scholar 

  • Chen, Y., Katuscak, P., & Ozdenoren, E. (2005). Why can’t a woman bid more like a man. Mimeo.

  • Cooper, R., DeJong, D. V., Forsythe, R., & Ross, T. W. (1996). Cooperation without reputation: experimental evidence from prisoner’s dilemma games,. Games and Economic Behavior, 12, 187–218.

    Article  Google Scholar 

  • Croson, R. (1996). Partners and strangers revisited. Economics Letters, 53, 25–32.

    Article  Google Scholar 

  • Dabbs, James M. J. (1990). Salivary testosterone measurements: reliability across hours, days, and weeks. Physiology & Behavior, 48(1), 83–86.

    Article  Google Scholar 

  • Davis, D. D., & Holt, C. A. (1993). Experimental economics. Princeton: Princeton University Press.

    Google Scholar 

  • Excoffier, L., Smouse, P. E., & Quattro, J. M. (1992). Analysis of molecular variance inferred from metric distances among DNA haplotypes: application to human mitochondrial DNA restriction data. Genetics, 131, 479–491.

    Google Scholar 

  • Fischbacher, U., Gächter, S., & Fehr, E. (2001). Are people conditionally cooperative? Evidence from a public goods experiment. Economics Letters, 71, 397–404.

    Article  Google Scholar 

  • Fréchette, G. R. (2009). Learning in a multilateral bargaining experiments. Journal of Econometrics, 153(2), 183–195.

    Article  Google Scholar 

  • Fréchette, G. R., Kagel, J. H., & Morelli, M. (2005). Behavioral identification in coalitional bargaining: an experimental analysis of demand bargaining and alternating offers. Econometrica, 73(6), 1893–1938.

    Article  Google Scholar 

  • Friedman, D., & Sunder, S. (1994). Experimental methods: a primer for economists. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Fudenberg, D., & Levine, D. K. (1999). Learning and evolution in games. Cambridge: MIT Press.

    Google Scholar 

  • Gneezy, U., Niederle, M., & Rustichini, A. (2003). Performance in competitive environments: gender differences. The Quarterly Journal of Economics, CXVIII, 1049–1074.

    Article  Google Scholar 

  • Ham, J. C., Kagel, J. H., & Lehrer, S. F. (2005). Randomization, endogeneity and laboratory experiments: the role of cash balances in private value auctions. Journal of Econometrics, 125(1–2), 175–205.

    Article  Google Scholar 

  • Heckman, J. J. (1981). Structural analysis of discrete data with econometric applications. In The incidental parameters problem and the problem of initial conditions in estimating a discrete time-discrete data stochastic process (pp. 179–195). Cambridge: MIT Press.

    Google Scholar 

  • Hsiao, C. (2003). Analysis of panel data, Cambridge books. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Ivanova-Stenzel, R., & Salmon, T. C. (2008). Revenue equivalence revisited. Games and Economic Behavior, 64, 171–192.

    Article  Google Scholar 

  • Lehrer, S. F., Tremblay, R. E., Vitaro, F., & Schaal, B. (2004). Raging hormones in puberty: do they influence adolescent risky behavior? Mimeo.

  • Lewejohann, L., Reinhard, C., Schrewe, A., Brandewiede, J., Haemisch, A., Görtz, N., Schachner, M., & Sachser, N. (2006). Environmental bias? Effects of housing conditions, laboratory environment and experimenter on behavioral tests. Genes, Brain and Behavior, 5, 64–72.

    Article  Google Scholar 

  • Roth, A. E., Prasnikar, V., Okuno-Fujiwara, M., & Zamir, S. (1991). Bargaining and market behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: an experimental study. American Economic Review, 81(5), 1068–1095.

    Google Scholar 

  • Sakata, S. (2002). Quasi-maximum likelihood estimation with complex survey data. Mimeo.

  • Schipper, B. C. (2011). Sex hormones and choice under risk. Working Paper, UC Davis.

  • Seinen, I., & Schram, A. (2006). Social status and group norms: indirect reciprocity in a repeated helping experiment. European Economic Review, 50, 581–602.

    Article  Google Scholar 

  • Stern, K., & McClintock, M. K. (1998). Regulation of ovulation by human pheromones. Nature, 392(12), 177–179.

    Google Scholar 

  • Van Huyck, J., Battalio, R. C., & Beil, R. (1991). Strategic uncertainty, equilibrium selection, and coordination failure in average opinion games. The Quarterly Journal of Economics, 885–910.

  • Vanberg, C. (2008). Why do people keep their promises? An experimental test of two explanations. Econometrica, 76(6), 1467–1480.

    Article  Google Scholar 

  • Warton, D. I., & Hudson, H. M. (2004). A MANOVA statistic is just as powerful as distance-based statistics, for multivariate abundances. Ecology, 85(3), 858–874.

    Article  Google Scholar 

  • Williams, R. L. (2000). A note on robust variance estimation for cluster-correlated data. Biometrics, 56, 645–646.

    Article  Google Scholar 

  • Wooldridge, J. M. (2002). Econometric analysis of cross section and panel data. Cambridge: MIT Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. New York University, New York, USA

    Guillaume R. Fréchette

Authors
  1. Guillaume R. Fréchette
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guillaume R. Fréchette.

Additional information

I wish to thank the editor and two anonymous referees, Marina Agranov, Jeffrey Carpenter, Gary Charness, Yan Chen, Vincent Crawford, Pedro Dal Bó, Mark Dean, John Duffy, Martin Dufwenberg, Drew Fudenberg, John Ham, Burkhard Schipper, Andrew Schotter, Chloé Tergiman, Kevin Thom, Isabel Treviño, and Emanuel Vespa for helpful comments and the NSF via grants SES-0519045, SES-0721111, and SES-0924780, and the Center for Experimental Social Science (CESS) for their financial support.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fréchette, G.R. Session-effects in the laboratory. Exp Econ 15, 485–498 (2012). https://doi.org/10.1007/s10683-011-9309-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10683-011-9309-1

Keywords

Search

Navigation