“Waiting for Life to Arrive”: A history of the regression-discontinuity design in Psychology, Statistics and Economics
Introduction
The regression discontinuity design (RDD) occurs when assignment to treatment depends deterministically on a quantified score on some continuous assignment variable. This score is then used as a covariate in a regression of outcome. When RDD is perfectly implemented, the selection process is fully observed and so can be modeled to produce an unbiased causal inference.
This paper is about the history of RDD. Although I am not a trained historian, I know enough to respect the primacy historians place on documenting events and trends. I also know that interpreting these events and trends has to be conditioned by independent knowledge of temporal sequence, by archived specifics that are relevant to the explanations offered, and by recourse to substantiated theories of individual and institutional behavior. Fortunately, most of this paper is about such events, trends and interpretations. But a few parts are not, and historians may well become nervous when I try to interpret events that might have happened but did not. Although some historians are developing a taste for counterfactual or virtual history (Ferguson, 1997), it is deservedly a minority taste. As an amateur historian, I will almost certainly fall into other traps professionals learn to avoid. Of those I can recognize, one is the teleological trap of inferring inevitable-seeming links between past events when, with more secure footing in the original time and place, these events might seem more contingent and many other futures possible. Another problem is that I am not an independent commentator on RDD. I have been peripherally involved in its history, albeit as a disseminator and not a theorist or practitioner. I was also marginally involved in the Northwestern University theory group that developed the design in the early 1970s after its discovery earlier (Thistlethwaite and Campbell, 1960). Doubtless I know more about RDD's history in that context than about other attempts to develop, disseminate or evaluate the design. While I have read many of the original sources reported on here, I have probably relied on secondary sources more than a real historian would. The relatively recent history of RDD has helped me, though, since many of the method's pioneers are still alive and have offered commentary on earlier drafts of this paper as it touched on their work. I have tried to incorporate their memories and sensitivities into this final version, sometimes even citing their notes to me about their work. Nonetheless, the following account is mine and not theirs; and while I respect the simplest norms of writing history, I cannot hope to dip deeply into the historian's bag of analytic tools. So, caveat lector.
This is not the first historical account of RDD. Donald Campbell, the design's originator, wrote his own version of the design's early history (Campbell, 1984), and various scholars have given snapshots of its history since then (Trochim, 1984, Trochim, 2001). However, the present account is more current, detailed and interdisciplinary than its predecessors. Indeed, it is organized around academic disciplines, tracing the history of the design in Psychology and Education, then in Statistics and Biostatistics, and then in Economics. The account speaks to many themes, including the repeated re-invention of the design across these disciplines. This was often done invoking different names for the design, the upshot being that RDD has not attained consistent “brand” status across the various behavioral, social and health sciences. Another theme speaks to the design's differential waxing and waning by discipline, trying to describe and explain what happened. RDD was invented and initially developed in Psychology and Education, but interest in it waned there after about 1990. It has never had much visible growth in Statistics, though its was acknowledged there. And in Economics RDD had a serendipitous birth, a long period of neglect, and then a renaissance after about 1995. This special journal number is part of that revival. Since its invention in 1960, RDD has been, in Samuel Beckett's words: “waiting for life to happen”. Will this revival breathe life into the design in Economics and, who knows, even beyond?
Section snippets
Psychology and Education
No doubt exists that the first publication on RDD was an application in education by two psychologists, Thistlethwaite and Campbell (1960). No doubt also exists that Campbell was the initiator and that he continued to work on the topic while Thistlethwaite did not. What is less clear is the intuition that led Campbell to develop the design. To probe this we go to Campbell and Stanley (1963) since it provides more conceptual clarification than the earlier paper. This clarification did not
Statistics
For the purposes of this paper I understand Statistics in terms of scholars trained in that field, whether in a Department of Statistics, Mathematics or Biostatistics. Also included are those scholars trained in other fields who later came to hold their major academic appointment in a Statistics Department.
Rubin (1977) is the first published article I could find in Statistics that mentions RDD. This paper has been portrayed as another independent invention of the design together with a formal
Economics
The earliest papers on RDD in Economics were by Goldberger, 1972a, Goldberger, 1972b. These unpublished papers represent two main accomplishments for RDD theory, though they were only incidental to Goldberger's main purpose. The first accomplishment was a proof of the basic design, showing formally what Campbell had only intuited. The Goldberger's papers were based on the distinction between non-equivalent groups whose difference depends on true ability in one case, and on measured ability in
Conclusions
Several themes stand out in the half century of RDD's history. One is its repeated independent discovery. While this augurs well for the design's validity and relevance across fields, one circumstance of the reinventions has been strange. Campbell first named the design regression-discontinuity; Goldberger referred to it as deterministic selection on the covariate; Sacks and Spiegelman studiously avoided naming it; Rubin first wrote about it as part of a larger discussion of treatment
Acknowledgments
Thanks are due to Richard Berk, Glen Cain, Arthur Goldberger, Guido Imbens, George Knafl, Thomas Lemieux, William Shadish, Clifford Spiegelman, William Trochim and Vivian Wong for feedback on prior drafts. They are not responsible for any errors of fact or taste.
References (83)
Regression and selection models to improve nonexperimental comparisons
- et al.
An illustrative statistical analysis of cut-off based randomized clinical trials
Journal of Clinical Epidemiology
(1994) - et al.
Model robust confidence intervals
Journal for Statistical Planning and Inference
(1982) Regression discontinuity design
- et al.
Cutoff assignment strategies for enhancing randomized clinical trials
Controlled Clinical Trials
(1992) - et al.
Comparison of a randomized and two quasi-experimental designs in a single outcome evaluation: efficacy of a university-level remedial writing program
Evaluation Review
(1998) - et al.
Identification of causal effects using instrumental variables
Journal of the American Statistical Association
(1996) - et al.
Does compensatory school attendance affect schooling and earnings
Quarterly Journal of Economics
(1991) - et al.
Using Maimonides’ rule to estimate the effect of class size on scholastic achievement
Quarterly Journal of Economics
(1999) - et al.
Issues in the analysis of selectivity bias
An evaluation of California's inmate classification system using a regression discontinuity design
Journal of the American Statistical Association
Evaluation and Experiment
Capitalizing on nonrandom assignment to treatments: a regression-discontinuity evaluation of a crime-control program
Journal of the American Statistical Association
Memo on the Evaluation Design of the Reading First National Impact Study
Coupling randomized experiments and approximations to experiments in social program evaluation
Sociological Methods and Research
An Evaluation of the Performance of Regression Discontinuity Design on PROGRESA
Reforms as experiments
American Psychologist
Forward
Making the case for randomized assignment to treatments by considering the alternatives: six ways in which quasi-experimental evaluations in compensatory education tend to underestimate effects
How regression artifacts in quasi-experiments can mistakenly make compensatory education look harmful
Experimental and quasi-experimental designs for research on teaching
Power analysis of cutoff-based randomized clinical trials
Evaluation Review
Ethical and scientific features of cutoff-based designs
Medical Decision Making
Quasi-Experimentation: Design and Analysis for Field Settings
Virtual History: Alternatives and Counterfactuals
Clinical and prophylactic trials with assured new treatment for those at greater risk: I. A design proposal
Journal of Public Health
Clinical and prophylactic trials with assured new treatment for those at greater risk: II. Examples
Journal of Public Health
Nonexperimental vs. experimental estimates of earnings impacts
The Annals of the American Academy
The effects of universal pre-k in Oklahoma: research highlights and policy implications
The Policy Studies Journal
Identification and estimation of treatment effects with a regression-discontinuity design
Econometrica
Identification and estimation of local average treatment effects
Econometrica
Evaluating the cost of conscription in the Netherlands
Journal of Business and Economic Statistics
The impact of teacher training on student achievement: quasi-experimental evidence from school reform efforts in Chicago
Journal of Human Resources
Cited by (216)
The impact of retirement on body mass index in China: An empirical study based on regression discontinuity design
2023, SSM - Population HealthCausal analysis of central bank holdings of corporate bonds under interference
2022, Economic ModellingThe regression discontinuity design: Methods and implementation with a worked example in health services research
2022, Zeitschrift fur Evidenz, Fortbildung und Qualitat im GesundheitswesenThe impact of the send-down experience on the health of elderly Chinese women: Evidence from the China family panel studies
2022, International Review of Economics and Finance