Economics of evaluation (with special reference to promotion and tenure committees)

Lucas (1988), p. 38.

Heckman (2014), p. 128.

It is possible that universities that are ranked lower might fear an RG more than an AB because tenuring one who becomes a star could have a big positive impact on such a department. The problem of using publication quality to evaluate faculty has received recent attention [Heckman (2017) and Powdthavee et al. (2017)].

The top seventy-five economics departments are not necessarily the top seventy-five universities. However, I prefer to use a ranking that is more familiar to economists, one of our own profession. Also, at least the top departments on the list are in universities that are generally highly ranked. I stopped at seventy-five universities because, after NYU (number six), excluding Cal Tech, whose policy I could not determine, all had outside committees. Although I know of lower ranked departments without outside committees, clearly the usual policy in a wide range of universities is to have an outside committee. Non-U.S. universities are not considered herein because they may have institutional features that differ from those in the U.S.

Some departments have more than two committees. I do not distinguish between universities other than if they have at least one external (to the department) committee. It was difficult to find procedures for some universities, and sometimes policies are not clearly delineated. A table containing the sources for these results is available from the author by request.

I do not consider outside letters in the analysis herein. Such letters represent information that is available at all levels of academic evaluation, and are thus inputs and not a formal vote as occurs with university committees.

See the articles by Sah and Stiglitz (1986, 1988) and the personnel economics textbooks by Lazear and Gibbs (2009, 2015).

I ignore the fact another entity (e.g. a board of trustees) technically may have the final decision, given that such entities rarely fail to follow the recommendation of the top administrator involved in promotion and tenure decisions.

Lazear (1995), p. 7. For additional discussion of how firms can learn from academic research, see Lazear and Shaw (2011).

Why tenure exists has been considered elsewhere. For example, see Carmichael (1988, 2001), Aghion and Jackson (2016), and Prendergast (2015). I discuss Carmichael’s argument for tenure in Sects. 3 and 9.

In the Sah and Stiglitz (1986) model, either one or two evaluators approve a project. With two evaluators, both must approve. Herein, technically, there is a hierarchy when just the department committee and the administration exist. However, as shown in Sect. 3, if the administration never goes against a department recommendation, then essentially there is a flat structure when there is only the department committee. Thus, to be consistent with the previous literature, I will refer to that case as a flat structure.

Lazear (1995) discusses how salespersons must report to a manager to get a final decision. With more knowledge about appropriate decisions, it is likely managers in business are more willing to overrule subordinates than are academic administrators.

I ignore budgetary issues herein. Faria et al. (2013) suggest that a department with a small budget is less likely to award tenure to someone is who highly paid. I expect that is not a serious problem. Someone who is highly paid prior to tenure is likely to be in a department with a relatively large budget. A department with a small budget would likely have lost a productive junior faculty member before tenure because such an individual would be a good candidate to be raided by another department. In general, budgetary problems may result in fewer individuals hired in a department, or fewer hired to tenure track positions. I am interested in the structure of evaluation for tenure and promotion given a department that has such decisions to make.

Akerlof and Michaillat (2017) have a different probability of making the two kinds of mistakes. They only have one level of evaluation.

Lazear and Gibbs (2009) consider a situation with two evaluators who each review N projects per period. With a flat structure, 2N projects are evaluated, but, with a hierarchy (each project reviewed by both evaluators), only N projects are evaluated. Although a hierarchy results in a higher rate of good applicants rejected, the total number rejected is lower with a hierarchy because only one half as many projects are evaluated with the hierarchy as are considered with the flat structure. In the problem herein, the number of candidates evaluated is the same regardless of which structure is used.

Faria et al. (2013) note that the ideal data set would “...follow individual faculty over time, and include faculty who were denied tenure and promotion at a particular institution. Unfortunately, the collection of such a sample would be prohibitively difficult, if not impossible.” Additionally, for my purposes, there would have to be a way of determining if an individual turned down for tenure was favorably viewed by any recommending committee, and if one who received tenure was subsequently viewed as a mistake by his or her department. This hypothetical data set is even less likely to be obtained than the one described by Faria et al. (2013).

The denominator of \(t_{B}\) in Eq. (3), call it D, is clearly positive. \(D=p + \rho {-} 2p \rho = (p- \rho )^{2} + p (1 {-}p) + \rho (1 {-}\rho ) > 0\).

Lazear and Gibbs (2009) refer to a hierarchy with \(t > 0\) as a second opinion structure. What they call a hierarchy has two levels of evaluators with \(t = 0\). One of their claims is that a second opinion structure has the lowest rate of rejecting good candidates. However, this cannot be true in general since, with t small enough, their second opinion structure is essentially the same as their hierarchy. I find a flat structure has the lowest likelihood of rejecting a good candidate if \(t < t_{G}\). Compared to the analysis herein, the second opinion structure is like having one committee and an administration, where the latter may accept the committee’s recommendation.

In a previous version of this paper, I considered the cases when the administration treats either committee as supreme in that tenure is never granted with a split vote if (1) the college committee had the favorable recommendation, or, alternatively, if (2) the department committee had the favorable recommendation. In the first case, I found \(t_{B}=t_{G} = 1\): a flat structure always accepts more bad candidates and rejects fewer good candidates than would occur with a hierarchy. In the second case, if \(p > \rho \), then \(t_{B} < t_{G} = 1\), so it is possible a flat structure accepts fewer bad candidates than would a hierarchy (when \(t > t_{B}\)). However, if \(p < \rho \), then \(t_{B}=t_{G} = 1\).

I assume the administration never goes against committee recommendations for tenure unless there are two committees that disagree. In hiring decisions, senior administrators rarely overrule department recommendations. One noteworthy case occurred in 1946. The Department of Economics at the University of Chicago wanted to hire George Stigler. The university president, Ernest C. Colwell, believed Stigler was too empirical, and vetoed the appointment (Wallis 1993). Also, in 1938, Harry Gideonese, an opponent of President Robert Hutchins, was unanimously recommended for tenure by the economics faculty at Chicago, but was rejected by the university administration (Ebenstein 2015).

Prendergast (2015) suggests universities differ in how much administrators intervene in the evaluation of candidates for tenure. Herein, such a difference implies that t varies among universities. Prendergast is interested in how different control rights affect the kinds of activities undertaken by candidates for tenure, an issue that is ignored herein.

For brevity, I will mainly consider the case when \(\rho > p\) in this section. Flipping the inequality simply flips which structure is superior.

Thus, \(\vert {\varDelta }_{\textit{RG}} \vert > \vert {\varDelta }_{\textit{AB}} \vert \) so, if \(C_{\textit{AB}} > C_{\textit{RG}}\), the RHS of in eq. (23) may exceed, equal, or be less than one.

Another question raised by a reader of this paper is, with only one committee, if \(\rho > p\), why is not the outside committee the one committee? The best answer to that point is the one just made in the text: without seeing the department’s honest evaluation, the outside committee would not likely be more accurate than the department committee.

If departments compete for faculty positions, faculty on an outside committee might be negative on a tenure decision in the belief that their departments would ultimately receive a position if someone is turned down for tenure. While possible, I do not see this as a serious problem. Since many departments would generally have faculty on an outside committee, there is not likely a high probability of any one department gaining a position if another department has someone turned down for tenure.

The administration does not behave strategically for the same reason the faculty committees do not do so: the administration’s goal is to tenure good candidates and not tenure bad candidates. Also, as argued in Sect. 7, the administration is likely to play a different role than do faculty committees, and is primarily worried about difficulties involved in either decision with a split recommendation from evaluating committees.