Finance journal rankings and tiers: An Active Scholar Assessment methodology

https://doi.org/10.1016/j.jbankfin.2010.07.034Get rights and content

Abstract

This study uses respondent data from a web-based survey of active finance scholars (45% response rate from 37 countries) to endogenously rank 83 finance journals by quality and importance. Journals are further tiered into four groups (A, B, C and D) and stratified into “upper”, “middle” and “lower” tier categories (e.g. A+, A and A−) by estimating a nested regression with random journal-within-tier effects. The comprehensive and endogenous ranking of finance journals based on the Active Scholar Assessment (ASA) methodology can help authors evaluate the strategic aspects of placing their research, facilitate assessment of research achievement by tenure and promotion committees; and assist university libraries in better managing their journal resources. Study findings from active researchers in the field also provide useful guidance to editorial boards for enhancing their journal standing.

Introduction

Academic journal rankings have become an important factor in assessing the significance of research in decisions regarding tenure, promotion, remuneration and research funding. These rankings frequently serve as a broad proxy for research quality and its impact. Prevailing methods for ranking journals may be broadly classified as (i) publication citation-based methods and (ii) peer assessment methods. The citation approach attempts to measure the impact of scholarship published in a journal by counting its papers referenced by other authors. Peer assessment-based studies survey select members of the finance academic community (e.g. Chairpersons of finance departments) and ask respondents to directly rank journals in the field.

This paper carries out a web-based survey of active scholars in finance and uses respondent data to rank and tier journals in the field. The sample of active scholars in the study consists of authors who published in the most recent issues of 83 finance journals at the time of the survey. To avoid subjectivity in journal selection, the study uses a list of finance journals created by the Association of Business Schools (ABS) in the United Kingdom.1 An email requesting authors to complete the on-line questionnaire was sent to an effective survey sample of 866 active scholars, with two subsequent follow-up reminders. The survey elicited 390 responses from active finance scholars in 37 countries, yielding a response rate of 45%.

The Active Scholar Assessment (ASA) methodology of this paper may be distinguished from other journal assessment studies in some important respects and the results can be useful to authors, promotion and tenure committees, libraries and editorial boards. First, the survey sample consists of active scholars who have published in recent issues of journals in the field and may be reasonably inferred as being more aware and current in their knowledge of journal quality. Second, the ASA methodology does not ask active scholars to sequentially rank journals as in other assessment studies, but determines relative rankings as an endogenous function of active scholar perceptions of quality and awareness of each journal. We believe that this imposes a much lower cognitive and memory burden on respondents and improves the quality of survey results (for example, can respondents asked to consecutively rank journals differentiate between journals ranked in positions 6–7, or 79–80, for that matter). Third, the ASA methodology also tiers journals into four groups (A, B, C and D) based on their quality and importance rankings and uses a nested random-effects regression model to further stratify them into upper and lower tier categories (e.g. A+, A and A−). These results can be useful to tenure and promotion committees who frequently evaluate candidate publication records in terms of such categories (e.g. does a publication fall in the “A” or “B+” journal group). The regression analysis also provides insights on the relation between respondent scores, tier-levels and respondent characteristics.

Fourth, in addition to ranking and stratifying journals by perceptions of journal quality, the ASA study also provides rankings by journal importance to the field. The importance to the field score for each journal is defined as the product of the journal’s average relative quality times its percent level of awareness by survey respondents and has a simple utility interpretation. Scholars publishing in academic journals may be seen as deriving utility from a journal’s perceived quality as well as its reach or awareness within the field (the latter is positively linked to the potential of increasing a paper’s citations and research impact). For instance, in considering journals following the premier journals (e.g. top 2–3 journals), an author of a quantitative paper may be indifferent between publishing in a technically rigorous journal with smaller readership and a broader journal with higher readership. This tradeoff may be represented by utility isoquants over journal quality and level of journal awareness. This utility interpretation of the importance score offers one justification for using it to rank academic journals. In the study, we report results for journal rankings (and tiers) using both quality and the importance scores.

The paper also compares journal ranking results from the Active Scholar Assessment study with other sources including the ABS Academic Journal Quality Guide and Thomson Reuters’ ISI Journal Citation Reports. The ISI Citation Report for “Business Finance” journals ranks 48 journals of which 24 are finance journals and the remainder are from accounting and other disciplines. Furthermore, we find a more monotone and less steep descent in both quality and importance measures after the top ranked finance journals in comparison to citation-based rankings. For example, while the Journal of Finance has average quality (importance) scores of 4.84 out of 5 (78.7 out of 100), the 5th, 10th and 20th ranked journals have quality (importance) scores of 4.03 (58.3), 3.66 (35.4) and 3.31 (28.7), respectively. In contrast, citation-based metrics exhibit a much sharper decline beyond the top few citation-ranked journals and their magnitude remains small and clustered over the remaining journals (Chung et al., 2001).2 For instance, the 2009 Thomson Reuters’ ISI citation impact factors for the 1st, 3rd, 5th, 10th and 20th ranked finance journals are 4.02, 3.55, 1.63, 1.21 and 0.57, respectively (Table 3). This suggests that the quality of finance journals following the premier three journals, as perceived by active finance scholars, is higher than what citation-based methods may appear to suggest.

Some researchers including Chan et al., 2000, Arnold et al., 2003, Krishnan and Bricker, 2004 have suggested that the steep decline may be due to a self-citation group-bias among authors publishing in the premier finance journals. The more monotonic decline in quality and importance measures over journal rankings and lack of clustering suggest that the active scholar peer assessment methodology may be less influenced by this type of potential citation bias. It has also been suggested that the more gradual decline in quality across journal ranks may be due to respondent subjectivity and bias. This is considered in more detail later (Section 5.2) and we argue that the ASA survey design minimizes the effect of such potential bias.

The remaining paper is organized as follows. Section 2 describes the relation of the proposed ASA methodology to previous studies on journal assessment. Section 3 describes the survey design and data collection and the journal assessment methodology follows in Section 4. Results on journal ranks and tiers are presented and discussed in Section 5. This section also reports the results from nested random-effects regression analysis used to stratify journals into upper and lower tier categories within tiers and evaluate the impact of respondent characteristics. Section 6 concludes the paper.

Section snippets

Literature review

Methodologies for ranking journals are typically categorized as (i) objective measurement or (ii) peer assessment. The most common objective measures are citation indices (e.g. Thomson Reuters ISI) or citations impact measures. More recent metrics include SSRN downloads (Brown, 2003) and Google Scholar citation numbers (Law and Van der Veen, 2008). Peer assessment methodology relies on assessments of journal and rankings by peers and qualified experts. They are increasingly used as a method for

Survey design and data collection

This section describes the survey design used to select the study’s active scholar sample and the data collected from the on-line survey. Response rates and summary statistics for respondent characteristics are also provided.

Methodology and data

The methods used to rank journals by quality and importance to the field and tier them into four groups (A, B, C and D) are described in this section. Journals within each tier are further stratified into upper, middle and lower categories (e.g. A+, A, A−, B+, B, B−, etc.) using nested random-effects regression modeling.

While the ranking and tiering procedure requires construction of journal-level metrics from respondent data, the regression analysis uses respondent-level data. In addition to

Results and discussion

The results from applying the journal assessment methodology in Section 3 are reported and discussed in this section. Journal rankings and tiers from the ASA study are also compared with the same from Thomson Reuters Journal Citation Reports (ISI) and ABS Academic Journal Quality Guide. The homogeneity of journals within each tier is also investigated using a nested regression specification with random journal-within-tier effects. This analysis provides insights into the relative position of

Conclusion

This study carries out a web-based peer assessment of active finance scholars and uses respondent data to rank and tier 83 finance journals. Within each tier, journals are further stratified into upper, middle and lower categories (e.g. A+, A and A−) using a nested random-effects regression. In addition to rankings by journal quality (1–5 scale), the study also provides rankings by journal importance to the field (defined as the product of average journal relative quality and awareness) which

References (28)

  • K.A. Borokhovich et al.

    A framework for journal assessment: the case of the Journal of Banking and Finance

    Journal of Banking and Finance

    (2011)
  • B. McKercher et al.

    Rating tourism and hospitality journals

    Tourism Management

    (2006)
  • J.C. Alexander et al.

    Relative significance of journals, authors, and articles cited in financial research

    Journal of Finance

    (1994)
  • T. Arnold et al.

    Impact: what influences finance research

    The Journal of Business

    (2003)
  • S.F. Borde et al.

    A note on perceptions of finance journal quality

    Review of Quantitative Finance and Accounting

    (1999)
  • K.A. Borokhovich et al.

    Finance research productivity and influence

    Journal of Finance

    (1995)
  • K.A. Borokhovich et al.

    Analysis of finance journal impact factors

    Journal of Finance

    (2000)
  • L.D. Brown

    Ranking journals using social science research network downloads

    Review of Quantitative Finance and Accounting

    (2003)
  • K.C. Chan et al.

    Production in the finance literature, institutional reputation and labor mobility in academia: a global perspective

    Financial Management

    (2002)
  • K.C. Chan et al.

    Citation-based finance journal rankings: an update

    Financial Practice and Education

    (2000)
  • K.H. Chung et al.

    Patterns of productivity in the finance literature: a study of bibliometric distributions

    Journal of Finance

    (1990)
  • K.H. Chung et al.

    Citation patterns in the finance literature

    Financial Management

    (2001)
  • R.A.K. Cox et al.

    Patterns of research output and author concentration in the economics literature

    The Review of Economics and Statistics

    (1991)
  • R.K. Coe et al.

    Evaluating the finance journals: the department chairperson’s perspective

    Journal of Financial Research

    (1983)
  • Cited by (0)

    View full text