Rating the Raters: Evaluating how ESG Rating Agencies Integrate Sustainability Principles

Abstract

Environmental, social, and governance (ESG) rating agencies, acting as relevant financial market actors, should take a stand on working towards achieving a more sustainable development. In this context, the objective of this paper is, on the one hand, to understand how criteria used by ESG rating agencies in their assessment processes have evolved over the last ten years and, on the other hand, to analyze whether ESG rating agencies are contributing to fostering sustainable development by the inclusion of sustainability principles into their assessment processes and practices according to the ESG criteria. This research is based on a comparative descriptive analysis of the public information provided by the most representative ESG rating and information provider agencies in the financial market in two periods: 2008 and 2018. The findings show that ESG rating agencies have integrated new criteria into their assessment models to measure corporate performance more accurately and robustly in order to respond to new global challenges. However, a deep analysis of the criteria also shows that ESG rating agencies do not fully integrate sustainability principles into the corporate sustainability assessment process.

1. Introduction

There has been a significant development of sustainable and responsible investment (SRI) in the last ten years. Investors, shareholders, governments and firms have benefitted from this since they request accurate information not only regarding financial performance but also about environmental, social and governance (ESG) aspects, which has become part of their competitive strategy [1]. These factors have given rise to the inevitable appearance of ESG rating agencies.

ESG rating agencies scrutinize businesses and assess corporate sustainability performance by using their own research methodologies. This expertise has turned ESG rating agencies into a key reference for companies, financial markets and academia in terms of corporate sustainability assessments. Consequently, the sustainability rating market has grown considerably in the last decade—in keeping with their influence—to the point of being studied not only as economic actors but also as social actors, which have an impact on the behavior of other social actors in society [2]. However, if social impact is not internalized and ESG rating agencies act only as economic actors, the messages launched by rating agencies about what could a sustainable company be or how corporate sustainability performance could be measured might be misrepresented. This could be overcome if the expectations held by society and rating agencies about sustainability and sustainable development are matched. Consequently, providing society with misleading information about corporate sustainability can affect the social legitimacy and trust of both companies and ESG rating agencies.

In this context and considering the remarkable changes the rating agency industry has undergone in the last years, some issues have arisen: What concept of corporate sustainability has consolidated regarding the ESG assessment criteria in the last decade? Have these assessment criteria used by ESG rating agencies changed over time considering the processes of mergers and acquisitions or the new challenges in terms of sustainability? Are these rating agencies contributing to achieving a more sustainable development?

Previous studies have already been conducted with the aim of analyzing the historical evolution, growth and consolidation of ESG rating agencies and their strategies [3,4], their assessment frameworks and weighing systems [5,6,7,8]. Moreover, the question regarding the implications in terms of the definition of corporate sustainability derived from the criteria adopted by ESG rating agencies has previously been addressed [9], by analyzing their contribution to more sustainable business models. However, to our knowledge, the analysis of their efforts to achieve adequate sustainable development has not yet been analyzed.

In this paper, the theoretical foundation for the study of the ESG rating agency market is based on the institutional theory, since this theory emphasizes the relationship between organizations and how companies conform to norms as a consequence of pressures placed by institutions on the setting to which they belong [10]. Specifically, the focus is over the study of the relationship between ESG rating agencies and society in general, beyond their business scope, by means of the reflection around how ESG rating agencies are operationalizing sustainability in their assessment frameworks. In this paper, we use the terms “sustainable development” and “sustainability” interchangeably even though the debate regarding possible differences between the terms remains unsolved; sustainability and sustainable development are commonly used as synonymous [11].

Therefore, the purpose of this paper is twofold: First, to know how the criteria used by ESG rating agencies in their assessment process have evolved over time and second, to analyze whether ESG rating agencies are contributing to achieving a more sustainable development by the inclusion of sustainability principles [12] into their assessment processes and practices. As a result, we analyze whether ESG rating agencies integrate the main sustainability principles into their assessment processes: (i) the sustainability dimensions (financial economic, environmental and social) and the balance among them (ii) the intergenerational perspective, (iii) the stakeholder approach and (iv) the life-cycle thinking.

This analysis is based on a comparative study of the most representative ESG rating and information provider agencies in the financial market. This paper first explores how their assessment criteria have evolved in the last 10 years, followed by an examination of how ESG rating agencies have worked towards achieving sustainable development.

The main findings of this paper allow us to understand the integration of sustainability principles into the financial markets system and to extend the prior insights from Escrig-Olmedo et al. [7] that present an overview on the different assessment criteria adopted by sustainability indices and ESG rating agencies; Avetisyan and Hockerts [4] explain the causes and impact of the ESG rating agency consolidation process, and Muñoz-Torres et al. [9] analyze whether assessment methods employed by ESG agencies are identifying and/or driving more sustainable business models.

The results of this work are especially relevant to understanding the behavior of these organizations and their evaluation processes, considering the EC Sustainable Finance Action Plan, The Sustainable Development Goals (SDGs), Paris 2016 Agreement on Climate Change and the increasing emphasis on the monitoring of corporate sustainability performance.

The paper is structured as follows. The next section presents the extant literature and includes a descriptive analysis of the ESG rating agencies industry evolution. After that, the data and methods are presented and the main findings are remarked. Finally, the implications, contributions and limitations are discussed in the concluding section of the paper.

2. Theoretical Background

According to Nawaz and Koc [13], who adopted the systematic methodology proposed by Tranfield et al. [14], we developed our literature review following the next steps:

Step 1—Identification of keywords: In particular, the search included the following terms: “rating agencies + corporate social responsibility” or “rating agencies + sustainability” or “rating agencies + sustainable development” or “rating + development goals”. Step 2—Screening criteria: The search was limited to papers published in the English language between 2000 and 2018. Step 3—Identification of search engines: The academic databases used by the literature search were Web of Science and Google Scholar. Step 4—Execution of search (total number of returned results): The search was carried out in November 2018 and led to the identification of 78 articles in Web of Science and 478 articles in Google Scholar. Step 5—Number of articles selected after initial screening: In this step, we reviewed the title and abstract to demonstrate a clear connection between ESG rating agencies and corporate sustainability. Finally, 45 articles were selected after this systematic review. Step 6-Number of articles selected after final screening and research gap detection: After a careful reading of the identified articles, 44 papers were selected as relevant publications to set the theoretical framework of this paper in order to develop the body of knowledge concerning the behavior of ESG rating agencies. As was stated in the previous section, most of the papers are focused on assessment methodologies and their results; however, scarce information is provided related to how aligned they are with the concept of sustainability.

2.1. ESG Rating Agency Industry

The current context demands companies to contribute to sustainable development [15] by means of corporate strategies that integrate sustainable practices into their activities with the aim of achieving corporate sustainability. Corporate sustainability, therefore, requires a commitment between the present environmental, social and economic needs of a firm’s stakeholders and their future needs.

According to Lozano [16], corporate sustainability refers to “corporate activities that proactively seek to contribute to sustainability equilibria, including the economic, environmental and social dimensions of today, as well as their interrelations within and throughout the time dimension (i.e., the short-, long- and longer-term), while addressing the company’s systems, i.e., operations and production, management and strategy, organizational systems, procurement and marketing, and assessment and communication, as well as with its stakeholders”.

Assessing corporate sustainability is gaining momentum since the financial market is paying growing attention to this issue [17]. However, most existing tools, frameworks and mechanisms to measure corporate sustainability are not adequate [18].

ESG rating and information provider agencies (the so-called corporate social responsibility (CSR) ratings, social ratings, sustainability ratings or SRI ratings agencies) have emerged in response to the demands of socially responsible investors that require social and environmental information of companies so to invest in more sustainable companies [4].

ESG rating agencies assess the corporate sustainability performance of a large number of companies. Some ratings are based exclusively on extra-financial information while others combine financial and extra-financial data to assess long-term value and sustainability [19].

They use a large amount of information obtained from the companies themselves through questionnaires that involve a laborious process and the analysis of public information (reports, news, etc.) examined by interdisciplinary work teams in different geographical areas.

In the last decade, the ESG rating industry has grown considerably and has already undergone a phase of consolidation, not only with merger and acquisition processes among the existing ESG rating agencies but also through the new entrance of financial rating and information provider agencies.

The beginning of the financial crisis in 2008 brought about a positive shift in capital market perceptions and attitudes towards corporate sustainability [20]. As Figure 1 shows, the ESG rating agency market has gone through a concentration process in the last 10 years (from 2008 to 2018). Financial stakeholders and investment decision makers tend to be the main incentives to this growth and concentration process [21,22], which shows the permanence of the corporate social responsibility movement [19].

ESG rating agencies stop being isolated market actors oriented to a limited financial market niche to become a promising and dynamic business even for “traditional” rating agencies. This trend could be illustrated, for example, by the case of Morgan Stanley Capital International (MSCI). MSCI is one of the current ESG rating agencies, and MSCI analyzes the environmental, social and governance-related business practices of thousands of companies worldwide.

MSCI is the result of the absorption of several ESG research providers. In 2010, MSCI acquired RiskMetrics Group, a provider of risk management and governance products and services. RiskMetrics had previously bought ISS (Institutional Shareholder Services) in 2007, Innovest Strategic Value Advisors in February 2009, and Kinder Lydenberg Domini (KLD) Research & Analytics in November 2009. The last two are now known as MSCI ESG Research. In addition, in July 2010, MSCI acquired MeasureRisk, a provider of risk transparency and risk measurement tools for hedge fund investors.

Afterwards, in August 2014, MSCI acquired Governance Holdings Co. (GMI Ratings), a provider of corporate governance research and ratings, and in January 2013, it bought InvestorForce, a provider of performance reporting tools to the institutional investment community in the United States.

MSCI is an example of how a big data provider has expanded their scope as a provider of ESG information for institutional investors, including the world’s most important mutual funds, pension funds and hedge funds. Furthermore, MSCI ESG Research data and ratings are used in the construction of the MSCI ESG Indices.

This example shows how a large number of agencies have cropped up, while others have disappeared—most often taken over by a competitor. According to Avetisyan and Hockerts [4], it is possible to observe two growth strategies: (i) organic growth and partnerships, which is the establishment of a network of alliances (e.g., RobecoSAM), and (ii) mergers and acquisitions, that happen when two or more ESG rating agencies merge and join forces (e.g., Vigeo-EIRIS merge) or when prior financial data providers and assessment managers decide to enter into the ESG rating industry (e.g., MSCI).

This concentration process has allowed ESG rating agencies to develop wider and integral assessments of corporate sustainability, considering that sustainability is a multidimensional concept. As Figure 1 shows, current ESG rating agencies have integrated specialized actors in corporate governance, data management, risk or communication into their systems. In addition, this market change has led to the emergence of more professional, multidisciplinary and multicultural work teams and the extension of the geographic and sectorial reach.

In this context, the first research question arises: How have the criteria used by ESG rating agencies in their corporate sustainability assessment process evolved over time, considering the processes of mergers and acquisitions?

The analysis of this evolution will evidence whether the concept of corporate sustainability understood by rating agencies has changed over time and in what sense.

2.2. Have ESG Rating Agencies Been Able to Integrate Sustainability Principles into Corporate Sustainability Assessment?

The development of the ESG industry is mainly due to the combination of financial factors and market power [4]. On the one hand, the efforts made by large listed companies in terms of sustainability management and reporting during the last years [23,24] confirm the existence of an active strategy for being well-positioned in ESG ratings. As Mackenzie et al. [25] highlight, from the institutional theoretical view, the propensity to invest in sustainability management may be influenced by factors such as pressure groups, regulation, the organization’s competitive position and the dialogue with stakeholders; in particular, the authors found that belonging to an ESG stock index encourages companies to improve their sustainability management since this fact sends positive signals to shareholders and the rest of the stakeholders. On the other hand, outcomes derived from ESG rating agencies assessments have been frequently used by academics and the research community (e.g., Reference [26]).

Consequently, ESG rating agencies influence not only the behavior of financial market actors [27] but also the institutionalization of sustainability management in companies and are being considered as “institutional entrepreneurs” [2]. Therefore, the responsibility of ESG rating agencies cross financial market boundaries since the consequences of dynamic changes in corporate sustainability assessments by ESG rating agencies go beyond the scope of financial markets, affecting the society as a whole.

Bearing this influence in mind, the analysis of how ESG rating agencies could make a positive contribution to the sustainable development is a key challenge to overcome. Consequently, a second research question arises: Are ESG rating agencies contributing to achieving a sustainable development by the inclusion of sustainability principles into their assessment processes and practices?

Each ESG rating agency uses its own corporate sustainability assessment methodology. The diverse methodologies seem to be associated to a market-led strategy of differentiation [22] and to cultural and ideological factors [28]. However, in the different assessment processes of these ESG rating agencies, three aspects of measurement are always considered: the high-level categories evaluated (environmental, social and governance) and the positive criteria included in each category; the controversial activities and practices evaluated; and the normalization process of their ratings by the industry [6].

The proliferation of these rating agencies and the diversity of their assessment methodologies have posed several challenges to be met:

(i)

Lack of transparency. ESG rating agencies do not offer complete and public information about the criteria and the assessment process developed by them to evaluate the corporate sustainability performance. This makes understanding what ESG rating agencies are measuring and making comparisons between them difficult [7,19,22,29].

(ii)

Commensurability. ESG rating agencies may measure the same concept in different ways. Therefore, if the assessments of ESG ratings are not consistent, which involves evidence of low commensurability, the hypothesized benefits of CSR cannot occur [6].

(iii)

Trade-Offs among criteria. ESG ratings methodologies may compensate higher scores in one domain with very low scores in another domain [17,30,31].

(iv)

Lack of an overall score. Most of the ESG rating agencies provide environmental, social and governance rates to each domain, but they do not provide an overall score of the corporate sustainability performance [32].

(v)

Stakeholders’ preferences. ESG rating agencies do not address the different stakeholders’ expectations in their evaluation processes, which influences their acceptance and usefulness [17,33].

These shortcomings recommend caution in the interpretation of ESG rating agency outcomes. As Busch et al. [34] wonder, “If ESG data do not reliably and validly reflect organizational reality […] how can sustainable investment practices contribute to sustainable development?” Moreover, taking into account the abovementioned evolution of the ESG rating industry in the last decades (Figure 1), together with the increase of power of the financial rating industry, “The desired new norms of the SRI movement are therefore at risk of being reabsorbed by the traditional collective norms of the financial rating industry,” as Avetisyan and Hockerts [4] highlight.

Our proposition is that the inclusion of sustainability principles into ESG rating agency assessment processes and practices is a necessary condition for minimizing this risk: that is also a risk in the process of the institutionalization of the corporate sustainability concept.

2.3. Sustainability Principles

In the last years, sustainable development has become a strategic objective for business and governments and since the launch of the UN Sustainable Development Goals (SDGs) in 2015, the global community has had a new framework to address the most urgent global problems.

Sustainable development is defined in the Anthropocene as “the development that meets the needs of the present while safeguarding Earth’s life-support system, on which the welfare of current and future generations depends” [35].

This definition describes sustainability within a broad and complex context. The work of Muñoz-Torres et al. [12] presents a debate regarding the main sustainability principles associated to the concepts of sustainability and sustainable development. The authors, based on seminal references in the sustainability field [36,37,38,39], propose four basic and common conceptual Principles of Sustainability, which could be generally accepted across different disciplines [40]. These principles are as follows:

(i)

Sustainability dimensions and the balance among them. Sustainability is a multidimensional concept [41] where three domains can be distinguished: financial economic, environmental and social. Achieving a balance among the three domains implies not prioritizing or undermining a dimension over the other ones.

(ii)

Intergenerational perspective. This principle implies identifying, evaluating and managing the future and the current needs [39], considering the long-term effects of today’s decisions and a balance between both short- and long-term ones.

(iii)

Stakeholder approach. Sustainability involves identifying the current stakeholders’ needs and expectations and the future generations’ needs.

(iv)

Life-cycle thinking (LCT). Decision-makers should shoulder economic, environmental and social responsibilities in order to achieve sustainability which cross legal organizational boundaries. In this regard, sustainability involves managing the impacts of upstream and downstream activities and, accordingly, adopting an LCT approach.

These principles are consistent with previous works such as Wass et al. [11], who highlight the following sustainability principles: i) the normativity principle: sustainable development is influenced by our values and reflects the kind of world we want; ii) the equity principle among generations and species; iii) the integration principle opposed to the concepts such as “balancing” or “trading-off” sustainability dimensions; and iv) the dynamism principle: sustainable development is a process submitted to change.

Nevertheless, the present paper follows Muñoz et al.’s [12] sustainability principles because they facilitate the operationalization of the sustainability concept in an organizational and life-cycle thinking context, and it is consistent with previous proposals.

3. Methodology

In order to answer our research questions, we developed a qualitative analysis in two stages.

First, we selected eight ESG rating and information provider agencies that are the most representatives of the European and USA SRI market, considering the number of companies analyzed (between 4,000 and 20,000 companies), their clientele, countries and markets covered (see

Table 1. The sample.

ESG Rating Agencies and Information Provider Agencies 1 (2008) → (2018)	Number of Companies Analyzed by ESG Rating Agencies	Other Important Facts
ASSET4 → REFINITIV ECP → ECP EIRIS → FTSE Russell ESG Ratings 2 KLD Research & Analytics Inc. → MSCI ESG Research Oekom → ISS-oekom SAM → RobecoSAM SIRI Company → Sustainalytics Vigeo → Vigeo EIRIS	Around 22,000 companies Around 4000 companies Around 4100 companies Around 7000 companies Around 20,000 companies Around 4500 companies Around 9000 companies Around 4000 companies	Companies across 87 countries Companies across 47 developed and emerging markets It is used by 46 of the top 50 asset managers and by 1200 investors worldwide. 2000 institutional clients, 115 markets covered. Companies across 60 countries Expanded to 11,000 companies in Q2 2019. A multicultural team of 145 analysts

Notes: ¹ Colum 1 shows the ESG rating agencies analyzed and their evolution over the time after a convergence process. ^2. In 2008, the FTSE4 Good Index was carried out by EIRIS. The current FTSE Russell uses its own methodology. Source: Own creation (data December 2018–January 2019).

). These rating agencies, despite their merger and acquisition processes, have maintained their assessment structures or have integrated their expertise in other comprehensive assessment structures (Figure 1). These allow the comparability of the ESG rating agencies criteria along the time.

The analysis was developed in two periods: 2008 and 2018. These two periods are relevant for the ESG rating agency industry because, after the fall of Lehman Brothers in 2008, which marked the beginning of the financial crisis and revealed the limitations of traditional measurement models of corporate performance and risk analysis, the financial market began to consider corporate sustainability as an important factor [20]. For instance, the way KLD Research & Analytics Inc., a leader in ESG research, was acquired by RiskMetrics (2009) and later by MSCI (2010) showed how the traditional financial information and risk analysis agencies began to consider ESG information necessary for giving a better assessment of corporate performance. Moreover, in 2018, the SRI market has grown exponentially due to stakeholders’ demands of accurate information regarding company performance. In fact, sustainable investing assets reached $12 trillion in the United States, which represented a 38% increase in 2016 [42]. In Europe, SRI assets represented around 11 trillion euros [43].

Continuing the process, we examined the relevant environmental, social and corporate governance criteria used by these ESG rating and information provider agencies to evaluate corporate sustainability through a content analysis method [44]. We developed a thematic content analysis that led to the identification of common themes in different types of texts. Furthermore, as Beattie et al. [45] highlight, it is a holistic content analysis in which the complete text is analyzed.

Following Bardin [46], the thematic content analysis is divided in three steps:

(i)

Pre-analysis: We established the objectives of the content analysis, and we selected the material for analysis. Concretely, we analyzed the qualitative and public information provided by the ESG rating agencies on their corporate websites concerning their corporate sustainability analysis criteria in 2008 (see Escrig-Olmedo et al. [7]) and 2018. We browsed the company website in order to seek information on the corporate sustainability assessment, sustainability and ESG data, ESG research and ratings until five clicks.

(ii)

Exploration: We defined the unit of analysis and codes. First, we identified themes for examination considering current sustainability global goals, and apart from this, following Escrig-Olmedo et al. [7] and Muñoz-Torres et al. [9], we defined certain codes (categories).

(iii)

Treatment and interpretation: We examined the relevance and the presence or absence of the themes in the content analyzed. We considered that a theme is relevant if it is public on the ESG rating agency website and, moreover, if it is highlighted as a key criterion in the assessment process of the ESG rating agency. Finally, we quantified the percentage of ESG rating agencies in the sample that offered information about specific assessment criteria.

Secondly, following Muñoz-Torres et al. [12], we examined whether ESG rating agencies integrate the main sustainability principles into their sustainability assessment frameworks. To that end, we studied whether the ESG criteria used by ESG rating and information provider agencies are consistent with these sustainability principles. To this analysis, three researchers acted as primary coders and after that, two researchers checked the result and resolved inconsistencies [47,48]. On the other hand, we examined specific aspects of their methodologies (such as the aggregation process of scores).

To determine if an ESG rating agency integrates sustainability principles into their assessment methodologies, the following arguments were taken into account:

(i)

Sustainability dimensions and the balance among them. An ESG rating agency integrates this principle if in its assessment process and methodologies, environmental, social and governance criteria are equally important.

(ii)

The intergenerational perspective. An ESG rating agency integrates this principle if the criteria or assessment methodology aspects such as the future and the current needs or specific risks considering both the short and long term are explicitly laid down.

(iii)

Stakeholder approach. An ESG rating agency integrates this principle if stakeholders’ needs and expectations are integrated in its criteria and assessment process.

(iv)

Life-cycle thinking (LCT). An ESG rating agency applies this principle if the way the company impacts upstream and downstream activities, in accordance with the adoption of a LCT approach, are examined in its criteria and assessment frameworks.

4. Results

This section first shows the results of the comparative study of the positive assessment criteria used by the ESG rating and information provider agencies in 2008 and 2018 and then discusses the main findings about the integration of the sustainability principles into their assessment processes and practices.

The ESG criteria in the two years analyzed are the same; however, the integration of these criteria in the assessment frameworks of ESG rating agencies has changed in these two periods.

By performing an analysis of the results by dimensions (environmental, social and governance), we can highlight that regarding the evolution of environmental criteria (Figure 2), a change of trend is observed in the screening criteria used by the ESG rating and information provider agencies. Specifically, in 2008, the most widely used analysis criteria were environmental policy/management (100%), emissions (62.5%) and climate change (50%) while in 2018, the main criteria analyzed were environmental policy/management (87.5%), water use and management (87.5%) and protection of biodiversity (87.5%). In addition, the aspects that have been incorporated into the assessment process of ESG rating agencies in 2018 were climate change (75%), emissions (75%) and waste management/reduction (75%). The results show a greater interest in environmental concerns than 10 years ago. As a result, in 2008, the assessment of the corporate environmental performance focused principally on the analysis of environmental management policies or systems while in 2018, this assessment framework was complemented by the inclusion of new criteria linked to the efforts of large companies to reduce emissions, consumption, etc. This greater interest in combating climate change and in mitigating greenhouse-gas emissions reflects how the agreements reached at the 21st Conference of the Parties (COP21) in Paris (December 2015) are having a strong impact on the assessment of corporate sustainability performance.

Regarding the social pillar criteria (see Figure 3), the aspects that have been mainly considered in the assessment process of ESG rating agencies analyzed in 2008 were human capital development and training (100%), human rights (87.5%) and community relations (87.5%). On the contrary, the aspects incorporated into the assessment process of all ESG rating agencies in 2018 were labor management, human rights and quality working condition, health and safety. After the definition of Sustainable Development Goals (SDGs), aspects related to the improvement of health and education, the reduction of inequalities and the necessity to spur economic growth seem to be crucial to measure how companies contribute to sustainable development.

However, issues related to business behavior, talent attraction and market ethics seem less important. It is also interesting to highlight that criteria related to supply-chain management and security and data protection have arisen in the last ten years, which reflects the new trends in sustainability assessment that focus on more complex and integrated productive configurations instead of on companies as isolated structures.

Finally, concerning the corporate governance aspects (Figure 4) considered by most ESG rating and information provider agencies in the period 2008, we can highlight that corporate governance functions and committees (100%), board structure (75%) and remuneration/compensation policy (62.5%) were the most valuable criteria. These criteria, generally included in the corporate governance codes, were regarded as important in 2018. By contrast, those criteria that involve abstract concepts with measurement challenges (e.g., brand management or vision and strategy) are not usually used as a key criterion in the public information of ESG rating agencies. However, the increase in the prevention of corruption and bribery issues and in the transparency issues stresses a significant difference between 2008 and 2018, since they are now the second most analyzed aspects in the assessment process (87.5%). As in the case of environmental concerns, aspects related to the governance of companies seem to be gaining importance into the assessment frameworks, especially those related to the prevention of corruption and bribery; international organizations such as OCDE are defining these concerns through a series of recommendations to successfully attack corruption both in the public and private sector.

However, are these criteria consistent with the sustainability principles? To answer this question, it is necessary to thoroughly examine each ESG rating agency individually.

By drawing on the concept of how each ESG rating agency integrates sustainability principles into their assessment processes and practices, we followed the conceptual framework developed by Muñoz-Torres et al. [9] (see

Table 2. The sustainability principles in ESG rating agencies.

ESG Rating Agencies	(1) Sustainability Dimensions 1	(2) Balance	(3) Intergenerational Perspective	(4) Stakeholder Approach	(5) Life-Cycle Thinking
REFINITIV	EC EN SO CG	✕	✕	✓	✕
ECP	EN SO CG	✓	✓	✓	✕
FTSE Russell ESG Ratings	EN SO CG	✓	✕	✓	✕
MSCI ESG Research	EC EN SO CG	✓	✓	✓	✕
ISS-oekom	EN SO CG	✓	✕	✓	✕
RobecoSAM	EN SO CG	✕	✓	✓	✕
Sustainalytics	EN SO CG	✕	✓	✓	✕
Vigeo EIRIS	EN SO CG	✓	✕	✓	✕
ESG rating agencies	EC EN SO CG	≠	≠	✓	✕

✓ Yes, ✕ No, ≠ Partially and —Not explicit; ¹ Financial Economic (EC), Environmental (EN), Social (SO) and Corporate Governance (CG). Source: Own work.

) who have been able to state that they do not fully integrate them into their corporate sustainability assessment process.

Although all ESG rating agencies incorporate the sustainability dimensions (or pillars) in their assessment process, not all of them evaluate sustainability in a balanced way. For example, RobecoSAM does not consider the three dimensions of sustainability in a balanced way, since each dimension has a specific weight in the assessment and it is not clear how it has been defined, as public information about assessment is based on sectoral matters.

Concerning the stakeholder approach principle, all the ESG rating agencies in the sample are gradually integrating the needs and expectations of a variety of stakeholders, as can be seen from the analysis of the assessment criteria used by them (Figure 2, Figure 3 and Figure 4). These criteria assess the degree of compliance with the expectations and needs of different stakeholders, such as the rights of indigenous people, local suppliers, customer relationship management and satisfaction, etc.

ESG rating agencies partially integrate risk assessment processes that incorporate the intergenerational perspective. The public information of Refinitiv, FTSE Russell, ISS-Oekom and Vigeo-EIRIS do not allow analysts to perceive how they manage risks connected to the future generation needs or expectations.

Finally, there is no evidence that ESG rating agencies integrate life-cycle thinking as an explicit principle in their sustainability assessment. As mentioned above, some of these agencies are, to some extent, beginning to analyze the company management of their supply chains, but this does not mean applying an explicit life-cycle thinking that requires different and specific assessment methodologies [12]. Therefore, there is a clear need to advance in the definition of tools and initiatives that allow practitioners to integrate sustainability principles into the assessment of corporate performances.

5. Discussion and Conclusions

This paper explores how the ESG rating agency industry and the criteria they use in the assessment process have evolved over the last ten years and examines whether ESG rating agencies are now contributing to a more sustainable development by the inclusion of sustainability principles into their assessment processes and practices. This analysis has been developed from a comparative study of the most representative ESG rating and information provider agencies in the financial market in two periods of time: 2008 and 2018.

The ESG rating agency industry has gone through a lengthy merger and acquisition process, which has redefined the industry map with bigger, more professionalized and finance industry-connected companies. Moreover, ESG rating agencies also have a clear commercial character [17], since they market diverse products and services (sustainability indices, sector and thematic research reports, benchmarks, etc.). The result is a clearer business case, and ESG rating agencies’ bargaining powers have grown exponentially. This could imply a biased concept of sustainability if sustainability principles are not guaranteed in the assessment business.

ESG rating agencies have integrated new criteria into their assessment models (mainly environmental and governance criteria) in order to assess corporate performance in a more robust and accurate way. These results support the findings of Saadaoui and Soobaroyen [22] who point out that ESG rating agencies focus on analyzing environmental, social and governance criteria, whereas the economic dimension is less studied; in addition, Attig et al. [49] have found that each ESG rating agency regards different individual components of CSR as relevant.

However, we can observe that ESG rating agencies do not fully integrate the sustainability principles into the corporate sustainability assessment process; therefore, they should include these principles in their work to contribute to the sustainable development of the companies that they rate. We encourage ESG rating agencies to improve the measurement of corporate sustainability performances by integrating sustainability principles into their assessment processes and practices. For instance, there are assessment methodologies like footprint analysis that allow appraisers to tackle the life-cycle thinking principle. Using those methodologies will contribute to achieving a more sustainable development.

This study makes a significant contribution to the extant literature on ESG rating agencies, offering information to academics about the metrics that they use in their researches. Moreover, from a practical standpoint, this paper is expected to contribute to the way the financial market faces extra-financial assessment, providing useful insights to policy-makers and other market actors such as fund managers or investors who want to build their portfolios and are more committed to social and environmental aspects. This paper also contributes to promoting an open debate over who rates the raters, adding sustainability principles as key elements to the analysis, which help to clarify which concept of corporate sustainability is being institutionalized in the SRI market and beyond.

Finally, a number of limitations need to be considered. The constant changes in the sector (mergers, acquisitions and disappearance of ESG rating agencies) and the selection of limited cases do not show all the particularities of ESG rating agencies, although our selection covers the main trend. The current study was limited by analyst bias in the content analysis developed. We encourage other researchers to reply and extend the sample of study. Moreover, this study is limited by the lack of public information made available by ESG agencies about their evaluation criteria.

Our research poses an important question that future studies could address: How are ESG rating agencies integrating sustainability principles into their assessments of corporate performance along supply chains? Current supply-chain performance-assessment systems are not suitable tools to integrate new global challenges. Therefore, further efforts are needed to develop more comprehensive corporate sustainability assessment frameworks which integrate sustainability principles along the whole supply chain.