What do ESG scores measure?

ESG ratings providers generally assign a single score to an entity or financial product based on environmental, social, and governance characteristics. Commenting on the growth of aggregated scoring, Sylvain Guyoton, Chief Rating Officer at EcoVadis, an ESG rating agency, put it well: “You can blend bananas and carrots together if you want, and you’ll end up with a nice smoothie … people want one score at the end so we give it to them, even when it makes no sense”.

Much of the criticism directed at ESG ratings stems from a misunderstanding of what they are.

ESG ratings are not a measurement of environmental or social impact. ESG ratings first and foremost indicate a company’s ability to create long-term value and its exposure to environmental, social, and governance risks. A good ESG rating is not an indication that a company is ‘making the world a better place’.

In short, ESG ratings are not impact ratings.

At the heart of this confusion is the question of materiality: are ratings based on sustainability factors that pose a financial risk to the company or are they based on the risk a company’s activities pose to society and the environment (this is known as “double materiality”)?

While some rating providers, such as ISS ESG and Moody’s, claim to incorporate “double materiality”, an absence of methodological transparency makes it difficult to determine how a corporate-level rating score corresponds to positive social/environmental impact.

Masque-rating

Investor confusion over the purpose of ESG ratings is one issue, but ESG scoring has also been criticised over subjectivity, lack of standardisation, and opaqueness.

A 2022 paper found an average correlation of 0.61 between prominent ESG rating agencies (Moody’s and S&P’s credit ratings, based on financial metrics, have a correlation of 0.92).

The primary source of the low correlation between ESG rating agencies is that each agency possesses its own methodology, meaning different rating agencies have fundamentally different views about what data should be analysed, and how it should be analysed.

The below table illustrates how two ESG rating agencies use different indicators to analyse various ESG categories.

According to a study from MIT Sloan, ESG ratings suffer from three sources of divergence: scope divergence, weight divergence, and measurement divergence, the latter of which accounted for 56% of the divergence between ratings in the study.

This research also indicates the presence of a “rater effect,” where a firm receiving a high score in one category is more likely to receive high scores in all other categories from that same rater. This results in companies receiving different scores depending on the rating agency. The below table demonstrates the varying level of divergence as correlations between rating agencies.

Scoring and methodology bias

ESG rating agencies’ reliance on ‘relative scoring’ can also lead to counter-intuitive results. This practice is what led electric vehicle company Tesla, to be removed from the S&P 500 ESG Index while the energy exploration and production company ExxonMobil was included. In response, CEO Elon Musk tweeted: “ESG is a scam.”

Given that Tesla’s S&P Dow Jones Indices ESG score has remained stable year-over-year, Musk may have a point. Because S&P DJI ESG scores are calculated on a “best-in-industry” approach, and Tesla’s auto industry peers had been increasing their scores, the EV company ended up finishing in the bottom 25% of its peer group, justifying its removal from the index. In contrast, ExxonMobil compares relatively well to its oil and gas industry peers.

Studies have found other built-in biases within ESG scoring methodologies. For example, companies that are domiciled in countries with robust ESG regulatory reporting requirements and larger market capitalisation are awarded more favourable ESG ratings.

A report by the Institute for Energy Economics and Financial Analysis (IEEFA) also found that of the 7,600 companies analysed, 80% of total small and medium-cap companies have low ESG ratings while 54% of large and mega-cap companies were assigned high ESG ratings. The theory behind this is that larger companies have greater resources to spend on ESG disclosure and are more accustomed to reporting requirements and regulatory scrutiny. This relationship is demonstrated in the below chart:

These biases are important considerations as they limit the empirical quality of ratings, reducing their usefulness to fund managers and decision markers. The lack of trust around ESG data and scoring is reflected in a 2021 survey by Deutsche Bank that found 27% of global investors did not incorporate ESG in their investment decisions due to fears that data is greenwashed.

Low-quality ESG ratings have a knock-on effect because when ESG ratings are calculated through predictive modelling (when raw data is unavailable) any biases present are reproduced in these models. For example, Sustainalytics’ Core Rating Framework uses a predictive model to calculate ESG scores for SMEs based on the scores of comparable large-cap companies.

Day-to-day challenges of ESG ratings

As previously mentioned, quantifying ESG risk is made difficult by the sheer volume of indicators and variables assessed. For example, Sustainalytics looks at a total of 163 indicators while Refinitiv assesses 282. This, combined with unclear methodologies, makes it hard to discern which risk areas a company’s score is derived from. The aggregation of dozens of sub-scores and variables can also hide glaring issues within a company.

For example, before Wirecard’s admission of accounting fraud, the group held a median-tier rating from MSCI and was included in several large ESG-focused ETFs.

The difficulties of relying on a single score are reflected amongst ESG finance professionals, with a sustainable finance think-tank survey finding that 86% of respondents think it should be mandatory for ESG scores to provide constituent “E”, “S”, and “G” scores as individual parameters.

Divergence amongst ESG rating agencies' analyses has resulted in a situation where financial market participants (FMPs) are using multiple ESG ratings, with an ESMA survey finding that 77% of respondents used more than one provider for ESG ratings. Because of the subjective nature of ESG ratings, using multiple ratings providers increases the likelihood that a fund manager will better understand the ESG implications of an investment decision.

ESG analysis is in its infancy, with considerable disagreement over what ideal reporting and performance should look like. Unlike basic sustainability reporting (eg., the Corporate sustainability reporting directive), ESG scoring has yet to be regulated, but there are signs that this is changing. For example, in April 2022 the European Commission opened a consultation on the functions of the ESG rating market. In June 2022, The UK’s Financial Conduct Authority also indicated its support for regulating ESG ratings.

Sustainable finance regulation in Europe has arrived and demand for tools to holistically incorporate ESG risk is now unprecedented. The EU’s Sustainable Finance Disclosure Regulation (SFDR), for example, requires asset managers to justify the classification of their ESG-focused funds (i.e., Article 8 and 9 funds). As outlined in a 2022 report by Goldman Sachs, asset managers have been using ESG ratings to meet the Article 8 requirement of “promoting environmental and social characteristics.”

This detracts from the purpose of the regulation as ESG ratings are complex empirical representations of enterprise risk, rather than a clear demonstration of a company’s ability to further environmental and social causes.

The verdict on scoring?

Despite challenges, ESG scoring is here to stay, and this could be beneficial. Scoring can be valuable when ratings are broken down into sub-indicators and the ratings come with a high degree of transparency.

For example, a score representing emissions performance or biodiversity can be a convenient way to understand one environmental component of a company. Instead of using a single aggregated score, multiple granular scores can be a more reliable way to gauge social and environmental performance.

At 9fin, we acknowledge the usefulness of scoring but also recognise the inability of a single score to holistically communicate the plethora of factors that ESG data can capture. Written qualitative analysis offers insights that cannot be translated into a score.

Our ESG QuickTakes are transparent and in-depth and provide our clients with the level of detail needed to make an informed investment decision. To request a sample ESG QuickTake complete your details here.