ESG data will always be imperfect, despite its critical role

Abandoning ESG due to imperfect data would be like abandoning the judicial system for the same reason, argued the author of the controversial ‘Aggregate Confusion’ paper which sent shockwaves around the finance world.

“It is important to understand the fact that because the data is noisy, does not imply that it is useless,” said Professor Roberto Rigobon, Society of Sloan Fellows Professor of Management, and Professor of Applied Economics at MIT Sloan School of Management. “It is a very, very important difference.”

Speaking at Conexus Financial’s Sustainability in Practice forum at Harvard University, Rigobon likened the use of ESG data to the judicial system, which plays a critical role in society but relies on incomplete data and evidence to make often controversial decisions.

Despite the noise, there “absolutely” are useful signals in the available data, and investors can gain alpha from using it to change their behaviour.

“The question is, how can we extract that in a smarter way,” Rigobon said.

In 2019 Rigobon wrote a paper titled ‘Aggregate Confusion: The Divergence of ESG Ratings,’ which looked at the divergence of ESG ratings between six rating agencies, revealing ESG data was noisy and unreliable. This meant investors did not trust the data, potentially leading to inaction, or to corporates gaming the system. The paper led to the Aggregate Confusion Project, a consortium whose aim is to make ESG integration more defensible and more workable for the industry.

ESG-related advertisements are increasingly common across the financial sector. Coming against a backdrop of “the world upset about how the private sector is behaving” on multiple fronts, and consumers demanding change, the reliability of ESG data is extremely important, Rigobon said.

industry needs to adapt to imperfection

But while the measures themselves can be improved over time, there is no way to produce perfect data for a single attribute such as CO2 emissions or diversity, particularly with societal values continually changing over time, he said.

“So we need to learn to deal with the fact that that the data is imperfect, that it is uncertain, that it is ambiguous, and that decisions are very rarely considered just by all,” Rigobon said.

This is similar to the judicial system, he said, where the data and evidence is incomplete and imperfect, and judicial decisions usually upset some people.

“That doesn’t mean that we should just disband the judicial system,” he said. “I come from Venezuela where the judicial system has been disbanded. I’m telling you [that was] a very bad outcome in general.”

Like the judicial system, ESG is a measurement of unethical behaviour, he said. It will inherit similar features to the judicial system such as the use of intermediaries — which are ratings agencies in the case of ESG — and the continuous improvement of the practices involved.

The industry, academia, data providers and regulators need to collaborate, and while the measures are imperfect, the goal has never been perfection but to “do the best that we can,” he said.

Transparency is also critical, he said, and while creating measurement is complicated and a job that needs to be delegated, “it’s much better if we actually explain that process more openly, that we are willing to be judged, and that will allow improvements in that process, not only of the data collection, but on the data aggregation.”

Also speaking on the panel was Michael Trotsky, chief investment officer at MassPRIM, one of five founding partners of the Aggregate Confusion Project led by academics at MIT which aims to improve the quality of ESG measurement and decision making in the financial sector.

Trotsky said Rigobon was initially “a skunk at the party, saying the data stinks,” when he published his controversial paper which sent shockwaves around the world of finance.

“Everyone in this room is very excited about ESG products, and trying to make a difference,” Trotsky told delegates at Top1000funds.com’s Sustainability in Practice conference at Harvard University. “And then when someone pointed out that their data was unreliable and therefore, it couldn’t be impactful, it wasn’t a great message. And we all kind of reeled from that.”

Now it is widely accepted that the data needs to get better, he said, noting a lot of improvements have been made to the data since that time, and work is ongoing.

“We believe that with a better signal, with preference selection, we will be able to reflect the view of our constituents in a better way that’s more impactful both financially, and from an ESG standpoint,” Trotsky said.