Integrating and Analyzing ESG Data on AWS Using CSRHub and Amazon QuickSight

By Colin Marden, Solutions Architect at AWS

Environmental, social, and governance (ESG) factors are increasingly important for financial institutions as they look to assess portfolio risk, meet investment mandates, align with customer values, and report on the sustainability of their portfolios.

According to a BNP Paribas-sponsored survey, 66 percent of asset managers and institutional investors identified the difficulty of ESG data integration as one of the biggest barriers for asset managers who want to get started with sustainable investing.

In response to this challenge, Amazon Web Services (AWS) launched a curated catalog of Sustainability Data on AWS Data Exchange in 2020, with the aim to make it easy to find, subscribe to, and use third-party sustainability-related data on the cloud.

This catalog includes sustainable finance and corporate social responsibility data, such as company-level ESG data products, as well as physical regional-level data like weather, air-quality, and hydrological data.

Working closely with CSRHub, an AWS Partner and data provider on AWS Data Exchange, we have produced a demonstration to illustrate how customers can analyze company-level ESG scoring data with Amazon QuickSight.

In this post, I will show how a portfolio analyst can use third-party data on AWS to generate ESG sustainability and portfolio insights to identify organizations with a sustained positive performance trend, or ones that may represent a risk to their portfolio.

I’ll also explore how we integrated ESG scoring data into QuickSight using AWS Glue and Amazon Athena.

Visualizing CSRHub’s ESG Data

The intention of this demonstration is to show AWS customers how the adoption of new data sources like CSRHub’s ESG data can augment asset managers’ existing data, and enhance their business decision making with actionable investment insight.

CSRHub provides historical ESG data dating back to 2008 and covers more than 18,000 public, private, and not-for-profit entities across 134 industries and 145 countries.

For each rated entity, CSRHub provides four top-level categories, spanning 12 subcategories of ratings and rankings:

• Community: Community Development and Philanthropy; Product; and Human Rights and Supply Chain
• Employee: Compensation and Benefits; Diversity and Labor Rights; and Training, Health, and Safety
• Environment: Energy and Climate Change; Environment Policy and Reporting; and Resource Management
• Governance: Board; Leadership Ethics; and Transparency and Reporting

Given the number of data points and size of the dataset from CSRHub, traditional spreadsheet tools struggle to give investors the speed, agility, and scalability necessary to analyze and visualize the data effectively.

In order to appreciate the richness of this dataset, quickly derive actionable insights, and scale to hundreds of thousands of users, we’ll be using Amazon QuickSight—a scalable, serverless, embeddable, machine learning-powered business intelligence (BI) service built for big data on the cloud.

Amazon QuickSight can cater for 500 GB datasets with up to 250 million rows, and lets authors easily create and publish interactive BI dashboards that include machine learning-powered insights.

Those dashboards can be accessed by readers from any device, and seamlessly embedded into applications, portals, and websites, with session-based pricing to optimize the cost of consumption.

From ESG Data to ESG Insight

In the following video, I demonstrate how AWS customers might use QuickSight as a tool for investment analysis. I walk through a dashboard that I created from the CSRHub ESG product, and explain how it can used by a business analyst or asset manager who is looking to gain an understanding of a portfolio of companies’ ESG performance.

The following screenshot shows an example visual from the QuickSight dashboard we created using CSRHub data. It highlights overall performance within sectors over time, with insights highlighting the top and bottom movers by sector.

The graph allows us to conduct simple, side-by-side comparison-based research across regions, countries, sectors, industries, and even individual companies. Data about the top and bottom 100 performers in the comparison is available for download to the reader for downstream analysis or distribution.

Figure 1 – Side-by-side comparison.

The screenshot in Figure 2 below is an illustrative view from our fictitious portfolio sheet. The key performance indicators enable users to assess month-over-month performance of the portfolio (percentage change), and provides breakout of top-level aggregates for Environment, Governance, Community, and Employee.

Below that, you can see auto-generated narratives that update as data changes and provides useful metrics and indicators. Insights include the top and bottom performing companies, sectors, and industries, and the three-month compound growth rate.

Figure 2 – Portfolio performance ratings.

Creating Your Own Data Pipeline

The architecture underpinning the Amazon QuickSight dashboard in the example above is represented below:

Figure 3 – Data pipeline diagram.

The left-to-right data pipeline starts once data is available in Amazon Simple Storage Service (Amazon S3) and finishes with consumable dashboards on QuickSight. This is a common pattern on AWS for a range of business use cases.

In the example above, we sourced CSRHub’s ESG Rating data from AWS Data Exchange. In the next video, I will demonstrate how you can build the data pipeline described above yourself.

I will use local ESG scoring files and show you how to upload data to S3, and how to use AWS Glue and Amazon Athena to enable the consumption of data (and generation of insights) with QuickSight.

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. In this example, I will use AWS Glue Crawlers to discover our data.

Amazon Athena is an interactive query service that makes it easy to analyze data in S3 using standard SQL. In this example, I’ll use Athena to query and join our data in order to prepare it for integration into QuickSight.

AWS Glue and Amazon Athena are key AWS services for asset managers getting started with ESG data that can help them overcome the integration barriers highlighted in this post.

Other Enhancements

In these videos, I have demonstrated how business analysts use cloud-based services to derive actionable insight from large datasets, and how you can build a data pipeline to create dashboards.

We have explored a number of essential AWS services to overcome the barrier of ESG data integration, including AWS Data Exchange to remove the friction of finding, licensing, and using sustainability-related data, and AWS Glue and Amazon Athena to create a unified metadata repository.

With this demonstration, I used high-level aggregates, generic processes, and basic visualization techniques so it’s accessible to a general audience. For production workloads, asset managers should continue to learn about sustainability data, and explore how other AWS functionality or services can provide a competitive edge.

Examples of meaningful enhancements to this solution may include:

Anomaly Detection

Amazon QuickSight uses proven technology to continuously run machine learning-powered anomaly detection across millions of metrics to discover hidden trends and outliers in your data. This anomaly detection enables you to get deep insights that are often buried in the aggregates and not scalable with manual analysis.

With ML-powered anomaly detection, you can find outliers in your data without the need for manual analysis, custom development, or machine learning domain expertise.

Learn more: Setting up ML-powered anomaly detection for outlier analysis

Forecasting and What-If Scenarios

Using ML-powered forecasting, you can forecast key business metrics like sustainability performance with point-and-click simplicity. No machine learning expertise is required.

The built-in ML algorithm in QuickSight is designed to handle complex real-world scenarios, and helps provide more reliable forecasts than available by traditional means.

Learn more: Using forecasting and what-if scenarios

Integrating Amazon SageMaker Models

QuickSight supports Amazon SageMaker models that use regression and classification algorithms. You can apply this feature to get predictions for just about any business use case. Examples include predicting the likelihood of customer churn, employee attrition, scoring sales leads, and assessing credit risks.

Learn more: Visualizing Amazon SageMaker machine learning predictions with QuickSight

Row- and Column-Level Security

QuickSight users in your organization should have access to only certain data for compliance and security reasons. Without the integrated features to enforce row- and column-level security within QuickSight, you don’t have to develop additional solutions such as views, data masking, or encryption, or try to integrate third-party solutions around your data to enforce security.

Learn more: Applying row-level and column-level security on QuickSight dashboards

Conclusion

This post demonstrated the power of the cloud for Environmental, social, and governance (ESG) data integration and Amazon QuickSight’s ability to quickly generate insight from massive datasets.

ESG data consumers are choosing AWS for their analytical workloads for its best-in-class security, seamless scalability, a broad portfolio of AWS and third-party ISV data processing tools available on day one, and virtually unlimited scalability of Amazon S3.

Additional Resources

If you’d like to find out more about ESG data available on AWS that’s ready to be integrated into QuickSight or other AWS services through Amazon S3, visit the AWS Data Exchange Sustainability landing page where you can find more than 35 ESG data products with more becoming available weekly.

In addition to company-level data, you’ll find physical regional-level data, such as weather and hydrological, much of which is free.

To list, request, or procure ESG or sustainability data through AWS Data Exchange, contact esg-investing@amazon.com. You can also reach out to us to learn more about AWS Partner SaaS offerings specialized in ESG investing that easily integrate with AWS architecture and ESG data on the cloud.

To learn more about AWS-recommended architectures, or how to use AWS services beyond QuickSight, the AWS Architecture Center can help you find accurate and up-to-date information to help you make the right decisions from the very beginning of your projects. It’s your one-stop destination for recommended guidance from AWS Solutions Architects.

To procure data from CSRHub on AWS, contact support@csrhub.com.

.

.

CSRHub – AWS Partner Spotlight

CSRHub is an AWS Partner that provides historical ESG data dating back to 2008 and covers more than 18,000 public, private, and not-for-profit entities across 134 industries and 145 countries.

Contact CSRHub | Partner Overview | AWS Marketplace

*Already worked with CSRHub? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.