How Amazon Achieves Near-Real-Time Renewable Energy Plant Monitoring to Optimize Performance Using AWS

As part of the goal to reach net-zero carbon emissions by 2040, Amazon is on a path to powering its operations with 100 percent renewable energy by 2025—five years ahead of the original target of 2030. In June 2021, Amazon became the world’s largest corporate purchaser of renewable energy, reaching 65 percent renewable energy across the business. In the future, Amazon will meet our total annual electricity demands with 100% renewable energy from projects all around the world.

Amazon collaborates with renewable energy providers across the world to bring online new renewable projects dedicated to serving our power needs, which is monitored by Amazon’s renewable energy optimization (REO) team. The REO team is also responsible for maximizing operational and financial performance of assets. These optimizations help Amazon achieve its sustainability goals at scale across its global footprint.

As our renewable energy fleet grows exponentially, so does our need to monitor renewable asset operational performance in near-real time to ensure we are achieving production goals. The challenge of monitoring the performance of hundreds of renewable assets in more than 30 countries increases each day. Some of these challenges include:

• Disparate equipment (turbines, solar panels) and systems (supervisory control and data acquisition (SCADA, historians) from various vendors and built at different times
• Multiple data formats (proprietary, XML, IEC) and protocols and interfaces (OPC-UA, Modbus, DNP3, APIs, no-APIs/flat files)
• Different data definitions (IEC-61850, IEC 61450-25, RDS-PP)
• Varied data source resolution (from near-real time to 1 minute, hourly, and daily)
• Extra-large datasets, with each asset in the set having up to 10,000 tags of data points and sampling frequency as low as subseconds

It’s clear there is more than one strategy to overcome these challenges and that one size does not fit all. Each use case is independent and often demands its own solution architecture, which is why we at Amazon Web Services (AWS) have always believed in using the right tools for the job. We learn from our own energy transition work at Amazon, and we share our learnings with customers and partners so they can tailor an approach to deploy, manage, and optimize their renewable energy solutions at the bulk generation level and at the grid-edge with our services, including compute, storage, artificial intelligence/machine learning (AI/ML), security, and Internet of Things (IoT).

In the next section, we describe how the REO team uses AWS IoT services for real-time performance monitoring of renewable energy assets. In the future, we will publish blog posts on how the REO team uses Amazon SageMaker—which helps data scientists and developers prepare, build, train, and deploy high-quality ML models quickly—and ML to maximize output and revenue from Amazon facilities. In addition, as Amazon scales up its renewable and storage footprint, we will develop advanced analytics based on ML to support Amazon’s sustainability goals and business objectives.

The REO Team’s Reference Architecture for Renewable Energy Asset Monitoring

The REO team developed a complete solution built entirely on AWS to perform near-real-time monitoring of asset performance. The reference architecture for the solution is shown and described in figure 1 below and includes a variety of AWS services used to stream, store, manage, calculate, and visualize renewable energy asset operations data.

The REO team used this AWS IoT–based reference architecture to create a highly secure, reliable, scalable, and low-cost solution capable of ingesting SCADA data from hundreds of renewable energy projects in near-real time.

Figure 1. The Amazon REO team’s reference architecture for renewable energy asset monitoring

1. Data sources at the edge include SCADA systems, third-party protocol converters, data historians (e.g., OSI PI), and Web APIs.
2.  Data is ingested directly using AWS Lambda or AWS IoT Greengrass 2.0 and AWS IoT Greengrass stream manager from the asset to Amazon Kinesis and processed by an AWS Lambda function. Amazon Simple Storage Service (Amazon S3) stores the data meta map, which is used to map SCADA data tag to AWS IoT SiteWise ID.
3. Data is ingested at scale either with AWS IoT SiteWise for data requiring asset modeling or into Amazon Timestream for unmodeled data to support diversity of asset data.
4. The REO team created a near-real-time analytics dashboard for critical asset performance monitoring using Amazon Managed Grafana.
5. Data from AWS IoT SiteWise/Amazon Timestream is used by AI/ML services (e.g., Amazon SageMaker) or third-party ML services for predictive health analysis and assessment.
6. Detailed business intelligence reporting is available to a wide variety of data-consuming persona.
7. All communication is fully secured, traceable, authenticated, and encrypted by AWS security, identity, and audit services.

Data Ingestion

The REO solution uses two main methods to stream SCADA data.

First, the REO solution uses AWS IoT Greengrass 2.0—a new version of AWS IoT Greengrass that makes it easy for device builders to build, deploy, and manage intelligent device software—and AWS IoT SiteWise, a managed service that simplifies collecting, organizing, and analyzing industrial equipment data—to stream data from SCADA systems. AWS IoT Greengrass 2.0 supports multiple native connectors to transfer high-volume, high-resolution data to the AWS Cloud. The REO team can then route the data to Amazon Kinesis using the AWS IoT Greengrass stream manager for its batching and exporting functions. The data-collection frequency is configurable from subseconds to minutes and depends on the specific sensor and tags as the edge. With this solution, REO can reliably stream thousands of data tags from a single renewable asset to the AWS Cloud at scale and at a low cost.

Alternatively, SCADA data from some operators may come through an API rather than direct device connectivity. An AWS Lambda function is scheduled to read data from the API and send the operational data to Amazon Kinesis. AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers.

Finally, another AWS Lambda function is initiated by Amazon Kinesis to process the raw data, map and transform the data, and then send the data to a target data storage endpoint—either AWS IoT SiteWise or Amazon Timestream, a fast, scalable, and serverless time series database service for IoT and operational applications that makes it easy to store and analyze trillions of events per day up to 1,000 times faster and at as little as 10 percent of the cost of relational databases.

Data Storage

The REO solution uses two levels of data storage: AWS IoT SiteWise and Amazon Timestream.

For tags with critical business functionality, the REO team chose AWS IoT SiteWise to serve as the historian to save the data in an organized pattern, as it met the majority of Amazon business requirements. Each renewable asset is organized into a hierarchy: wind/solar farm, substation, turbine, inverter, or meteorological station.

The following is the reference AWS IoT SiteWise asset model for wind and solar farms.

Figure 2: Reference AWS IoT SiteWise asset model for wind and solar farms

In addition, each turbine or inverter has a corresponding reference data model:

Figure 3: Reference data model for turbines and inverters

Amazon chose to use AWS IoT SiteWise as it provides the following:

• Fully managed serverless architecture that eliminates maintenance hours for the REO team
• A fast, highly efficient API interface to write and query data
• A standardized asset model that efficiently and flexibly organizes the renewable assets
• The ability to quickly compute common industrial performance metrics
• The capability to ingest data with different time granularities
• Native connectors to Grafana through Amazon Managed Grafana

Amazon Timestream is used to store all the raw data from data sources and is valuable due to its

• fast, scalable, and serverless time series database service;
• high performance at a low cost, with most data being saved in magnetic mode;
• easy query and aggregate-use of SQL-type queries; and
• its connector to Grafana.

Operational Near-Real-Time Dashboards

AWS IoT SiteWise serves as a data source to Amazon Managed Grafana, the visualization tool selected for:

• native integration with multiple AWS security services to meet Amazon corporate compliance requirements;
• seamless integration with industrial AWS services, including AWS IoT SiteWise and Amazon Timestream; and
• its powerful visualization features, including third-party plugin options.

Below are a few screenshots of near-real-time dashboards powered by AWS services as described in the previous reference architecture for a wind farm containing 13 wind turbines (rated 3,600 kW each with 110 tags sampled every 20 seconds).

Wind Turbine Power Heat Map Dashboard

This plot shows a 48-hour period power heat map of the wind farm. The scale on the bottom shows the power range from about 0–3,600 kW. The plot clearly shows that turbines 1, 5, 8, and 12 have anomalies and are out of service frequently during this period.

Figure 4: Wind turbine power heat map dashboard

Single Turbine Plot Dashboard

This screen shows a 24-hour period single turbine performance. This turbine is actively turned off every night for 2 hours, starting at midnight.

Figure 5: Single turbine plot dashboard

Turbine Availability Dashboard

This screenshot shows 7 days of availability metrics, including overall wind farm availability, individual turbine status, turbine availability in the past hour, and turbine availability in the last 7 days. From the dashboard, it is clear that turbines 4, 6, 8, and 11 are offline more often than others.

Figure 6: Turbine availability dashboard

Wind Farm Performance Dashboard

This screen shows 7 days of wind farm performance, including overall current wind farm performance and weather conditions, daily energy production, wind farm level power and wind speed for the last 7 days, and a histogram of wind speed distribution

Figure 7: Wind farm performance dashboard

Conclusion

As highlighted at the beginning of this blog, each use case is unique, and customers are free to choose which combination of services best meet their requirements. The reference architecture demonstrates one such solution—how the REO team uses AWS IoT services to achieve a highly available, highly scalable, near-real-time operational data monitoring system for large-scale renewable energy assets such as wind and solar plants. By using the full gamut of AWS services and best-practice cloud technologies, we can efficiently serve the needs of operation/maintenance supervisors, data engineers, and data scientists and provide near-real-time situational awareness of renewable energy generation assets. Furthermore, we can analyze the data to provide more insight into asset performance, such as anomaly detection and preventive maintenance use cases. The solution provides a foundation to help Amazon achieve our operations commitment of powering our business operations with 100% renewable energy by 2030, a target we are on track to meet five years early, by 2025.

For more information, please visit Amazon renewable energy or AWS Power & Utilities.