How Greenko does near-real-time monitoring and analytics of 2,200 wind turbines on AWS

Greenko Group (Greenko) is a leading energy solution provider in India with a total installed capacity of 7.5 GW, which includes wind (over 2,000 wind turbines), solar (over 5 million solar panels) and hydro (25 sites). Greenko has electrified more than 6.3 million households and has helped avoid the equivalent of over 17 million tons of carbon dioxide in the atmosphere.

Renewable energy assets produce high-velocity Internet of Things (IoT) data in the form of tags, events, and alerts. Greenko needed robust, reliable, and secure IoT solutions as it embarked on its cloud journey with Amazon Web Services (AWS) in 2021. AWS IoT services scale to accommodate demanding renewable energy use cases, and the pay-as-you-go model means Greenko pays for only what it uses. A blog showcasing how Greenko uses AWS IoT and serverless technology for the monitoring of 100 wind turbines was published in January 2022.

For example, AWS IoT Core lets you connect billions of IoT devices and route trillions of messages to AWS services without managing infrastructure. Amazon Kinesis Data Firehose offers customers an extract, transform, and load (ETL) service that reliably captures, transforms, and delivers streaming data to data lakes, data stores, and analytics services. Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can store and protect any amount of data for virtually any use case, such as data lakes, cloud-native applications, and mobile apps. With cost-effective storage classes and easy-to-use management features, you can optimize costs, organize data, and configure fine-tuned access controls to meet specific business, organizational, and compliance requirements.

These were some of the core AWS services that were used to build the Greenko minimum viable product (MVP) on 100 wind turbines in 2021. Although the MVP was done on 100 wind turbines, the architecture was designed to be scalable and robust to support multiple sites with thousands of assets. It’s important to note that the solution was completely serverless with minimal to no infrastructure to manage for the Greenko team. Furthermore, the solution incorporates use of open-source technologies on the cloud to achieve an extremely competitive cost of ownership for Greenko.

Today after exactly 1 year of being operational, Greenko has successfully migrated its entire wind fleet of 3.2 GW, consisting of 2,200 wind turbines, to AWS.

Let us quantify this increase in scale, compared to the MVP in 2021.

Greenko scaled the architecture designed for 100 wind turbines during the MVP to more than 2,200 wind turbines with zero downtime and zero incidents over 24 months, demonstrating the high availability and resilience of AWS services. Greenko has increased the scale of the solution by an average of 16 times compared to the original situation and introduced new functionalities, including data analytics, asset hierarchy definition, and data visualization.

For this initiative, Greenko selected India AWS consulting partner, Locuz Enterprise Solutions, for delivery of the solution. Locuz has more than two decades of experience and 350+ deployments in High-Performance Computing (HPC).

New functionalities introduced

1. asset hierarchy definition

To meet the situation awareness and visualization requirements of the business and SCADA teams, four levels of asset hierarchies were created. These were:

I. fleet level (consolidated performance of 2,200 assets)
II. cluster level (consolidated performance across seven states of India)
III. state level (consolidated wind farm performance within a state)
IV. site level (consolidated wind turbine performance of a wind farm)
V. asset level (individual wind turbine performance within a wind farm)

The following critical metrics were identified for visualization:

I. total installed capacity
II. PLF or performance load factor
III. active power generated
IV. wind speed
V. actual power
VI. expected power
VII. 1 million units of energy exported
VIII. wind turbine status: online, ready, offline, and no connection

a. online—blades spinning
b. ready—blades static, waiting for wind to pick up
c. offline—down for maintenance activity
d. no connection—loss of connectivity

These eight metrics were then visualized at every level of the four hierarchies. See the below examples of site- and asset-level dashboards.

I. site level (consolidated wind turbine performance of a wind farm)

Figure 2. Greenko site-level landing page

II. asset level (individual wind turbine performance within a wind farm)

Figure 3. Greenko asset-level landing page

2. Business intelligence and analytics

Greenko continuously monitors the machine health of its assets to identify underperforming assets, make decisions to bring them offline, or schedule maintenance. It ingests approximately 800,000 process values (tags) from more than 60 wind farms every minute. It aggregates telemetry data for key metrics to ensure the fleet of turbines is performing optimally. Greenko needed a reliable and scalable analytics service that could run SQL analytics in the cloud. There was also a demand to separate compute from storage, depending upon the workload.

Expanded solution architecture with new functionalities

The expanded Greenko solution architecture adds the new functionalities to the architecture of the MVP, including:

• added business intelligence (BI) reporting and self-serve analytics (block 5)
• Amazon Redshift, which uses SQL to analyze structured and semi-structured data across data warehouses, instead of Amazon EMR

Figure 4. Greenko AWS reference architecture

Let us see the new functions Greenko has introduced in block 5.

Block 5: BI reporting and self-serve analysis

1. Amazon Redshift – Tens of thousands of customers use Amazon Redshift every day to run SQL analytics in the cloud, processing exabytes of data for business insights. Whether data is residing in operational data stores, data lakes, streaming data services, or third-party datasets, using Amazon Redshift, you can securely access, combine, and share data with minimal movement or copying. Amazon Redshift is deeply integrated with AWS databases, analytics, and machine learning (ML) services to employ zero-ETL approaches or help you access data in place for near-real-time analytics and build ML models in SQL. Amazon Redshift is a fully managed service and offers both provisioned and serverless options, making it more efficient to run and scale analytics without having to manage your data warehouse. Amazon Redshift delivers on all your SQL analytics needs, with up to five times better price performance than other cloud data warehouses.
2. Wind Turbine Analytics – Greenko switched from Amazon EMR to Amazon Redshift so the Greenko team could use existing skills with SQL to aggregate and ELT the data. Using Amazon Redshift, Greenko analysts have more flexibility than with Amazon EMR for wind turbine analytics.

Approximately 800,000 process values (tags) are ingested to Amazon S3 every minute. Incoming values arrive in Amazon S3 as JSON objects and are pushed to Amazon Redshift through the Amazon Redshift API using functions from AWS Lambda, a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers.

The updating process provides live telemetry data to Greenko analysts every 1 minute. Greenko analysts typically work with 2 months of historical data, so a lifecycle policy, through a stored procedure in Amazon Redshift, purges records that are more than 2 months old. The original data persists in Amazon S3, so analysts can use federated queries in Amazon Redshift to access recent and historical values across their Amazon Redshift and Amazon S3 environments.

Because Amazon Redshift fully supports SQL, the Greenko analytics team uses SQL in Amazon Redshift to perform complicated queries, indexing, joins, and more on demand. In addition, the Greenko team uses scheduled jobs to create 10-minute data aggregations within Amazon Redshift.

This 10-minute aggregation function was earlier being carried out in Amazon EMR through Apache Spark. Not only has the introduction of Amazon Redshift empowered the Greenko analytics team to do more, but it has also simplified the architecture, with no compromise on the reliability or scalability.

Greenko uses Amazon Redshift to analyze performance and health of its wind turbines. It carries out triage on low-performing wind turbines and takes corrective action. See the below example on how this is done:

I. first-level triage: identification of a low-performing turbine

Figure 5. Identification of a low-performing asset

II. second-level triage: power-performance curve

Figure 6. Turbine power curve

III. third-level triage: pitch-angle versus wind speed versus power output

Figure 7. Turbine pitch-angle versus wind speed versus power output

IV. fourth-level triage: ambient, nacelle, generator, radiator temperature

Figure 8. Turbine temperature scatter chart

The above charts are available to the engineering team 24/7. They are located in the central control room at the Greenko head office in Hyderabad, India. By using AWS IoT and analytic services, the Greenko engineering team is fully empowered to take remediation actions based on what it observes on the analytic charts. It takes remote corrective action from the central control room and attempts to return the asset to the optimal state. In case this fails, a work order is issued for a physical inspection of the asset at site.

Business value delivered

1. reduced OPEX – Since moving to AWS, Greenko has abandoned the licensed proprietary software that was originally being used for near-real-time dashboarding. This was provided by the original equipment manufacturer (OEM) and incurred a recurring cost year on year, contributing to operating expenses (OPEX). AWS services have emancipated Greenko from the clutches of such software. Greenko also uses open-source technologies like InfluxDB and Grafana. As a result, Greenko has saved $200,000 USD on licensing costs alone.
2. improved employee productivity – By introducing Amazon Redshift for SQL analytics, Greenko has used the internal talent of the organization that was well versed in SQL. Rather than moving to a new BI analytics software that might entail training and licensing costs, the Greenko analytics team invested in developing SQL-powered BI reports. Amazon Redshift fully supports SQL and provides commercial-grade data warehouse performance at a fraction of the cost. Using Amazon Redshift saved valuable time and improved employee productivity. It allowed Greenko to focus on improving its business, in contrast to having to train and invest in a new BI software.
3. ready for artificial intelligence (AI) and ML – The accuracy of any AI/ML function depends on the quality and granularity of the data that the AI/ML algorithm is trained on. Greenko’s data lake is built on Amazon S3 and is hydrated by AWS IoT Core. The data lake has been hydrated with 10-second interval telemetry data of an average of 1,200 wind turbines during the past 1 year. Greenko also has kept a separate repository of weather data regarding its fleet for the past several years. This combination of 10-second interval telemetry data enriched with near-real-time weather data is the optimal foundation for the training of an AI/ML model. As a result, Greenko has already started exploring Amazon Forecast, a time-series forecasting service based on ML, for next-day energy forecasting. The plan is to develop an in-house prediction accuracy of better than 95 percent. Also, Amazon Redshift comes with native integration with Amazon SageMaker so Greenko can quickly create, train, and build ML models for nearly any use case with fully managed infrastructure, tools, and workflows.
4. consolidated operational monitoring – The Greenko engineering team now has a single consolidated view of its entire fleet of 2,200 wind turbines. This is further enriched and customized for the user based on the asset hierarchy maintained in InfluxDB. All this information is now available at the Greenko control room. At a click of a button, the engineers can drill down from fleet level to an individual asset, irrespective of its geographical location in India.
5. one data lake for all users – Different users and personas within the organization have different needs. The analytics and visualization needs of a control room engineer are very different from that of a data scientist. A control room engineer might be more interested in near-real-time streaming data from the asset and needs flexibility to expand or contract a time window to identify anomalies. Data scientists, on the other hand, might want to run a mathematical formula on a set of 10-minute aggregated data, which is more consumable for their AI/ML model training. The data lake built on Amazon S3 is capable of fulfilling the needs of all Greenko users and acts as the single point of truth for the organization. The data lake is the centralized repository that allows Greenko to store all its structured and unstructured data at nearly any scale. It gives the company the freedom to run different types of analytics, from dashboards and visualizations to big data processing, near-real-time analytics, and ML, to guide better decisions.

What’s next for Greenko

Greenko operates in three renewable economies of wind, solar, and hydro. Its entire wind fleet was successfully onboarded to AWS in 2022. In 2023 Greenko will be gradually onboarding its solar fleet to the cloud. The solar fleet comprises of over five million solar panels, spread across 54 plants in India, with a combined capacity of 1.5 GW. Greenko will also be investing in AI/ML technology for next-day generation forecasting and predictive maintenance of its assets. An initial proof of concept on Amazon Forecast has been initialed on one wind and one solar farm. By the end of 2023, Greenko expects to carry out near-real-time monitoring and analytics of its entire wind and solar fleets on AWS.

For more information, visit Greenko Group. And to find out more about how AWS is empowering digital utilities, visit here.