GE Renewable Energy’s Digital Services Platform Achieves 99.9% Data Availability on AWS
Experiencing growing demand for carbon-free electricity, GERE needed to advance its digital capabilities. To improve data management, processing, and analysis, GERE chose to modernize the Digital Services platform on Amazon Web Services (AWS). The company developed a highly available platform that consumes and analyzes terabytes of asset data per day. On AWS, GERE can deliver faster insights, reduce operational costs, and improve asset performance in near real time.
Chief Digital Officer of Digital Services
GE Renewable Energy
Processing IoT Data from Over 40,000 Assets
Managing thousands of assets to maximize power generation while controlling cost is a concern for all renewable energy companies, and GERE is no exception. Although it had a system in place to manage assets, its expanding fleet and growing customer demand led to operating cost, scalability, and reliability challenges. Siloed applications led to inconsistent user experiences and overlapping information across GERE’s enterprise.
GERE chose to create a new, fully managed, and user-centric Digital Services platform that collects, consumes, and analyzes asset data. It also wanted to present data to customers on a new persona-driven interface. To achieve these goals, GERE turned to AWS. “It was very clear that AWS had all the building blocks that we needed to solve the challenges that we were facing,” says Tarigopula. “For example, we could scale our solution quickly on AWS to serve a large number of users without having to provision, manage, or set up infrastructure.”
By using cloud-native, fully managed AWS services, GERE can focus on building the capabilities at scale that deliver direct value to customers while maintaining agility and speed.”
Chief Technology Officer of Digital Engineering
GE Renewable Energy
Building a Highly Available Asset Management Platform on AWS
In 2020, GERE built its Digital Services platform on AWS. The solution uses a containerized architecture on Amazon Elastic Kubernetes Service (Amazon EKS), a managed container service for running and scaling Kubernetes applications in the cloud or on premises. “Because we have a container-based architecture on Amazon EKS, it is simpler for us to add more applications and digital capabilities to support our solution,” says Tarigopula. GERE used AWS Enterprise Support, which provides companies with a concierge-like service to help them achieve their outcomes and find success in the cloud, to help design its architecture and better understand AWS service capabilities.
GERE’s solution also uses Amazon Managed Streaming for Apache Kafka (Amazon MSK), a fully managed Apache Kafka service that makes it simple to ingest and process logs and other forms of streaming data in near real time. Using Amazon MSK, GERE processes terabytes of data from over 40,000 assets each day, a 500 percent higher volume compared to the previous solution. The platform’s availability also improved on AWS, increasing from 89–92 percent to 99.9 percent. “By migrating to AWS, our team improved our deployment frequency by over five times,” says Brian Case, chief digital officer of Digital Services at GERE. “We can deploy capabilities at a pace that matches the growth of our industry and the needs of our customers.”
In GERE’s new system, after data is processed, it is sent to AWS Glue—a serverless data-integration service that makes it simple to discover, prepare, and combine data for analytics, machine learning (ML), and application development—where it is processed through extract, transform, load jobs. Next, the data is sent to an analytics solution powered by Amazon EMR, which is used to rapidly process, analyze, and apply ML to big data using open-source frameworks. GERE uses Amazon EMR to run analytics on its wind turbine data, which helps it accurately detect and troubleshoot anomalies and deploy preventative maintenance before a turbine’s performance degrades.
After being analyzed, the data can be consumed by more than 300 microservices running on Amazon EKS, helping serve valuable, actionable insights to GERE’s customers. “Using Amazon EKS has fundamentally changed the way we build our services and applications,” says Tarigopula. “We used to have multiple big applications that ran on their own monolithic architecture. Now that we use a container-based architecture, we have a single unified solution where we can bring lots of different applications together in a cohesive way.” GERE has also improved its upgrade and application deployment cycles on AWS. Under its previous solution, deployments were limited to once or twice in 2 weeks. Now, GERE has multiple deployments per day and can deploy new changes on demand.
GERE also improved its ability to scale on AWS. Previously, the company scaled manually when it needed more capacity, which was labor and resource intensive. GERE now automatically scales on demand, removing the need to provision infrastructure. GERE also migrated from a single-tenant to a multitenant solution. Now, it can quickly scale to support new users without the high infrastructure overhead or upfront cost previously required to create a new tenancy on its previous solution. “By using cloud-native, fully managed AWS services, GERE can focus on building the capabilities at scale that deliver direct value to customers while maintaining agility and speed,” says Anwar Ahmed, chief technology officer - Digital Engineering at GERE.
Chief Architect of Digital Engineering
GE Renewable Energy
Expanding Data Analytics Capabilities on AWS
GERE also plans to use Amazon SageMaker—which data scientists and developers can use to prepare, build, train, and deploy high-quality ML models quickly—to strengthen its ML capabilities. “We have a natural cadence when working with the AWS team,” says Tarigopula. “Using AWS Enterprise Support, we received the help we needed to build this solution and unlock some of the challenges we faced. It has been a great experience using AWS services during this journey.”
About GE Renewable Energy
Benefits of AWS
- Processes terabytes of data per day
- Increased data processing volume by 500%
- Scales to support over 40,000 assets
- Supports over 5,000 global users
- Improved availability from 89%–92% to 99.9%
- Increased deployment frequency from once or twice every 2 weeks to multiple times per day
- Unlocked on-demand deployment
AWS Services Used
Amazon Elastic Kubernetes Service (Amazon EKS) is a managed container service to run and scale Kubernetes applications in the cloud or on-premises.
Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that makes it easy for you to build and run applications that use Apache Kafka to process streaming data.
AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development.
Amazon EMR is the industry-leading cloud big data platform for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, and Presto.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.