PGE Migrates to AWS, Significantly Improves Energy Loss Detection Performance

Like its customers, Portland General Electric (PGE) is committed to a greener future. In fact, 25 percent of PGE residential and business customers pay extra to fund clean energy initiatives—and the investor-owned utility strives to deliver innovative energy solutions to them. However, PGE operated using a physical data center, which prevented it from meeting its energy goals. “We were looking for a solution that would be highly performant, scalable, and cost effective,” says Hema Sundaram, enterprise data strategy lead at PGE. “We could not view different datasets to see how we could enrich or build different digital threads.”

To increase its innovation speed and help accelerate Oregon’s transition to clean energy, PGE chose to migrate to a hybrid on-premises/cloud model on Amazon Web Services (AWS) using services such as Amazon DynamoDB—a key-value and document database that delivers subsecond performance at any scale. PGE increased operational efficiency, reduced costs, and improved customers’ digital experiences.

PGE is a regulated investor-owned utility that has operated out of Portland, Oregon, for more than 130 years. It supplies electricity to 51 cities and serves approximately 900,000 customers in a service area population of 2 million Oregonians. In 2019, PGE realized that its on-premises system could not support its evolving customer base and corporate data needs. “We only had one enterprise data warehouse,” says Aravind Murugesh, principal data architect at PGE. “We wanted to enable self-service that was faster and less costly than our on-premises solution.”

A tightly coupled data warehouse limited the speed at which PGE could deliver innovative solutions, preventing the utility from quickly querying data. The PGE team realized that migrating to the cloud and using modern data lake, analytics, and machine learning solutions would provide it with the speed, scalability, and resiliency it needed to innovate. “One thing we like about AWS is the ability to interact with the marketplace on AWS and use different AWS services, which expand our cloud capabilities,” says Uma Venkatachalam, principal information architect at PGE. The utility began its migration to AWS in March 2019.

PGE’s data engineers and scientists had no prior experience on AWS. However, they were able to build their cloud capabilities by participating in AWS Immersion Days in Seattle. Through the AWS Training and Certification process, PGE has increased the number of employees who are proficient in AWS technologies and significantly advanced its employees’ skill sets. The team began its migration by developing APIs for microservices, seeing an exponential increase in their output. “Within the first year, we built 9 APIs in 9 months,” says Sundaram. “In the next 6 months, our team built another 15 APIs. By the end of 2020, we had built over 100 APIs.” Now PGE has an integrated API and microservices system powered by multiple AWS services, including Amazon DynamoDB, which serves data to web, mobile, and other digital channels in near real time.

By migrating to AWS, PGE has the elasticity, scalability, and resiliency to serve its customers during high-traffic times, such as power outages. “Over the last 2 years, we encountered some historic events,” says Venkatachalam. “In 2020, PGE confronted historic wind, wildfire, and weather events. In 2021, we had a series of storms that caused unprecedented damage. Our capabilities on AWS helped us support our customers better than we could have otherwise.” With cloud-based APIs, PGE can respond to customers faster. For example, during the storm, its APIs were able to serve five times the traffic at a 25 percent faster rate compared to APIs on premises.

PGE’s digital channels can now support thousands of customers while delivering low latency, with most of the company’s APIs responding in less than a second. PGE’s digital channels can also scale to support peak traffic during critical events. “At one point, we had about 200,000 customers reporting outages on our website within a 6-hour time window,” says John Pedapalli, lead API and data engineer at PGE. “We were able to serve them efficiently, and our digital channels did not crash. When weather and other events lead to high customer traffic, it’s reassuring to know we can meet unprecedented load.”

PGE built a data lake with the help of AWS Advanced Technology Partner Snowflake and its Data Cloud, a global network where thousands of organizations mobilize data with near-unlimited scale, concurrency, and performance. The utility built several solutions on top of this data lake. For example, PGE previously had used less-effective vendor software. To improve energy loss detection, PGE built an advanced analytics solution in the data lake, using AWS services such as Amazon Simple Storage Service (Amazon S3), an object storage service that offers industry-leading scalability, data availability, security, and performance. “We developed a solution to replace the vendor software,” says Sundaram. “This helped us not only save costs but also identify potential energy theft with better accuracy.” With this solution, PGE significantly improved energy loss detection performance.

PGE also developed an algorithm to identify splits, woodpecker holes, and decay in wooden electrical poles with 80 percent accuracy. This algorithm was built using Amazon SageMaker, which helps data scientists and developers prepare, build, train, and deploy high-quality models quickly. Using historical inspection data, plus data such as pole characteristics, weather, and geospatial information, PGE’s algorithm can prioritize pole inspections in a proactive manner.

By migrating to AWS, PGE saved costs, increased its speed of innovation, and built solutions to automatically monitor its physical infrastructure. It hopes to reduce its greenhouse gas emissions by 2030 and achieve company-wide net-zero emissions by 2040. It also plans to complete production on the first plant in the United States that combines wind and solar generation with battery storage in a single location.

In the future, PGE plans to continue to use AWS services to drive growth and innovation. “We plan to use data and analytics to further strengthen our resiliency, reduce customer friction, and improve operational efficiency,” Sundaram says.