AWS Partner Network (APN) Blog
IBM Energy Anomaly Detection for Energy and Utilities Companies Leveraging Personalized AI
By Elena Angelone, Chiara Betti, Giulia Franco, and Davide Toma, Data Scientists – IBM
By Simone Romano, Technical Solution Architect – IBM
By Gianluca Vergaretti, Service Line Leader, Partner Data & Technology Transformation – IBM
By Diego Colombatto, Principal Partner Solutions Architect – AWS
 |
| IBM |
 |
Energy and utilities companies face the issue of optimizing energy consumption to maximize benefits for the climate and helping customers to use energy in concert with a dynamic power grid. In this context, the identification of abnormal consumptions is the first important step which leads to consumption awareness.
Regardless of the source of energy, demand is growing. With the global population expected to increase by about two billion over the next two decades, and with improving standards of living it’s estimated that by 2050 electricity generation is expected to increase by 50%.
As the world grows, concerns over pollution and climate change are forcing businesses to redesign how they produce and use energy. Energy efficiency can play an important role in helping the world meet its demand for power and mobility, while generating new revenues given by enhanced resource management.
A notable approach for achieving this objective is through informing end users of their power usage patterns. Accordingly, consumers can improve their behavior and change consumption habits with the aim of reducing wasted energy and contributing to the promotion of sustainable and green energy ecosystems.
The good news is that, in last two decades, smart metering solutions have been implemented and are up and running for major worldwide utilities providers, collecting data daily from up to millions of devices with current energy consumption information.
Given the uniqueness of each user, traditional analysis methods often provide inaccurate results when it comes to detecting abnormal consumption.
In this post, we’ll discuss how IBM’s anomaly detection solution for energy and utilities helps companies increase energy efficiency leveraging a personalized artificial intelligence (AI) paradigm and to calculate environmental, social, and governance (ESG) metrics related to energy consumption.
IBM Consulting is an AWS Premier Tier Services Partner and Managed Service Provider (MSP) that offers comprehensive service capabilities addressing both business and technology challenges that clients face today.
Business Needs and Opportunities
IBM’s anomaly detection solution for energy and utilities enables companies to be aware of abnormal energy consumptions. For each entity of the system (it can be a smart meter, current clamp, or other device able to monitor energy consumption), the system will train a specific AI-based anomaly detection model that will detect abnormal patterns.
To give you the understanding of the benefit of personalized AI approach embedded in IBM anomaly detection system for energy and utilities, consider the situation where you need to identify anomalies for two identical monitored appliances, where each appliance has its own specific consumption pattern.
In this kind of situation, it’s often difficult to implement a rule-based engine to identify abnormal consumption patterns or use a traditional machine learning (ML) model trained with cross-appliances data to identify abnormal consumption patterns.
One of the biggest obstacles encountered in this type of problem lies in predicting entity-specific behavior in cases where this is closely tied to its unique habits. A personalized AI approach, the foundation of IBM’s anomaly detection solution for energy and utilities, aims to overcome this by proposing an entity-specific anomaly detection system.
Figure 1 – Personalized AI high-level overview.
The system can calculate environmental ESG metrics related to energy consumption, give advice on the current state of energy efficiency for the organization, and energy efficiency measures it can take to achieve its objectives.
IBM anomaly detection is the right tool in a scenario where:
- There are Internet of Things (IoT) devices collecting energy data for a set of equipment.
- Each device/equipment has specific usage pattern (there is no deterministic approach to identify anomalies).
- There’s a need for an automatic report with anomalies highlights.
- Calculate environmental ESG metrics to measure energy efficiency for the organization.
Solution Overview
One of most important concepts in IBM’s anomaly detection solution, from an end-user point of view, is the concept of pipeline that enables the following capabilities:
- Ingestion process to collect data from external IoT platform and/or other data layers. IBM anomaly detection can be integrated with existing smart metering solutions (such as IoT platforms) using the external data collection layer.
- Select the AI model you want to use to predict abnormal usage, choosing from a catalog of pre-trained AI models or defining your own AI model, according with IBM solution programmatic interface.
- Detect anomalous consumption patterns for each user and for each time window.
- Calculate environmental ESG metrics.
- Visualize model output in a business user-oriented dashboard.
Once the end users have defined the pipeline, they just need to check prediction results which are available on web dashboard.
It’s important to note that IBM’s system allows you to customize processing layer injecting custom AI anomaly detection models and environmental ESG metrics modules. This can be useful if you have the need to implement a custom AI model (like for a domain-specific AI model) and you need to integrate it in system pipeline.
Architecture
IBM’s anomaly detection solution for energy and utilities can be grouped, from an architectural point of view, into the following logical layers:
- Extract, transform, load (ETL) layer
- Storage layer
- Management and processing layer
- Presentation layer
Figure 2 – IBM anomaly detection solution architectural overview.
- ETL layer uses AWS Glue to connect to customer data repository (external IoT data sources in Figure 2) and store raw data in Amazon Simple Storage Service (Amazon S3) bucket (raw data bucket). In case an IoT platform collecting energy consumption data is already is in place, IBM’s solution can integrate it using this ETL layer. Depending on the customer scenario, data ingestion to S3 can be achieved in additional ways, including the usage of AWS no-code services like Amazon AppFlow.
- Storage layer uses Amazon S3 and is composed by these data layers:
- Raw data: Energy usage data collected from external systems.
- Trainable datasets: Automatically generated dataset, with consumption data organized per entity (per IoT device).
- Curated zone: This layer includes entity-specific trained ML models and prediction outputs, visualized using Amazon QuickSight.
- Management and processing layer includes an AI module and ESG module. The AI module is composed by AI algorithms, custom and pre-built, developed and deployed using Amazon SageMaker. The ESG module includes the business logic to calculate environmental ESG metrics, using SageMaker.
- Presentation layer, implemented using Amazon QuickSight, includes a business dashboard (see Figure 3) where users can visualize prediction output, and an administration board that provides capabilities required to configure end-to-end pipeline, from data acquisition to model training.
Figure 3 – Business dashboard summary of devices monitored and anomalies detected.
Conclusion
As energy and utilities companies face the issue of optimizing energy consumption while dealing with expanded regulatory compliance and pressures from key stakeholders, companies are realizing they must be transparent about sustainability efforts.
Leading companies around the world are now committed to validating measured ESG reporting and non-financial performance disclosures. ESG performance and reporting summarize efforts and provide stakeholders and investors with documented performance and insights to make more holistic and sustainable decisions. In this context, the identification of abnormal consumptions is the first important step which leads to consumption awareness.
IBM has a ready-to-use anomaly detection solution for energy and utilities to automatically identify abnormal energy usage pattern leveraging a personalized AI paradigm to reach high accuracy metrics and calculate environmental ESG metrics.
The solution can be extended to additional use cases in energy and utilities domains and integrate with many third-party IoT devices and applications for IoT data collection.
.
.
IBM – AWS Partner Spotlight
IBM Consulting is an AWS Premier Tier Services Partner and MSP that offers comprehensive service capabilities addressing both business and technology challenges that clients face today.