The Internet of Things on AWS – Official Blog
Detect water leaks in near real time using AWS IoT
Introduction
Water is one of the most precious resources needed for the sustenance of life. However, only 2% of the global water supply is suitable for human consumption. The United States Environmental Protection Association (EPA) estimates that 1.7 trillion gallons, roughly 30 percent of all treated water, is wasted every year in the United States.
According to New York City Department of Environmental Protection, a leaking fire hydrant can waste up to 1,000 gallons of water per minute. Utilities have to deploy manual resources to identify these leaks, which can be a time consuming and labor-intensive process. Moreover, if these leaks are not addressed, then utilities can be penalized with heavy fines for non-compliance with environmental laws. These risks are an outcome of the water supply infrastructure seen in most cities, where supply lines are run underground, creating natural challenges to quickly identify and repair leaks. In most instances, leaks are never detected until the next scheduled maintenance call or during an emergency situation where use of fire hydrant is verified. A typical maintenance call can take up to 3 months and may require a minimum of 3 visits. Imagine how much water is wasted during this period. The solution described in this blog can help reduce leakage waste and maintenance costs for the utilities.
Solution overview
Consider a scenario, where water is flowing through fire hydrants across cities and rural areas, and somewhere along such long routes, a minor leak occurs. The leak remains undetected for several weeks to months. Even after the leak is detected, it could take a few more weeks to identify the exact location and then fix it. Today, most fire hydrants can be upgraded with an IoT sensor to communicate statistics on the status and usage of the fire hydrant. These sensors can help identify water leaks in almost real time to trigger proactive maintenance actions. In the proposed architecture, an IoT-enabled, 5G-capable fire hydrant communicates using MQTT protocols to AWS. These fire hydrants use AWS IoT Core to authenticate with AWS.
Architecture diagram
Before we deep dive into the architecture, let’s understand the 4 stages represented by the architectural diagram.
The first stage establishes communication from IoT devices to AWS over MQTT protocol.
The second stage stores the collected data and maintains it for long-term compliance reasons.
The third stage dispatches notifications to field teams for active leak situations and enables them to take actions ASAP.
The fourth stage provides back-office operators control over the entire system. Back-office operators can utilize Amazon QuickSight dashboards to view active leaks and actions being performed, device health, and much more based on the collected data.
Stage 1: IoT Communications
5G-capable fire hydrants can establish communications with AWS over MQTT protocols.
- AWS IoT Core helps establish communication. Bulk on-boarding of devices is possible with AWS IoT Device Management.
- AWS IoT Device Management will securely access IoT devices, monitor health, detect and remotely troubleshoot problems, and manage software and firmware updates.
- AWS IoT Device Defender helps maintain security of all on-boarded IoT devices, monitors security metrics, and generates alerts based on deviations from the expected behavior of each device.
- Once this communication is established, the fire hydrant can send hydrant health status, geo-location and water flow data over secure MQTT protocol in JSON format to AWS.
Stage 2: Storage
Once IoT devices start sending events, there will be more data to collect and process.
- Events are stored into Amazon Simple Storage Service (Amazon S3) and then periodically offloaded to an archival service such as Amazon S3 Glacier.
Stage 3: Dispatch & Fix
Events contain vital information about possible leaks. Now, connected fire hydrants can communicate with AWS so back-office teams can be notified in near real-time and alert field workers for immediate action.
- Fire hydrant publishes a notification to AWS IoT Core over an MQTT protocol after a set interval.
- AWS IoT Core Rules Engine retrieves the notification from the MQTT topics.
- AWS IoT Core Rules Engine then sends the notification to AWS IoT Events.
- AWS IoT Events has a detector model that monitors incoming IoT events, (e.g. a fire hydrant’s water flow and pressure level), by sending a request back to the AWS IoT Core MQTT topic.
- AWS IoT Events detects the water flow and pressure abnormalities based on defined thresholds. If the water pressure and flow fall outside of defined thresholds, AWS IoT Events sends a notification message to an Amazon Simple Notification Service (Amazon SNS) topic.
- The field operations team receives a notification message to inform field operators of the possible leak situation.
- In addition to triggering an alert, AWS IoT Events sends the same message to Amazon DynamoDB to create a support case. The back-office team tracks the status of fire hydrants using an Amazon QuickSight dashboard.
- Once the leak is fixed, the field operations team updates the status of the support case.
Stage 4: Insight Reporting
All the collected events stored in Amazon S3 enable reporting capabilities.
- Amazon Athena is used to analyze and query on collected events.
- Amazon QuickSight supports insightful dashboards for back-office operators to help them visualize active leaks, actions being performed for active leaks, geographical distribution of leaks, as well as help them monitor health status for devices and much more.
Prerequisites
To follow along and set up the asset inspection solution, be sure to have the following:
- An AWS account.
- A device or laptop/computer with an access to your AWS account, Python version 2.7.18+ installed, and the AWS IoT Device SDK for Python version 1.3.1+.
Setting up AWS IoT Events to manage fire hydrant leaks
An AWS IoT rule needs to be configured to forward device data from AWS IoT Core (MQTT topic) to AWS IoT Events.
- Go to the AWS Management Console and select AWS IoT Core.
- Select Message Routing, then Rules, and then choose Create rule. Rule description is an optional field.
IoT Rule set up
- Set the Name for the rule and set the rule query statement to SELECT * FROM ‘iot/topic’. Sample query below.
Sample Query
- Choose Add rule action.
- Select IoT Events option and enter Input name.
- Select the Input previously created.
- Select Create new role and enter a role name.
- Select Add rule action.
- Select Create rule.
- A sample of the rule created in AWS Console.
Create Rule
In AWS IoT Events, create the following components to start the fire hydrant’s water flow and pressure:
- Select AWS IoT Events from the services menu. On the AWS IoT Events page, select an industry-specific template from the section shown following.
Create Detector Model
- From that screen select Simple Alarm and choose Start.
IoT Event template selection
- Create a detector model with 3 states each, with 2 transitions as shown following.
Create Detector Model
- Upon receipt of a notification, the device state is changed to “ActiveLeak“. This state is used to trigger the alert to field worker and back-office dashboard.
Creating an input in AWS IoT Events
You can create an input in AWS IoT Events by following the guide to create an input. In our example, we create an input with the following details:
- An example Input name set to “deviceNotificationInput”
- Upload a JSON file with following example JSON payload:
{
"geoLocation": "42.3928258265305, -71.07754968042828",
"timeStamp": "2022-05-31 08:47:44.870092",
"cityName": "Boston",
"state": "MA",
"deviceId": "BOS0”,
"sensorHealth": "OK",
"inputFlow": "10",
"outputFlow": "10"
}
Create Device Notification Input
Creating and publishing a detector model in AWS IoT Events
In our example, we create a detector model with the following details and you can find a sample at AWS IoT Events Developer Guide.
- Three states (Normal, ActiveLeak and Snooze).
- Each with transitions that switches the device from one state to another.
- Upon receiving a notification from the device, the normal transition triggers the sending of a notification to the outbound AWS IoT Core’s MQTT topic and changes the device state to out_of_range and pushing state to ActiveLeak.
- If the repair is taking longer, then device state could be pushed to Snooze state while the fix is being performed.
Normal State | ActiveLeak State | Snooze State |
States for the detector model
Creating a detector model
- Create an IAM role for the Detector Model. For more information, see the documentation for setting up permissions for AWS IoT Events.
- Create the sample detector model as shown in the previous image with states and transitions.
Analyzing data using Amazon QuickSight
Following are the type of visualizations you can create with the data using Amazon QuickSight.
Device health status: The below chart shows the device status along with device locations of fire hydrants. These charts will assist back-office teams to identify faulty fire hydrants and send geolocation to field operators to fix those faulty devices.
Device health report – Keeps back office up to date with compliance and device health status near real time.
Active leaks: The below chart shows water leaks across 3 example cities – New York, Chicago, and Boston.
Active leak report – Shows active leaks across cities for the back office
Geo-locations of active leaks: The below chart shows active water leaks on a map, plotted using device Geo coordinates. This could be useful for back-office teams to actively look at streets where there is an issue.
Geo location of fire hydrants – Shows fire hydrants in map/street view with their status
Cleaning up
To avoid incurring unwanted charges, delete the following resources:
- Clean up IoT devices as explained in this guide.
- Delete AWS Kinesis Delivery Streams as explained in this guide. .
- Delete Amazon S3 bucket as explained in this guide.
- Delete Amazon DynamoDB tables as explained in this guide.
- Delete Amazon Athena schema as explained in this guide.
- Delete Amazon QuickSight charts/visuals as explained in this guide. Delete Dashboard explained in this guide, and delete account explained in this guide.
- Delete Amazon SNS topics as explained in this guide.
Conclusion
In this blog, we showed how AWS IoT services can be used to detect real-time leaks in fire hydrants and pin-point the exact location of the leaks. This solution can lead to a reduction in time to fix the leaks, thereby reducing water wastage and improving environmental impact. The same solution can be extended to detect leaks in home water appliances or leaks in oil/ natural gas pipelines.
To learn more about how to use AWS IoT Core, you can refer to the documentation.
AWS welcomes feedback. Please connect with us on LinkedIn if you have thoughts or questions.
Authors
Mrunal Daftari is an Enterprise Senior Solutions Architect at Amazon Web Services. Mrunal is based in Boston, MA. He is a cloud enthusiast and very passionate about finding solutions for customers that are simple and addresses their business outcomes. He loves working with cloud technologies providing simple, scalable solutions that drive positive business outcomes, cloud adoption strategy, design innovative solutions and drive operational excellence. | |
Vaibhav Sabharwal is a Senior Solution Architect with Amazon Web Services (AWS). He is part of the Financial Service Technical Field Community. He helps AWS customers to build cloud adoption strategy, design innovative solutions and drive operational excellence. |