Using AWS IoT to Create a Smart Home Water-Monitoring Solution

2018 saw the fourth year of drought and the worst in recorded history for the city of Cape Town, South Africa. “Day zero” was a term coined by the city for the day when they would have to turn the water off for citizens. Fortunately, “day zero” was never realized, and Cape Town didn’t go down in history as the first major city to run out of water. Water restrictions and augmentation plans alleviated the crisis until rainfall returned to the city, however, the city remains sensitive to water crises. Citizen behaviour must permanently change in order to use water in a sustainable manner.

AWS presented this session at the AWS Summit in Cape Town where I met the CTO of a company called Apex Innovation who are now registered AWS partners. Apex Innovation has commercialized the entire architecture above. They spent time resolving the problem of using a hall-effect sensor in a commercial use case. Hall effect sensors are susceptible to a phenomenon called pulse drift which produces inaccurate pulse counts over time, rendering the water flow measurement inaccurate. Apex Innovation has many installations of this solution in commercial buildings across the continent and is working with city municipalities for large scale installations of more than 50,000 units to assist municipalities with accurate water consumption data.

In this blog post, I share an AWS IoT water solution that provides real-time water consumption information to home owners. The information can be used to encourage a change in consumption behavior to help the city conserve water. This solution uses AWS IoT core, Amazon Kinesis Data Firehose, Amazon S3, AWS Lambda, Amazon DynamoDB and an iOS application to create a real-time water consumption application.

Architecture

The diagram illustrates how simple it is to design a real-time water consumption application. I will delve into more detail of each component in the diagram in the sections below.

The physical install

I installed a water flow sensor available on Amazon for $11.19 on the water mains supply line into my house. As the water flows through the sensor, it turns a cog. As the cog turns, it passes over a coil. Each pass produces an electrical pulse. By counting the pulses in a given time period and factoring the diameter of the pipe it’s possible calculate the volume of water flowing through the sensor.

These photos show the installation of the water flow sensor on the main water supply.

Each pulse is sent to a micro-controller board, which is similar to the ESP32 series of low-cost, low-power system on a chip microcontrollers. Pulses are counted over a period of one minute. The volume is calculated and sent to AWS IoT Core for processing. Before any communication can take place between the AWS Hex board and AWS IoT Core, you must register the AWS Hex board as a thing.

A JavaScript method running on the micro-controller board:

• Runs the timer and the code to execute the flow calculation.
• Sends an MQTT message to the AWS topic.

The timer method stores the number of pulses received in the last minute. Every minute, the flow is calculated by factoring the flow rate for 1L per minute, which for this size sensor is a factor of 4.5. Divide that by 60 to calculate the liters per minute, as follows:

(pulse count / 4.5) / 60

Here is the source code for the timer function and helper functions:

// Load  Mongoose OS API
load('api_timer.js');
load('api_uart.js');
load('api_sys.js');
load('api_mqtt.js');
load('api_config.js');
 
// Uart  number used for this example
let uartNo = 1;
//  Accumulated Rx data, will be echoed back to Tx
let rxAcc = "";
 
let value = false;
 
// Set the  global pulse counter
let pulseCounter = 0;
 
// Set the  conversion factor for the hall effect calculation
let HEFactor = 4.5;
 
/*
 * Configure UART at 115200 baud
 * Configure the water flow sensor pin: 22
 */
UART.setConfig(uartNo, {
    baudRate: 115200,
    esp32: {
        gpio: {
            rx: 22,
            tx: 23,
        },
    },
});
 
/*
 * Set dispatcher callback, it will be called  whenver new Rx data or space in
 * the Tx buffer becomes available.
 * The number of messages received per minute  x 4.5 is the L/min
 * Count the number of messages per minute.
 */
UART.setDispatcher(uartNo, function (uartNo, ud) {
    let ra = UART.readAvail(uartNo);
    if (ra > 0) {
        // Received new data:  print it immediately to the console, and also
        // accumulate in the  "rxAcc" variable which will be echoed back to UART later
        let data = UART.read(uartNo);
        pulseCounter++;
        print("Received  UART data:", data);
        rxAcc += data;
    }
}, null);
 
// Enable  Rx
UART.setRxEnabled(uartNo, true);
 
/**
 * Create a timer to run every minute.
 * This timer method will store the number of  pulses received in the last minute.
 * Once the timer is done, send data to AWS  IoT
 * Reset the pulse counter.
 * Rinse & repeat.
 * 
 * Calculating the flow rate per hour.
 * For every litre of liquid passing through  the sensor per minute,
 * it pulses 4.5 times. Divide the pulseCount  by 4.5 to get the litres per minute.
 * Divide that by 60 to get litres per hour.
 * 
 * (pulseCount / 4.5) / 60
 * 
 */
Timer.set(60000 /*  milliseconds */, true /* repeat  */, function () {
    // Send the MQTT message
    // Hard-coded device ID was  866191037731759
    let topic = '/things/waterflowcontrollers/' + Cfg.get('device.id');
    // Set the date and time
    let Timestamp = Timer.now();
    // Date format day/month/year  hour:minute:second
    let datetime = Timer.fmt("%d/%m/%Y  %H:%M:%S", Timestamp);
    let flowRate = (pulseCounter / HEFactor) / 60;
 
    let res = MQTT.pub(topic, JSON.stringify(
        {
            "SerialNumber": Cfg.get('device.id'),
            "BatteryVoltage": "2080mV",
            "QCCID": "8944538523012471069",
            "GSN": Cfg.get('device.id'),
            "FlowRate": flowRate,
            "FlowCalibration": "4.5",
            "Date": datetime
        }
    ), 1);
    print('Published  flow rate: ', res ? flowRate : 'no');
 
    // Reset the pulse counter
    pulseCounter = 0;
}, null);

Because you configure most of this solution in AWS IoT Core, here is some background about the registry, AWS IoT policies, and X.509 certificates.

Registering a device in the registry

The registry allows you to keep a record of all of the devices that are registered to your AWS IoT account. It tracks metadata, such as device attributes and capabilities. In the case of the example used in this post, the registry supports metadata that describes whether a sensor reports temperature and if the data is Fahrenheit or Celsius. The registry assigns a unique identity to each device that is consistently formatted, regardless of the device type or the way the device connects.

For more information, see Register a Device in the AWS IoT Core Developer Guide.

AWS IoT Policy

X.509 certificates are used to authenticate your device with AWS IoT Core. AWS IoT Core policies are used to authorize your device to perform operations, such as subscribing or publishing to MQTT topics. Your device presents its certificate when it sends messages to AWS IoT Core. To allow your device to perform operations, you must create an AWS IoT policy and attach it to your device certificate.

For more information, see Create an AWS IoT Policy in the AWS IoT Core Developer Guide.

Attach a certificate to your device

A device must have a X.509 certificate, private key, and root CA certificate to authenticate with AWS IoT Core. AWS recommends that you also attach the device certificate to the thing that represents your device in AWS IoT Core. This way you can create AWS IoT Core policies that grant permissions based on certificates attached to your things.

For more information, see Attach a Certificate to a Thing in the AWS IoT Core Developer Guide.

Data

Incoming messages from the thing are sent to an AWS IoT topic configured for the thing. Amazon Kinesis Data Firehose subscribes to the topic to send every message to Amazon S3 for long-term durable storage. When the message is received in Amazon S3, an S3 event triggers an AWS Lambda function that writes a copy of the message into Amazon DynamoDB, where analysis and visualization is performed. After the raw message has been copied to Amazon DynamoDB, the message is archived in Amazon Glacier to reduce storage costs.

Data from the thing is sent in JSON format for ease of use with applications. The timestamp is important for the downstream applications to collate the data into comprehensible time series data blocks. The flow rate is the calculated volume of water flowing through the water flow sensor per minute. A serial number uniquely identifies the thing and the longitude and latitude are the GPS coordinates of the installed device.

Here is a sample JSON message:

{
    "Date": "21/02/2019 16:14:10",
    "FlowRate": 28,
    "SerialNumber": "esp32_39B3F8",
    "latitude": -33.9399299,
    "longitude": 18.4751000
}

Visualization

Visualizing the data from the water flow sensors in a format that is easily interpreted is crucial to the success of the application. A mobile application and a web application make it easy for the user to access and read the data. Different graphs are important for interpreting different trends in water usage.

An hourly graph is useful for leak detection, where a sudden spike in water usage over the last hour can alert the user to a potential leak. The user is notified of the high water usage with an SMS using Amazon SNS.

A weekly and a monthly graph is useful for the user to quickly identify trends over the last few days or the last month. Additional visual space in the web dashboard allows us to provide comparative graphs for water consumption over years so the user can compare his or her water consumption at the same time in previous years. This is useful for seasonal changes in water consumption.

Mobile application

In this example, an iOS application was developed in XCode to visualize the data in graph format by connecting to a gateway in Amazon API Gateway which exposes AWS Lambda functions to fetch the data required to render each graph.

Here are some photos of the mobile application screens.

The mobile app and the website both need security credentials in order to make programmatic requests to AWS. Amazon Cognito is ideal for this scenario. You can use this service with the AWS Mobile SDK for iOS and the AWS Mobile SDK for Android and Fire OS to create unique identities for users and authenticate them for secure access to your AWS resources.

Web application

The same AWS Lambda functions used by the iOS mobile application are accessed through Amazon API Gateway for the web application dashboard to render the graphs on the website. The website, hosted in Amazon S3, uses Node.js to access the gateway in Amazon API Gateway and render the graphs.

Here is a dashboard in the web application.

Next steps

Being able to accurately measure the water consumption in our house has changed my family’s consumption of this most precious resource. We now actively reduce our water consumption. Now that you know how easy it is to measure important environmental data, like water usage using AWS IoT services, I recommend you find opportunities in your everyday life to leverage IoT technologies. Whether it’s temperature, smart home utilities or metering. The ability for you to build IoT applications with simple devices unlocks a world of data and insights into your environment.