The Internet of Things on AWS – Official Blog

How to detect anomalies in device metrics and improve your security posture using AWS IoT Device Defender custom metrics

Introduction

IoT applications and devices can be diverse and are used across industries such as utilities, agriculture, manufacturing, mining, and consumer electronics. With the exponential growth of IoT devices and the increasing threat landscape, it also means that IoT security needs to be accounted for and designed into the solution from the ground up.

AWS IoT Device Defender is a service that helps secure your IoT device fleet and can be used to audit and monitor your IoT devices at scale. By default, the service enables you to monitor 17 network-related metrics, such as changes in connection patterns, devices that communicate to unauthorized or unrecognized endpoints, and changes in inbound and outbound device traffic patterns. You can learn how to leverage these metrics to monitor your fleet of IoT devices.

But what happens if you need to monitor metrics that are unique to your device fleet or use case? For example, the number of devices connected to wi-fi gateways, charge levels for batteries, or security-related metrics such as domains being contacted by devices, detecting changes to running applications or processes on your devices, changes in configuration of your devices, remote logins, or any other application-specific behavior.

In this blog post, you will learn the steps involved in monitoring security metrics specific to your IoT application. As an IoT administrator, you’ll be able to set up security profiles to define the expected behavior of your devices based on custom metrics, monitor behavior patterns, and receive alerts when devices violate the expected behavior. AWS IoT Device Defender custom metrics give you the flexibility to monitor operational health and security metrics that are unique to your device fleet or use case and enable you to respond to issues in a timely manner. It’s easy to configure and use on devices which connect to AWS IoT Core and helps you improve the security posture of your IoT devices and system. Understanding the state of your devices is important for ensuring the reliability, security, health and performance of your IoT system. Device monitoring can provide the information you need to help your development and operations teams react to issues. It can help you understand your IoT system’s state using a predefined set of metrics and custom metrics. We will now show you how to create a sample custom metric to monitor for changes in processes running on an IoT device.

Solution overview and use case

For the purposes of this post, let’s assume:

  1. You are building a Linux-based device. Let’s call this device mything1
  2. You have authored an application myapp that performs all the business operation in the device.

You have identified that since myapp is communicating over the network, monitoring myapp‘s behavior is important. From a process behavior perspective, you know that myapp should never launch a child process. For example, launching a child process such as a shell that’s controlled by an unauthorized user to execute arbitrary commands or a crypto-miner for mining cryptocurrency using the device’s compute resources, are common indicators of compromise. With this context in mind, we will build a solution for monitoring the number of child processes launched by myapp and receive alerts from AWS IoT Device Defender when myapp launches any new process.

Solution prerequisites

  1. AWS account
  2. You can use the AWS IoT quick connect guide to register a thing, apply policies, attach certificates and download the sample device agent. Choose Python SDK for the AWS IoT Device SDK under step 2 of the above guide.
  3. AWS IoT Device Defender Agent SDK (Python)
  4. A computer with the latest browser – like Firefox or Chrome
  5. Basic understanding of Linux (e.g. create directories, set file permissions) and programming (compiling code)

Note: You will find code screenshots to indicate where code additions need to be made in the existing AWS IoT Device Defender Agent SDK

Solution architecture

Solution walkthrough

Cloud-side changes

1.     Create a custom metric representing the number of child processes of myapp:

a. Go to the Device Defender Detect Metrics section – Under AWS IoT on the left panel under Detect click Metrics.

b. Click Create beside custom metrics.

c. In the definition section, specify the name, a description and number as the metric type:

d. Successful custom metric creation:

2. Create security profile

a. Go to the Security profiles section, under the Detect drop down select Security Profiles

b. Under Create Security Profile click on “Create Rule-based anomaly Detect profile”

c. Since we know that myapp should never launch a child process, you should define the expected behaviors by picking the Metric as: Number of Child Processes of myapp and setting the expected value to be less than or equal to 0:

d. Also, add your custom metric by clicking the drop down “Additional Metrics to retain”:

e. Click Next. Keep default settings for Alert target section.

f. Click Next. Attach Security profile to All things, if this rule is a fleet-wide expectation. Please note that you have the option to pick specific thing groups to apply this profile too.

g. Click Next. Click Save and re-check all settings then click continue.

h. The Security profiles page should list your newly created security profile:

3. First check you are in the right region. Then update the IoT policy in the AWS IoT Policies page to allow Device Defender metrics scoping the privileges to things prefixed by mything1 only.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iot:Publish",
        "iot:Receive"
      ],
      "Resource": [
        "arn:aws:iot:eu-west-1:111122223333:topic/$aws/things/mything1/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iot:Subscribe"
      ],
      "Resource": [
        "arn:aws:iot:eu-west-1:111122223333:topicfilter/$aws/things/mything1/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iot:Connect"
      ],
      "Resource": [
        "arn:aws:iot:eu-west-1:111122223333:client/mything1"
      ]
    }
  ]
}

Device-side change

1. Download sample agent from Github:

git clone https://github.com/aws-samples/aws-iot-device-defender-agent-sdk-python.git

2. Structure of the sample agent:

a. collector.py is the Python module responsible for

i. Collecting metrics that you’re interested in, in this case: the number of child processes of myapp. Note that the collection of metrics occurs at intervals defined by the command line argument: -i

ii. It formats the collected metrics in the format required by AWS IoT Device Defender Detect using the metrics.py module. metrics.py uses the tags.py module to specify the metric name to be sent to AWS IoT Device Defender Detect

b. agent.py is the high-level module that combines the collector and the awsiot SDK used for communicating with AWS IoT

3. Modify tags.py to include a new metric as a property of the class Tags:

@property
def num_child_processes(self):
    return "num_child_procs_myap

4. Modify metrics.py to include num_child_processes:

a. Update the constructor function (init) to set a default value: self.num_child_processes = []

b. Create a member function of class Metrics to set up your metric for sending over the network

def add_num_child_processes(self, num_child_processes):
    self.num_child_processes = {"number": num_child_processes}

c. Lastly, convert the metric to the previously specified Tag property in the member function _v1_metrics:

if self.num_child_processes:
    report[t.custom_metrics] = {t.num_child_processes: [self.num_child_processes] }

5. Update collector.py  to include the functions required for finding the number of child processes of myapp:

a. We will use two functions here:

i. one for finding the process object representing myapp . This function should be defined outside the collector class

def find_process(process_name):
    # Return the first process object 
    # which matches `process_name` exactly
    for proc in ps.process_iter():
        if process_name == proc.name():
            return proc

ii. one as a staticmethod member function of class Collector for finding myapp‘s child processes:

@staticmethod
def get_num_child_processes(metrics):
    process_name = "myapp"
    my_process = find_process(process_name)
    num_child_processes = 0
    if my_process:
        num_child_processes = len(my_process.children(recursive=True))
    metrics.add_num_child_processes(num_child_processes)

b. In the member function collect_metrics, add a line to call get_num_child_processes if custom_metrics are enabled

          c.if self._use_custom_metrics:
self.get_num_child_processes(metrics_current)

6. Install the package: pip install ./aws-iot-device-defender-agent-sdk-python --upgrade

7. Test by running collector.py module independently just to ensure that there are no errors:

a. Note that I have passed the command line argument: -cm here to enable custom metrics collection

b. Create a fake myapp by creating a copy of your current shell and renaming it to myapp:

                       i. cp `which sh` ./myapp

 ii. Launch myapp: ./myapp

                      iii.     Launch a long running process like cat that will wait for user input: cat

8. Run agent.py to continuously monitor the number of process spawned by myapp with the required parameters, in addition to -cm (For enabling custom metrics):

python aws-iot-device-defender-agent-sdk-python/AWSIoTDeviceDefenderAgentSDK/agent.py -f json -e <your-endpoint> -r <root_cert_path> -c <cert_path> -k <private_key_path> -cm -id mything1

9. You should shortly be able to view the number of child processes of myapp by navigating to the Defender Metrics tab in the Things page (recheck you are in the right AWS region):

10. You should also be able to see any alarms generated in case there are any violations:

Conclusion

In this blog post, we demonstrated how to define a custom metric in AWS IoT Device Defender by creating a rule-based security profile and the changes required in the sample agent in order to send this information from the device to AWS IoT Device Defender. Now you can get started with creating your own custom metrics unique to your device fleet or use case, get alerts, investigate issues and take mitigation actions. AWS IoT Device Defender’s built-in mitigation actions can be used to perform mitigation steps on alerts such as adding things to a thing group, replacing default policy version, and updating device certificate.

Learn more

About the authors

Eknath Venkataramani is a security engineer on the AWS IoT team. He currently focuses on helping to secure multiple AWS IoT service releases by identifying and designing new IoT features that make security easier for IoT customers.
Ryan Dsouza Ryan Dsouza is a Principal Solutions Architect for IoT at AWS. Based in New York City, Ryan helps customers design, develop, and operate more secure, scalable, and innovative solutions using the breadth and depth of AWS capabilities to deliver measurable business outcomes. Ryan has over 25 years of experience in digital platforms, smart manufacturing, energy management, building and industrial automation, and OT/IIoT security across a diverse range of industries. Before AWS, Ryan worked for Accenture, SIEMENS, General Electric, IBM, and AECOM, serving customers for their digital transformation initiatives.