How do I publish MQTT messages to AWS IoT Core from my device when using Python?

Last updated: 2020-06-16

I can't send or receive MQTT (MQ Telemetry Transport) messages between AWS IoT Core and my device or MQTT client. How do I publish MQTT messages to AWS IoT Core?

Short Description

Follow these instructions to confirm that your AWS IoT thing is correctly configured and its certificates are properly attached. To test your setup, you can use the AWS IoT MQTT client and the example Python code provided in this article.

Resolution

Set up a directory to test MQTT publishing

1.    Create a working directory (for example, iot-test-publish) in your development environment.

2.    Create a sub-directory for certificates (for example, certificates) in your new working directory.

3.    From the command line, change the directory to your new working directory.

Install pip and the AWS IoT SDK for Python

1.    If you haven't already, install pip for Python 3 packaging. For more information, see Installation on the Python Packaging Authority (PyPA) website.

2.    Install the AWS IoT SDK for Python v2 by running the following from the command line:

pip install awsiotsdk

-or-

Install the AWS IoT Device SDK for Python (the previous SDK version) if you prefer by running the following command:

pip install AWSIoTPythonSDK

For more information, see AWS IoT SDK for Python v2 or AWS IoT Device SDK for Python on GitHub.

Note: These SDKs are recommended for connecting to AWS IoT Core, but they aren't required. You can also connect using any compliant third-party MQTT client.

Create an AWS IoT Core policy

1.    Open the AWS IoT Core console.

2.    In the left navigation pane, choose Secure.

3.    Under Secure, choose Policies.

4.    If you have any existing AWS IoT Core policies, choose Create to create a new policy.
-or-
On the You don't have any policies yet page, choose Create a policy.

5.    On the Create a policy page, enter a Name for your policy. For example, admin.

6.    Under Add statements, do the following:
For Action, enter iot:*.
Note: Allowing all AWS IoT actions (iot:*) is useful for testing. However, it's a best practice to increase security for a production setup. For more secure policy examples, see Example AWS IoT policies.
For Resource ARN, enter *.
For Effect, select the Allow check box.

7.    Choose Create.

For more information, see Create an AWS IoT Core policy and AWS IoT Core policies.

Create an AWS IoT thing

Note: You don't need to create a thing to connect to AWS IoT. However, things allow you to use additional security controls, as well as other AWS IoT features such as Fleet Indexing, Jobs, or Device Shadow.

1.    In the AWS IoT Core console, in the left navigation pane, choose Manage.

2.    If you have any existing things, choose Create to create a new thing.
-or-
On the You don't have any things yet page, choose Register a thing.

3.    On the Creating AWS IoT things page, choose Create a single thing.

4.    On the Add your device to the thing registry page, do the following:
Enter a Name for your thing. For example, Test-Thing.
(Optional) Under Add a type to this thing, choose or create a thing type.
(Optional) Under Add this thing to a group, choose or create a group. For more information about groups, see Static thing groups and Dynamic thing groups.
(Optional) Under Set searchable thing attributes (optional), add attributes as key-value pairs.
Choose Next.

5.    On the Add a certificate for your thing page, choose Create certificate. You see notifications confirming that your thing and a certificate for your thing are created.

6.    On the Certificate created page, do the following:
Under In order to connect a device, you need to download the following, choose Download for the certificate, public key, and private key.
Save each of the downloaded files to the certificates sub-directory that you created earlier.
Under You also need to download a root CA for AWS IoT, choose Download. The Server authentication page opens to CA certificates for server authentication.

7.    Under Amazon Trust Services Endpoints (preferred), choose Amazon Root CA 1. The certificate opens in your browser.

8.    Copy the certificate (everything from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE-----) and paste it into a text editor.

9.    Save the certificate as a .pem file named root.pem to the certificates sub-directory.

10.    On the Certificate created page in the AWS IoT Core console, choose Activate. The button changes to Deactivate.

11.    Choose Attach a policy.

12.    On the Add a policy for your thing page, do the following:
Select the AWS IoT Core policy that you previously created. For example, admin.
Choose Register Thing.

For more information, see the following pages:

Copy the AWS IoT Core endpoint URL

1.    In the AWS IoT Core console, in the left navigation pane, choose Settings.

2.    On the Settings page, under Custom endpoint, copy the Endpoint. This AWS IoT Core custom endpoint URL is personal to your AWS account and Region.

Create a Python program file

Save one of the following Python code examples as a Python program file named publish.py. If you installed the AWS IoT SDK for Python v2 earlier, use this example code:

Note: Replace customEndpointUrl with your AWS IoT Core custom endpoint URL. Replace certificates with the name of your certificates sub-directory. Replace a1b23cd45e-certificate.pem.crt with the name of your client .crt. Replace a1b23cd45e-private.pem.key with the name of your private key.

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0

from awscrt import io, mqtt, auth, http
from awsiot import mqtt_connection_builder
import time as t
import json

# Define ENDPOINT, CLIENT_ID, PATH_TO_CERT, PATH_TO_KEY, PATH_TO_ROOT, MESSAGE, TOPIC, and RANGE
ENDPOINT = "customEndpointUrl"
CLIENT_ID = "testDevice"
PATH_TO_CERT = "certificates/a1b23cd45e-certificate.pem.crt"
PATH_TO_KEY = "certificates/a1b23cd45e-private.pem.key"
PATH_TO_ROOT = "certificates/root.pem"
MESSAGE = "Hello World"
TOPIC = "test/testing"
RANGE = 20

# Spin up resources
event_loop_group = io.EventLoopGroup(1)
host_resolver = io.DefaultHostResolver(event_loop_group)
client_bootstrap = io.ClientBootstrap(event_loop_group, host_resolver)
mqtt_connection = mqtt_connection_builder.mtls_from_path(
            endpoint=ENDPOINT,
            cert_filepath=PATH_TO_CERT,
            pri_key_filepath=PATH_TO_KEY,
            client_bootstrap=client_bootstrap,
            ca_filepath=PATH_TO_ROOT,
            client_id=CLIENT_ID,
            clean_session=False,
            keep_alive_secs=6
            )
print("Connecting to {} with client ID '{}'...".format(
        ENDPOINT, CLIENT_ID))
# Make the connect() call
connect_future = mqtt_connection.connect()
# Future.result() waits until a result is available
connect_future.result()
print("Connected!")
# Publish message to server desired number of times.
print('Begin Publish')
for i in range (RANGE):
    data = "{} [{}]".format(MESSAGE, i+1)
    message = {"message" : data}
    mqtt_connection.publish(topic=TOPIC, payload=json.dumps(message), qos=mqtt.QoS.AT_LEAST_ONCE)
    print("Published: '" + json.dumps(message) + "' to the topic: " + "'test/testing'")
    t.sleep(0.1)
print('Publish End')
disconnect_future = mqtt_connection.disconnect()
disconnect_future.result()

If you installed the AWS IoT Device SDK for Python (the previous SDK version), use this example code:

Note: Replace customEndpointUrl with your AWS IoT Core custom endpoint URL. Replace certificates with the name of your certificates sub-directory. Replace a1b23cd45e-certificate.pem.crt with the name of your client .crt. Replace a1b23cd45e-private.pem.key with the name of your private key.

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0

import time as t
import json
import AWSIoTPythonSDK.MQTTLib as AWSIoTPyMQTT

# Define ENDPOINT, CLIENT_ID, PATH_TO_CERT, PATH_TO_KEY, PATH_TO_ROOT, MESSAGE, TOPIC, and RANGE
ENDPOINT = "customEndpointUrl"
CLIENT_ID = "testDevice"
PATH_TO_CERT = "certificates/a1b23cd45e-certificate.pem.crt"
PATH_TO_KEY = "certificates/a1b23cd45e-private.pem.key"
PATH_TO_ROOT = "certificates/root.pem"
MESSAGE = "Hello World"
TOPIC = "test/testing"
RANGE = 20

myAWSIoTMQTTClient = AWSIoTPyMQTT.AWSIoTMQTTClient(CLIENT_ID)
myAWSIoTMQTTClient.configureEndpoint(ENDPOINT, 8883)
myAWSIoTMQTTClient.configureCredentials(PATH_TO_ROOT, PATH_TO_KEY, PATH_TO_CERT)

myAWSIoTMQTTClient.connect()
print('Begin Publish')
for i in range (RANGE):
    data = "{} [{}]".format(MESSAGE, i+1)
    message = {"message" : data}
    myAWSIoTMQTTClient.publish(TOPIC, json.dumps(message), 1) 
    print("Published: '" + json.dumps(message) + "' to the topic: " + "'test/testing'")
    t.sleep(0.1)
print('Publish End')
myAWSIoTMQTTClient.disconnect()

Test your setup

1.    In the AWS IoT Core console, in the left navigation pane, choose Test.

2.    On the MQTT client page, for Subscription topic, enter test/testing.

3.    Choose Subscribe to topic. A test topic named test/testing is ready for test message publication. For more information, see View device MQTT messages with the AWS IoT MQTT client.

4.    Run the following from your command line:

python3 publish.py

The Python program publishes 20 test messages to the topic test/testing that you created in the AWS IoT Core console. View the topic in the console to see the published messages.

Tip: You can also test other features of the SDKs, such as subscribing and connecting via WebSockets, using the included pubsub samples. For more information, see pubsub (AWS IoT SDK for Python v2) or BasicPubSub (AWS IoT Device SDK for Python) on GitHub.

(Optional) Enable AWS IoT logging to Amazon CloudWatch

You can monitor event logs for MQTT messages that you publish to AWS IoT Core. For setup instructions, see Configure AWS IoT logging and Monitor AWS IoT using CloudWatch Logs.


Did this article help you?

Anything we could improve?


Need more help?