How to encrypt and sign DynamoDB data in your application

August 31, 2021: AWS KMS is replacing the term customer master key (CMK) with AWS KMS key and KMS key. The concept has not changed. To prevent breaking changes, AWS KMS is keeping some variations of this term. More info.

If you store sensitive or confidential data in Amazon DynamoDB, you might want to encrypt that data as close as possible to its origin so your data is protected throughout its lifecycle.

You can use the DynamoDB Encryption Client to protect your table data before you send it to DynamoDB. Encrypting your sensitive data in transit and at rest helps assure that your plaintext data isn’t available to any third party, including AWS.

You don’t need to be a cryptography expert to use the DynamoDB Encryption Client. The encryption and signing elements are designed to work with your existing DynamoDB applications. After you create and configure the required components, the DynamoDB Encryption Client transparently encrypts and signs your table items when you call PutItem and verifies and decrypts them when you call GetItem.

You can create your own custom components, or use the basic implementations that are included in the library. We’ve made sure that the classes that we provide implement strong and secure cryptography.

You can use the DynamoDB Encryption Client with AWS Key Management Service (AWS KMS) or AWS CloudHSM, but the library doesn’t require AWS or any AWS service.

The DynamoDB Encryption Client is now available in Python, as well as Java. All supported language implementations are interoperable. For example, you can encrypt table data with the Python library and decrypt it with the Java library.

The DynamoDB Encryption Client is an open-source project. We hope that you will join us in developing the libraries and writing great documentation.

How it works

The DynamoDB Encryption Client processes one table item at a time. First, it encrypts the values (but not the names) of attributes that you specify. Then, it calculates a signature over the attributes that you specify, so you can detect unauthorized changes to the item as a whole, including adding or deleting attributes, or substituting one encrypted value for another.

However, attribute names, and the names and values in the primary key (the partition key and sort key, if one is provided) must remain in plaintext to make the item discoverable. They’re included in the signature by default.

Important: Do not put any sensitive data in the table name, attribute names, the names and values of the primary key attributes, or any attribute values that you tell the client not to encrypt.

How to use it

I’ll demonstrate how to use the DynamoDB Encryption Client in Python with a simple example. I’ll encrypt and sign one table item, and then add it to an existing table. This example uses a test item with arbitrary data, but you can use a similar procedure to protect a table item that contains highly sensitive data, such as a customer’s personal information.

You can see the complete example in the examples directory of the aws-dynamodb-encryption-python repository.

Step 1: Create a table

I’ll start by creating a DynamoDB table resource that represents an existing table. If you use the code, be sure to supply a valid table name.


# Create a DynamoDB table
table = boto3.resource('dynamodb').Table(table_name)

Step 2: Create a cryptographic materials provider

Next, create an instance of a cryptographic materials provider (CMP). The CMP is the component that gathers the encryption and signing keys that are used to encrypt and sign your table items. The CMP also determines the encryption algorithms that are used and whether you create unique keys for every item or reuse them.

The DynamoDB Encryption Client includes several CMPs and you can create your own. And, if you’re in doubt, we help you to choose a CMP that fits your application and its security requirements.

In this example, I’ll use the Direct KMS Provider, which gets its cryptographic material from the AWS Key Management Service (AWS KMS). The encryption and signing keys that you use are protected by an AWS KMS key (KMS key) in your AWS account that never leaves AWS KMS unencrypted.

To create a Direct KMS Provider, you specify a KMS key. Be sure to replace the fictitious KMS key ID (the value of aws-cmk-id) in this example with a valid one.


# Create a Direct KMS provider. Pass in a valid KMS key.
aws_cmk_id = '1234abcd-12ab-34cd-56ef-1234567890ab'
aws_kms_cmp = AwsKmsCryptographicMaterialsProvider(key_id=aws_cmk_id)

Step 3: Create an attribute actions object

An attribute actions object tells the DynamoDB Encryption Client which item attribute values to encrypt and which attributes to include in the signature. The options are: ENCRYPT_AND_SIGN, SIGN_ONLY, and DO_NOTHING.

This sample attribute action encrypts and signs all attributes values except for the value of the test attribute; that attribute is neither encrypted nor included in the signature.


# Tell the encrypted table to encrypt and sign all attributes except one.
actions = AttributeActions(
    default_action=CryptoAction.ENCRYPT_AND_SIGN,
    attribute_actions={
        'test': CryptoAction.DO_NOTHING
    }
)

If you’re using a helper class, such as the EncryptedTable class that I use in the next step, you can’t specify an attribute action for the primary key. The helper classes make sure that the primary key is signed, but never encrypted (SIGN_ONLY).

Step 4: Create an encrypted table

Now I can use the original table object, along with the materials provider and attribute actions, to create an encrypted table.


# Use these objects to create an encrypted table resource.
encrypted_table = EncryptedTable(
    table=table,
    materials_provider=aws_kms_cmp,
    attribute_actions=actions
)

In this example, I’m using the EncryptedTable helper class, which adds encryption features to the DynamoDB Table class in the AWS SDK for Python (Boto 3). The DynamoDB Encryption Client in Python also includes EncryptedClient and EncryptedResource helper classes.

The DynamoDB Encryption Client helper classes call the DescribeTable operation to find the primary key. The application that runs the code must have permission to call the operation.

We’re done configuring the client. Now, we can encrypt, sign, verify, and decrypt table items.

Step 5: Add an item to the table

Let’s add an item to the DynamoDB table.


plaintext_item = {
    'partition_key': 'key1',
    'sort_key': 'key2'
    'example': 'data',
    'numbers': 99,
    'binary': Binary(b'\x00\x01\x02'),
    'test': 'test-value'
}

When we call the PutItem operation, the item is transparently encrypted and signed, except for the primary key, which is signed, but not encrypted, and the test attribute, which is ignored.


encrypted_table.put_item(Item=plaintext_item)

And, when we call the GetItem operation, the item is transparently verified and decrypted.


decrypted_item = encrypted_table.get_item(Key=partition_key)['Item']

To view the encrypted item, call the GetItem operation on the original table object, instead of the encrypted_table object. It gets the item from the DynamoDB table without verifying and decrypting it.


encrypted_item = table.get_item(Key=partition_key)['Item']

Here’s an excerpt of the output that displays the encrypted and signed item:

Figure 1: Output that displays the encrypted item

Client-side or server-side encryption?

The DynamoDB Encryption Client is designed for client-side encryption, where you encrypt your data before you send it to DynamoDB.

But, you have other options. DynamoDB supports encryption at rest, a server-side encryption option that transparently encrypts the data in your table whenever DynamoDB saves the table to disk. You can even use both the DynamoDB Encryption Client and encryption at rest together. The encrypted and signed items that the client generates are standard table items that have binary data in their attribute values. Your choice depends on the sensitivity of your data and the security requirements of your application.

Although the Java and Python versions of the DynamoDB Encryption Client are fully compatible, the DynamoDB Encryption Client isn’t compatible with other client-side encryption libraries, such as the AWS Encryption SDK or the S3 Encryption Client. You can’t encrypt data with one library and decrypt it with another. For data that you store in DynamoDB, we recommend the DynamoDB Encryption Client.

Encryption is crucial

Using tools like the DynamoDB Encryption Client helps you to protect your table data and comply with the security requirements for your application. We hope that you use the client and join us in developing it on GitHub.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about the Amazon DynamoDB Encryption Client, file an issue in the GitHub repos for Java or Python, or read and post on the AWS Crypto Tools Discussion Forum.

Want more AWS Security news? Follow us on Twitter.

AWS Security Blog