AWS Developer Tools Blog

Using atomic counters in the Enhanced DynamoDB AWS SDK for Java 2.x client

We are pleased to announce that users of the enhanced client for Amazon DynamoDB in AWS SDK for Java 2.x can now enable atomic counters, as well as add custom DynamoDB update expressions through the enhanced client extension framework.

Customers have told us that they want improved performance and consistency when updating table records. The record update workflow in the DynamoDB enhanced client often means reading a record from the database to access current values before writing it back. This overhead can be painful if the records are large, and it can incur additional costs. Furthermore, there’s no guarantee that the record isn’t updated between a read and a write. This means that it isn’t an atomic set of actions.

This change means that you can tag a numeric record attribute as an atomic counter and use DynamoDB to update the attribute with a specific value each time that you call the updateItem operation on your table. Moreover, we exposed an API that models update expressions. This lets you write your own extensions that directly create schema-level update expressions for DynamoDB.

Concepts

DynamoDB UpdateExpression – This is the syntax used by DynamoDB when calling its UpdateItem operation. Use the enhanced DynamoDB client to automatically generate this expression when you supply an item to update.

Enhanced client UpdateExpression API – This is the abstraction representing DynamoDB UpdateExpression in the enhanced client. Read more under Introduction to the enhanced client UpdateExpression API later in this post.

Enhanced client extension – This is a class implementing the DynamoDbEnhancedClientExtension interface that hooks into the logic of operations, such as updateItem, and provides the ability to modify requests or response parameters.

Extension chain – All of the extensions that are activated for an enhanced client. The extensions in a chain are applied in order.

Using Atomic Counters

When you want to create an atomic counter and update DynamoDB every time that you call the updateItem operation, create a table attribute of the type Long that represents the counter, and tag it as an atomic counter.

You enable atomic counter functionality when you instantiate an enhanced client. This is because it automatically loads the corresponding extension, AtomicCounterExtension. By default, a counter starts at 0 and increments by 1 each time the record in the table is updated. You can customize it by changing the start value and/or the increment value, including negative values.

The start value is set either when

  • you call the updateItem operation and the attribute doesn’t exist, or
  • you call the putItem operation.

The following example shows how to create and use atomic counters for both bean-based table schemas and static table schemas.

Step 1: Define a schema with a tagged attribute

Option 1: Bean-based table schema
Create an attribute of the type Long, and annotate it with @DynamoDbAtomicCounter.

@DynamoDbBean
public class Customer {
  
    @DynamoDbPartitionKey
    public String getId() { ... }
    public void setId(String id) { ... }

    @DynamoDbAtomicCounter
    public Long getUpdateCounter() { ... }
    public void setUpdateCounter(Long counter) { ... }

    @DynamoDbAtomicCounter(delta = 5, startValue = 10)
    public Long getCustomCounter() { ... }
    public void setCustomCounter(Long counter) { ... }
}

Option 2: Static immutable table schema
Create a StaticAttribute attribute with the attribute type Long, and use one of the StaticAttributeTags.atomicCounter() methods to tag the attribute.

The following code example assumes that the Customer.class exists and defines a schema to reference the class.

static final StaticTableSchema<Customer> TABLE_SCHEMA=
    StaticTableSchema.builder(Customer.class)
                     .newItemSupplier(Customer::new)
                     .addAttribute(String.class, a -> a.name("id") ... )
                     .addAttribute(Long.class, a -> a.name("defaultCounter")
                                                     .getter(Customer::getDefaultCounter)
                                                     .setter(Customer::setDefaultCounter)
                                                     .addTag(atomicCounter()))
                     .addAttribute(Long.class, a -> a.name("customCounter")
                                                     .getter(Customer::getCustomCounter)
                                                     .setter(Customer::setCustomCounter)
                                                     .addTag(atomicCounter(5, 10)))
                     .build();

Step 2: Create a client and table resource

Instantiate the enhanced DynamoDB client, and create the table resource by giving it the previously defined schema:

DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.create();
TableSchema tableSchema = TableSchema.fromBean(Customer.class); // or TABLE_SCHEMA for static schema 
DynamoDbTable<Customer> customerTable = enhancedClient.table("customers_table", tableSchema);

Step 3: Call DynamoDB

Instantiate a Customer object and set the key attribute. Then, call updateItem and getItem to verify the automatic updates of the counter attributes:

Customer customer = Customer.builder().id("SOME_ID").build():

customerTable.updateItem(customer); //both putItem and updateItem can be used to add records
Customer retrievedCustomer = customerTable.getItem(customer)

retrievedCustomer.defaultCounter(); //the value is 0
retrievedCustomer.customCounter();  //the value is 10

customerTable.updateItem(customer); 

retrievedCustomer = customerTable.getItem(customer)
retrievedCustomer.defaultCounter(); //the value is 1
retrievedCustomer.customCounter();  //the value is 15

As you can see, we don’t reference the attributes in the record that’s sent to DynamoDB. However, the record is continually updated with values each time that you call the database. Manually setting a value on the record will cause DynamoDB to throw a DynamoDBException, saying “Invalid UpdateExpression: Two document paths overlap […]”, because the update expression with counters auto-generated by the AtomicCounterExtension will collide with the expression that was created for the record itself.

Creating a custom UpdateExpression extension

Write your own extension that takes advantage of the new option to provide a custom UpdateExpression in the extension framework.

Update expressions in the extensions are applicable for use cases where you want to do the same thing with an attribute every time that you call the database, such as atomic counters. However, if you need a one-time effect for a single request, then leveraging the extension framework isn’t useful. Before considering support for single-request update expressions, we’ll evaluate the usage of the new UpdateExpression API included in this release, as well as the feedback that we get.

Introduction to the enhanced client UpdateExpression API

An enhanced client UpdateExpression consists of one or more UpdateAction that correspond to the DynamoDB UpdateExpression syntax. Before sending an update request to DynamoDB, the enhanced client parses an UpdateExpression into a format that DynamoDB understands.

For example, you can create a RemoveAction that will remove the attribute with the name “attr1” from a record:

RemoveAction removeAction = 
    RemoveAction.builder()
                .path("#attr1_ref")
                .putExpressionName("#attr1_ref", "attr1")
                .build();

Note that, while usage of ExpressionNames is optional, we recommend it to avoid name collisions.

Step 1: Create an extension class

Create an extension class that implements the beforeWrite extension hook:

public final class CustomExtension implements DynamoDbEnhancedClientExtension {

    @Override
    public WriteModification beforeWrite(DynamoDbExtensionContext.BeforeWrite context) {
        return WriteModification.builder()
                                .updateExpression(createUpdateExpression())
                                .build();
    }
}

You can use the context object to retrieve information about the following:

  • The transformed item
  • Table metadata, such as custom tags
  • The name of the operation that is being invoked

Step 2: Create the UpdateExpression

In our extension, a SetAction changes the value of “attr2”, which we can infer is a String attribute:

private static UpdateExpression createUpdateExpression() {
    AttributeValue newValue = AttributeValue.builder().s("A new value").build();
    SetAction setAction = 
    SetAction.builder()
                .path("#attr1_ref")
                .value(":new_value")
                .putExpressionName("#attr1_ref", "attr1")
                .putExpressionValue(":new_value", newValue)
                .build();

    UpdateExpression.builder()
                    .addAction(setAction)
                    .build();
}

Step 3: Add the extension to a client

Add your custom extension to the client:

DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder()
                                                              .extensions(new CustomExtension()))
                                                              .build();

Step 4: Call DynamoDB

Retrieve a reference to the table and call updateItem:

DynamoDbTable<MyRecord> table = enhancedClient.table("tableName", tableSchema);
table.updateItem(new MyRecord());

Under the hood

It can be useful to understand how the extension framework and the UpdateItem operation work together to combine the output from the extensions with request-level item information to form a cohesive DynamoDB request:

  • If there are several extensions that add UpdateExpressions in the extension chain, then these are merged without any checks to form a single expression.
  • The item to be updated that is provided by the enhanced UpdateItemRequest is transformed by the operation to an internal UpdateExpression.
  • The internal UpdateExpression in the UpdateItem operation is merged with the one from the extension framework – if it exists.
  • DynamoDB allows for only one action to manipulate a single attribute. Therefore, any duplicate actions referencing the same attribute will fail locally in the client when the UpdateExpression is parsed into the low-level DynamoDB request.
  • The enhanced client generates remove statements for any attribute that isn’t explicitly set on an item supplied to the UpdateItem operation. Because this default behavior (controlled by the ignoreNulls flag) interferes with extension functionality, the client automatically filters those attributes by checking if they’re present in an extension UpdateExpression. If they are, then the client doesn’t create remove statements for them.

Conclusion

In this post, you’ve learned to use atomic counters in the enhanced DynamoDB client, and you’ve seen how extensions can work with the enhanced UpdateExpression API for custom applications. The enhanced client is open-source and resides in the same repository as the AWS SDK for Java 2.0.

We hope you found this post useful, and we look forward to your feedback. You can always share your input on our GitHub issues page, or up-vote other ideas for features that you want to see in the DynamoDB enhanced client or the AWS SDK for Java in general.

Anna-Karin Salander

Anna-Karin is a maintainer of AWS SDK for Java. She has a passion for writing maintainable software and infrastructure, as well as enjoying gardening, hiking and painting. You can find her on GitHub @cenedhryn.