Tag: Transaction


Performing Conditional Writes Using the Amazon DynamoDB Transaction Library

by Wade Matveyenko | on | in Java | Permalink | Comments |  Share

Today we’re lucky to have another guest post by David Yanacek from the Amazon DynamoDB team. David is sharing his deep knowledge on the Amazon DynamoDB Transactions library to help explain how to use it with the conditional writes feature of Amazon DynamoDB.


The DynamoDB transaction library provides a convenient way to perform atomic reads and writes across multiple DynamoDB items and tables. The library does all of the nuanced item locking, commits, applies, and rollbacks for you, so that you don’t have to worry about building your own state machines or other schemes to make sure that writes eventually happen across multiple items. In this post, we demonstrate how to use the read-modify-write pattern with the transaction library to accomplish the same atomic checks you were used to getting by using conditional writes with the vanilla DynamoDB API.

The transaction library exposes as much of the low-level Java API as possible, but it does not support conditional writes out of the box. Conditional writes are a way of asking DynamoDB to perform a write operation like PutItem, UpdateItem, or DeleteItem, but only if certain attributes of the item still have the values that you expect, right before the write goes through. Instead of exposing conditional writes directly, the transaction library enables the read-modify-write pattern—just like the pattern you’re used to with transactions in an RDBMS. The idea is to start a transaction, read items using that transaction, validate that those items contain the values you expect to start with, write your changes using that same transaction, and then commit the transaction.  If the commit() call succeeds, it means that the changes were written atomically, and none of the items in the transaction were modified by any other transaction in the meantime, starting from the time when each item was read by your transaction.

Transaction library recap

Let’s say you’re implementing a tic-tac-toe game. You have an Item in a DynamoDB table representing a single match of the game, with an attribute for each position in the board (Top-Left, Bottom-Right, etc.). Also, to make this into a multi-item transaction, let’s add two more items—one per player in the game, each with an attribute saying whether it is currently that player’s turn or not. The items might look something like this:

Games Table Item Users Table Items
{
  " GameId": "cf3df",
  "Turn": "Bob",
  "Top-Right": "O"
}
{
  " UserId": "Alice",
  "IsMyTurn": 0
}
{
  " UserId": "Bob",
  "IsMyTurn": 1
}

Now when Bob plays his turn in the game, all three items need to be updated:

  1. The Bob record needs to be marked as "Not my turn anymore."
  2. The Alice record needs to be marked as "It’s my turn now."
  3. The Game record needs to be marked as "It’s Alice’s turn, and also the Top-Left has an X in it."

If you write your application so that it performs three UpdateItem operations in a row, a few problems could occur. For example, your application could crash after doing one of the writes, and now something else in your application would need to notice this and pick up where it left off before doing anything else in the game. Fortunately, the transaction library can make these three separate operations happen together in a transaction, where either all of the writes go through together, or if there is another transaction overlapping with yours at the same time, only one of those transactions happens.

The code for doing this in a transaction looks like this:

// Start a new transaction
Transaction t = txManager.newTransaction();
 
// Update Alice's record to let him know that it is now her turn.
t.updateItem(
  new UpdateItemRequest()
    .withTableName("Users")
    .addKeyEntry("UserId", new AttributeValue("Alice"))
    .addAttributeUpdatesEntry("IsMyTurn",
            new AttributeValueUpdate(new AttributeValue("1"), AttributeAction.PUT)));
 
// Update Bob's record to let him know that it is not his turn anymore.
t.updateItem(
  new UpdateItemRequest()
    .withTableName("Users")
    .addKeyEntry("UserId", new AttributeValue("Bob"))
    .addAttributeUpdatesEntry("IsMyTurn",
            new AttributeValueUpdate(new AttributeValue("0"), AttributeAction.PUT)));
 
// Update the Game item to mark the spot that was played, and make it Alice's turn now.
t.updateItem(
  new UpdateItemRequest()
    .withTableName("Games")
    .addKeyEntry("GameId", new AttributeValue("cf3df"))
    .addAttributeUpdatesEntry("Top-Left", 
            new AttributeValueUpdate(new AttributeValue("X"), AttributeAction.PUT))
    .addAttributeUpdatesEntry("Turn",
            new AttributeValueUpdate(new AttributeValue("Alice"), AttributeAction.PUT)));
 
// If no exceptions are thrown by this line, it means that the transaction was committed.
t.commit();

What about conditional writes?

The preceding code makes sure that the writes go through atomically, but that’s not enough logic for making a move in the game. We need to make sure that, when the transaction goes through, there wasn’t a transaction right before it where Bob already played his turn. In other words, how do we make sure that Bob doesn’t play twice in a row—for example, by trying to sneak in two turns before Alice has a chance to move? If there was only a single item involved, say the "Games" item, we could accomplish this by using conditional writes (the Expected clause), like so:

// An example of a conditional update using the DynamoDB client (not the transaction library)
dynamodb.updateItem(
  new UpdateItemRequest()
    .withTableName("Games")
    .addKeyEntry("GameId", new AttributeValue("cf3df"))
    .addAttributeUpdatesEntry("Top-Left", 
    		new AttributeValueUpdate(new AttributeValue("X"), AttributeAction.PUT))
    .addAttributeUpdatesEntry("Turn",
    		new AttributeValueUpdate(new AttributeValue("Alice"), AttributeAction.PUT))
    .addExpectedEntry("Turn", new ExpectedAttributeValue(new AttributeValue("Bob"))) // A condition to ensure it's still Bob's turn
    .addExpectedEntry("Top-Left", new ExpectedAttributeValue(false)));               // A condition to ensure the Top-Left hasn't been played

This code now correctly updates the single Game item. However, conditional writes in DynamoDB can only refer to the single item the operation is updating, and our transaction contains three items that need to be updated together, only if the Game is still in the right state. Therefore, we need some way of mixing the original transaction code with these “conditional check” semantics.

Conditional writes with the transaction library

We started off with code for a transaction that coordinated the writes to all three items atomically, but it didn’t ensure that it was still Bob’s turn when it played Bob’s move. Fortunately, adding that check is easy: it’s simply a matter of adding a read to the transaction, and then performing the verification on the client-side. This is sometimes referred to as a "read-modify-write" pattern:

// Start a new transaction, just like before.
Transaction t = txManager.newTransaction();
 
// First, read the Game item.
Map game = t.getItem(
    new GetItemRequest()
        .withTableName("Games")
        .addKeyEntry("GameId", new AttributeValue("cf3df"))).getItem();
 
// Now check the Game item to ensure it's in the state you expect, and bail out if it's not.
// These checks serve as the "expected" clause.  
if (! "Bob".equals(game.get("Turn").getS())) {
    t.rollback();
    throw new ConditionalCheckFailedException("Bob can only play when it's Bob's turn!");
}
 
if (game.containsKey("Top-Left")) {
    t.rollback();
    throw new ConditionalCheckFailedException("Bob cannot play in the Top-Left because it has already been played.");
}
 
// Again, update Alice's record to let her know that it is now her turn.
t.updateItem(
    new UpdateItemRequest()
        .withTableName("Users")
        .addKeyEntry("UserId", new AttributeValue("Alice"))
        .addAttributeUpdatesEntry("IsMyTurn",
            new AttributeValueUpdate(new AttributeValue("1"), AttributeAction.PUT)));
 
// And again, update Bob's record to let him know that it is not his turn anymore.
t.updateItem(
    new UpdateItemRequest()
        .withTableName("Users")
        .addKeyEntry("UserId", new AttributeValue("Bob"))
        .addAttributeUpdatesEntry("IsMyTurn",
            new AttributeValueUpdate(new AttributeValue("0"), AttributeAction.PUT)));
 
// Finally, update the Game item to mark the spot that was played and make it Alice's turn now.
t.updateItem(
    new UpdateItemRequest()
        .withTableName("Games")
        .addKeyEntry("GameId", new AttributeValue("cf3df"))
        .addAttributeUpdatesEntry("Top-Left", 
            new AttributeValueUpdate(new AttributeValue("X"), AttributeAction.PUT))
        .addAttributeUpdatesEntry("Turn",
            new AttributeValueUpdate(new AttributeValue("Alice"), AttributeAction.PUT)));
 
// If no exceptions are thrown by this line, it means that the transaction was committed without interference from any other transactions.
try {
    t.commit();
} catch (TransactionRolledBackException e) {
    // If any of the items in the transaction were changed or read in the meantime by a different transaction, then this will be thrown.
    throw new RuntimeException("The game was changed while this transaction was happening. You probably want to refresh Bob's view of the game.", e);
}

There are two main differences with the first approach.

  • First, the code calls GetItem on the transaction and checks to make sure the item is in the state your application expects it to be in. If not, it rolls back the transaction and returns an error to the caller. This is done in the same transaction as the subsequent updates. When you read an item in a transaction, the transaction library locks the item in the same way as when you modify it in the transaction. Your application can still read an item without interfering with it while it is locked, but it must do so outside of a transaction, using one of the read isolation levels on the TransactionManager. More about read isolation levels is available in the design document for the transaction library.
  • Next, the code checks for TransactionRolledBackException. This check could have been done in the first example as well, but it’s called out in this example to show what will happen if another transaction either reads or writes any of the items involved in the transaction while yours was going on. When this happens, you might want to retry the whole transaction (start from the beginning—don’t skip any steps), or refresh your client’s view so that they can re-evaluate their move, since the state of the game may have changed.

While the preceding code doesn’t literally use the conditional writes API in DynamoDB (through the Expected parameter), it functionally does the same atomic validation—except with the added capability of performing that check and write atomically across multiple items.

More info

You can find the DynamoDB transaction library in the AWS Labs repository on GitHub. You’ll also find a more detailed write-up describing the algorithms it uses. You can find more usage information about the transaction library in the blog post that announced the library. And if you want to see some working code that uses transactions, check out TransactionExamples.java in the same repo.

For a recap on conditional writes, see part of a talk called Amazon DynamoDB Design Patterns for Ultra-High Performance Apps from the 2013 AWS re: Invent conference. You may find the rest of the talk useful as well, but the segment on conditional writes is only five minutes long.