Build resilient applications with Amazon DynamoDB global tables: Part 2

In the first post of this series, you learned about the differences between zonal, Regional, and global services, and how they affect theoretical application availability. In this post, you’ll learn more about some important Amazon DynamoDB characteristics and how they impact multi-Region design.

Properties of DynamoDB tables in a single Region

DynamoDB is a NoSQL database that provides durable writes and the option of eventually consistent or strongly consistent reads in a single AWS Region. When you write a new item into a DynamoDB table and receive an HTTP 200 (okay) response back, the new item is written and is durable. The new item might not be immediately available in all Availability Zones because DynamoDB uses a two-thirds quorum rule during write operations. However, when you retrieve an item, you can request a strongly consistent read, which reflects all previous writes.

DynamoDB doesn’t provide the same type of schema enforcement or SQL support as a relational database. However, it does offer conditional writes and transactions. Conditional writes let you implement logical checks to determine whether an item should be modified during an update. For example, you can decrement an inventory count attribute of a product on the condition that the existing inventory count is more than zero. If that check fails, you can’t sell the item because it’s out of stock. Transactions let you read or modify multiple items in one or more tables concurrently, with the usual ACID properties.

For example, in the order processing schema in Figure 1 that follows, you create an order and mark the associated product as sold in a single transaction. The basic schema shown in Figure 1 is based on the DynamoDB transactions example. It includes three tables—one for customers, another for orders, and a third for the product catalog. The partition keys are customer ID, order ID, and product ID, respectively. When creating an order, you confirm that the customer ID is valid and that the item is in stock, mark an item as sold, and finally create the order. When reading an order, you read the order from the order table and the related product details from the product catalog table.

Figure 1: Order processing schema

Properties of DynamoDB global tables

The persistence tier is a challenging part of a multi-Region architecture. Handling replication of databases is difficult and requires a deep understanding of the consistency and availability trade-offs. Coordination of writes between Regions where inter-Region latency is hundreds of milliseconds is a much different task than coordinating writes into multiple storage locations in a single Region.

That’s why DynamoDB offers global tables, which provide asynchronous replication with approximately 1-second replication latency for tables between two or more Regions. When using global tables, you should understand how the design choices that global tables make will impact your application. There are five important characteristics to review:

• Replication latency – The replication latency is around 1 second. From the perspective of a database user in another Region, it’s possible to read stale data. That’s different from working in a single Region where you can request a strongly consistent read.
• Conflict resolution – If two writes occur to the same item from different Regions, DynamoDB reconciles that conflict in favor of the last writer. This conflict resolution happens at the item level, not at the attribute level. For example, say a user updates the attribute foo in an item identified by partition key p1 at time t₁, and this write happens in the Region us-east-1. Another user updates the attribute bar in the same item at time t₂, and this write happens in the Region us-west-2. Time t₂ is later than but very close to time t₁. After both writes occur, the item will have the new value for attribute bar but the original value for attribute foo. If you normally use DynamoDB conditional checks to protect against conflicting writes, you have to consider that the data that the conditional expression is evaluating might be out of date or has updates pending replication from another Region.
• Transactions – Transactions aren’t supported across Regions in global tables. From the perspective of a database user in another Region, it’s possible to read inconsistent data from a transaction. Consider a transaction from the order processing sample application that updates items in the ProductCatalog and Order tables in one transaction. A database user in another Region could read the original value of the ProductCatalog item and the new value of the Order item if the read operation happens after the Order item is replicated but before the ProductCatalog item is replicated.
• Observability – Global tables version 2019.11.21 doesn’t provide a metric showing how many items have not yet been replicated, or a direct way to know if a specific item was replicated or not. It does provide a metric showing replication latency, which is the average time for an item to be successfully replicated from one Region to another. If you need to measure how many items have not yet been replicated or observe which specific items are still pending replication, you need to implement your own custom metrics.
• Recovery after degradation – If a global table replica in one Region is degraded, DynamoDB keeps track of writes made to that Region that haven’t yet been replicated. When the replica is operating normally again, DynamoDB resumes replicating writes from the replica to replicas in other Regions, and resumes replicating writes from other Regions to the impacted replica. If you see Regional degradation of one of your global tables replicas, you should consider whether you need to evacuate that Region until the replica is fully recovered and in sync with other Regions.

Although there are workarounds for some of the issues discussed here, they’re difficult to generalize. If your application is a write-once, read-many application where new items always have unique IDs, you’ll find global tables can be set up with minimal effort. An example of such an application is a clickstream processing system, where each new event gets a unique partition key in a DynamoDB global table. Your frontend system can use global tables to record clickstream events in the global table replicas in individual Regions, and the stream processing code can have access to the set of clickstream records from all Regions.

If your application doesn’t fit comfortably into the boundaries of global tables, you need to carefully weigh the trade-offs involved in writing complex application code to work around issues like potentially stale reads. These decisions must be made on a case-by-case basis according to your business requirements.

Conclusion

In this post, you learned how DynamoDB behaves in a single Region and in with global tables a multi-Region situation. You learned about some of the support that DynamoDB provides in a single-Region context, like strongly consistent reads, that aren’t available with global tables when working across Regions. In Part 3, you’ll review a design pattern for building a multi-Region application using global tables.

Special thanks to Todd Moore, Parker Bradshaw and Kurt Tometich who contributed to this post.

About the author

Randy DeFauw is a Senior Principal Solutions Architect at AWS. He holds an MSEE from the University of Michigan, where he worked on computer vision for autonomous vehicles. He also holds an MBA from Colorado State University. Randy has held a variety of positions in the technology space, ranging from software engineering to product management. In entered the Big Data space in 2013 and continues to explore that area. He is actively working on projects in the ML space and has presented at numerous conferences including Strata and GlueCon.