Megabytes in milliseconds: How FireTV uses parallel queries and vertical partitioning to serve millions of customers in Amazon DynamoDB

This post describes how FireTV redesigned their DynamoDB data model for Continue Watching to eliminate item size constraints, reduce write costs by 97 percent, and maintain single-digit-millisecond average reads (sub-50 ms at p99) regardless of profile size.

If you’ve used a FireTV, you’ve almost certainly used the Continue Watching row, even if you’ve never given it much thought. Resuming a movie or episode exactly where you left off is a core function that everyone expects to be available, fast, reliable, and work across devices. Continue Watching is powered by a watch progress system that uses Amazon DynamoDB to store every play, pause, and resume event, which adds up to billions of records across millions of customer profiles.

We need to retrieve a customer profile’s complete watch history (every item in the partition key’s collection) on every home screen load in single-digit milliseconds. For some customers, this watch history can be quite large. Supporting these large customers and working with large items was a primary motivation in the modernization of our DynamoDB data model.

This post describes how we redesigned our schema using vertical partitioning combined with hash-prefixed sort keys to run parallel segmented reads. This approach supported profiles of any size, reduced write costs by 97 percent, and maintained single-digit millisecond average read latency regardless of profile size.

In its initial design, FireTV stored each customer’s watch progress as a single compressed item in Amazon DynamoDB. We chose this design because it aligned well with the requirement to fetch entire customer profiles and was fast to read. Our retention period was also short enough at the time that item size was not expected to become a problem. As the service evolved and customer profiles grew, three problems related to these large items became apparent.

First, DynamoDB has an item size limit of 400 KB. While most customers were not yet at this limit, an upcoming requirement to enrich each record with additional identifiers for content catalog changes would effectively double customer profile sizes. With this doubling, many existing customers would be over the 400 KB limit. We needed a schema that could scale without limits while maintaining the same read performance.

Second, throttling became more likely. DynamoDB scales horizontally by adding additional partitions to a table, each responsible for a portion of the total items in the table. This allows a DynamoDB table to scale to handle millions of requests per second. Each individual partition, however, provides at most 1,000 write capacity units (WCUs) and 3,000 read capacity units (RCUs) of throughput. As items get very large, even a relatively modest update rate of three times per second can consume all the write capacity of a partition. Customers with large watch histories are often the most likely customers to have high update rates, making throttling more likely.

Finally, write cost began to increase. Writes to DynamoDB are priced by the WCU, which represents one write per second for an item up to 1 KB in size. Updating a 1 KB item consumes 1 WCU, and updating a 100 KB item consumes 100 WCU. As our profile sizes increased, so did our write costs to update them.

Vertical partitioning

Upon identifying the item size constraint, we scheduled a conversation with a DynamoDB Solutions Architect to discuss potential design changes. In that conversation we learned that Solutions Architects commonly address item size concerns, particularly when they work with customers migrating from database engines with different limits. Most customers solve these concerns with two design patterns.

First, customers should implement compression on any large non-indexed attributes. This can significantly reduce the size of items, lowering costs and keeping them below 400 KB. Using an algorithm optimized for speed (such as LZ4) can also improve performance. Unfortunately for us, we were already compressing profiles.

Second, customers should consider a data model which uses vertical partitioning. Vertical partitioning is a design which breaks individual large items into a collection of smaller items which together represent the whole profile. For example, a single item design may have a list attribute which contains watch data for five different movies.

Figure 1: Single-item design, where each customer profile is one compressed item that grows toward the 400 KB limit

A vertically partitioned data model might break that item into six separate items. One item to represent the profile metadata, and 5 individual items each containing watch data for a single movie. One goal behind a design such as this is to keep updates as small and efficient as possible. Separating out larger, infrequently updated elements such as profile metadata into their own items keeps updates to frequently updated elements such as watch position small and efficient.

Figure 2: Vertically partitioned design, with one metadata item and one small item per content entry

A model like this solves the challenges around large items. Write costs are reduced as updates now target small individual items instead of the entire profile. These smaller writes consume much less partition capacity, reducing the chance of throttling and allowing DynamoDB to split hot partitions to add more capacity. Item collections have no size limit (assuming a table doesn’t use a local secondary index), eliminating the 400 KB limit.

In addition, vertical partitioning would allow us to prune old watch data using the intrinsic time-to-live (TTL) capabilities of DynamoDB rather than alternative solutions.

The 1 MB page limit

While vertical partitioning was a good fit, our service has some unique requirements that required us to evolve the approach. With a new model uncapping profile size, coupled with service changes increasing the volume of data stored in a profile, we expected some customers’ profiles to grow quite large. While many other DynamoDB use cases only need to query a portion of their vertically partitioned item collections, we need the whole profile. These large profiles would begin pushing up against a different limit: the 1 MB page size for queries.

While this was of course a much less impactful limit, it did mean that large profiles would require multiple sequential requests to DynamoDB. Each request returns 1 MB of data and the LastEvaluatedKey where the next query should begin. Large profiles would take significantly longer to query, breaking our service’s single-digit millisecond target for those customers.

To keep performance consistent, we needed a way to parallelize reads.

Parallel reads with hash-prefixed sort keys

Suppose we could divide the items for a partition key into smaller, exclusive segments. If joining them after parallel retrieval produces the same result as a full partition read, then profile size stops mattering. And if each segment is small enough to fit within a single 1 MB response page, pagination never kicks in. Every query returns in one round trip. The total read time becomes the latency of the slowest single segment, not the sum of sequential pages.

The question is how to define those segments in a way that DynamoDB can execute efficiently, and in a way that both writers and readers can agree on without coordination. When a write arrives, it must land in a predictable segment. When a read arrives, it must know exactly which segments to query without first inspecting the data. The assignment has to be deterministic: given a content ID, both sides independently compute the same segment every time.

DynamoDB gives us the building block for this. Items within a partition key are stored in lexicographic order by their sort key. The items form a sorted sequence with well-defined boundaries. If we can control where an item lands in that sequence, we can pre-define non-overlapping ranges and query each one independently. The writer places the item in the right position. The reader queries the right range. No coordination needed.

The problem is that this only works if items are distributed roughly evenly across the segments. Content IDs assigned by streaming partners tend to share prefixes within a provider. A customer who watches mostly from one service will have sort keys concentrated in the same region of the lexicographic space. Split that space into four segments and one segment might contain 70 percent of the items while another contains almost nothing. The parallel queries return at wildly different times, and the slowest one still exceeds our latency target.

We needed a way to spread items uniformly across the sort key space regardless of what the underlying content IDs look like. The solution is to prepend each sort key with a hash of the content ID. We use the first eight hexadecimal characters of a MurmurHash3, giving us a sort key structure of {hash}#{contentId}. Because hash functions distribute inputs evenly across their output range, items now land uniformly between 00000000 and ffffffff in the sort key space. The segment boundaries become fixed hex ranges. Every segment holds roughly the same number of items.

On write, when a watch event arrives for a content ID, we compute its hash and construct the sort key. The item lands in its segment automatically. There’s no bookkeeping, no index to update, no coordination with any other component. A single PutItem of roughly 600 bytes, consuming 1 WCU.

On read, when we need the full profile, we know the segment boundaries upfront. For four segments, we issue four parallel queries: one covering 00000000 to 3fffffff, the next 40000000 to 7fffffff, and so on. Each query returns its portion of the collection, and we merge the results client-side. The total latency is the time of the slowest segment, not the sum of all pages.

Figure 3: The sort key space divided into fixed segments, with independent write and read paths

The number of segments is tunable. We chose our initial segment count based on the data size of our largest customer profiles and the 1 MB page limit. After launch, we monitor for pagination events and overall profile size growth, then adjust the segment count up or down without any schema changes. The sort key space remains the same. Only the boundaries we query against change. Going from four segments to eight is a client-side configuration change, not a migration.

Figure 4: Four parallel queries over fixed hex ranges, merged client-side into the complete profile

This design removed the constraints that we started with. The 400 KB item size limit no longer applies, item collections have no size cap, and we now support profiles with tens of thousands of entries. Write operations dropped from 50 WCUs per update (linearly increasing) to 1 WCU (constant), removing the read-modify-write cycle entirely.

Read throughput did increase. The old design stored a single compressed blob per profile, so a 50 KB compressed item might represent 200 KB of raw data. In the flat schema, each 600-byte item is stored uncompressed because compression yields negligible savings at that size. A pre-computed dictionary would be an option to allow compression on small items to be more efficient, but wasn’t deemed necessary for our case. The total bytes read per profile are higher as a result. In practice, the latency improvement from parallel reads more than compensates. Total read time averages 8 milliseconds regardless of profile size, retrieving over a megabyte of customer data in under 50 milliseconds at the 99th percentile.

The technique we used can be applied to multiple use cases. While we applied it to watch progress data, the same pattern works for any workload that needs full-collection retrieval with predictable latency. A few design principles are worth calling out for anyone considering this approach.

Keys design tradeoffs

Optimizing for full-collection retrieval with hash-prefixed sort keys comes at a cost: you lose efficient fine-grained access patterns within the partition. With a plain vertically partitioned schema, if a client needed watch progress for a specific provider, we could query by a predictable sort key prefix. With hashed sort keys, items for the same provider are scattered uniformly across the hex range. A query like “get all Partner1 entries for this customer” now has two options: retrieve the entire profile and filter in memory, or maintain a secondary index that supports the narrower access pattern.

For our workload, this tradeoff was acceptable. The primary access pattern (full profile retrieval on every home screen load) accounts for the vast majority of traffic. The few use cases that need per-provider or per-content lookups either filter client-side from the already-fetched full profile, or use a global secondary index (GSI). But this is a fundamental constraint of the pattern: if your workload requires both full-collection retrieval and frequent fine-grained queries on the same table, you will need an index to support the latter.

The partition key choice matters too. A broader partition key like customerId means a single query returns all data across child entities (profiles, sessions, and more) without a secondary index, which simplifies account-level operations. A narrower partition key like customerId_profileId gives each child entity dedicated partition throughput but requires multiple sets of parallel queries to aggregate all data or a GSI structured to support the aggregate access pattern. The decision comes down to whether the overhead justifies the narrower queries and throughput isolation, or whether the simpler single-partition-key model fits your access patterns.

Choosing your segment count

To find the right count, work backward from two constraints.

First, each segment must return under 1 MB to avoid pagination. Divide your target max profile size by 1 MB and round up. That’s your minimum segment count.

Second, all segments for a profile are generally read from the same partition. DynamoDB split for heat isolates frequently accessed partition keys into their own partitions and can even split further along between sort keys. These splits are reactive and aren’t possible in every case (see Scaling DynamoDB: How partitions, hot keys, and split for heat impact performance for an in depth discussion on split for heat). Since splits are reactive, it’s a good idea to consider the throughput of an individual unsplit partition. This isolated partition supports approximately 24 MB/s of read throughput (3,000 RCU per partition × 8 KB per eventually-consistent read ≈ 24 MB/s). Your parallel queries share that budget. This puts a practical ceiling on how many segments you can query concurrently before you risk throttling the partition itself and depending on split for heat for additional capacity.

Between these two bounds, cost helps narrow the choice. An RCU represents a 4 KB read, and the total cost of a query is the aggregated data returned rounded up to the nearest 4 KB boundary. For example, a 1 KB read and a 3 KB read both cost 1 RCU, while an 11 KB read would cost 3 RCU. Splitting that data across multiple queries means less aggregation, and more rounding. Splitting an 11 KB read across 4 queries means each query consumes 1 RCU, or 4 RCU in total. That is a slight increase over a single-query read. The increase is more substantial for the smaller reads, however. A 1 KB or 3 KB read split across 4 queries now consumes 4 RCU as well. Pick the fewest segments that keep your largest profiles under 1 MB per segment, and the economics take care of themselves.

Migration and validation

Migrating between two schemas with different data models is not a cutover you can do in one shot. A single compressed item per profile and one item per content entry are structurally different representations of the same data. You need to prove they produce equivalent results before switching production reads.

We ran both tables in parallel. Every new write went to both the legacy and new table simultaneously, ensuring incoming events landed in both from that point forward. We then backfilled historical data using an offline job that exported the legacy table, decomposed each compressed profile into individual items, and wrote them to the new table. After the backfill completed, both tables held the same logical dataset in their respective formats. Only then did we introduce shadow reads: production traffic continued reading from the legacy table, but a configurable percentage of requests also read from the new table in the background. We compared the results without exposing the new table’s responses to customers.

To validate at full scale, we ran an Amazon EMR job that joined both tables across billions of matched records. For each (customer profile, content ID) pair, we compared event timestamps, playback positions, and provider metadata. We defined a tolerance threshold of 10 seconds for timestamp drift, since events processed milliseconds apart through different write paths can legitimately differ by small amounts. The results gave us confidence: 99.45 percent of matched records were identical within tolerance. Of the 0.55 percent showing drift, over 95 percent followed expected patterns where the direction of timestamp drift correlated with the direction of playback position change.

With data quality validated, we dialed up production reads from the new table gradually using a feature flag, monitoring API latency, error rates, and downstream metrics (like the number of tiles in the Continue Watching row) at each step.

Conclusion

While on the surface the 400 KB item size limit of DynamoDB is a constraint, it also encourages more efficient designs that optimize for smaller writes. For most workloads, the vertical partitioning design pattern (one item per entity in an item collection) fulfills this requirement. It allows entities of any size and shifts large single-item writes into a collection of smaller, more efficient ones. For workloads that demand complete collection retrieval at consistent latency regardless of data size, hash-prefixed sort keys transform sequential pagination into parallel range queries. This provides the same small-item write optimization along with optimized read performance.

We started with a single compressed item per customer that was approaching its size limit, costing 50 WCUs per write, and heading toward a wall. We ended with a flat schema that writes at 1 WCU, scales without limit, and reads in single-digit milliseconds on average whether a profile has 100 entries or 30,000.

If you’re hitting similar constraints, start by reading Use vertical partitioning to scale data efficiently in Amazon DynamoDB. If your access pattern requires the full collection and pagination becomes your bottleneck, the hash-prefix technique described here is the next step. For workloads where the problem is multi-dimensional filtering rather than full-collection retrieval, Z-order indexing applies the same core insight (computed sort keys over lexicographic ranges) to a different access pattern.

About the authors