DynamoDB Document Model Manual Pagination

In version 3.1.1.2 of the DynamoDB .NET SDK package, we added pagination support to the Document Model. This feature allows you to use a pagination token returned by the API to paginate a set of Query or Scan results across sessions. Until now, it was not possible to resume pagination of Query or Scan results without retrieving the already encountered items. This post includes two simple examples of this new functionality.

The first example makes the initial Query call to retrieve all movies from the year 2012, and then saves the pagination token returned by the Search.PaginationToken property. The second example retrieves the PaginationToken and continues the same query. For these examples, we will assume we have functions (void SaveToken(string token) and string LoadToken()) to persist the pagination token across sessions. (If this were an ASP.NET application, the functions would use the session store to store the token, but this can be any similar environment where manual pagination is used.)

Initial query:

var client = new AmazonDynamoDBClient();
Table moviesTable = Table.LoadTable(client, "MoviesByYear");

// start initial query
var search = moviesTable.Query(new QueryOperationConfig
{
    Filter = new QueryFilter("Year", QueryOperator.Equal, 2012),
});

// retrieve one pages of items
List<Document> items = search.GetNextSet();

// get pagination token
string token = search.PaginationToken;

// persist the token in session data or something similar
SaveToken(token);

Resumed query:

var client = new AmazonDynamoDBClient();
Table moviesTable = Table.LoadTable(client, "MoviesByYear");

// load persisted token
string token = LoadToken();

// use token to resume query from last position
var search = moviesTable.Query(new QueryOperationConfig
{
    Filter = new QueryFilter("Year", QueryOperator.Equal, 2012),
    PaginationToken = token,
});
List<Document> items = search.GetNextSet();

// pagination token changed, persist new value
SaveToken(search.PaginationToken);

DataModel support

Although this functionality has not yet been added to the Object Persistence Model, it is possible to work around this limitation. In the following code sample, we can use the DocumentModel API to manually paginate our data, and then use DynamoDBContext to convert the retrieved Documents into .NET objects. Because we are using DynamoDBContext and don’t want to stray too far into the Document Model API, we’re going to use DynamoDBContext.GetTargetTable to avoid the manual construction of our Table instance.

// create DynamoDBContext object
var context = new DynamoDBContext(client);

// get the target table from the context
var moviesTable = context.GetTargetTable<Movie>();

// use token to resume query from last position
var search = moviesTable.Query(new QueryOperationConfig
{
    Filter = new QueryFilter("Year", QueryOperator.Equal, 2012),
    PaginationToken = token,
});
List<Document> items = search.GetNextSet();

// pagination token changed, persist new value
SaveToken(search.PaginationToken);

// convert page of Documents in .NET objects and enumerate over them
IEnumerable<Movie> movies = context.FromDocuments<Movie>(items);
foreach (var movie in movies)
    Log("{0} ({1})", movie.Title, movie.Year);

As you can see, even though we executed our Query using a Table object, we can continue working with familiar .NET classes while controlling the pagination of our data.