AWS AppSync releases Pipeline Resolvers, Aurora Serverless support, Delta Sync

AWS AppSync, a Serverless GraphQL backend for providing data to mobile and web applications, has been steadily releasing features over the past year since launching at re:Invent 2017. Today, we’re happy to release several new service and client features for AWS AppSync that can greatly enhance the way you build client applications and provide data to them.

Pipeline resolvers – Enable execution of one or more operations against multiple data sources in order, on a single GraphQL field. This allows orchestration of actions by composing code into a single resolver, or sharing code across resolvers.

Delta Sync – Enables synchronization to the client cache of only database items that have changed since a client was last online, with built-in retry and connection management logic and updates to the offline cache. This reduces extra computations and overhead in your client devices by shifting the work into the backend data storage layers.

Amazon Aurora Serverless data source – A built-in resolver for executing GraphQL operations with the new Aurora Serverless Data API. It includes connection management functionality.

These new capabilities are available in any AWS AppSync API immediately.

Pipeline resolvers

If you’re not familiar with AWS AppSync or GraphQL, a “resolver” is essentially a function that’s responsible for fetching data from a location to fulfill a request. For instance, a resolver might query from a database to get a stored record, or it could just compute a value directly. Resolvers are attached to fields on a “type” in a GraphQL schema. These are executed at runtime, depending on the request that comes from a client.

AWS AppSync follows this notion by executing resolvers on a GraphQL field that’s defined in your schema. For instance, “getPosts” might execute a resolver to pull all blog posts from an Amazon DynamoDB table with a query or scan. In some cases, applications require executing multiple operations to resolve a single GraphQL field.

Using the previous example, you might want to have “getPost(user: String)” return all blog posts for a specific user, but only if you have permissions. These permissions could be defined in that same DynamoDB table. However, if your authorization logic is more complex, with rules in a separate table or system, this becomes difficult. Previously, this required attaching resolvers to nested fields, which affected the data model that was exposed through the schema.

Today’s release includes both “unit” and “pipeline” resolvers, which makes solving the previously mentioned problem much easier. Unit resolvers are the original way of attaching a request and response template directly to a data source. With pipeline resolvers, developers can now compose operations (called functions) and can execute one or more in sequence. A function is a first-class object in AWS AppSync that you can reuse across pipeline resolvers. This enables you to consolidate common logic in one place that’s applied consistently in your GraphQL execution.

Continuing with the previous example, you’re now able to have a pipeline resolver that runs a function for authorization logic, as a first step. This function could be a DynamoDB operation, Lambda function, or any AWS AppSync data source. After this function executes, you can then have another function that performs your actual data fetching for the GraphQL response. These results could then be passed to another function, which either transforms the data further, or performs additional operations against another data source.

Pipeline resolvers also include the concept of a “before” and “after” template. The template enables you to perform actions (such as data sanitization) before running the pipeline or right after completion. The new functionality also enables you to exit a function in the pipeline at any time with a “return” keyword. This enables you to perform custom branching workflows. You can also store arbitrary objects in $context.stash between function execution in a pipeline, to reuse results at different points in your business logic.

Delta Sync

Client applications in AWS AppSync store data by caching GraphQL responses locally to disk in a mobile or web application. For an example of this architecture for iOS, see Client Architecture. The process is the same for other platforms.

The Delta Sync feature gives customers the ability to specify two separate queries in this process—a “base” query and a “delta” query. This allows clients to hydrate their local cache with results from one base query that might have many records, and then receive only the data altered since their last query (the “delta updates”). This is substantially more efficient and less onerous on the clients when they regularly switch between online and offline states.

In addition, Delta Sync clients can also receive a subscription query as an argument. The client coordinates subscription reconnects and writes between offline to online transitions. Delta Sync performs this by automatically resuming subscriptions (including exponential backoff and retry with jitter through different network error scenarios), and storing events in a queue. The appropriate delta or base query is then run before merging any events from the queue, and finally processing subscriptions as normal.

Delta Sync is a client feature that can use any GraphQL query, but it works best in conjunction with pipeline resolvers. By allowing clients to separate the base hydration of the cache with one query and incremental updates in another query, you can move the computation from your client application to the backend. This enables you to have a base query run a scan on one table, and then a query with a conditional expression on the same table to pull changed events.

For larger workloads, you can partition the base query and delta query storage into different locations and then perform create, update, or delete operations with a pipeline resolver. This lets you shard your GraphQL responses with multiple AWS AppSync data sources—and have one function update your base table and another function update your delta table. When combined with Amazon DynamoDB TTLs, the net result is that the second table becomes a journal of changes that clients can query for their missed updates. If clients are offline for long periods of time and records have been purged from the delta table, the base query catch-up ensures that they converge to the one source of truth.

With pipeline resolvers, you can control how long items are kept in the second table. With the client API, you decide how often the base query catchup process runs. These offer you maximum control over the tombstone eviction process and client reconciliation. For some organizations, grooming the delta table might be a daily event or it might be more frequent. Similarly, you might want the base query to run more or less frequently than the 24-hour default.

This new Delta Sync feature works for iOS, Android, and JavaScript platforms. As an advanced feature, Delta Sync adds a bit of extra complexity to the development and configuration process. We recommend that you confirm that you need this level of fine-grained control before you implement it. To read more about this feature and use the built-in sample to get started quickly, see the client documentation on the Amplify website or the backend AppSync documentation.

Aurora Serverless data source

Before today, using relational databases with AWS AppSync was only possible with an AWS Lambda function resolver. We previously provided a template for the resolver in the console, which you can continue to use. However, you would still need to manage the connection between the function and the database at scale. With the release of the Data API for Aurora Serverless, you now have a first-class data source for executing GraphQL operations against a relational database in AWS AppSync.

Using this data source is similar to adding other AWS AppSync data sources, but now you can choose your cluster endpoint and can specify role connection details—as well as an ARN for AWS Secret Store, which has the SQL user name and password.

After you set up a data source, you can attach resolvers to GraphQL fields (including pipeline resolvers with functions) and execute a SQL statement in a response template like below:

{
  "version": "2018-05-29",
  "statements": [
    "select * from Pets WHERE  type = '$ctx.args.type'"
  ]
}

Then, resulting data from the SQL statement is available in your response template. AWS AppSync provides helper utilities for the response template for easy formatting into a GraphQL response:

$utils.toJson($utils.parseJson($utils.rds.toJsonString($ctx.result))[0])

GraphQL mutations operate slightly differently based on the semantics of SQL databases. Request templates for writes can optionally contain an extra SQL statement to perform the read after writing a record—which you can use to populate the selection set of the GraphQL response. For more information, see the AppSync documentation.

Next steps

Take some time to look at the examples and tutorials provided in the documentation, as these are advanced features that require a bit of understanding. We’re excited to see how you use these new capabilities for new GraphQL use cases—as well as how you use them to provide data to mobile and web applications in unique ways. The AWS AppSync forums are a great place to go for help if you have any questions around customization or general usage. Please reach out to us with any feedback.