Invoking AWS Step Functions short- and long-running workflows from AWS AppSync

AWS AppSync is a fully managed GraphQL service allowing developers to easily build GraphQL APIs. AWS AppSync lets developers connect their API various data sources, such as Amazon DynamoDB, AWS Lambda, Amazon OpenSearch, and HTTP Data Sources. By utilizing HTTP Data Sources, your AWS AppSync API can be connected to other AWS services.

You can invoke AWS Step Functions workflows from AWS AppSync API by adding it as an HTTP data source. Note that AWS AppSync supports Pipeline resolvers utilized for executing functions in sequence and allowing orchestrating requests to multiples data sources. Some common Pipeline resolver use cases are authorization checks before fetching a field, checking server side data before inserting into the database, conducting nested queries, and integrating with multiple data sources.

Step Functions is a serverless orchestration service allowing you to interact with AWS services by implementing long-running or short-running workflows. Step function workflows manage failures, retries, parallelization, AWS service integrations, and observability.

There are two types of Step Functions Express Workflows: Asynchronous Express Workflows and Synchronous Express Workflows. Synchronous Express Workflows return results immediately after completion, and they can be useful for short-running tasks. Asynchronous Express Workflows are useful for long-running tasks because once the workflow has been kicked they return the execution details in the response. Then, this can be utilized to poll the workflow status. This post explains how Synchronous and Asynchronous Express Workflows can be integrated with AWS AppSync.

Moreover, these integrations are explained with the help of an Airline Booking System that offers functionalities such as search flights, book flights, and generate boarding pass. The basic skeleton GraphQL schema for this Airline Booking System is shown below:

type Flight {
    flightId: ID!
    departureTime: AWSDateTime!
    arrivalTime: AWSDateTime!
    departureAirportCode: String!
    arrivalAirportCode: String!
    price: Int!
    flightNumber: Int!
    seatAllocation: Int
    seatCapacity: Int!
}

type Booking {
    transactionId: ID!
    bookingId: ID!
    status: Status!
    flight: Flight
    paymentDetails: String
    checkedIn: Boolean
    customer: String
    createdAt: String
    message: String
}

enum Status {
    PENDING
    CONFIRMED
    FAILED
}

Short-running workflows

Synchronous express workflows are utilized for short-running tasks. Once the request is received, the workflow starts executing, waits for completion, and then the result is immediately returned back without having to poll for it. Note that Synchronous Express Workflows can run for as long as five minutes. In AWS AppSync, GraphQL request execution timeout is set to 30 seconds. Therefore, this pattern can apply to workflows requiring parallelization, handing failures, and retries, and it can return results within 30 seconds.

Synchronous Express Workflow can be invoked from AWS AppSync API by adding it as an HTTP data source and making a StartSyncExecution API call from the resolver. For AWS to identify and authorize HTTP requests, they must be signed with the Signature Version 4 process. AWS AppSync signs the request on your behalf, which is based on the IAM role that you provide as part of the HTTP data source configuration.

For our Airline Booking System, generating the boarding pass can be completed in a short duration and involves multiple steps. AWS AppSync API will receive the booking ID and customer in the request and invoke the state machine. The state machine will utilize these values to generate a boarding pass and send the result back.

We can add the below mutation for generating a boarding pass to the above schema:

type BoardingPass {
    bookingId: ID
    customer: String
    boardingPassLink: AWSURL
    message: String
}

input generateBoardingPassInput {
    bookingId: ID!
    customer: String!
}

type Mutation {
    generateBoardingPass(input: generateBoardingPassInput!): BoardingPass
}

A Synchronous Express Workflow can be useful in this scenario. Based on the booking ID and customer details received in the request, the state machine will retrieve the reservation from the database and, if the reservation exists, it will assign a seat and retrieve the terminal and gate details. After every detail is retrieved, the next step will be generating a boarding pass and allowing the user to receive the boarding pass image. If the reservation does not exist, then the workflow will generate a response with a failure message and end.

If the boarding pass is generated successfully, then the state machine will generate a response with a booking ID, customer details, boarding pass URL, and a message indicating successful execution.

The request begins with the client calling the generateBoardingPass mutation with user and booking information, and then AWS AppSync will invoke the state machine by making a StartSynchronousExecution API call from HTTP resolver. Then, the Step Function will run the workflow, and once all the information has been retrieved, AWS AppSync API will receive the result from the state machine and forward the details to the user.

Below is a sample request mapping template for invoking Synchronous Express Workflows from AWS AppSync API. A detailed implementation of this pattern (AppSync to Step Functions) is available on Serverless Land.

{
    "version": "2018-05-29",
    "method": "POST",
    "resourcePath": "/",
    "params": {
      "headers": {
        "content-type": "application/x-amz-json-1.0",
        "x-amz-target":"AWSStepFunctions.StartSyncExecution"
      },
      "body": {
        "stateMachineArn": "arn:aws:states:<REGION>:<ACCOUNT_ID>:stateMachine:boarding-pass",
        "input": "{ \"details\": \"$context.args.input\"}"
      }
    }
  }

The given architecture diagram shows the integration between Synchronous Express Workflows and AWS AppSync API in order to check the user in and generate a boarding pass.

Long-running workflows

Some queries can take over 30 seconds to execute and cannot return immediate responses. For long-running queries requiring multi-step processing, utilize Step Functions to orchestrate the tasks by using Asynchronous Express Workflows. They can also run for up to five minutes.

Asynchronous Express Workflows can be triggered using a StartExecution API call. Once the workflow is invoked and has started executing, it does not wait for completion and simply returns the ARN and startDate of the execution in response.

To invoke these workflows from AWS AppSync API, add Step Functions as a HTTP Data Source. The workflow can be triggered from HTTP resolver by making a signed request to StartExecution API. However, invoking the workflow and then receiving the results back is a multi-step process for long-running queries. Let’s look at how we would handle this use case via the example of an airline booking system.

Tasks such as booking a flight involve steps like processing the payment, which can sometimes take more than 30 seconds to finish. Therefore, for such long-running workflows where it is required to poll for response, we must have two mutations and one subscription. One mutation is required to actually invoke the state machine workflow, and another mutation is to mark workflow completion. Then, the subscription will subscribe to events from the latter.

The client will generate a transaction ID that will be utilized to identify a particular state machine execution from the client side. The client fires the subscription to get updates from a run by using the transaction ID. And then a mutation is invoked with the same transaction ID in order to invoke the workflow. Once the workflow has finished executing, another mutation will be invoked that will send a status update to the subscription.

Now we will see how to integrate a long-running workflow like creating a booking with AWS AppSync API. By using the analogy above, the mutation createBooking will invoke the state machine to book the flight and process the payment, and the mutation updateBookingStatus will mark the completion of the state machine workflow. The subscription that will subscribe to updateBookingStatus will be called onUpdateBookingStatus. updateBookingStatus will have a None type data source attached to it, as the only purpose of this mutation is to send events to the onUpdateBookingStatus subscription.

The schema below can be appended for adding creating booking functionality:

input CreateBookingInput {
    transactionId: ID!
    flightId: ID!
    paymentDetails: String!
    customer: String!
}

input UpdateBookingStatusInput {
    transactionId: ID!
    message: String!
    bookingId: ID
    status: Status
    customer: String
}

type Mutation {
    createBooking(input: CreateBookingInput!):Booking
    updateBookingStatus(input: UpdateBookingStatusInput!):Booking
}

type Subscription {
    onUpdateBookingStatus(transactionId: ID!): Booking
        @aws_subscribe(mutations: ["updateBookingStatus"])
}

To make a call to StartExecution API from createBooking mutation, you can attach the given HTTP resolver:

{
  "version": "2018-05-29",
  "method": "POST",
  "resourcePath": "/",
  "params": {
    "headers": {
      "content-type": "application/x-amz-json-1.0",
      "x-amz-target":"AWSStepFunctions.StartExecution"
    },
    "body": {
      "stateMachineArn": "arn:aws:states:<REGION>:<ACCOUNT_ID>:stateMachine:create-booking",
      "input": "{ \"details\": \"$ctx.args.input\" }"
    }
  }
}

Detailed steps for the whole process are outlined below, along with an architectural diagram for the same:

1. Client will generate a transactionID and subscribe to events from that run by firing onUpdateBookingStatus subscription.
2. Client calls createBooking mutation with user and payment information along with the transactionID generated from step 1.
3. HTTP resolver attached to createBooking will make StartExecution API call in order to start the state machine workflow. The input from the createBooking mutation will be forwarded to the state machine.
4. StartExecution API returns Execution ARN in response, and the workflow will keep running in the background.
5. Once all steps of the workflow have completed, the last step will be to fire updateBookingStatus mutation with the same transactionID and the mutation will have a None data source attached to it which will simply forward the input from the request to the response. UpdateBookingStatus will set the appropriate booking status.
6. onUpdateBookingStatus subscription will receive an event from updateBookingStatus mutation and the client can be notified of the workflow completion.

Utilize AWS Lambda or Amazon EventBridge to make a call to AppSync API and fire the updateBookingStatus mutation. For more information on the integrations, visit patterns for AWS Lambda to AWS AppSync and Eventbridge to AWS AppSync on Serverless Land.

Invoking the API

Below is a sample query for generateBoardingPass mutation:

For long-running workflow CreateBooking, we subscribe to events for a given transactionID, and then fire the mutation createBooking with the same transactionID. This will call UpdateBookingStatus mutation as part of the workflow, and the subscription will receive events from that mutation.

Conclusion

In this post, we have described patterns for integrating Synchronous and Asynchronous Express Workflows with AWS AppSync. By integrating Step Functions as an HTTP data source with AWS AppSync API, we can easily provide a GraphQL interface for invoking state machine workflows through HTTP resolvers. For short-running workflows that can finish executing within 30 seconds, Synchronous Express Workflows can be utilized, and they can be invoked from AppSync API by making the StartSyncExecution call from HTTP resolver. For complex workflows that take longer than 30 seconds to complete, we can utilize Asynchronous Express Workflows. Once the workflow is invoked, it will return the execution details, continue running in the background, and then a subscription can be used to get the status update of the run.

For additional details, refer to Step Function Developer Guide and AppSync Developer Guide.

To get started with HTTP endpoint integration with AWS AppSync, visit HTTP Resolvers tutorial.

About the author