Simplify access to multiple microservices with AWS AppSync and AWS Amplify

This article was written by Faraz Masood, Cloud Architect, AWS

Modern applications and architectures are created with microservices in mind, and the ever evolving nature of each service makes it difficult to build and maintain a single API for multiple clients. Rapid iteration in the development cycle can benefit from an unified API interface exposed to both external and internal customers, internal microservices are integrated to this unified interface and teams can work on each system independently.

GraphQL is a query language for APIs where a client can specify exactly the data it needs by interacting with a single endpoint exposed by a GraphQL backend that is able to retrieve data from multiple sources. It makes it a very suitable technology for unifying multiple existing services behind a single, coherent and unified gateway that manages clients access and can retrieve data from multiple sources with a single network call.

The AWS Toolbox

AWS provides scalable and flexible services leveraging different technologies that allow developers to build applications for any business needs. In this article we’re using the following serverless services to showcase how we can work with different technologies in the same application, focusing on security and business logic instead of the infrastructure to maintain and operate the different services:

AWS AppSync is a managed service that uses GraphQL to make it easy for applications to get exactly the data they need by letting you create a flexible API to securely access, manipulate, and combine data from one or more data sources.
The Amplify Framework allows developers to create, configure, interact and implement scalable mobile and web apps powered by AWS. Amplify seamlessly provisions and manages your mobile backend and provides a simple framework to easily integrate your backend with your iOS, Android, Web, and React Native frontends. Amplify also automates the application release process of both your frontend and backend allowing you to deliver features faster.
AWS Amplify Console provides a Git-based workflow for deploying and hosting fullstack serverless web applications.
AWS Amplify CLI is a unified toolchain to create, integrate, and manage the AWS cloud services for your app.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure REST APIs at any scale.
AWS Fargate is a serverless compute engine for containers that works with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS).
AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume. With Lambda, you can run code for virtually any type of application or backend service.
Amazon Cognito is a user management service with rich support for users authentication and authorization. You can manage those users within Amazon Cognito or from other federated IdPs.
Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale.

We will build and host a WebStore application using AWS Amplify and the AWS Amplify Console, managing WebStore users with Amazon Cognito and using AWS AppSync to create an unified API layer to integrate and consolidate different microservices that compose the application.

The application code can be found in this GitHub repository. It can be fully deployed with one-click to the Amplify Console after creating an ECR registry and adding an image to the container registry in your account, which is a requirement for one of the microservices based on containers backed by AWS Fargate.

Architecture Overview

The solution is powered by 3 different backend microservices based on different technologies: a REST service, a secondary GraphQL service and a service running on containers in a VPC. All microservices are accessed through a single GraphQL API powered by AppSync.

The workflow can be better understood with the following steps:

Users authenticate with their user name and passwords managed by Cognito User Pools
Authenticated clients make API calls to AppSync using valid JWT tokens generated by Cognito
AppSync uses Resolvers to make direct calls to different microservices. HTTP Resolvers connect to either the REST or GraphQL endpoints of the User service and the Order service, respectively. A Lambda Resolver directs calls to the private Payment Service in a VPC.
The communication between the Resolvers and the HTTP endpoints are protected with temporary IAM credentials based on assumed IAM roles. The JWT token specific to the authenticated user is also forwarded to each microservice.
A Lambda function is invoked to access the private service hosted in a VPC. All 3 services are secured in a way that only the main AppSync API is granted access.
The REST microservice returns the requested information in XML based on the user details in the JWT Token, the XML payload is automatically converted to JSON by built-in utilities in the AppSync resolver.
The GraphQL microservice returns information about orders from a user by another AppSync API
Finally, the Payment service is hosted on Fargate containers in a private VPC and returns payment information for the user.

AWS AppSync built-in features are used to securely access, modify, consolidate and map data from different microservices as a single gateway providing data to clients. The data is correlated and linked by different services to a specific user using the unique JWT tokens that Cognito generates for each session.

Application Overview

The WebStore application allows authenticated users to query their profile information, add or view orders and manage payment details. It interacts with 3 different microservices called User, Order and Payment services through the interface exposed by the GraphQL Store API to manage users, orders and payment details.

The Amplify CLI is used to create all the backend services with a single amplify push command, triggering built-in and custom CloudFormation templates. The Amplify client provides useful higher order components that allow to add authentication features to the application with a couple of lines of code. Users can easily authenticate, sign up, sign in to the app and retrieve JWT tokens from Amazon Cognito with 2 lines of code:

The client web application is very simple with very basic functionality. Users can access their details (user name, email, phone number), place and view orders and add payment methods, nothing else. It’s not a complete front-end store experience as the main purpose of the application is to demonstrate how to securely access multiple polyglot independent backend microservices using a single AppSync GraphQL API.

For instance, when adding a payment method there’s only a choice to add a type of payment and its details. On the backend, AppSync will forward the user ID retrieved from Cognito as well as the payment details to the Fargate service via Lambda. The user payment data is then stored via Fargate to a separate DynamoDB table, which is the data store for the Payments microservice.

Let’s take a look at different microservices supporting the application functionality.

Backend Microservices Overview

User Service

The User service retrieves user profile information such as username, email and phone number. It is behind a REST API implemented with AWS API Gateway and AWS Lambda. You can find the service CloudFormation implementation details here.

The input to this service is an user name and it just returns profile information retrieved from the Cognito JWT Token in a XML format.

<body>
 <userDetails>
   <userName>uname</userName>
   <email>abc@xyz.com</email>
   <phoneNumber>phone_number</phoneNumber>
 </userDetails>
</body>

AppSync then uses handy XML helpers to automatically translate the payload to JSON.

Payment Service

This service allows a valid user to add a new payment method (credit card, cash etc) or update/retrieve existing ones. It is deployed inside an Amazon VPC, and is implemented using AWS Fargate with Amazon DynamoDB as a data store. You can find the service CloudFormation implementation details here.

This is a completely private service and only accessible in a private VPC.

Order Service

The Order service lets users place new orders or retrieve existing ones. It is a secondary GraphQL API, also built on AWS AppSync. The service data is defined using the following GraphQL schema:

type Order
    @model(mutations: { create : "addOrder", delete: "cancelOrder"}, subscriptions: {onCreate: ["addedOrder"], onDelete: ["cancelledOrder"]})
    @auth(rules: [{allow: private, provider: iam}])
    @key(fields: ["userId", "orderDateTime", "status"] ) {

    userId: ID!
    status: Status!
    orderDateTime: String!
    details: String!
    orderId: String!
}

enum Status {
    DELIVERED
    IN_TRANSIT
    PENDING
    PROCESSING
    CANCELLED
}

In addition to the usual SDL type definitions, you will notice directives such as @model, @auth and @key (such directives annotate parts of the GraphQL schema that are evaluated by the GraphQL Transform).

GraphQL objects annotated with @model are top-level entities in the generated API and are stored in Amazon DynamoDB. The @key directive makes it simple to configure custom index structures. The first field in the list is the partition key and any additional field(s) (if provided) will be the sort key(s). This is very useful if you want to implement the Adjacency List pattern as CRUDL queries and mutations will be automatically configured. Objects annotated with @auth are protected by a set of authorization rules.

Note how we can customize these directives to selectively add mutations, queries and subscriptions.

Now we have a better understanding on supporting microservices, let’s discuss the Store API that will consolidate all of them in a single endpoint.

A GraphQL Layer for integrating with multiple microservices

As any other GraphQL API, you start with the schema definition and we will use the AWS Amplify CLI to deploy it:

schema {
  # query (read-only fetch)
  query: Query

  # mutation (write followed by a fetch)
  mutation: Mutation

  # subscription (long-lived requests that receive data in response to events)
  subscription: Subscription
}

type User {
    userName: ID!
    email: String
    phoneNumber: String
}

type Order {
    userId: ID!
    status: OrderStatus
    orderDateTime: String!
    details: String!
    orderId: String
}

type PaymentAccount {
  userId: ID!
  type: String!
  details: String!
}

enum OrderStatus {
    DELIVERED
    IN_TRANSIT
    PENDING
    PROCESSING
    CANCELLED
}

type Query {
  # Get user profile information by userName
  getUserInfo(userName: ID!): User

  # Get payment accounts by userId
  getPaymentAccounts(userId: ID!): [PaymentAccount]

  # List recent orders by orderDateTime
  listRecentOrders(userId: ID!, orderDateTime: String!) : [Order]

  # List recent orders by orderDateTime and status
  listRecentOrdersByStatus(userId: ID!, orderDateTime: String!, status: OrderStatus! ) : [Order]

}

type Mutation {
  # add/update Payment method
  addPaymentAccount(userId: ID!, paymentAccountType: String!, paymentAccountDetails: String!): PaymentAccount

  # add a new order
  addOrder(userId: ID!, orderDateTime: String!, details: String!) : Order
}

type Subscription {
  addedPaymentAccount: PaymentAccount
  @aws_subscribe(mutations: ["addPaymentAccount"])

  addedOrder: Order
  @aws_subscribe(mutations: ["addOrder"])

}

Comparing the schema above with the Order API schema, you will notice that we have not used GraphQL Transform directives. You can take advantage of the AWS Amplify CLI flexibility to customize the backend using escape hatches such as custom CloudFormation templates, allowing you to further tailor the configuration of the data sources for each service to fit your business needs.

Connecting the Store API to backend services

AWS AppSync translates GraphQL requests and fetches information from the different data sources using resolvers that consist of request and response mapping templates.

A data source can be a persistent storage system (Amazon Dynamo DB, Amazon Elastic Search Domain, Amazon Aurora Serverless) or a trigger (HTTP endpoint including other AWS services own endpoints, REST API, AWS Lambda). In order to connect securely to the data sources, AppSync can use IAM roles to authorize the connection to the data sources and, in case of HTTP endpoints, it can also forward headers. The mapping templates translate a GraphQL request received by AppSync into a format that the data source can understand and vice versa.

The Store API defines two HTTP data sources and a Lambda data source to interact with User, Order and Payment services respectively. Let’s take a look at different configuration options available for these data sources.

HTTP Data Sources

The HTTP data sources comprise of an endpoint to access the service and optional authorization details. They are public endpoints however they are protected and secured so they can only be accessed with proper IAM credentials.

"UserServiceDataSource": {
	"Type": "AWS::AppSync::DataSource",
	"Properties": {
		"ApiId": {
			"Ref": "AppSyncApiId"
		},
		"Description": "User service data source",
		"HttpConfig": {
			"Endpoint": {
				"Ref": "UserServiceEndpoint"
			},
			"AuthorizationConfig": {
				"AuthorizationType": "AWS_IAM",
				"AwsIamConfig": {
					"SigningRegion": {
						"Fn::Sub": "${AWS::Region}"
					},
					"SigningServiceName": "execute-api"
				}
			}
		},
		"Name": {
			"Fn::Sub": "${env}UserService"
		},
		"Type": "HTTP",
		"ServiceRoleArn": {
			"Fn::GetAtt": [
				"UserServiceDataSourceIamRole",
				"Arn"
			]
		}
	}
}

You can also use the request mapping template to add custom authentication headers. We are mapping the userName as part of the resourcePath, as well as forwarding additional headers. Additionally AppSync also performs a check to confirm the identity of the user based on the received JWT token ($context.identity). For the User service, the Endpoint (defined in HTTP data source) and resourcePath will make up the REST API backend service URL connecting to API Gateway:

Lambda Data Source

Recent improvements to networking and performance make Lambda the perfect serverless managed service to connect to VPC resources. The Payment service is not publicly accessible, it is a completely private service and for that reason we’ll use a Lambda data source to connect to the VPC where the service is hosted. Similarly to the HTTP Data Sources, an IAM service role is used to secure the access to Lambda and allow AppSync to invoke the function.

"PaymentServiceDataSource": {
	"Type": "AWS::AppSync::DataSource",
	"Properties": {
		"ApiId": {
			"Ref": "AppSyncApiId"
		},
		"Description": "Payment Service data source",
		"LambdaConfig": {
			"LambdaFunctionArn": {
				"Fn::GetAtt": [
					"PaymentServiceDataSourceLambda",
					"Arn"
				]
			}
		},
		"Name": {
			"Fn::Sub": "${env}PaymentService"
		},
		"Type": "AWS_LAMBDA",
		"ServiceRoleArn": {
			"Fn::GetAtt": [
				"PaymentServiceDataSourceIamRole",
				"Arn"
			]
		}
	}
}

In the Lambda resolver, anything passed as a payload will be available as part of the Lambda’s event object. Same as before AppSync performs an additional check to confirm the identity of the user based on the received JWT token ($context.identity).

You can find more examples of HTTP and AWS Lambda resolvers in our documentation. For a more details of the configuration of these data sources you can check the custom CloudFormation template here.

Securing the Store API and its interaction with backend microservices

Users will typically interact with the Store API through a client (web or mobile) application. In the sample application, the access to the Store API is secured using an Amazon Cognito User Pool. Users will log in first to retrieve a JWT token and pass this token along the request to AWS AppSync. AWS AppSync will then validate the token and reject all unauthenticated requests.

AWS AppSync does not store any data, and the authorization metadata/logic is often best determined by the backend services. You can still conditionally forward or reject requests based on the information in authentication token (for example, user A cannot retrieve user profile information of user B), as we saw in the request mapping templates in the previous section.

All 3 microservices are completely protected and isolated and only the main Store API can access any of the services, it doesn’t matter if it’s a protected public API or a completely private service running on a VPC.

Conclusion

You can use AWS AppSync as a single interface to access and combine data from multiple microservices in your application, even if they’re running in different environments such as containers in a VPC, behind a REST API on Amazon API Gateway, or behind a GraphQL API on another AWS AppSync endpoint.

This article walked through an example on how you can implement and secure an API composition layer using AWS AppSync to connect to multiple microservices on containers in a VPC, REST APIs or even another GraphQL API allowing to expose a single reliable and scalable endpoint to interact with different services. Best of all, you don’t need to operate and maintain any services and can spend your time focusing on your business logic instead.

Front-End Web & Mobile