Front-End Web & Mobile

Building a real-time stock monitoring dashboard with AWS AppSync

This article was written by Jan Michael Go Tan, Specialist SA at AWS

Building real-time applications normally requires two types of endpoints. The first type of endpoint typically involves a standard stateless request–response type of interaction, which is handled over traditional HTTP/HTTPS protocol. The second type of endpoint enables persistent connectivity and allows the server to push information to the clients. This is normally accomplished by means of a stateful WebSocket protocol

With AWS, you can accomplish these in a number of different ways. Some (but not all) of the approaches that customers can use are as follows:

1. Amazon EC2 and Elastic Load Balancing – The customer is responsible for configuring, patching, maintaining, implementing high availability, and automatically scaling the backend infrastructure.

2. Amazon API Gateway and AWS Lambda – Customer focus shifts up the stack and concentrate on implementing the backend that would respond to the routes that are defined at the gateway level. AWS manages the infrastructure of both the API Gateway and the backend compute that would be running in Lambda. The customer still must define two API Gateways, one for the standard request–response model and one for the WebSocket.

3. AWS AppSync and Amplify Framework – Customer focus shifts even higher up the stack for both frontend and backend. Instead of defining routes, customers define a schema, the various data sources that they require, and then use Amplify Framework, an opinionated framework, to interact with AWS AppSync. The underlying detail of which protocol (HTTP/HTTPS or WebSocket) is abstracted away from the developer. This is the focus of this post.

Overview

The application uses a number of AWS services:

  • Amplify Framework – This allows you to rapidly bootstrap the project’s integration with other AWS services. You use the following modules: Auth and API. The Auth module integrates with Amazon Cognito, and the API module integrates with AWS AppSync.
  • Amazon Cognito user pools – This allows the application to handle user sign-ups and sign-ins. The service returns a JWT token after a successful sign-in, and you use this JWT token to identify the user when you’re making a request to AWS AppSync.
  • AWS AppSync – This is the primary way that the app interacts with the backend. AWS AppSync enables developers to interact with their data by using a managed GraphQL service.
  • AWS Amplify Console – When you’re ready to deploy the app, you can just connect your GitHub repository. AWS Amplify Console automatically determines the frontend framework used, and then builds and deploys the app to a globally available content delivery network (CDN). AWS Amplify Console detects backend functionality added using the Amplify Framework and can deploy the necessary AWS resources in the same deployment as the frontend.
  • Amazon DynamoDB – All the data of the application is stored in DynamoDB. AWS AppSync has direct integration with DynamoDB through resolvers.
  • AWS Lambda: This allows the backend to regularly fetch the latest intraday prices from the data feed and then use AWS AppSync to insert into DynamoDB.
  • Amazon CloudWatch Events – This allows you to schedule the execution of the Lambda function that fetches the latest intraday prices.

Before going deeper into the application, here’s an overview of GraphQL and AWS AppSync.

GraphQL overview

GraphQL is a data language that was developed to enable apps to fetch data from servers. It has a declarative, self-documenting style. In a GraphQL operation, the client specifies how to structure the data when it is returned by the server. This makes it possible for the client to query only for the data it requires, in the format that it requires.

GraphQL supports the following operations:

  • Query: Fetch data
  • Mutation: Write data and then fetch
  • Subscription: Long-lived connection for receiving data

AWS AppSync overview

AWS AppSync is a managed GraphQL service. Users define the GraphQL schema and attach resolvers to either fields or types to resolve them. Examples follow:

  • A field resolver can be used to retrieve specific relationships. For example, when building a blog application, a field resolver can be used to retrieve the comments for a particular blog post.
  • A type resolver can be used in association with a specific operation (query, mutation). For example, a resolver can be used to retrieve all the latest blog posts.

Frontend

The frontend uses ReactJS with Amplify Framework to integrate with the different AWS services that you are using.

Getting started

Bootstrapping the project

First bootstrap the React application using the create-react-app tool. After the bootstrap process is done, you can execute npm start to test if everything is working. This would open up the React version of “hello world” in your browser.

The next step is to bootstrap the Amplify project using the amplify init command. This command must be executed at the root directory of the project. To configure the project, follow the guided setup.

Adding capabilities

After the project has been bootstrapped, you can start adding capabilities like authentication and API support to the project. Amplify automatically generates the necessary integration into your React frontend.

For Auth, run the following command: amplify add auth

  • Select Default configuration.
  • Choose email as a way for users to sign in.
  • Select No, I am done when asked about configuring advanced settings.

For API, run the following command: amplify add api

  • Select GraphQL.
  • Provide an API name or leave as the default.
  • Use Amazon Cognito User Pool as the authorization type.
  • Select No for annotated GraphQL schema.
  • Select No for guided schema creation.
  • Select Single object with fields.
  • Select No for editing the schema now.

This automatically generates a default GraphQL schema that you update later on.

At this point, all these changes are still in your local machine. To set up the AWS resources to support these capabilities, you must execute the following command: amplify push. This generates and executes an AWS CloudFormation template to build the necessary AWS resources.

Dependencies

The following are additional Node.js dependencies that must be installed as part of the project:

To install the dependencies, run the following command:

npm install aws-amplify aws-amplify-react

Integrate Amplify with React

After running the amplify push command, Amplify automatically generates a config file: src/aws-exports.js. To use this config file, open the src/App.js and do the following.

Import the dependencies:

import Amplify from 'aws-amplify';
import awsExports from './aws-exports';

You can then use the config file to configure the Amplify components in the app:

Amplify.configure(awsExports);

Optionally, to use the Amazon Cognito hosted UI to provide sign-up and sign-in functionality, you can take the following steps.

Import the UI component:

import {withAuthenticator} from 'aws-amplify-react';

Use the wrapper with the default export, for example:

export default withAuthenticator(App);

Integrating the backend

Data model

Amplify Framework has a built-in functionality called GraphQL Transform that provides a simple to use abstraction that helps you quickly create backends for your web and mobile applications on AWS. With the GraphQL Transform, you define your application’s data model using the GraphQL schema definition language (SDL), and the library handles converting your SDL definition into a set of fully descriptive AWS CloudFormation templates that implement your data model.

To build the data model, you use the following built-in directives in addition to the standard GraphQL SDL:

  • @model – These are top-level entities and are stored in DynamoDB.
  • @auth – Automatically connect the ownership of the object with the Amazon Cognito user making the request.
  • @key – Create a secondary index in DynamoDB.

The GraphQL schema file can be found in the following folder: amplify/backend/api/<projectName>/schema.graphql.

The application uses the following data model:

type IntradayStockPrice @model(subscriptions: null)
	@key(name: "BySymbol", fields: ["symbol", "data_timestamp"])
{
	id: ID!
	symbol: String!
	data_timezone: String!
	data_timestamp: Int!
	open_price: Float
	high_price: Float
	low_price: Float
	close_price: Float
	volume: Int
}

type Subscription {
	intradayStockPriceCreated(symbol: String): IntradayStockPrice @aws_subscribe(mutations: ["createIntradayStockPrice"])
}

type StockSymbol @model(subscriptions: null) @auth(rules: [{allow: owner}]) {
	id: ID!
	symbol: String!
	owner: String
}

type StockSymbolConnection {
	items: [StockSymbol],
	nextToken: String
}

type IntradayStockPriceConnection {
	items: [IntradayStockPrice],
	nextToken: String
}

type Query {
	listAllStockSymbols(limit: Int, nextToken: String): StockSymbolConnection
	retrieveLatestIntradayPrices(symbol: String, ts: Int, limit: Int, nextToken: String): IntradayStockPriceConnection
}

StockSymbol model

Different users within the application can declare their own set of stock symbols to monitor. That’s why the StockSymbol model is annotated with @auth, which means that query and mutation operations include the owner information from Amazon Cognito.

IntradayStockPrice

The intraday stock price represents the data coming from the data feed. This doesn’t have the @auth annotation because data stored here is shared across users that are monitoring the same symbol to reduce duplicate data.

By default, Amplify automatically generates basic CRUD query and mutation operations both on the backend and on the client side. The code generation would also generate subscriptions for the mutation operations. Using the generated subscriptions would mean that users would get notified for all stock updates even though they’re not monitoring those symbols. To keep the volume of incoming notifications from subscriptions low, AWS AppSync allows the subscriptions to filter based on a field.

In the previous example, you created a new subscription in the schema that would filter by symbol (intradayStockPriceCreated), which means you only receive incoming data if there are new intraday stock prices created for that symbol. New subscriptions can be annotated with @aws_subscribe and then indicate the mutations that would trigger the notification.

Even though you’re receiving data in real time from subscriptions, you must still retrieve the latest intraday prices from the backend to build the UI when the page gets loaded. To speed up retrieving intraday prices by symbol, create a secondary index through the @key annotation. This secondary index can then be used by the DynamoDB resolver when calling the retrieveLatestIntradayPrices query.

Custom resources

For AWS AppSync operations that aren’t covered by the default query and mutation operations that are automatically generated, Amplify provides a way to build custom resources. Defining these custom resources requires the following:

  • Entry in the GraphQL schema file
  • Define how AWS AppSync would resolve the request
  • Update the backend resources through AWS CloudFormation

Using the retrieveLatestIntradayPrices query as an example, you already defined the query in the preceding schema. Next, create two files in the amplify/backend/api/<projectName>/resolvers folder. These are as follows:

// Query.retrieveLatestIntradayPrices.req.vtl

{
    "version" : "2017-02-28",
    "operation" : "Query",
    "query" : {
        "expression" : "symbol=:symbol and data_timestamp>=:ts",
        "expressionValues" : {
            ":symbol": $util.dynamodb.toDynamoDBJson($ctx.args.symbol),
            ":ts": $util.dynamodb.toDynamoDBJson($ctx.args.ts)
        }
    },
    "index" : "BySymbol",
    "nextToken" : $util.toJson($util.defaultIfNullOrBlank($ctx.args.nextToken, null)),
    "limit" : $util.defaultIfNull(${ctx.args.limit}, 20),
    "scanIndexForward" : false,
    "consistentRead" : false
}

This file is responsible for defining the request resolver. As you can see, you’re using the secondary index that you defined through the @key annotation to query the DynamoDB table where the data is stored.

// Query.retrieveLatestIntradayPrices.res.vtl

{
    "items": $util.toJson($ctx.result.items),
    "nextToken": $util.toJson($util.defaultIfNullOrBlank($context.result.nextToken, null))
}

This file is responsible for defining the response mapping. Because this is a basic list query operation, just wrap the results in a paginated payload indicating a nextToken to be used to request for the next page.

Lastly, the amplify/backend/api/<projectName>/stacks/CustomResources.json file contains the AWS CloudFormation template to be executed. In this particular example, you are adding a new resolver in the “Resources” section:

"QueryRetrieveLatestIntradayPrices": {
    "Type": "AWS::AppSync::Resolver",
    "Properties": {
        "ApiId": {
        "Ref": "AppSyncApiId"
        },
        "DataSourceName": "IntradayStockPriceTable",
        "TypeName": "Query",
        "FieldName": "retrieveLatestIntradayPrices",
        "RequestMappingTemplateS3Location": {
        "Fn::Sub": [
            "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Query.retrieveLatestIntradayPrices.req.vtl",
            {
            "S3DeploymentBucket": {
                "Ref": "S3DeploymentBucket"
            },
            "S3DeploymentRootKey": {
                "Ref": "S3DeploymentRootKey"
            }
            }
        ]
        },
        "ResponseMappingTemplateS3Location": {
        "Fn::Sub": [
            "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Query.retrieveLatestIntradayPrices.res.vtl",
            {
            "S3DeploymentBucket": {
                "Ref": "S3DeploymentBucket"
            },
            "S3DeploymentRootKey": {
                "Ref": "S3DeploymentRootKey"
            }
            }
        ]
        }
    }
}

After the files have been modified, run amplify push at the root of the project folder to update the backend and create the resources.

Data feed

In the sample application, you implemented a Lambda function that is scheduled to run every 30 minutes, connect to the feed source, download the data, and use standard GraphQL mutation operation to write the data back to DynamoDB.

Conclusion

Using a mixture of different AWS services, you can quickly implement a completely serverless solution that enables both standard request–response type and real-time WebSocket connectivity. You don’t have to worry about low-level details, such as provisioning the infrastructure and managing client-side connectivity and messaging.

To deploy the entire solution, see the stock_dashboard GitHub repository, and follow the instructions on how to do both local deployment and deploying through Amplify Console.