AWS Compute Blog
Building a serverless URL shortener app without AWS Lambda – part 1
When building applications, developers often use a standard multi-tier architecture pattern that generally includes a presentation, processing, and data tier. When building such an application using serverless technologies on AWS, it might look like the following:
In this three-part series, I am going to challenge you to approach this a different way by building a functionless or “backend-less” URL shortener application, that looks like this:
In part one, I discuss configuring a service integration between Amazon API Gateway and Amazon DynamoDB, removing the need for AWS Lambda entirely. I also demonstrate using Apache’s Velocity Templating Language (VTL) to apply business logic and modify the API request and response as needed. In part two, I show how to use API Gateway to increase security. In part three, I demonstrate how to improve response time and configure observability to get insights into application performance and client usage.
At AWS re:Invent 2019, the new HTTP API for Amazon API Gateway was announced. At the time of this writing, this new service does not support VTL or some of the other features discussed, so instead I use a REST API. When HTTP API gains feature parity, we will publish an additional follow up to this post.
Throughout this blog series, there are deep links to AWS SAM and OpenAPI configurations to show how to build this application using infrastructure as code (IaC). To refer to the full application, visit https://github.com/aws-samples/amazon-api-gateway-url-shortener. The template.yaml file is the AWS SAM configuration for the application, and the api.yaml is the OpenAPI configuration for the API. I have included instructions on how to deploy the full application, including a simple web client, in the README.md file.
Why would I do this?
AWS Lambda is the standard compute resource for serverless applications. With a Lambda function, I can process complex business logic in any of the AWS supported runtimes or even in my own custom runtime. However, do I really need to use a Lambda function when the business logic is minimal, and the main purpose becomes the transportation of data? Instead, I can turn to API Gateway to transport the data and process minimal amounts of business logic, as needed, with VTL. This allows me to minimize my application resources and cost.
API Gateway service integration
While each request to an API Gateway REST endpoint follows the same path, to understand how service integrations work, I show the integration for /app – POST. This represents the lifecycle of a request made to http://myexampleapi.com/api using a POST method. The purpose of this endpoint is to post new short links to the database.
The Method Request and Method Response mainly handle authorization, modeling, and validation, and are covered in detail in part two of this blog. For now, I focus on the Integration Request and Integration Response. The Integration Request is responsible for service integrations, and looks like this:
The Integration type is AWS Service and the AWS Region is my closest Region, us-west-2. For AWS Service, I choose DynamoDB from the long list of available services. For the HTTP Method, when interacting with the DynamoDB API, the POST method is required to take action on the underlying table.
For the Action, I choose UpdateItem. The action is the same here as you would use in the CLI or SDK to interact with DynamoDB. Generally, when adding new items to the DynamoDB table, I use the PutItem command. However, in this instance I must use UpdateItem to get a specific set of return data from DynamoDB.
When creating a new record in DynamoDB, the PutItem action does not return the completed record in the single request. If I want to obtain the new record, I need to make a secondary call to DynamoDB to fetch the record. However, the API Gateway request lifecycle does not have the ability to call the database a second time. I need to make sure I get everything I need the first time around. The nature of the UpdateItem is to update an existing item or create a new one if it doesn’t exist. Additionally, it returns the newly created object which I can then return to the client.
Finally, I configure the execution role. On this method, API Gateway needs permission to read and write from DynamoDB. Here is the policy section of the DDBCrudRole:
Policies:
- PolicyName: DDBCrudPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
Action:
- dynamodb:DeleteItem
- dynamodb:UpdateItem
Effect: Allow
Resource: !GetAtt LinkTable.Arn
This simple policy is used for all create, read, update, and delete (CRUD) operations, and UpdateItem is used for both create and update. This policy is part of the SAM template, and dynamically references the DynamoDB table name for the resource. This follows the principles of least privilege, only allowing access to the required table.
Modifying the request
Now that I have configured the integration from API Gateway to DynamoDB, I modify the incoming request to a format that DynamoDB understands. Further down the page on the Integration Request, you see the Mapping Template option:
The mapping template evaluates incoming request body and looks for existing templates to apply. I have created a template for application/json to match the incoming body. Here is a summarized version of the template:
{
"TableName": "URLShortener-LinkTable-QTK7WFAJ11YS",
"ConditionExpression":"attribute_not_exists(id)",
"Key": {
"id": { "S": $input.json('$.id') }
},
"ExpressionAttributeNames": {
"#u": "url",
"#o": "owner",
"#ts": "timestamp"
},
"ExpressionAttributeValues":{
":u": {"S": $input.json('$.url')},
":o": {"S": "$context.authorizer.claims.email"},
":ts": {"S": "$context.requestTime"}
},
"UpdateExpression": "SET #u = :u, #o = :o, #ts = :ts",
"ReturnValues": "ALL_NEW"
}
If you have worked with the DynamoDB SDK, this might look familiar. The TableName indicates which table to use in the call. The ConditionExpression value ensures that the id passed does not already exist. The value for id is extracted from the request body using $input.json(‘$.id’).
To avoid colliding with reserved words, DynamoDB has the concept of ExpressionAttributeNames and ExpressionAttributeValues. In the ExpressionAttributeValues I have set ‘:o’ to $context.authorizer.claims.email. This extracts the authenticated user’s email from the request context and maps it to owner. This allows me to uniquely group a single user’s links into a global secondary index (GSI). Querying the GSI is much more efficient than scanning the entire table.
I also retrieve the requestTime from the context object, allowing me to place a timestamp in the record. I set the ReturnValues to return all new values for the record. Finally, the UpdateExpression maps the values to the proper names and inserts the item into DynamoDB.
Modifying the response
Before I discuss the Integration Response, let’s examine the Method Response:
The Method Response is responsible for modeling the response to the client. In most cases, DynamoDB returns a status code of either 200 or 400. Therefore, I configure a 200 response and a 400 response.
When DynamoDB returns a 200 response, the data looks like the following:
{
"id": {"S": "aws"},
"owner": {"S": "me@mydomain.com"},
"timestamp": {"S": "27/Dec/2019:21:21:17 +0000"},
"url": {"S": "http://aws.amazon.com"}
}
In the Integration Response, I have a template that converts this to a structure that the client is expecting. The template looks like this:
#set($inputRoot = $input.path('$'))
{
"id":"$inputRoot.Attributes.id.S",
"url":"$inputRoot.Attributes.url.S",
"timestamp":"$inputRoot.Attributes.timestamp.S",
"owner":"$inputRoot.Attributes.owner.S"
}
This template has a variable called $inputRoot to contain the root data. I then build out the return object, formatted for the client:
{
"id": "aws",
"url": http://aws.amazon.com,
"timestamp": "27/Dec/2019:21:21:17 +0000",
"owner": "ericdj@amazon.com"
}
For a 400 status, I must evaluate the issue and respond accordingly. The mapping template looks like this:
#set($inputRoot = $input.path('$'))
#if($inputRoot.toString().contains("ConditionalCheckFailedException"))
#set($context.responseOverride.status = 200)
{"error": true,"message": "URL link already exists"}
#end
This template checks for the string, “ConditionalCheckFailedException”. If it exists, then I know that the conditional check “attribute_not_exists(id)”, from the UpdateItem template in the Integration Request failed. To return a 200 response, I use the “#set($context.responseOverride.status = 200)” override andset the response with the error details.
With my integration and mapping templates in place for the /app – POST method, I now have the ability to create new short links for my URL shortener. Taking this same approach for reading, updating, and deleting short links, I now have a fully functioning backend for the URL shortener that only uses API Gateway and DynamoDB.
Conclusion
In this post, I walked through using VTL to manage simple business logic at the processing tier with API Gateway. I covered configuring the service integration with DynamoDB and modifying the request and response payloads as needed. In part 2, I discuss different options for configuring Amazon API Gateway security.
To deploy the URL shortener, visit https://github.com/aws-samples/amazon-api-gateway-url-shortener. The README.md file contains instructions for launching the application.
Happy coding!