Category: Amazon API Gateway

Nick Corbett, AWS Professional Services, Big Data Consultant

Amazon API Gateway and AWS Lambda empower developers to deliver a microservice architecture without managing infrastructure.  Building scalable, secure, and durable applications has never been easier.  However, managing the deployment of a large project is not always easy.  A global app, deployed across multiple AWS regions in multiple environments will collect API Gateway resources, AWS Lambda functions, Amazon Identity and Access Management (IAM) roles and other AWS resources.  As your project grows, so will the number of resources.  The need to coordinate and organize your efforts quickly becomes apparent.

In this post I will introduce JAWS, an open source application framework that you can use to develop massively scalable and complex apps using API Gateway and AWS Lambda whilst helping you manage your codebase and deployments.  I will show you how to build a simple microservice that you can use to manage users for a sample application.  You will build CRUD methods to support the management of users and persist details in Amazon DynamoDB.

To get started, you’ll first need to install Node.js.  Once you’ve done that, you can install JAWS using node.js’ package manger from a command prompt (note that on some systems you may need to run this command as super user):

npm install jaws-framework -g

Now that JAWS is installed, you are ready to create your first project. Navigate to the directory where you want to create your project and type:

jaws project create

JAWS will walk you through the process of creating a project. When prompted, enter the following information:

• Project Name: Specify “userManagement” as value. Camel case is recommended here. JAWS uses AWS CloudFormation to deploy your project and some items use the project name. CloudFormation tokenizes the project name with hyphens so it is best to avoid adding any more.
• Project Domain: Use any domain you own. It is important to make this unique for your project. The project domain is used as part of the name for a new Amazon Simple Storage Service (Amazon S3) bucket. This Amazon S3 bucket is used to deploy your solution.
• Email Address for CloudWatch Alarms: Your email address.
• Stage: Specify “dev“. A stage is an environment, such as dev, UAT, or production. Each region can have multiple stages. You can easily add more stages after the project is created.
• Region: Any AWS region. The AWS region in which you will deploy your solution. You can add other regions after the project is created. Regardless of the region you pick, API Gateway will create a global Amazon CloudFront distribution for your project to provide your users with the lowest possible latency for their API requests.
• Profile: Your AWS profile. JAWS uses a profile in your AWS Command Line Interface credentials file (in ~/.aws/credentials) to make API calls. If you have multiple profiles defined, you can select the one to use.

As it creates your project, the framework builds and runs a CloudFormation script containing some shared resources that are needed to support your project, such as IAM roles and the Amazon S3 bucket named after the project domain.

After this is complete, you are ready to create your first AWS Module (awsm). An awsm, is how JAWS describes your microservice and includes references to both your API Gateway endpoints and AWS Lambda functions. To create a module, navigate to the userManagement project directory that JAWS created and type:

jaws module create users create

This creates a new endpoint (users) with a method behind it for creating a new user (create). The following folders and files are created in the aws_modules directory of your project:

The create directory that JAWS made contains 4 files:

• awsm.json: Contains configuration for the API Gateway endpoint and AWS Lambda method
• index.js: Contains the code you write to implement the method
• handler.js
• event.json: Defines the event that is used when your code is tested with the jaws run command

JAWS creates a thin wrapper around your code (in handler.js) to integrate with AWS Lambda. This means that you can develop and test your code (in index.js) before deploying to AWS Lambda. To demonstrate this, go to the index.js file in the create directory and update the code to:

// Export for Lambda Handler
module.exports.run = function(event, context, cb) {
  return cb(null, action(event));
};

// Your code
var action = function(event) {
  return {
    message: 'You have created user ' + event.username
  };
};

Next, edit the event.json file to read:

{
  "username" : "Nick"
}

Finally, from the create directory, type the following command:

jaws run

The JAWS framework uses the event that is defined in event.json to test your code. The following message is returned:

JAWS: {"message":"You have created user Nick"}

The run command is good for simple testing but as your project grows, a unit test framework, such as Mocha, is recommended.

As well as developing an AWS Lambda function to implement your business logic, you also need to configure a REST endpoint in API Gateway. In our sample application, users are created using the following url:

/users		called with POST

Go into awsm.json in the create directory and find the apiGateway section. Update the Path to users and the Method to POST. This indicates to JAWS that it should create a users resource in API gateway with a POST method that is integrated to your AWS Lambda code. There are other settings in the awsm.json file to control how your project is deployed, although there is no need to change anything else at the moment.

You are now ready to deploy the first iteration of your project to AWS. At the command line, type:

jaws dash

Use the arrow keys and enter to highlight both the endpoint and AWS Lambda function in yellow before navigating to deploy selected and pressing enter. Your code is then packaged, using Browserify and Minify to improve run-time efficiency, and zipped. This package is then uploaded to an S3 bucket.

For each project, JAWS maintains two CloudFormation stacks. The first stack, containing shared resources, was deployed when you made the project. JAWS now creates a second stack that contains your new AWS Lambda function (the code in the S3 bucket is used as a source). Any additional AWS Lambda functions that you write are added to this stack. After this stack is deployed, JAWS creates the API Gateway resources and methods.

You are now ready to test the deployment. Go to the AWS Management Console and open the Amazon API Gateway console. Click the userManagement-dev API and then click the POST method that JAWS created for the users resource. Click Test to test the function and use the JSON object from event.json in the request body. If everything has worked, you will see a response:

{
  "message": "You have created user Nick"
}

You can view more detailed logging from your AWS Lambda function in Amazon CloudWatch Logs.

The next step is to add a similar stub for the GET function. This is accessed by the url:

/users/		called with GET

To create the endpoint and method, go to your command line and, from the project directory, type:

jaws module create users get

JAWS creates another sub-directory in your AWSM for the new method. Go into awsm.json in the get subdirectory and update the apiGateway section of the file by making the following changes to the cloudformation section:

"Path": "users/{username}"
"Method": "GET"
"RequestTemplates": {
  "application/json": "{\"username\": \"$input.params('username') \"}"
}

This change indicates to JAWS that the AWS Lambda function will be invoked when the path users/{username} is called. It also specifies the format of the JSON event sent to the AWS Lambda function. For example, if the url users/Anna is called with a GET verb, then your AWS Lambda function is called with the following event:

{
  "username": "Anna"
}

Go into index.js for the GET method and change the code to the following:

// Export For Lambda Handler
module.exports.run = function(event, context, cb) {
  return cb(null, action(event));
};

var action = function(event) {
  return {
    message: "User requested: " + event.username
  };
};

You are then ready to deploy your project again (using the jaws dash command). When you test this method in API Gateway console, you are asked for the username:

You’ve now created stub functions for the CREATE and GET methods. Hopefully you can see how this process can be used to make the UPDATE and DELETE methods to complete the set of CRUD functions. Its now time to replace your stub code with something more meaningful.

Before replacing your stub code, you need a data store for your users. Open the resources-cf.json file in the cloudformation folder for your stage and region. This file contains the shared resources CloudFormation stack that JAWS deployed when you created your project. Add the following to resource:

"myDynamoDBTable" : {
  "Type" : "AWS::DynamoDB::Table",
  "Properties" : {
    "AttributeDefinitions": [
      {
        "AttributeName": "username",
        "AttributeType": "S"
      }
    ],
    "KeySchema": [
      {
        "AttributeName": "username",
        "KeyType": "HASH"
      }
    ],
    "ProvisionedThroughput": {
      "ReadCapacityUnits": "5",
      "WriteCapacityUnits": "5"
    },
    "TableName": {
      "Fn::Join": [
        "-",
        [
          "users",
          {
            "Ref": "aaDataModelStage"
          }
        ]
      ]
    }
  }
}

In addition to creating the DynamoDB table, you also need to update the IAM role used by your AWS Lambda functions so they have permission to use the service. Find the IAMPolicyLambda policy in the same file and add the following extra statement to the policy document:

{
  "Effect": "Allow",
  "Resource": "*",
  "Action": [
    "dynamodb:*Item",
    "dynamodb:Query",
    "dynamodb:Scan"
  ]
}

You can deploy changes to the resources for your stage by running the following JAWS command:

jaws deploy resources dev

JAWS updates the CloudFormation stack for your resources, creating a DynamoDB table (named users-dev) and updating the IAM role used by your AWS Lambda functions. The only remaining task is to inject this dependency into your AWS Lambda function, so it can find the location to write the data. You can use the recently released API Gateway stage variables for this, or you can use an environment variable in JAWS.

Environment variables are set for each stage and region, allowing you to run the same code across multiple regions and environments and configure the code at runtime. In this case, for example, you can have a different users table for dev and production.

Open the aswm.json file for the GET function and modify the envVars section at the top of the file:

"lambda": {
  "envVars": [
    "TABLE_NAME",
    "JAWS_STAGE"
  ],

This indicates to the framework to include environment variables called TABLE_NAME and JAWS_STAGE in the deployment package for your AWS Lambda function. Your code can access these as environment variables of the runtime:

// Export For Lambda Handler
module.exports.run = function(event, context, cb) {
  return cb(null, action(event));
}

// Your Code
var action = function(event) {
  const tableName = process.env.TABLE_NAME + "-" + process.env.JAWS_STAGE;

  return {
    message: "User requested: " + event.username,
    table: tableName
  };
};

The final task is to set the environment variable for the stage and region. To do this, use the following JAWS command:

jaws env set dev <region> TABLE_NAME users

JAWS maintains a file in S3 that contains the environment variables. You can now deploy your project again using the jaws dash command. Before packaging your code, JAWS pulls down the environment variables from S3 and includes them in your distribution.

At runtime, your code builds the name for the DynamoDB table by combining TABLE_NAME and the JAWS_STAGE environment variable. The JAWS_STAGE environment variable is maintained by the framework. If you test your method now in API Gateway, you should see a result similar to this:

Its now a simple task to update the logic of your AWS Lambda functions to read and write from the DynamoDB table.

After you finish building your sample app, you can tidy up by deleting the two CloudFormation stacks that JAWS created. You also need to manually delete the userManagement-dev API from API Gateway.

Summary

In this post I have shown you how to get started with JAWS, a framework for building microservices using API Gateway and AWS Lambda. As you build your project you can focus on producing exciting functionality since all of the infrastructure you need is fully managed. You can start to build your microservice application, leaving the heavy lifting to AWS and the organization of your project to JAWS.

If you have any questions or suggestions, please leave a comment below.