24 open source tools for the serverless developer: Part 1

This article is a guest post from AWS Serverless Hero Yan Cui.

The mindset of a serverless developer is one of a minimalist: Don’t take on undifferentiated heavy-lifting, and leverage services as much as possible so we can focus on the things that actually differentiate our product and deliver value to our customers. In the same vein, we want to leverage open source tools that are battle-tested rather than building our own.

In this two-part article series, we will review open source tools you should consider adding to your toolbox. The tools include deployment frameworks, CLIs, libraries, and AWS Serverless Application Repository applications.

In part 1, we will learn about deployment frameworks and highlight the key difference between the two most popular frameworks in the Serverless Framework and AWS Serverless Application Model (AWS SAM). We will explore top plugins for the Serverless Framework, and then learn about two CLI tools you might find invaluable.

In part 2, we will check out popular libraries for writing AWS Lambda functions. Covering popular libraries for every language runtime is not possible, so we’ll focus only on Node.js. And finally, we will explore useful AWS Serverless Application Repository apps that can take care of many common chores for you.

Deployment frameworks

When developing serverless applications, the most important tool in your toolbox is the deployment framework. Many open source alternatives are available, including the Serverless Framework, AWS SAM, AWS Cloud Development Kit (AWS CDK), Terraform, Claudia.js, Zappa, Up, and Architect.

I compared some of these frameworks in a previous article, “AWS Lambda deployment frameworks compared”. In my opinion, the framework you use doesn’t really matter, as long as you use one instead of writing your own by cobbling together a bunch of scripts, which is the kind of undifferentiated heavy lifting we want to avoid with serverless technologies.

Keep in mind that you should understand how to provision resources in AWS before you use these tools to automate provisioning for you. For example, if you use the Serverless Framework or AWS SAM, then you should understand how AWS CloudFormation works and what the frameworks are doing for you. Being over-reliant on tools without understanding how they work is dangerous because you will be hopelessly stuck when problems arise and you won’t know how to resolve them.

The moral is: You shouldn’t automate what you don’t understand.

That said, once you have grasped the underlying mechanics and how things work, you should look for tools that can automate work for you so you can be more productive.

I’m a big fan of the Serverless Framework because it offers me sane defaults and a productive abstraction to work from. At the same time, it gives me the option to extend the framework’s capabilities and disagree with its decisions via plugins. To me, this is the big advantage that Serverless Framework holds over AWS SAM and other similar frameworks. With SAM, I once had to create an AWS CloudFormation macro just to change one setting in its generated AWS CloudFormation template. I wrote about that experience in the article “AWS SAM + Cloudformation macros, a patch made in heaven”.

Serverless Framework plugins

There is a wealth of community-driven plugins for the Serverless Framework that you can draw from. Here are a few of my favorites:

serverless-iam-roles-per-function

For the best security, you should follow the principle of least privilege and create a dedicated IAM role for each function. These roles should grant the least amount of permissions possible.

While SAM allows you to easily configure per-function IAM roles, the default behavior in Serverless Framework is to use a shared IAM role. Its built-in mechanism for per-function IAM requires you to create the per-function roles as custom AWS CloudFormation resources in the serverless.yml, which is far more verbose than the syntax to create the shared role.

Thankfully, the serverless-iam-roles-per-function plugin lets you do just that. Visit the GitHub repo for an example screenshot of defining iamRoleStatements definitions at the function level.

serverless-webpack

For Node.js functions, bundling helps reduce their cold start time and reduce the size of the deployment artefacts. The serverless-webpack plugin lets you bundle your functions whenever you run sls deploy.

serverless-offline

The serverless-offline plugin lets you run a local version of Amazon API Gateway by running sls offline in your project folder. This is basically what AWS SAM offers through the sam local start-api command. Visit the GitHub repo for a list of usage and command-line options.

I seldom find the need for this myself, as it’s just as easy to use sls invoke local to execute the Lambda function locally and inspect its JSON output. And I can attach a debugger and step through the code to help me debug issues more easily. However, this plugin comes in handy when I need to do server-side rendering because it lets me point a browser to a local endpoint and see how the returned HTML would render in the browser.

serverless-domain-manager

The serverless-domain-manager plugin makes it easy to configure custom domain names for your APIs and the plugin can even create the Route53 recordset for you too.

serverless-step-functions

The serverless-step-functions plugin lets you define your state machines in YML as part of your serverless.yml.

There are a ton of other value-add features, too, including:

• Configure API Gateway, Amazon EventBridge/Amazon CloudWatch Events, and scheduled triggers.
• Configure CloudWatch alarms on execution timeout/failed/aborted/throttled.
• Configure notifications to deliver success and failure responses to Amazon Simple Notification Service (Amazon SNS), Amazon Simple Queue Service (SQS), Amazon Kinesis, etc. through CloudWatch Events.
• Perform schema validation before you deploy.
• Perform blue-green deployment.

Also, it supports the new Express Workflows.

serverless-finch

The serverless-finch plugin makes hosting a static website with S3 a breeze. Just put your static site’s content in a distribution folder, name the S3 bucket, and run sls client deploy.

CLIs

Here are two CLI tools that I use on an almost daily basis.

org-formation

This is arguably the most impressive and useful open source tool I have seen in a while. Org-formation is a CLI tool that lets you manage your entire AWS organization using Infrastructure-as-Code (IAC). It sports an AWS CloudFormation-like syntax and has more than 50 features, including:

• Create new AWS accounts in code.
• Create AWS organizational units (OU) and adding accounts to them.
• Define Service Control Policies (SCPs), configure cross-account AWS CloudTrail collection, etc.
• If you have an existing AWS Organization, you can run org-formation init to pull them into a template so you can start managing them with org-formation going forward.
• You can use org-formation update-stacks to apply CloudFormation templates to multiple accounts and regions.
• You can use pseudo-functions such as **!Ref**, **!GetAtt**, and **!Sub** at the AWS Organization level as well as to reference accounts, regions, and organization units.
• You can use the org-formation init-pipeline command to bootstrap a CI/CD pipeline with AWS CodeCommit and AWS CodePipeline to automatically deploy updates to your AWS Organization.

lumigo-cli

The lumigo-cli is a Swiss army knife of helpful commands for working with AWS. There are a few commands that are especially useful if you do a lot of work with Lambda.

For example, you can run lumigo-cli list-lambda to list all your functions in all regions; it gives you a complete overview of what you have in your account and can help you quickly identify inactive functions (i.e. functions that haven’t been invoked in more than 30 days).

The analyze-lambda-cold-starts command analyzes the cold start performance of your functions in all regions, including their Provisioned Concurrency utilization. This helps you identify functions to optimize for better cold start performance, or to enable Provisioned Concurrency on them.

To apply a data-driven approach to finding the best memory setting for a function, use the powertune-lambda command to find the best setting for performance, cost, or a balance of the two.

If you use Lambda with SNS, SQS, Amazon DynamoDB, or Kinesis streams, then seeing what events your functions are triggered with can be challenging sometimes. The following commands let you tail these event sources and see what actually is going on:

• lumigo-cli tail-cloudwatch-events-bus
• lumigo-cli tail-cloudwatch-events-rule
• lumigo-cli tail-dynamodb
• lumigo-cli tail-eventbridge-bus
• lumigo-cli tail-eventbridge-rule
• lumigo-cli tail-kinesis
• lumigo-cli tail-sns
• lumigo-cli tail-sqs

If you work with lots of different accounts or profiles, then you will appreciate the whoami and switch-profile commands. They allow you to identify what profile you’re currently using and quickly switch to another.

All in all, there are more than 20 commands, with something for nearly everyone.

Summary

So that’s it for Part 1. Here are the open source tools I highlighted in this post and where you can find them:

Deployment frameworks

• Serverless Framework
• AWS SAM
• AWS CDK
• Terraform
• Claudia.js
• Zappa
• Architect

Serverless Framework plugins

• serverless-iam-roles-per-function
• serverless-webpack
• serverless-offline
• serverless-domain-manager
• serverless-step-functions
• serverless-finch

CLIs

• org-formation
• lumigo-cli

Open source tools depend on their communities to survive and thrive. If these tools are helpful for you, please consider giving them a star on GitHub, joining their communities, and contributing to them.

In part 2, we will look at popular libraries for writing AWS Lambda functions in Node.js and we will explore useful AWS Serverless Application Repository apps.

The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.

Feature image via Pixabay.