Announcing nested applications for AWS SAM and the AWS Serverless Application Repository

Serverless application architectures enable you to break large projects into smaller, more manageable services that are highly reusable and independently scalable, secured, and evolved over time. As serverless architectures grow, we have seen common patterns that get reimplemented across companies, teams, and projects, hurting development velocity and leading to wasted effort. We have made it easier to develop new serverless architectures that meet organizational best practices by supporting nested applications in AWS SAM and the AWS Serverless Application Repository.

How it works

Nested applications build off a concept in AWS CloudFormation called nested stacks. With nested applications, serverless applications are deployed as stacks, or collections of resources, that contain one or more other serverless application stacks. You can reference resources created in these nested templates to either the parent stack or other nested stacks to manage these collections of resources more easily.

This enables you to build sophisticated serverless architectures by reusing services that are authored and maintained independently but easily composed via AWS SAM and the AWS Serverless Application Repository. These applications can be either publicly or privately available in the AWS Serverless Application Repository. It’s just as easy to create your own serverless applications that you then consume again later in other nested applications. You can access the application code, configure parameters exposed via the nested application’s template, and later manage its configuration completely.

Building a nested application

Suppose that I want to build an API powered by a serverless application architecture made up of AWS Lambda and Amazon API Gateway. I can use AWS SAM to create Lambda functions, configure API Gateway, and deploy and manage them both. To start building, I can use the sam init command.

$ sam init -r python2.7
[+] Initializing project structure...
[SUCCESS] - Read sam-app/README.md for further instructions on how to proceed
[*] Project initialization is now complete

The sam-app directory has everything that I need to start building a serverless application.

$ tree sam-app/
sam-app/
├── hello_world
│   ├── app.py
│   ├── app.pyc
│   ├── __init__.py
│   ├── __init__.pyc
│   └── requirements.txt
├── README.md
├── template.yaml
└── tests
    └── unit
        ├── __init__.py
        ├── __init__.pyc
        ├── test_handler.py
        └── test_handler.pyc

3 directories, 11 files

The README in the sam-app directory points me to using the new sam build command to install any requirements of my application.

$ cd sam-app/
$ sam build
2018-11-21 20:41:23 Building resource 'HelloWorldFunction'
2018-11-21 20:41:23 Running PythonPipBuilder:ResolveDependencies
2018-11-21 20:41:24 Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml
...

At this point, I have a fully functioning serverless application based on Lambda that I can test and debug using the AWS SAM CLI locally. For example, I can invoke my Lambda function directly, as in the following code.

$ cd .aws-sam/build/
$ sam local invoke --no-event
2018-11-21 20:43:52 Invoking app.lambda_handler (python2.7)

....trimmed output....

{"body": "{\"message\": \"hello world\", \"location\": \"34.239.158.3\"}", "statusCode": 200}

Or I can use the actual API Gateway interface for it with the sam local start-api command. I can also now use the sam package and sam deploy commands to get this application running on Lambda and begin letting my customers consume it. What I really want to do, though, is expand on this API by adding an authorization mechanism to add some security. API Gateway supports several methods for doing this, but in this case, I want to leverage a basic form of HTTP Basic Auth. Although not the best way to secure an API, this example highlights the power of nested applications.

To start, I search an existing serverless application that meets my needs. I can access the AWS Serverless Application Repository either directly or via the AWS Lambda console. Then I search for “http basic auth,” as shown in the following image.

There is already an application for HTTP Basic Auth that someone else made.

I can review the AWS SAM template, license, and permissions created in the AWS Serverless Application Repository. Reading through the linked GitHub repository and README, I find that this application meets my needs, and I can review its code. The application enables me to store a user’s username and password in Amazon DynamoDB and use them to provide authorization to an API.

I could deploy this application directly from the console and then plug the launched Lambda function into my API Gateway configuration manually. This would work, but it doesn’t give me a way to more directly relate the two applications together as one application, which to me it is. If I remove the authorizer application, my API will break. If I decide to shut down my API, I might forget about the authorizer. Logically thinking about them as a single application is difficult. This is where nested applications come in.

The launch of nested applications comes with a new AWS SAM resource, AWS::Serverless::Application, which you can use to refer to serverless applications that you want to nest inside another application. The specification for this resource is straightforward and at a minimum resembles the following template.

application-alias-name:
  Type: AWS::Serverless::Application
  Properties:
    Location:
    Parameters:

For the full resource specification, see AWS::Serverless::Application on the GitHub website.

Currently, the location can be one of two places. It can be in the AWS Serverless Application Repository.

Location:
  ApplicationId: arn:aws:serverlessrepo:region:account-id:applications/application-name
  SemanticVersion: 1.0.0

Or it can be in Amazon S3.

Location: https://s3.region.amazonaws.com/bucket-name/sam-template-object

The sam-template-object is the name of a packaged AWS SAM template.

Parameters is the parameters that the nested application requires at launch as defined by its template. You can discover them via the template that the nested application references or in the console under Application Settings. For this Basic HTTP Auth application, there are no parameters, and so there is nothing for me to do here.

We have included a feature to help you to figure out what you need in your template. First, navigate to the Review, configure and deploy page by choosing Deploy from the Application details page or when you choose an application via the Lambda console’s view into the app repository. Then choose Copy as SAM Resource.

This button copies to your clipboard exactly what you need to start nesting this application. For this application, choosing Copy as SAM Resource resulted in the following template.

lambdaauthorizerbasicauth:
  Type: AWS::Serverless::Application
  Properties:
    Location:
      ApplicationId: arn:aws:serverlessrepo:us-east-1:560348900601:applications/lambda-authorizer-basic-auth
      SemanticVersion: 0.2.0

Note: The name of this resource defaults to lambdaauthorizerbasicauth, but best practice is to give it a name that is more descriptive for your use case.

If I stopped here and launched the application, this second application’s stack would launch as a nested application off my application. Before I do that, I want to use the function that this application creates as the authorizer for the API Gateway endpoint created in my existing AWS SAM template. To reference this Lambda function, I can check the template for this application to see what (if any) outputs are created.

I can get the Amazon Resource Name (ARN) from this function by referencing the output named LambdaAuthorizerBasicAuthFunction directly as an attribute of the nested application.

!GetAtt lambdaauthorizerbasicauth.Outputs.LambdaAuthorizerBasicAuthFunction

To configure the authorizer for my existing function, I create an AWS::Serverless::Api resource and then configure the authorizer with the FunctionArn attribute set to this value. The following template shows the entire syntax.

MyApi:
  Type: AWS::Serverless::Api
  Properties:
    StageName: Prod      
    Auth:
      DefaultAuthorizer: MyLambdaRequestAuthorizer
      Authorizers:
        MyLambdaRequestAuthorizer:
          FunctionPayloadType: REQUEST
          FunctionArn: !GetAtt lambdaauthorizerbasicauth.Outputs.LambdaAuthorizerBasicAuthFunction
          Identity:
            Headers:
              - Authorization

This creates a new API Gateway stage, configures the authorizer to point to my nested stack’s Lambda function, and specifies what information is required from clients of the API. In my actual function definition, I refer to the new API stage definition and the authorizer. The last three lines of the following template are new.

Events:
   HelloWorld:
      Type: Api
      Properties:
        Method: get
        Path: /hello
        RestApiId: !Ref MyApi
        Auth:
          Authorizers: MyLambdaRequestAuthorizer

Finally, because of the code created by the sam init command that I ran earlier, I need to update the HelloWorldApi output to use my new API resource.

Outputs:
  HelloWorldApi:
    Description: API Gateway endpoint URL for Prod stage for Hello World function
    Value:
      Fn::Sub: https://${MyApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/

At this point, my combined template file is 70 lines of YAML that define the following:

My Lambda function
The API Gateway endpoint configuration that I’ll use to interface with my business logic
The nested application that represents my authorizer
Other bits such as the outputs, globals, and parameters of my template

I can use the sam package and sam deploy commands to launch this whole stack of resources.

sam package --template-file template.yaml --output-template-file packaged-template.yaml --s3-bucket my-bucket

Successfully packaged artifacts and wrote output template to file packaged-template.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file packaged-template.yaml --stack-name <YOUR STACK NAME>

To deploy applications that have nested applications that come from the app repository, I use a new capability in AWS CloudFormation called auto-expand. I pass it in by adding CAPABILITY_AUTO_EXPAND to the –capabilities flag of the deploy command.

sam deploy --template-file packaged-template.yaml --stack-name SimpleAuthExample --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - SimpleAuthExample

With the successful creation of my application stack, I can find what I created. The stack can be in one of two places: the AWS CloudFormation console or the Lambda console’s Application view. Because I launched this application via AWS SAM, I can use the Application view to manage serverless applications. That page displays two application stacks named SimpleAuthExample.

The second application stack has the appended name of the nested application that I launched. Because it was also a serverless application launched with AWS SAM, the Application view enables me to manage them independently if I want. Selecting the SimpleAuthExample application shows me its details page, where I can get more information, including the various launched resources of this application.

There are four top-level resources or resource groupings:

My Lambda function, which expands to show the related permissions and role for the function
The nested application stack for my authorizer
The API Gateway resource and related deployment and stage
The Lambda permission that AWS SAM created to allow API Gateway to use my nested application’s Lambda function as an authorizer

Selecting the logical ID of my nested application opens the AWS CloudFormation console, where I note its relation to the root stack.

Testing the authorizer

With my application and authorizer set up, I want to confirm its functionality. The Application view for the authorizer application shows the DynamoDB table that stores my user credentials.

Selecting the logical ID of the table opens the DynamoDB console. In the Item view, I can create a user and its password.

Back in the Application view for my main application, expanding the API Gateway resource displays the physical ID link to the API Gateway stage for this application. It represents the API endpoint that I would point my clients at.

I can copy that destination URL to my clipboard, and with a tool such as curl, I can test my application.

curl -u foo:bar https://dw0wr24jwg.execute-api.us-east-1.amazonaws.com/Prod/hello
{"message": "hello world", "location": "34.239.119.48"}

To confirm that my authorizer is working, I test with bad credentials.

curl -u bar:foo https://dw0wr24jwg.execute-api.us-east-1.amazonaws.com/Prod/hello
{"Message":"User is not authorized to access this resource with an explicit deny"}

Everything works!

To delete this entire setup, I delete the main stack of this application, which deletes all of its resources and the nested application as well.

Conclusion

With the growth of serverless applications, we’re finding that developers can be reusing and sharing common patterns, both publicly and privately. This led to the release of the AWS Serverless Application Repository in early 2018. Now nested applications bring the additional benefit of simplifying the consumption of these reusable application components alongside your own applications. This blog post demonstrates how to find and discover applications in the AWS Serverless Application Repository and nest them in your own applications. We have shown how you can modify an AWS SAM template to refer to these other components, launch your new nested applications, and get their improved management and organizational capabilities.

We’re excited to see what this enables you to do, and we welcome any feedback here, in the AWS forums, and on Twitter.

Happy coding!

AWS Compute Blog

Announcing nested applications for AWS SAM and the AWS Serverless Application Repository

How it works

Building a nested application

Testing the authorizer

Conclusion

Resources

Follow