Deploying application configuration to serverless: Introducing the AWS AppConfig Lambda extension

At AWS, we feel strongly that separating application configuration from application code is a best practice. Being able to deploy configuration independently from code makes it possible to build services like Service Quotas and launch new services and features right as we announce them. If we didn’t separate these, even a simple configuration change would result in a significantly longer and more involved build, test, and deployment process. This is not to say we consider configuration changes any less important than we do code changes, only that the requirements and mechanisms are different than those for code.

It was with this in mind that we launched AWS AppConfig, an externalized version of the tool we use internally to implement this best practice. AWS AppConfig allows customers to manage dynamic application configuration and includes:

Support for validation of configuration data before it starts rolling out to your applications.
Deployment strategies that specify the duration and rate at which updated configurations are available to your application.
Additional monitoring during a deployment that can roll back a change as soon as a threshold is breached.

You can already use AWS AppConfig with your Lambda-based serverless apps. We want to make it even easier for you by providing a Lambda extension that manages the interaction between your functions and AWS AppConfig. This extension runs as an agent alongside your function. It checks for new configuration data on a periodic basis and provides the latest to your code right as you need it. We’ll talk a bit more about what the extension offers later in the post.

Let’s get started

In this blog post, we create a simple Hello World function. By default, our function responds “Hello world!” when invoked. When we’re feeling especially enthusiastic, though, we’d like to up the number of exclamation points and say something like “Hello world!!!!” We use dynamic configuration so we can easily update this behavior without needing to go through an edit, commit, build, test, and deploy process for every change.

We have found that it’s best to manage a configuration like this with two values: a Boolean that indicates if a configurable capability is enabled and the value for the configurable attribute. This Boolean+value approach provides better organization and hygiene of your configurations. It makes your configurations easier to manage because it’s clear what each configuration does and how to adjust it. For our application, we have two attributes for our exclamation points: enableExclamationPoints and numberOfExclamationPoints. With that, on to some code!!!! (Sorry, we couldn’t help ourselves).

Step 1: Create a dynamically configurable “Hello World” Lambda function

To highlight the differences (and benefits) of using AWS AppConfig and the AWS AppConfig Lambda extension, we start with a version of our function that, while configurable, uses Lambda environment variables instead.

Open the AWS Lambda console and on the Functions page, choose Create function.
On the next page, keep Author from scratch selected. Enter a name for the function (for example, hello world). From the Runtime dropdown menu, choose Python 3.8. Let Lambda create a new execution role specific to this function, and then choose Create function.

In a few seconds, the console displays a page with a Function code section. Paste the following code into the editor window (replace any code that might already be there) and choose Deploy.

import os

def lambda_handler(event, context):
    num_exclamations = get_configured_number_of("ExclamationPoints", 1)
   
    return f'Hello world{"!" * num_exclamations}'

def get_configured_number_of(configuration_type, default):
    if os.getenv("enable" + configuration_type, "False") == "True":
        return int(os.getenv("numberOf" + configuration_type, str(default)))
    else:
        return default

On the upper right, choose Test. If this is the first time you’ve tested a Lambda function from the console, you are prompted to create a test event. You can accept the default that’s there (it’s not used by our function), give it a name, and then choose Create. If you already have a test event, you can reuse it. Choose Test again. A message that says Execution result: succeeded should appear. Expand the details and confirm it shows “Hello world!” as the result.

Step 2: Add the Lambda environment variables

Although it’s working, our application isn’t yet using any dynamic configuration data. It’s okay because, by default, it falls back to a value of 1. To update things, we must add two environment variables.

In the AWS Lambda console, scroll down to the Environment variables section. Choose Edit or Manage environment variables. (Both do the same thing.)
On the next page, choose Add environment variable. Under Key, enter enableExclamationPoints. Under Value, enter True.
Choose Add environment variable again. Under Key, enter numberOfExclamationPoints. Under Value, enter 2. Your entries should look like this:

Figure 1: Edit environment variables
Choose Save and then choose Test again. The details in the new execution result should now show “Hello world!!”

At this point, we might be feeling good about updating our function’s behavior without making any code changes, but there are a couple of things to be aware of.

First, even when just an environment variable change is made to a function, Lambda stops using all of its existing execution contexts that contain the prior values and starts new execution contexts with the new ones. So even though there’s no code change, there is still a cold start penalty every time you make a change to an environment variables.

Second, environment variable updates can only be made to your unpublished function code (that is, $latest). If you are using function versions (a recommended practice to ensure you have immutable versions of your function that can’t accidentally break), you’ll first want to ensure the current unpublished code is the code you think it is. Then, make any environment variable changes you want while specifying the unpublished function’s revision-id. Then, test it to make sure they values are valid. Finally, publish the function with your new environment variables as a new version, again specifying the function’s revision-id to confirm that everything is what you think it is.

Oh, and aliases. If you’re using aliases so callers of your function don’t need to change anything on their end when you publish an update, you’ll need to change the alias to refer to the newly published version. That’s a fair bit of choreography just to update a configuration value. Let’s switch over to AWS AppConfig and its extension to see what it can do for us.

Step 3: Add the AWS AppConfig Lambda extension to the function

A Lambda extension is a special type of Lambda layer that is built to interact with Lambda’s extension API. They have a lifecycle related to but independent from your function. To add the extension, let’s go back to the AWS Lambda console page for the function we just created.

In the Designer section, choose Layers, as shown in Figure 2.

Figure 2: Layers section of the AWS Lambda console
Choose Add a layer, and on the subsequent Add layer page, choose Specify an ARN. Find the AWS AppConfig extension ARN for your region here, and paste that value into the input field.

Figure 3: Add layer page of the AWS Lambda console
Choose Add. The previous page is displayed. You should now see a 1 on the right side of the Layers button. This indicates that the layer is now included with your function.

Next, we need to give our Lambda function (well, technically the extension layer we just added) the permission to call the AWS AppConfig StartConfigurationSession and GetLatestConfiguration APIs.

In the AWS Lambda console, choose the Configuration tab, then Permissions.
In the Execution role section, verify that the role name corresponds to the role you created for this function. It should be of the form <your_function_name>-role-<random_string>.
If the specified role does not correspond to this function, under Existing role, choose the role you created at the start of this post and then choose Edit. When you choose the Role name link, the IAM console opens and the Summary page for this role is displayed.
On the Summary page, expand the section that contains the policy name that starts with AWSLambdaBasicExecutionRole- and then choose Edit.
On the next page, on the Visual editor tab, choose Add additional permissions.
In the Select a service section, for Service, choose AppConfig. For Actions, expand Read, and then choose GetLatestConfiguration. Expand Write, and then choose StartConfigurationSession. Expand Resources and select the Any in this account.
At the bottom of the page, choose Review policy, and then choose Save changes.

Step 4: Update your Lambda function to use the AWS AppConfig extension

Our last step in the AWS Lambda console is to update the function code with a new version that interacts with the AWS AppConfig extension instead of Lambda environment variables. Before we do that, though, let’s quickly go over a bit about how the extension works.

When the AWS AppConfig extension starts, two main components are created:

The first, the proxy, exposes a localhost HTTP endpoint that can be called from your Lambda code to retrieve a piece of configuration data. The proxy does not call AWS AppConfig directly. Instead, it uses an extension-managed cache that contains the freshest configuration data available. Because the data is already available locally, the HTTP endpoint can be called on every invocation of your function (or even multiple times if you have long-running functions that you want to update mid-flight).

The second component, the retriever, works in the background to keep your configuration data fresh. It checks for potential updates even before your code asks for it. It tracks which configurations your function needs, whether the data is potentially stale, and makes appropriate calls to AWS AppConfig to retrieve fresher data, if available. It ensures the right metadata is passed to avoid any unnecessary data delivery and support various types of rollout strategy.

The determination of “how fresh is fresh” can be configured using…Lambda environment variables! Wait! Before you flip a table, realize that these values change rarely, if ever, over the lifetime of a function, so it is much less dynamic than the application-specific data we’ve been talking about.

Here are the three extension configuration environment variables:

AWS_APPCONFIG_EXTENSION_POLL_INTERVAL_SECONDS, which defaults to 45 seconds, specifies the frequency with which the extension checks for new configuration data.
AWS_APPCONFIG_EXTENSION_POLL_TIMEOUT_MILLIS, which defaults to 3000 milliseconds, specifies the maximum time the extension waits for a piece of configuration data before giving up and trying again during the next poll interval.
AWS_APPCONFIG_EXTENSION_HTTP_PORT, which defaults to 2772, specifies the port that the proxy’s HTTP endpoint uses.

Okay, to the code. As you now understand, the way to get data from the AWS AppConfig extension is to make a call to the localhost proxy endpoint, so head back to the Lambda editor for your function and replace the current code with the following code. (The lambda_handler is the same. Only get_configured_number_of has been changed.)

import json
import urllib.request

def lambda_handler(event, context):
    num_exclamations = get_configured_number_of("ExclamationPoints", 1)
   
    return f'Hello world{"!" * num_exclamations}'

def get_configured_number_of(configuration_type, default):
    try:
        url = f'http://localhost:2772/applications/helloworld/environments/demo/configurations/' + configuration_type
        config = json.loads(urllib.request.urlopen(url).read())
        
        if config.get("enable" + configuration_type, False):
            return config.get("numberOf" + configuration_type, default)
        else:
            return default
    except:
        return default

In the URL, we’ve hardcoded the values for application (helloworld) and environment (demo). We’ll use these in our final step, but for now, choose Deploy, and then choose Test. The function should again return “Hello world!” just as it did at the start of the post.

Step 5: Setting up and using AWS AppConfig

For our last step, we want to set up AWS AppConfig so we can update and deploy our configuration data. Because any changes deployed through AWS AppConfig do not result in a Lambda cold start, we can do this as often as we want.

In a new window or tab, open the AWS AppConfig console.
If this is your first time, choose Create configuration data. Otherwise, choose Create application.
For the application name, enter the value in the URL (helloworld) and then choose Create application.
This opens a page with two tabs. Environments should already be selected.
Chose Create environment. For a name, use the other URL value (demo). If you expand the Monitors section, you can see where you can add one or more Amazon CloudWatch alarms to your environment. These alarms are monitored while a configuration deployment is underway. If an alarm indicates a problem, AWS AppConfig rolls back the change immediately. We’ll skip setting up alarms, but we wanted to point them out for future reference.
Choose Create environment, and from the breadcrumb on the next page, choose helloworld.
You should now be back on the page with the two tabs where Environments is selected. Choose the Configuration profiles tab and then choose Create configuration profile. For the name of our configuration profile, we want to use the value that we have in our call to get_configured_number_of. Enter ExclamationPoints and then choose Next.
On this page, you see options for Configuration source. Use the default, AWS AppConfig hosted configuration. In the Content section, choose JSON for the type, paste the following into the content box, and then choose Next.
```
{
  "enableExclamationPoints": true,
  "numberOfExclamationPoints": 5
}
```
On this last page, you can configure zero or more validators to run before a deployment is successfully started. These validators can provide confidence that your configuration is correct syntactically (through JSON schema) or semantically (through Lambda) before it is ever made available to your application. If you want to check it out, choose the JSON schema option, paste the following, and then choose Create configuration profile.
```
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "enableExclamationPoints": {
      "type": "boolean"
    },
    "numberOfExclamationPoints": {
      "type": "integer",
      "minimum": 0
    }
  },
  "required": [
    "enableExclamationPoints",
    "numberOfExclamationPoints"
  ]
}
```
Now that we have everything set up, let’s start a deployment. Choose Start deployment.
On the next page, for the environment, choose demo. For Hosted configuration version, choose 1. For Deployment strategy, choose AppConfig.Linear50PercentEvery30Seconds. Choose Start deployment.
Go back to the AWS Lambda console page for your function and choose Test. Depending on the client ID that the AppConfig extension generated during initialization, your function might receive the deployed configuration data right away, or it may need to wait 45 seconds to receive the new data (assuming you didn’t change the poll interval). Once your function has the newest configuration, you’ll see the result in your function’s Test response. Feel free to play around with new configuration values, different deployment strategies, and so on. If you added the JSON Schema validator, try putting a bad value (for example, -3) for numberOfExclamationPoints and try to deploy it.

By now you’ve seen just how easy it is to add dynamic configuration to your Lambda functions. If you wanted to add another type of configuration (for example, how many extra o’s in hello), update your lambda_handler with this:

num_extra_os = get_configured_number_of("ExtraOs", 0)

return f'Hello{"o" * num_extra_os} world{"!" * num_exclamations}'

And create a new configuration profile named ExtraOs:

{
  "enableExtraOs": true,
  "numberOfExtraOs": 12
}

Deploy your Lambda function and the new configuration profile.

Conclusion

Beyond adding exclamation points and o’s, AWS AppConfig can be used for more, well, useful things. Our customers have been using AWS AppConfig for:

Feature flags: You can dark deploy features onto production that are hidden behind a feature flag. Toggling the feature flag turns on the feature immediately, without doing another code deployment.
Allow lists: You might have some features in your app that are for specific callers only. Using AWS AppConfig, you can control access to those features and dynamically update access rules without another code deployment
Verbosity of logging: You can control how often an event occurs by setting a variable limit. For example, you can adjust the verbosity of your logging during a production incident to better analyze what is going on. You would want to do another full deployment in the case of a production incident, but a quick configuration change gets you what you need..

There are many other use cases for AWS AppConfig. Remember that the value is not just in externalizing your configuration from your app, but also the management of those configurations, including validations, rollbacks, and deployment strategies.

Additional resources for AWS AppConfig and Lambda Extensions below. Happy extending!

About the Authors

	Steve Rice is the Principal Product Manager for AWS AppConfig. He loves building products that improve people’s lives, and is passionate about dynamic configuration and helping developers get their code to users as quickly and safely as possible.
	Jonathan (JT) Thompson is a Principal Engineer at AWS. He has worked with many teams across AWS to build and extend their services, with a particularly deep focus on API Gateway, Lambda, and, most recently, AWS AppConfig.
	Tim Yao is a Software Development Engineer within AWS AppConfig. He is an avid Go developer and has industry experience in IoT and LOB application development.

AWS Cloud Operations Blog