AWS Lambda Update – Python, VPC, Increased Function Duration, Scheduling, and More

We launched AWS Lambda at re:Invent 2014 and the reception has been incredible. Developers and system architects quickly figured out that they can quickly and easily build serverless systems that need no administration and can scale to handle a very large number of requests. As a recap, Lambda functions can run in response to the following events:

• Changes to Amazon Simple Storage Service (Amazon S3) buckets – see Handling Amazon S3 Events.
• Updates to Amazon DynamoDB tables – see Processing Events from Amazon DynamoDB Streams.
• Messages arriving in Amazon Kinesis streams – see Processing Events from an Amazon Kinesis Stream.
• Notification to Amazon Simple Notification Service (Amazon SNS) topics – see Invoking Lambda Functions Using Amazon SNS Notifications.
• HTTP requests (via the Amazon API Gateway) – see Walkthrough: API Gateway and Lambda Functions.
• Evaluation of AWS CloudFormation templates – see AWS::Lambda::Function.
• Firing of Amazon Cognito events – see Amazon Cognito Events.
• Activation of an Alexa Skill – see Developing an Alexa Skill as a Lambda Function.
• Calls from Amazon Simple Workflow Service (SWF) activities – see AWS Lambda Tasks.
• Incoming email messages – see Receive and Process Incoming Email with Amazon SES.
• CloudWatch Logs – see Near Real-time Processing of Amazon CloudWatch Logs with AWS Lambda.

Over the past year we have added lots of new features to Lambda. We launched in three AWS regions (US East (N. Virginia), US West (Oregon), and Europe (Ireland)) and added support for Asia Pacific (Tokyo) earlier this year. Lambda launched with support for functions written in Node.js; we added support for Java functions earlier this year. As you can see from the list above, we also connected Lambda to many other parts of AWS. Over on the AWS Compute Blog, you can find some great examples of how to put Lambda to use in powerful and creative ways, including (my personal favorite), Microservices Without the Servers.

New Features for re:Invent
Today we are announcing a set of features that will Lambda even more useful. Here’s a summary of what we just announced from the stage:

• VPC Support
• Python functions
• Increased function duration
• Function versioning
• Scheduled functions

As you can see, it is all about the functions! Let’s take a look at each of these new features.

Accessing Resources in a VPC From a Lambda Function
Many AWS customers host microservices within a Amazon Virtual Private Cloud (Amazon VPC) and would like to be able to access them from their Lambda functions. Perhaps they run a MongoDB cluster with lookup data, or want to use Amazon ElastiCache as a stateful store for Lambda functions, but don’t want to expose these resources to the Internet.

You will soon be able to access resources  of this type by setting up one or more security groups within the target VPC, configure them to accept inbound traffic from Lambda, and attach them to the target VPC subnets. Then you will need to specify the VPC, the subnets, and the security groups when your create your Lambda function (you can also add them to an existing function). You’ll also need to give your function permission (via its IAM role) to access a couple of EC2 functions related to Elastic Networking.

This feature will be available later this year. I’ll have more info (and a walk-through) when we launch it.

Python Functions
You can already write your Lambda functions in Node.js and Java. Today we are adding support for Python 2.7, complete with built-in access to the AWS SDK for Python.  Python is easy to learn and easy to use, and you’ll be up and running in minutes. We have received many, many requests for Python support and we are very happy to be able to deliver it. You can start using Python today. Here’s what it looks like in action:

Increased Function Duration
Lambda is a great fit for Extract-Transform-Load (ETL) applications. It can easily scale up to ingest and process large volumes of data, without requiring any persistent infrastructure. In order to support this very popular use case, your Lambda functions can now run for up to 5 minutes. As has always been the case, you simply specify the desired timeout when you create the function. Your function can consult the context object to see how much more time it has available.

Here’s how you can access and log that value using Python:

print(" Remaining time (ms): " + str(context.get_remaining_time_in_millis()) + "\n")

Functions that consume all of their time will be terminated, as has always been the case.

Function Versioning & Aliasing
When you start to build complex systems with Lambda, you will want to evolve them on a controlled basis. We have added a new versioning feature to simplify this important aspect of development & testing.

Each time you upload a fresh copy of the code for a particular function, Lambda will automatically create a new version and assign it a number (1, 2,3, and so forth). The Amazon Resource Name (ARN) for the function now accepts an optional version qualifier at the end (a “:” and then a version number).  An ARN without a qualifier always refers to the newest version of the function for ease of use and backward compatibility. A qualified ARN such as “arn:aws:lambda:us-west-2:123456789012:function:PyFunc1:2” refers to a particular version (2, in this case).

Here are a couple of things to keep in mind as you start to think about this new feature:

• Each version of a function has its own description and configuration (language / runtime, memory size, timeout, IAM role, and so forth).
• Each version of a given function generates a unique set of CloudWatch metrics.
• The CloudWatch Logs for the function will include the function version as part of the stream name.
• Lambda will store multiple versions for each function. Each Lambda account can store up to 1.5 gigabytes of code and you can delete older versions as needed.

You can also create named aliases and assign them to specific versions of the code for a function. For example, you could initially assign “prod” to version 3, “test” to version 5, and “dev” to version 7 for a function. Then you would use the alias as part of the ARN that you use to invoke the function, like this:

• Production – “arn:aws:lambda:us-west-2:123456789012:function:PyFunc1:prod”
• Testing – “arn:aws:lambda:us-west-2:123456789012:function:PyFunc1:test”
• Development – “arn:aws:lambda:us-west-2:123456789012:function:PyFunc1:dev”

You can use ARNs with versions or aliases (which we like to call qualified ARNs) anywhere you’d use an existing non-versioned or non-aliased ARN.  In fact, we recommend using them as a best practice.

This feature makes it easy to promote code between stages or to rollback to earlier versions if a problem arises. For example, you can point your prod alias to version 3 of the code, and then remap it to point to version 5 (effectively promoting it from test to production) without having to make any changes to the client applications or to the event source that triggers invocation of the function.

Scheduled Functions (Cron)
You can now invoke a Lambda function on a regular, scheduled basis. You can specify a fixed rate (number of minutes, hours, or days between invocations) or you can specify a Cron-like expression:

This feature is available now in the console, with API and CLI support in the works.

— Jeff;