Why Use AWS Lambda in a Custom VPC?

By Akash Jain. Akash is a Partner Solutions Architect (SA) at AWS. 

As a Partner Solutions Architect (SA), I work closely with APN Partners as they look to use AWS services in innovative ways to address their customers’ use cases. Recently, I came across an interesting use case with an APN Partner who configured an AWS Lambda function to access resources running in a custom virtual private cloud (VPC) to call an internal API over the virtual private network (VPN). In this post, I’ll walk through how this APN Partner is evolving an existing architecture to take advantage of AWS Lambda’s capabilities and to optimize the managed services they provide to their customers.

Existing architecture

For those who are new to AWS Lambda, it is a compute service that lets you run code (written up as “Lambda functions”) without provisioning or managing servers. These functions execute in response to a wide variety of AWS service events and custom events, and can be used in a variety of scenarios. AWS Lambda executes your code only when needed and scales automatically from a few requests per day to thousands per second. With AWS Lambda, you pay only for the requests served and the compute time required to run your code.

The particular use case I’ll discuss today involves an APN Partner who needed to integrate a customer’s AWS environment with their own on-premises environment to provide managed services. Several VPN connections were set up between their on-premises environment and different AWS Regions. As a part of the integration, all system alerts in a region needed to be consolidated in one place. In this case, that was the APN Partner’s on-premises environment.

To make this happen, they set up Amazon CloudWatch alerts to trigger a Lambda function. The job of the Lambda function was to call an externally hosted web service and pass the alert as payload. The web service could then convert the CloudWatch alerts to a format that the Netcool API, which was hosted on premises, could understand.

The following diagram outlines the setup of this architecture. For simplicity’s sake, I’ve chosen not to represent components like subnets, customer gateway, and VPN gateway.

After we reviewed this architecture with the APN Partner, they chose to re-evaluate and optimize it, for a few reasons:

• Extra cost – A dedicated system (VM) was in place to host the web service, and its job was to convert the message from the CloudWatch alert format to the Netcool API format. Getting the VM, OS, and other software in place required an upfront cost.
• Maintenance – Managing and maintaining this server added an extra layer of maintenance. The team had to patch the server regularly to keep it up-to-date.
• Security complexity – The API for converting the format was exposed externally, so it resulted in an additional security layer for authentication, authorization, and DoS/DDoS protection.
• Low scalability – The web service could not auto-scale because it was installed on a single VM.
• Fault tolerance – If the server went down, all the alerts would be lost.

Accessing resources in a VPC from a Lambda function

Working with the APN Partner, we decided to take advantage of AWS Lambda in a way that would alleviate these concerns with the existing architecture. We asked ourselves two questions: “What if we move the format conversion logic of the web service into AWS Lambda itself?” and then, “How can the modified Lambda function call the Netcool API, which is not exposed externally?”

The answer is to access resources in a VPC from an AWS Lambda function, a helpful feature that was introduced by AWS in early 2016. With this feature, the AWS Lambda function can call the Netcool API over the existing VPN connection, which was established for secure administrative access. When the Lambda function accesses resources in a VPC, it gets a private IP address from the VPC. It can then communicate with any service within the VPC or with any other system accessible from that VPC.

The benefits of this approach are:

• Manageability – The Lambda function automatically runs your code without requiring you to provision or manage servers. This means that your support team can focus on important system alerts instead of managing and maintaining the infrastructure around it.
• Minimized cost – You pay for what you use. For Lambda, you’re charged based on how many requests the Lambda function receives and how long your code executes. Since we’re working with system alerts in this scenario, I don’t expect the Lambda function to be more expensive than monthly charges for running a server.
• Security – Because the Lambda function is VPC-enabled, all communications between AWS and the on-premises environment will be over a secure tunnel.
• High scalability – Lambda can launch as many copies of the function as needed to scale to the rate of incoming events.
• Fault tolerance – Lambda can be a part of multiple subnets spanning multiple Availability Zones.

Lambda functions automatically scale based on the number of events they process. For VPC-enabled Lambda functions, you should make sure that your subnet has enough elastic network interfaces (ENIs) and IP addresses to meet the demand. For details on calculating ENI capacity, see the AWS Lambda documentation.

A Lambda function enabled to access resources within a VPC may take some time to instantiate, because an ENI needs to be initialized and attached to it. In the case of inactivity over a long period of time, the alerts may take some time to process. A workaround I suggest is to keep the Lambda function warm by triggering dummy CloudWatch alerts, as explained in a post on the A Cloud Guru blog.

Driving managed service optimization

As the APN Partner in this example is a Managed Service Provider (MSP), I’d like to tie this example back to how next-gen MSPs can drive workload optimizations and cost efficiencies for their customers.

Service delivery quality is a key value next-gen MSPs bring to customers. An essential goal for MSPs is to try to develop the tools, processes, and governance required to deliver reliable services cost-effectively. By accessing resources in a custom VPC from a Lambda function and leveraging an existing VPN connection, an MSP can send alerts more securely, reliably, and cost-effectively.

Conclusion

In this post, we briefly discussed the benefits of running Lambda functions that can access resources inside your private VPC. Your Lambda functions can access Amazon Redshift data warehouses, Amazon ElastiCache clusters, Amazon Relational Database Service (Amazon RDS) instances, and service endpoints that are accessible only from within a particular VPC, such as resources over VPN. I recommend that you keep sufficient ENI and IP addresses under a subnet for auto-scaling purposes, and keep your Lambda function warm if you need a quicker response in case of longer inactivity.

For more information, I recommend that you take a look at the AWS documentation. Do you have comments? Talk to me in the comments section. I’d love to hear from you.