AWS Cloud Operations Blog

Building a central Amazon CloudWatch Dashboard to monitor Lambda@Edge logs and metrics

Introduction

Lambda@Edge is a powerful feature of Amazon CloudFront that allows you to execute serverless code closer to your application users, resulting in improved performance and reduced latency. By distributing Lambda@Edge functions to edge locations worldwide, AWS ensures that the code executes closer to end users, providing faster response times. Moreover, the serverless nature of Lambda@Edge eliminates the need to manage application compute infrastructure in multiple locations, and you only pay for the compute time you consume. However, with the distributed compute footprint, Lambda@Edge logs are stored in Amazon CloudWatch in the Region closest to where the Lambda@Edge function executes. This dispersal of logs across different Regions in Amazon CloudFront makes it challenging to have a unified view of errors and metrics for Lambda@Edge executions.

In this blog post, we will explore a solution that allows us to collect logs, errors, and metrics from Lambda@Edge running in various regional edge caches and present them in a centralized Amazon CloudWatch dashboard. The centralization of logs and metrics in Amazon CloudWatch ensures ease of monitoring, simplifies troubleshooting, and enables you to make data-driven decisions to optimize your web application’s performance and user experience.

Prerequisites

Before proceeding, make sure you have an AWS account with the necessary roles and policies to create and manage Amazon CloudFront, Amazon CloudWatch, AWS Lambda, and AWS IAM resources.

Solution Overview

The diagram below illustrates Lambda@Edge execution in different regions, each logging to its respective Region in Amazon CloudWatch. As the number of regions increases, it becomes essential to set up similar alarms in each Region, making the process inefficient.

Lambda@Edge executing in different AWS Regions create AWS CloudWatch logs locally in the Region where they execute.
Figure 1: Lambda@Edge in different regions

In this blog post, we will use a simple use-case scenario to demonstrate how to monitor Lambda@Edge executions.

Use Case Scenario for Lambda@Edge

  1. We have a web application with various functions, each enabled by a different REST API.
  2. Access to web resources for application users is managed by a custom Authorization API.
  3. When a user logs in to the application and navigates to the home page, the application is loaded from CloudFront.
  4. When a user attempts to access a web resource, the request passes through Amazon CloudFront, where a Lambda@Edge function checks user authorization by calling the Authorization API.

– If the user has access to the resource, the request is allowed to proceed to the origin.

– If the user does not have access to the resource, the AWS Lambda generates an error and denies the request.

Requirements

To monitor the Lambda@Edge executions effectively, we need the following:

  1. Visibility of the AWS Regions where Lambda@Edge functions are running.
  2. A single dashboard displaying all the Regions and the number of authorization denials in each Region.
  3. Setting up Amazon CloudWatch alerts to trigger when authorization denials exceed a certain threshold.
This image represents the logic for a single AWS Region. Based on the users authorization outcome that is performed by a custom authorization service, the user will either be granted access to the resource or the Lambda@Edge will deny access. The same flow of logic happens in every AWS Region, that the Lambda@Edge is running in.
Figure 2: Lambda@Edge Use Case

                                                     

Let’s dive into the technical aspects of the proposed architecture step by step:

Lambda@Edge Invocation

When a user accesses your web application through Amazon CloudFront, CloudFront triggers the appropriate Lambda@Edge function associated with the request. These Lambda@Edge functions are automatically deployed to multiple AWS Regions to ensure they execute closer to the end users, reducing latency and improving performance.

Logging to CloudWatch

As the Lambda@Edge functions execute, they generate logs and errors. By default, these logs are stored in Amazon CloudWatch log groups specific to each AWS Region where the Lambda@Edge executes. Each log group contains logs generated by the Lambda@Edge function within that Region. This regional segregation of logs ensures that logs from different locations are isolated for efficient monitoring and analysis.

Subscription Filter for Log Aggregation

To consolidate the logs and errors from multiple regional log groups into a central location, we use Amazon CloudWatch Logs Subscription Filters. For each log group, we add a Subscription Filter that specifies a specific string pattern. The Subscription Filter is configured to forward log events that match the specified pattern to a central AWS Lambda function, called “Subscription Lambda.” This central AWS Lambda function will act as a log aggregator.

Centralized “Subscription Lambda” Function

The “Subscription Lambda” function is deployed in the primary region or a designated centralized region. It is designed to receive log events from various log groups across different Regions. As these log events arrive, the “Subscription Lambda” inspects each event, extracts relevant information, and generates custom metrics based on the content of the logs. These custom metrics represent aggregated data from all Lambda@Edge executions across different Regions.

Custom Metrics to CloudWatch

Once the “Subscription Lambda” function has generated the custom metrics, it sends these metrics to Amazon CloudWatch. These custom metrics are now available for visualization and monitoring through the Amazon CloudWatch service.

Centralized CloudWatch Dashboard

To provide a unified view of Lambda@Edge executions across multiple Regions, we create a centralized Amazon CloudWatch dashboard. This dashboard incorporates the custom metrics generated by the “Subscription Lambda” function, presenting a comprehensive overview of the Lambda@Edge performance and any encountered errors.

By following this architecture, you can efficiently monitor Lambda@Edge executions, gain valuable insights into their performance, and proactively address any issues that may arise. The centralization of logs and metrics in Amazon CloudWatch ensures ease of monitoring, simplifies troubleshooting, and enables you to make data-driven decisions to optimize your web application’s performance and user experience.

The Final Architecture. In here, the AWS CloudWatch logs that are generated in each AWS Region will be forwarded to a central Subscription Lambda based on the AWS CloudWatch Subscription Filter. The Subscription Lambda will generate custom metrics and forward in it’s own AWS Region’s CloudWatch, thus unifying all the logs generated in different Regions into a single, central Region.
Figure 3: Centralize logging with Subscription Filters

                                        

Cost Considerations

When you use a subscription filter, data is transferred from the Amazon CloudWatch logs in multiple Regions worldwide to the Region where you deploy the subscription Lambda. Data Transfer out from Amazon CloudWatch Logs is priced the same as Data Transfer out from Amazon EC2 and Data Transfer out from Amazon EC2 to Internet. This EC2 Pricing page has details about Data Transfer costs.

To minimize the cost of cross-Region data transfer

  1. Deploy the AWS Lambda in the Region where you expect most of your traffic to be.
  2. Optimize your subscription filter to only select the precise log events that are required for the dashboard.

Cleanup

It’s essential to clean up resources after completing the monitoring exercise. This involves deleting any unused log groups, AWS Lambda functions, and other associated resources.

Conclusion

Monitoring Lambda@Edge executions across various regional edge caches is crucial to ensure optimal performance and identify potential issues proactively. By centralizing logs and metrics into a single Amazon CloudWatch Dashboard, we gain valuable insights and simplify the monitoring process. AWS provides a robust set of tools and services that facilitate this monitoring task, and by following the steps outlined in this blog post, you can successfully build a central Amazon CloudWatch Dashboard to monitor Lambda@Edge Logs and Metrics efficiently. Learn more about Lambda@Edge in our Lambda Developer Guide. To learn more about how to create custom CloudWatch Metrics and build Dashboards, visit our CloudWatch Developer Guide.

About the authors

Samrat Lamichhane

Samrat is a Sr. Solutions Architect and has been at AWS for 4+ years. He has worked with multiple segments of customers from Enterprise, SMB and Startup, where he helped our customers architect their solutions and migrate to AWS. He specializes in Serverless, Storage and Configuration, Compliance, and Auditing in AWS.

Sagar Das

Sagar Das is a Solutions Architect at AWS. Focused on solving business problems, Sagar’s technical skills are targeted at today’s fast changing solutions. In addition, Sagar’s skills are enhanced with excellent communication and personal skills.
Outside of work Sagar loves spending time with his family along with playing Cricket and Tennis whenever possible. Sagar is based out of Atlanta, GA.

Vipan Kumar

Vipan Kumar is Senior Solutions Architect at Amazon Web Services, where he advices strategic customers. He has extensive experience in designing, building, and modernizing enterprise applications. He is passionate about solving complex problems with Microservices, Serverless technologies and Event-Driven architectures.