Monitor your Lambda function and get notified with AWS Chatbot

AWS Lambda is a serverless compute service that helps you run code without provisioning or managing hardware. You can run AWS Lambda function to execute a code in response to triggers such as changes in data or system state. For example, you can use Amazon S3 to trigger AWS Lambda to process data immediately after an upload. By combining AWS Lambda with other AWS services, developers can build powerful web applications that automatically scale up and down and run in a highly available configuration. Due to its transitory nature and handiness, Lambda has become a popular and integral part of many solutions or architectures. Lambda can have execution failures due to failure in logic or condition in the code or connectivity issue with other services. Such failures in Lambda function can have a significant impact on the overall workflow and architecture.

In this blog post, we describe the process of setting up notifications in Slack using AWS Chatbot to notify you in case of failures of Lambda. AWS chatbot helps you get notification in real time in slack or chime and also enables you to respond swiftly by issuing commands in your slack channel itself.

We don’t describe a comprehensive monitoring solution for Lambda in this blog post. You can visit here to learn details on holistic monitoring of AWS Lambda. Instead, we focus here on catching and notifying Lambda failures quickly on Slack. This solution also allows you to onboard Lambda functions for monitoring based on their tags in a specified AWS account and region.

Lambda functions are well integrated with CloudWatch. You can see runtime logs plus metrics such as number of invocations, duration, error count, and more. With these metrics you can monitor the state and health of your Lambda code and increase the availability of its executions. Here we focus only on the error count metric for lambda to catch its failures.

Solution overview

The solution uses CloudWatch, SNS, and AWS Chatbot as shown below. It leverages metrics stored in Amazon CloudWatch for your lambda function. If error count is more than desired, it triggers the CloudWatch alarm. As soon as alarm triggers, it notifies the topic in AWS SNS. The AWS chatbot which is subscribed to the SNS topic, pulls the information and puts it in slack channel.

Figure 1: Services used in the solution

Let’s look at individual services in details now.

CloudWatch is monitoring and observability service built to collect monitoring and operational data in the form of logs, metrics, and events. Lambda automatically monitors Lambda functions on your behalf and reports metrics through CloudWatch. To help you monitor your code when it runs, Lambda automatically tracks the number of requests, the invocation duration per request, and the number of requests that result in an error, as shown in Figure 2.

Figure 2: CloudWatch metrics

To monitor the failure of the execution of your Lambda function, you would need to create a CloudWatch alarm that is triggered when the value in error count goes above 0. Please see Figure 3 for details on alarm.

Figure 3: CloudWatch alarm

Amazon SNS is a fully managed messaging service. Using SNS topics, you can publish your message to Amazon SQS, Lambda, an HTTPS endpoint, or email. This solution uses SNS to notify you on Slack and through email.

Figure 4: SNS topic

AWS Chatbot is service that manages the integration between AWS services and your Slack channels or Amazon Chime chat rooms. With just a few clicks, you start receiving notifications and can issue AWS CLI commands in a Slack channel to perform operations. You and your team don’t have to switch contexts to collaborate.

Figure 5: aws cli command

The solution is available here to download. It has CloudFormation template with the Lambda code embedded in it so entire solution is deployed through one CloudFormation stack. It has an option to enable monitoring on many Lambda functions based on tags. You can extend this Lambda code to perform other functions while creating the alarms.

Prerequisites

You’ll need the workspace ID and Slack channel ID to establish connectivity from the chatbot to Slack and to send messages to the desired channel.

To get workspace ID, please visit AWS chatbot service in console.

• To start configuring AWS Chatbot with the chat client, in the AWS Chatbot console, choose Configure new client, and then choose Amazon Chime or Slack.
• The setup wizard redirects you to the Slack OAuth 2.0 page. In the upper-right corner, choose the Slack workspace to configure and then choose Agree. Your Slack workspace installs the AWS Slack App.
• Make a note of the workspace ID (for example, T01UXHUCRMW). You will need it later when you deploy the CloudFormation template.

To get the Slack channel ID, please visit desktop app or slack on web. Right-click on your channel and then copy the link. The last portion of the copied link is the channel ID (for example, C01U80K9KPD).

Solution can be deployed to monitor group of lambda functions either by entering their names or just the tag name. If you want to use it for later, please make sure that you have tagged desired lambda functions with same tag.

Deploy the solution

Follow these steps to deploy the solution:

1. Download the CloudFormation template from here.
2. Sign in to the AWS Management Console and open the CloudFormation console.
3. Choose Create Stack with new resources (standard).
4. In Templates, choose to upload a template file (CloudFormationTemplate.yml) from your local drive, and then choose Next.
5. Enter the following parameters:
   • AlarmEmail: If you chose to get emails for Lambda failures, enter an email address here.
   • LambdaFunctionNames:  A comma-separated list of Lambda function names in the Region. This parameter becomes optional if you provide value for LambdaFunctionTags and LambdaFunctionTagsValue parameters.
   • LambdaFunctionTags: The name of the tag that will be evaluated to find and add Lambda functions for monitoring in your account. This parameter becomes optional along with LambdaFunctionTagsValue if you provide names of the functions in LambdaFunctionNames.
   • LambdaFunctionTagsValue: The value of the tag that will be matched to find and add Lambda function for monitoring. This parameter is needed if you have provided tag key at LambdaFunctionTags.
   • SlackWorkspaceId: Enter the Slack workspace ID.
   • SlackChannelId: Enter the channel ID.

Figure 6: Parameters

6. Deploy the stack and wait for 3-4 minutes.
7. Check that the resources (Lambda function, Lambda execution role, SNS topic, Chatbot configuration) have been created by the stack. You can see CloudWatch metric alarms have been created for all Lambda functions with a tag value, mentioned in parameters.

Figure 7: Tags

In the CloudWatch console, on the Alarms page, you can verify the CloudWatch alarm been created:

Figure 8: CloudWatch alarm

If you have chosen to get emails by providing an email-id during the CloudFormation deployment, you would also get an email to confirm the subscription. Please check your mailbox and click on the link to confirm.

Cleanup

To avoid charges to your account, delete the CloudFormation stack and resources. For more information, see Deleting a stack on the AWS CloudFormation console in the AWS CloudFormation User Guide.

Conclusion

In this blog post, we showed you how to leverage AWS Chatbot to get Slack notifications in the event of Lambda execution failures. Due to its serverless and tight integration with rest of the AWS services, AWS Lambda is widely used in every organization. A failure in Lambda execution due to any reason would affect your operation or cause disruption to your service. By leveraging AWS chatbot, you or your team can get notified in real time and can troubleshoot it by running AWS commands in slack itself.

Although this solution helps you to track Lambda execution failures, you can modify or extend it to monitor other metrics that are important to you.

About the authors