How can I configure a CloudWatch subscription filter to invoke my Lambda function?

Short description

With Amazon CloudWatch Logs, you can use a subscription filter that sends log data to your Lambda function. CloudWatch Logs subscription filters are base64 encoded and compressed with the GZIP format.

Before you create your Lambda function, calculate the volume of log data that will be generated. Be sure to create a function that can manage the volume amount. If the function doesn't have enough volume, then the log stream is throttled. For more information, see Lambda quotas.

Note: Streaming large amounts of CloudWatch Logs data might result in high usage charges. It's a best practice to use AWS Budgets to track spending and usage. For instructions, see How can I use AWS Budgets to track my spending and usage?

Resolution

Create a CloudWatch Logs subscription filter that sends log data to your AWS Lambda function.

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

1.    To provide CloudWatch Logs permission to invoke your Lambda function, run the AWS CLI command add-permission similar to the following:

aws lambda add-permission \
    --function-name "helloworld" \
    --statement-id "helloworld" \
    --principal "logs.amazonaws.com" \
    --action "lambda:InvokeFunction" \
    --source-arn "arn:aws:logs:region:123456789123:log-group:YourLogGroup:*" \
    --source-account "123456789012"

Important: Replace "helloworld" with your Lambda function name, "YourLogGroup" with your log group, and the example account number with your account.

2.    Create a subscription filter using the AWS CLI command put-subscription-filter to send log events that contains a keyword. In the following example, the keyword "ERROR" is used for the Lambda function:

Important: Replace "YourLogGroup" with your log group and the example account number with your account.

aws logs put-subscription-filter \
    --log-group-name YourLogGroup \
    --filter-name demo \
    --filter-pattern "ERROR" \
    --destination-arn arn:aws:lambda:region:123456789123:function:helloworld

The CloudWatch log group "YourLogGroup" invokes the Lambda function when it receives a log event that contains the keyword "ERROR" similar to the following:

{
  "awslogs": {
    "data": "H4sIAAAAAAAAAHWPwQqCQBCGX0Xm7EFtK+smZBEUgXoLCdMhFtKV3akI8d0bLYmibvPPN3wz00CJxmQnTO41whwWQRIctmEcB6sQbFC3CjW3XW8kxpOpP+OC22d1Wml1qZkQGtoMsScxaczKN3plG8zlaHIta5KqWsozoTYw3/djzwhpLwivWFGHGpAFe7DL68JlBUk+l7KSN7tCOEJ4M3/qOI49vMHj+zCKdlFqLaU2ZHV2a4Ct/an0/ivdX8oYc1UVX860fQDQiMdxRQEAAA=="
  }
}

Related information

Filter and pattern syntax