How do I store Amazon SNS notifications for Amazon SES emails in DynamoDB using Lambda?

8 minute read

I use Amazon Simple Notification Service (Amazon SNS) to receive notifications about emails sent through the Amazon Simple Email Service (Amazon SES). I want to set up an AWS Lambda function to store these notifications in an Amazon DynamoDB table.

Resolution

Note: The code examples in the following steps work only with a Lambda runtime of Node.js versions 16.x and earlier.

(Prerequisite) Set up an Amazon SES email or domain with an Amazon SNS topic that's configured to receive notifications from Amazon SES

For more information, see Receiving Amazon SES notifications using Amazon SNS.

Create a DynamoDB table

1. Create a table in DynamoDB that has the following attributes:
For the Table-name, enter SESNotifications.
For the primary Partition key, enter SESMessageId.
For the primary Sort key, enter SnsPublishTime.

2. To allow Lambda to query the table and create an Amazon SES report, set up the following secondary indexes:

Index name	Partition key	Sort key
SESMessageType-Index	SESMessageType (String)	SnsPublishTime (String)
SESMessageComplaintType-Index	SESComplaintFeedbackType (String)	SnsPublishTime (String)

Note: You can add more secondary indexes as needed.

For information about creating a table in DynamoDB, see Create a DynamoDB table.

Add permissions to your Lambda function's IAM role that allow it to call the DynamoDB table

Create a new AWS Identity and Access Management (IAM) role. Configure the role to allow your Lambda function to call the DynamoDB:PutItem API:

Note: It's a best practice to create and use a new IAM role for different Lambda functions. Avoid reusing roles across multiple functions.

1. In the navigation pane of the IAM console, choose Roles.

2. Choose Create Role.

3. For Select type of trusted entity, choose AWS service.

4. For Choose a use case, choose Lambda. Then, choose Next: Permissions.

5. For Attach permissions policies, choose the check box next to AWSLambdaBasicExecutionRole managed policy. Then, choose Next: Tags.

6. (Optional) Add IAM tags to the role for your use case. For more information, see Tagging IAM resources.

7. Choose Next: Review.

8. For Role name*, enter lambda_ses_execution.

9. Choose Create role.

10. Return to the IAM Roles view, and then choose the role that you created.

11. In the Permissions tab, choose Add inline policy.

12. In the Visual editor tab, select Choose a service.

13. Choose DynamoDB.

14. In the Actions search field, enter PutItem. In the dropdown list that appears, choose the check box next to PutItem.

15. For Resources, choose Specific.

16. Choose Add ARN. Then, in the text box that appears, enter your DynamoDB table's Amazon Resource Name (ARN).

17. Choose Review policy.

18. For Name, enter a name for the policy.

19. Choose Create policy.

Example inline IAM policy that grants access to a DynamoDB table

{
    "Version": "2012-10-17",
    "Statement": \[
         {
            "Sid": "Stmt1428510662000",
            "Effect": "Allow",
            "Action": \[
                "DynamoDB:PutItem"
            \],
            "Resource": \[
                "arn:aws:DynamoDB:us-east-1:12345678912:table/SESNotifications"
            \]
        }
    \]
}

Create a Lambda function to process Amazon SES and Amazon SNS notifications

Use the following example function code to create a Lambda function that's named sesnotificationscode. You can use the following example Lambda function as a template for writing data to a customer relationship management (CRM) system or other destinations.

Important: Make sure that you assign the lambda_ses_execution role to the function.

Example Lambda function code that checks for Amazon SNS notifications and puts the associated Amazon SES notifications in a DynamoDB table

console.log("Loading event");

var aws = require("aws-sdk");
var ddb = new aws.DynamoDB({ params: { TableName: "SESNotifications" } });

exports.handler = function (event, context, callback) {
  console.log("Received event:", JSON.stringify(event, null, 2));

  var SnsPublishTime = event.Records\[0\].Sns.Timestamp;
  var SnsTopicArn = event.Records\[0\].Sns.TopicArn;
  var SESMessage = event.Records\[0\].Sns.Message;

  SESMessage = JSON.parse(SESMessage);

  var SESMessageType = SESMessage.notificationType;
  var SESMessageId = SESMessage.mail.messageId;
  var SESDestinationAddress = SESMessage.mail.destination.toString();
  var LambdaReceiveTime = new Date().toString();

  if (SESMessageType == "Bounce") {
    var SESreportingMTA = SESMessage.bounce.reportingMTA;
    var SESbounceSummary = JSON.stringify(SESMessage.bounce.bouncedRecipients);
    var itemParams = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESreportingMTA: { S: SESreportingMTA },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESbounceSummary: { S: SESbounceSummary },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParams, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Delivery") {
    var SESsmtpResponse1 = SESMessage.delivery.smtpResponse;
    var SESreportingMTA1 = SESMessage.delivery.reportingMTA;
    var itemParamsdel = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESsmtpResponse: { S: SESsmtpResponse1 },
        SESreportingMTA: { S: SESreportingMTA1 },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamsdel, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Complaint") {
    var SESComplaintFeedbackType = SESMessage.complaint.complaintFeedbackType;
    var SESFeedbackId = SESMessage.complaint.feedbackId;
    var itemParamscomp = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESComplaintFeedbackType: { S: SESComplaintFeedbackType },
        SESFeedbackId: { S: SESFeedbackId },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamscomp, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  }
};

Note: Replace the TableName parameter SESNotifications with your DynamoDB table's name.

Subscribe your Lambda function to one or more Amazon SNS topics

Using the Amazon SNS console

You must manually add permissions to the function resource policy to allow SNS to invoke the function. To add these permissions, run the following AWS CLI command:

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

aws lambda add-permission --function-name my-function --action lambda:InvokeFunction --statement-id sns-my-topic \\  
\--principal sns.amazonaws.com --source-arn arn:aws:sns:us-east-1:123456789012:my-topic

Note: Replace the values my-function, sns-my-topic, and arn:aws:sns:us-east-1:123456789012:my-topic with the ID of your function, topic, and full ARN.

After you add the necessary permissions, complete the following steps to subscribe your function to an SNS topic.

1. In the navigation pane of the Amazon SNS console, choose Topics.

2. Identify the SNS topic that's used in Amazon SES for bounce notifications. For example: An SNS topic named ses_notifications_repo.

3. Choose the SNS topic's ARN. The Topic Details page opens.

4. Choose Create Subscription.

5. For Protocol, choose AWS Lambda.

6. For Endpoint, enter your Lambda function's ARN.

7. Choose Create Subscription.

8. (Optional) Repeat steps 1–7 for each additional notification topic that you want to subscribe to your function.

Using the Lambda console

1. Open the Functions page in the Lambda console.

2. Choose your Lambda function that you created.

3. On the Function overview pane, choose +Add trigger.

4. In the Trigger configuration dropdown list, choose SNS. A configuration panel appears.

5. In the SNS topic dropdown list, choose the SNS topic that you want to subscribe the function to.

6. Choose Add.

7. (Optional) Repeat steps 1–6 for each additional notification topic that you want to subscribe to your function.

Test the set up: Send an Amazon SES message to invoke your Lambda function

To send a test Amazon SES message, use one of the available mailbox simulator addresses.

Note: When you send test messages, using one of the mailbox simulator addresses prevents a negative impact on your SES deliverability metrics.

When you send the test message, Amazon SES publishes a notification to the SNS topic. Then, Amazon SNS delivers the notification to Lambda as a JSON-escaped SES event notification object in the SNS event object.

To use the Lambda console to create sample events for local testing, see Examples of event data that Amazon SES publishes to Amazon SNS.

Important: You can't use these examples of event data as they're written to send test messages in the Lambda console. To use the examples for testing in the Lambda console, you must change the eventType message key to notificationType. If you don't change the message key, then the test fails.

Download a report from DynamoDB to view Amazon SES notifications

To query, sort, and download the contents of the DynamoDB table as a .csv file, complete the following steps:

1. Open the DynamoDB console, and then choose the SESNotifications table.

2. Choose the Items tab.

3. Create a Query or Scan search. For more information, see Best practices for querying and scanning data.

Note: You can use DynamoDB table export to schedule a download of the file to an Amazon Simple Storage Service (Amazon S3) bucket at regular intervals. For more information, see DynamoDB data export to Amazon S3: how it works.