How do I store Amazon SNS notifications for Amazon SES emails in DynamoDB using Lambda?

Last updated: 2021-11-11

I use Amazon Simple Notification Service (Amazon SNS) to receive notifications about emails sent through the Amazon Simple Email Service (Amazon SES). How do I set up an AWS Lambda function to store these notifications in an Amazon DynamoDB table?

Resolution

Note: The following example Lambda function can be used as a template for writing data to a customer relationship management (CRM) system or other destinations.

(Prerequisite) Set up an Amazon SES email or domain with an Amazon SNS topic configured to receive notifications from Amazon SES

Create a DynamoDB table

1.    Create a table in DynamoDB that has the following attributes:
For the Table-name, enter SESNotifications.
For the primary Partition key, enter SESMessageId.
For the primary Sort key, enter SnsPublishTime.

2.    To allow Lambda to query the table and create an Amazon SES report, set up the following secondary indexes:

Index name Partition key Sort key
SESMessageType-Index SESMessageType (String) SnsPublishTime (String)
SESMessageComplaintType-Index SESComplaintFeedbackType (String) SnsPublishTime (String)

Note: You can add more secondary indexes as needed.

For information about creating a table in DynamoDB, see Getting started with Amazon DynamoDB.

Add permissions to your Lambda function's IAM role that allow it to call the DynamoDB table

Create a new AWS Identity and Access Management (IAM) role. Configure the role to allow your Lambda function to call the DynamoDB:PutItem API by doing the following:

Note: It's a best practice to create and use a new IAM role for different Lambda functions. Avoid reusing roles across multiple functions.

1.    In the left navigation pane of the IAM console, choose Roles.

2.    Choose Create Role.

3.    For Select type of trusted entity, choose AWS service.

4.    For Choose a use case, choose Lambda. Then, choose Next: Permissions.

5.    For Attach permissions policies, choose the check box next to AWSLambdaBasicExecutionRole managed policy. Then, choose Next: Tags.

6.    (Optional) Add IAM tags to the role for your use case. For more information, see Tagging IAM resources.

7.    Choose Next: Review.

8.    For Role name*, enter lambda_ses_execution.

9.    Choose Create role.

10.    Return to the IAM Roles view and choose the role you created.

11.    In the Permissions tab, choose Add inline policy.

12.    In the Visual editor tab, select Choose a service.

13.    Choose DynamoDB.

14.    In the Actions search field, enter PutItem. In the dropdown list that appears, choose the check box next to PutItem.

15.    For Resources, choose Specific.

16.    Choose Add ARN. Then, in the text box that appears, enter your DynamoDB table's Amazon Resource Name (ARN).

17.    Choose Review policy.

18.    For Name, enter a name for the policy.

19.    Choose Create policy.

Example inline IAM policy that grants access to a DynamoDB table

------------------------IAM Policy Begins---------------------------
{
    "Version": "2012-10-17",
    "Statement": [
         {
            "Sid": "Stmt1428510662000",
            "Effect": "Allow",
            "Action": [
                "DynamoDB:PutItem"
            ],
            "Resource": [
                "arn:aws:DynamoDB:us-east-1:12345678912:table/SESNotifications"
            ]
        }
    ]
}
------------------------IAM Policy Ends-----------------------------

Create a Lambda function to processes Amazon SES and Amazon SNS notifications

Create a Lambda function named sesnotificationscode using the following example function code.

Important: Make sure that you assign the function the lambda_ses_execution role you created.

Example Lambda function code that checks for Amazon SNS notifications and puts the associated Amazon SES notifications in a DynamoDB table

Important: Replace the TableName parameter SESNotifications with your DynamoDB table's name.

--------------------------Lambda Code Begins------------------------
console.log("Loading event");

var aws = require("aws-sdk");
var ddb = new aws.DynamoDB({ params: { TableName: "SESNotifications" } });

exports.handler = function (event, context, callback) {
  console.log("Received event:", JSON.stringify(event, null, 2));

  var SnsPublishTime = event.Records[0].Sns.Timestamp;
  var SnsTopicArn = event.Records[0].Sns.TopicArn;
  var SESMessage = event.Records[0].Sns.Message;

  SESMessage = JSON.parse(SESMessage);

  var SESMessageType = SESMessage.notificationType;
  var SESMessageId = SESMessage.mail.messageId;
  var SESDestinationAddress = SESMessage.mail.destination.toString();
  var LambdaReceiveTime = new Date().toString();

  if (SESMessageType == "Bounce") {
    var SESreportingMTA = SESMessage.bounce.reportingMTA;
    var SESbounceSummary = JSON.stringify(SESMessage.bounce.bouncedRecipients);
    var itemParams = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESreportingMTA: { S: SESreportingMTA },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESbounceSummary: { S: SESbounceSummary },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParams, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Delivery") {
    var SESsmtpResponse1 = SESMessage.delivery.smtpResponse;
    var SESreportingMTA1 = SESMessage.delivery.reportingMTA;
    var itemParamsdel = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESsmtpResponse: { S: SESsmtpResponse1 },
        SESreportingMTA: { S: SESreportingMTA1 },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamsdel, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Complaint") {
    var SESComplaintFeedbackType = SESMessage.complaint.complaintFeedbackType;
    var SESFeedbackId = SESMessage.complaint.feedbackId;
    var itemParamscomp = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESComplaintFeedbackType: { S: SESComplaintFeedbackType },
        SESFeedbackId: { S: SESFeedbackId },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamscomp, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  }
};
------------------------Lambda Code Ends----------------------------

Subscribe your Lambda function to one or more Amazon SNS topics

To subscribe a function to an Amazon SNS topic using the Amazon SNS console

1.    In the left navigation pane of the Amazon SNS console, choose Topics.

2.    Identify the SNS topic that is used in Amazon SES for bounce notifications. For example: An SNS topic named ses_notifications_repo.

3.    Choose the SNS topic's ARN. The Topic Details page opens.

4.    Choose Create Subscription.

5.    For Protocol, choose AWS Lambda.

6.    For Endpoint, enter your Lambda function's ARN.

7.    Choose Create Subscription.

8.    (Optional) Repeat steps 1-7 for each additional notification topic that you want to subscribe to your function.

To subscribe a function to an Amazon SNS topic using the Lambda console

1.    Open the Functions page in the Lambda console.

2.    Choose your Lambda function that you created earlier.

3.    On the Function overview pane, choose the +Add trigger button.

4.    In the Trigger configuration dropdown list, choose SNS. A configuration panel appears.

5.    In the SNS topic dropdown list, choose the SNS topic that you want to subscribe the function to.

6.    Choose Add.

7.    (Optional) Repeat steps 1-6 for each additional notification topic that you want to subscribe to your function.

Test the set up by sending an Amazon SES message to invoke your Lambda function

To send a test Amazon SES message, use one of the available mailbox simulator addresses.

Note: Using one of the mailbox simulator addresses when sending test messages prevents a negative impact on your SES deliverability metrics.

When you send the test message, Amazon SES publishes a notification to the SNS topic. Then, Amazon SNS delivers the notification to Lambda as a JSON-escaped SES event notification object in the SNS event object.

To create sample events for local testing using the Lambda console, see Examples of event data that Amazon SES publishes to Amazon SNS.

Important: You can't use the Examples of event data that Amazon SES publishes to Amazon SNS as they're written to send test messages in the Lambda console. To use the examples for testing in the Lambda console, you must change the eventType message key to notificationType. If you don't change the message key, the test fails.

Download a report from DynamoDB to view Amazon SES notifications

To query, sort, and download the contents of the DynamoDB table as a CSV file, do the following:

1.    In the DynamoDB console, choose the SESNotifications table.

2.    Choose the Items tab.

3.    Create a Query or Scan search. For more information, see Best practices for querying and scanning data.

Note: You can use DynamoDB table export to schedule a download of the file to an Amazon Simple Storage Service (Amazon S3) bucket at regular intervals. For more information, see Exporting DynamoDB table data to Amazon S3.