How do I use a Lambda function to store Amazon SES notifications sent through Amazon SNS in DynamoDB?

Last updated: 2021-02-22

I use Amazon Simple Notification Service (Amazon SNS) to receive notifications about emails sent through the Amazon Simple Email Service (Amazon SES). How can I store these notifications in an Amazon DynamoDB table using an AWS Lambda function?

Resolution

Note: The following example Lambda function can be used as a template for writing data to a CRM or other destinations, based on where you host your mailing list.

If you haven't done so already, set up an Amazon SES account (email or domain) that has an Amazon SNS topic configured to receive notifications from Amazon SES. For more information, see Receiving Amazon SES notifications using Amazon SNS.

Then, do the following:

Create a DynamoDB table

1.    Create a table in DynamoDB with the following attributes:
For the Table-name, enter SESNotifications.
For the primary Partition key, enter SESMessageId.
For the primary Sort key, enter SnsPublishTime.

2.    Set up secondary indexes as described in the table below. This configuration will allow Lambda to query the table and create an Amazon SES report.

Index name Partition key Sort key
SESMessageType-Index SESMessageType (String) SnsPublishTime (String)
SESMessageComplaintType-Index SESComplaintFeedbackType (String) SnsPublishTime (String)

Note: You can add more secondary indexes as needed.

For information about creating a table in DynamoDB, see Getting Started with Amazon DynamoDB.

Add permissions to the Lambda function's IAM role that allow it to call the DynamoDB table

Create a new AWS Identity and Access Management (IAM) role that allows your Lambda function to call the DynamoDB:PutItem API by doing the following:

Note: It's a best practice to create and use a new IAM role for different Lambda functions. Avoid reusing roles across multiple functions.

1.    In the left navigation pane of the IAM console, choose Roles.

2.    Choose Create Role.

3.    For Select type of trusted entity, choose AWS service.

4.    For Choose a use case, choose Lambda, and then choose Next: Permissions.

5.    For Attach permissions policies, choose the check box next to AWSLambdaBasicExecutionRole managed policy. Then, choose Next: Tags.

6.    (Optional) Add IAM tags to the role for your use case. For more information, see Tagging IAM resources.

7.    Choose Next: Review.

8.    For Role name* enter lambda_ses_execution.

9.    Choose Create role.

10.    Return to the IAM Roles view and choose the role you created.

11.    In the Permissions tab, choose Add inline policy.

12.    In the Visual editor tab, select Choose a service.

13.    Choose DynamoDB. Then, for Actions, enter PutItem in the search field and choose the check box next to PutItem in the dropdown list that appears.

14.    For Resources, choose Specific. Then, choose Add ARN and enter provide the ARN of the DynamoDB table you created.

15.    Choose Review policy.

16.    For Name, enter a name for the policy. Then, choose Create policy.

Example inline IAM policy that grants access to a DynamoDB table

------------------------IAM Policy Begins---------------------------
{
    "Version": "2012-10-17",
    "Statement": [
         {
            "Sid": "Stmt1428510662000",
            "Effect": "Allow",
            "Action": [
                "DynamoDB:PutItem"
            ],
            "Resource": [
                "arn:aws:DynamoDB:us-east-1:12345678912:table/SESNotifications"
            ]
        }
    ]
}
------------------------IAM Policy Ends-----------------------------

Create a Lambda function to processes Amazon SES and Amazon SNS notifications

Create a Lambda function using the following sample code, and then name it sesnotificationscode. When creating the function, make sure that you assign to it the lambda_ses_execution role you created.

Example Lambda function code to process Amazon SES and Amazon SNS notifications

The following example Lambda function code checks for the three types of Amazon SNS notifications. The function also puts the associated Amazon SES notification into a DynamoDB table. For more information on the types of notifications, see Amazon SNS notification examples for Amazon SES.

Important: Replace the TableName parameter SESNotifications with the name of your DynamoDB table.

--------------------------Lambda Code Begins------------------------
console.log("Loading event");

var aws = require("aws-sdk");
var ddb = new aws.DynamoDB({ params: { TableName: "SESNotifications" } });

exports.handler = function (event, context, callback) {
  console.log("Received event:", JSON.stringify(event, null, 2));

  var SnsPublishTime = event.Records[0].Sns.Timestamp;
  var SnsTopicArn = event.Records[0].Sns.TopicArn;
  var SESMessage = event.Records[0].Sns.Message;

  SESMessage = JSON.parse(SESMessage);

  var SESMessageType = SESMessage.notificationType;
  var SESMessageId = SESMessage.mail.messageId;
  var SESDestinationAddress = SESMessage.mail.destination.toString();
  var LambdaReceiveTime = new Date().toString();

  if (SESMessageType == "Bounce") {
    var SESreportingMTA = SESMessage.bounce.reportingMTA;
    var SESbounceSummary = JSON.stringify(SESMessage.bounce.bouncedRecipients);
    var itemParams = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESreportingMTA: { S: SESreportingMTA },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESbounceSummary: { S: SESbounceSummary },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParams, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Delivery") {
    var SESsmtpResponse1 = SESMessage.delivery.smtpResponse;
    var SESreportingMTA1 = SESMessage.delivery.reportingMTA;
    var itemParamsdel = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESsmtpResponse: { S: SESsmtpResponse1 },
        SESreportingMTA: { S: SESreportingMTA1 },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamsdel, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Complaint") {
    var SESComplaintFeedbackType = SESMessage.complaint.complaintFeedbackType;
    var SESFeedbackId = SESMessage.complaint.feedbackId;
    var itemParamscomp = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESComplaintFeedbackType: { S: SESComplaintFeedbackType },
        SESFeedbackId: { S: SESFeedbackId },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamscomp, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  }
};
------------------------Lambda Code Ends----------------------------

Subscribe the Lambda function to one or more Amazon SNS topics

If you haven't already done so, create at least one Amazon SNS topic and configure an Amazon SES email domain to use that SNS topic for feedback notifications. Then, do the following, using either the Amazon SNS console or the Lambda console.

To subscribe a function to an Amazon SNS topic using the Amazon SNS console

1.    In the left navigation pane of the Amazon SNS console, choose Topics. Then, identify the SNS topic that is used in Amazon SES for bounce notifications. For example, identify an SNS topic named ses_notifications_repo.

2.    Choose the ARN of the SNS topic to open the Topic Details page.

3.    Choose Create Subscription.

4.    For Protocol, choose AWS Lambda.

5.    For Endpoint, enter the ARN of the Lambda function that you created. Then, choose Create Subscription.

To subscribe a function to an Amazon SNS topic using the Lambda console

1.    In the Lambda console, choose the Lambda function that you created earlier.

2.    On the Configuration page, in the Designer pane, choose the +Add trigger button.

3.    In the Trigger configuration dropdown list, choose SNS. A configuration panel appears below the designer.

4.    In the SNS topic dropdown, choose the SNS topic that you want to subscribe the function to. Then, choose the Add button.

Repeat this process to add the different notification topics that you want to subscribe this Lambda function.

Test the set up by sending an Amazon SES message to invoke the Lambda function

To send a test message, use one of the available mailbox simulator addresses to avoid a negative impact on your SES deliverability metrics. For more information, see Using the mailbox simulator.

After the Amazon SES message is sent, SES publishes a notification to the SNS topic. Amazon SNS then delivers the notification to Lambda as a JSON-escaped SES event notification object in the SNS event object.

To create sample events for local testing using the Lambda console, see Examples of event data that Amazon SES publishes to Amazon SNS.

For SES bounce, complaint, and delivery notification examples, see Examples of event data that Amazon SES publishes to Amazon SNS.

Download a report from DynamoDB to view Amazon SES notifications

To query, sort, and download the contents of the DynamoDB table as a CSV file, do the following:

1.    In the DynamoDB console, choose the SESNotifications table.

2.    Choose the Items tab.

3.    Create a Query or Scan search. For more information, see Best practices for querying and scanning data.

Note: You can use DynamoDB table export to schedule a download of the file to an Amazon Simple Storage Service (Amazon S3) bucket at regular intervals. For more information, see Exporting DynamoDB table data to Amazon S3.