How do I store Amazon SNS notifications for Amazon SES emails in DynamoDB using Lambda?

Last updated: 2022-11-10

Note: The following example Lambda function can be used as a template for writing data to a customer relationship management (CRM) system or other destinations.

For more information, see Receiving Amazon SES notifications using Amazon SNS.

1.    Create a table in DynamoDB that has the following attributes:
For the Table-name, enter SESNotifications.
For the primary Partition key, enter SESMessageId.
For the primary Sort key, enter SnsPublishTime.

2.    To allow Lambda to query the table and create an Amazon SES report, set up the following secondary indexes:

Note: You can add more secondary indexes as needed.

For information about creating a table in DynamoDB, see Getting started with Amazon DynamoDB.

Create a new AWS Identity and Access Management (IAM) role. Configure the role to allow your Lambda function to call the DynamoDB:PutItem API by doing the following:

Note: It's a best practice to create and use a new IAM role for different Lambda functions. Avoid reusing roles across multiple functions.

1.    In the left navigation pane of the IAM console, choose Roles.

2.    Choose Create Role.

3.    For Select type of trusted entity, choose AWS service.

4.    For Choose a use case, choose Lambda. Then, choose Next: Permissions.

5.    For Attach permissions policies, choose the check box next to AWSLambdaBasicExecutionRole managed policy. Then, choose Next: Tags.

6.    (Optional) Add IAM tags to the role for your use case. For more information, see Tagging IAM resources.

7.    Choose Next: Review.

8.    For Role name*, enter lambda_ses_execution.

9.    Choose Create role.

10.    Return to the IAM Roles view and choose the role you created.

11.    In the Permissions tab, choose Add inline policy.

12.    In the Visual editor tab, select Choose a service.

13.    Choose DynamoDB.

14.    In the Actions search field, enter PutItem. In the dropdown list that appears, choose the check box next to PutItem.

15.    For Resources, choose Specific.

16.    Choose Add ARN. Then, in the text box that appears, enter your DynamoDB table's Amazon Resource Name (ARN).

17.    Choose Review policy.

18.    For Name, enter a name for the policy.

19.    Choose Create policy.

Example inline IAM policy that grants access to a DynamoDB table

------------------------IAM Policy Begins---------------------------
{
    "Version": "2012-10-17",
    "Statement": [
         {
            "Sid": "Stmt1428510662000",
            "Effect": "Allow",
            "Action": [
                "DynamoDB:PutItem"
            ],
            "Resource": [
                "arn:aws:DynamoDB:us-east-1:12345678912:table/SESNotifications"
            ]
        }
    ]
}
------------------------IAM Policy Ends-----------------------------

Create a Lambda function named sesnotificationscode using the following example function code.

Important: Make sure that you assign the function the lambda_ses_execution role you created.

Example Lambda function code that checks for Amazon SNS notifications and puts the associated Amazon SES notifications in a DynamoDB table

Important: Replace the TableName parameter SESNotifications with your DynamoDB table's name.

--------------------------Lambda Code Begins------------------------
console.log("Loading event");

var aws = require("aws-sdk");
var ddb = new aws.DynamoDB({ params: { TableName: "SESNotifications" } });

exports.handler = function (event, context, callback) {
  console.log("Received event:", JSON.stringify(event, null, 2));

  var SnsPublishTime = event.Records[0].Sns.Timestamp;
  var SnsTopicArn = event.Records[0].Sns.TopicArn;
  var SESMessage = event.Records[0].Sns.Message;

  SESMessage = JSON.parse(SESMessage);

  var SESMessageType = SESMessage.notificationType;
  var SESMessageId = SESMessage.mail.messageId;
  var SESDestinationAddress = SESMessage.mail.destination.toString();
  var LambdaReceiveTime = new Date().toString();

  if (SESMessageType == "Bounce") {
    var SESreportingMTA = SESMessage.bounce.reportingMTA;
    var SESbounceSummary = JSON.stringify(SESMessage.bounce.bouncedRecipients);
    var itemParams = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESreportingMTA: { S: SESreportingMTA },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESbounceSummary: { S: SESbounceSummary },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParams, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Delivery") {
    var SESsmtpResponse1 = SESMessage.delivery.smtpResponse;
    var SESreportingMTA1 = SESMessage.delivery.reportingMTA;
    var itemParamsdel = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESsmtpResponse: { S: SESsmtpResponse1 },
        SESreportingMTA: { S: SESreportingMTA1 },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamsdel, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  } else if (SESMessageType == "Complaint") {
    var SESComplaintFeedbackType = SESMessage.complaint.complaintFeedbackType;
    var SESFeedbackId = SESMessage.complaint.feedbackId;
    var itemParamscomp = {
      Item: {
        SESMessageId: { S: SESMessageId },
        SnsPublishTime: { S: SnsPublishTime },
        SESComplaintFeedbackType: { S: SESComplaintFeedbackType },
        SESFeedbackId: { S: SESFeedbackId },
        SESDestinationAddress: { S: SESDestinationAddress },
        SESMessageType: { S: SESMessageType },
      },
    };
    ddb.putItem(itemParamscomp, function (err, data) {
      if (err) {
        callback(err)
      } else {
        console.log(data);
        callback(null,'')
      }
    });
  }
};
------------------------Lambda Code Ends----------------------------

To subscribe a function to an Amazon SNS topic using the Amazon SNS console

You must manually add permissions to the function resource policy to allow SNS to invoke the function. You can add these permissions with the following AWS CLI command:

aws lambda add-permission --function-name my-function --action lambda:InvokeFunction --statement-id sns-my-topic \<br>--principal sns.amazonaws.com --source-arn arn:aws:sns:us-east-1:123456789012:my-topic

Note: Be sure to replace the values my-function, sns-my-topic, and arn:aws:sns:us-east-1:123456789012:my-topic with the ID of your function, topic, and full ARN.

After you add the necessary permissions, follow the steps below to subscribe your function to an SNS topic.

1.    In the left navigation pane of the Amazon SNS console, choose Topics.

2.    Identify the SNS topic that is used in Amazon SES for bounce notifications. For example: An SNS topic named ses_notifications_repo.

3.    Choose the SNS topic's ARN. The Topic Details page opens.

4.    Choose Create Subscription.

5.    For Protocol, choose AWS Lambda.

6.    For Endpoint, enter your Lambda function's ARN.

7.    Choose Create Subscription.

8.    (Optional) Repeat steps 1-7 for each additional notification topic that you want to subscribe to your function.

To subscribe a function to an Amazon SNS topic using the Lambda console

1.    Open the Functions page in the Lambda console.

2.    Choose your Lambda function that you created earlier.

3.    On the Function overview pane, choose the +Add trigger button.

4.    In the Trigger configuration dropdown list, choose SNS. A configuration panel appears.

5.    In the SNS topic dropdown list, choose the SNS topic that you want to subscribe the function to.

6.    Choose Add.

7.    (Optional) Repeat steps 1-6 for each additional notification topic that you want to subscribe to your function.

To send a test Amazon SES message, use one of the available mailbox simulator addresses.

Note: Using one of the mailbox simulator addresses when sending test messages prevents a negative impact on your SES deliverability metrics.

When you send the test message, Amazon SES publishes a notification to the SNS topic. Then, Amazon SNS delivers the notification to Lambda as a JSON-escaped SES event notification object in the SNS event object.

To create sample events for local testing using the Lambda console, see Examples of event data that Amazon SES publishes to Amazon SNS.

Important: You can't use Examples of event data that Amazon SES publishes to Amazon SNS as they're written to send test messages in the Lambda console. To use the examples for testing in the Lambda console, you must change the eventType message key to notificationType. If you don't change the message key, the test fails.

To query, sort, and download the contents of the DynamoDB table as a CSV file, do the following:

1.    In the DynamoDB console, choose the SESNotifications table.

2.    Choose the Items tab.

3.    Create a Query or Scan search. For more information, see Best practices for querying and scanning data.

Note: You can use DynamoDB table export to schedule a download of the file to an Amazon Simple Storage Service (Amazon S3) bucket at regular intervals. For more information, see DynamoDB data export to Amazon S3: how it works.