Category: How To


Introducing the AWS Lambda Blueprint for Filtering Emails Received Through Amazon SES

by Mahae Koh | on | in Announcements, How To | | Comments

When we set out to build an email receiving service for AWS, we wanted to make receiving email easy, but we also wanted to empower you to make your own decisions about how your inbound emails should be handled. With the goal of striking a balance between ease of use and flexibility, we integrated SES with AWS Lambda, which allows you to easily define your own filter function with arbitrary logic.

Today, we’ll discuss how you can build a simple customized filter function using the new AWS Lambda blueprint for filtering inbound email.

Defining your email filter

When you log into the AWS Lambda console and select “Create a Lambda Function,” you are presented with a list of blueprints you can use. One of these blueprints is called “inbound-ses-spam-filter”—we’ll use this blueprint in our example as a springboard to jumpstart our custom filter function.

Spam filter Lambda blueprint

The blueprint is simple: it checks whether the email has passed SPF and DKIM validations, and whether the email is deemed spam or virus. If an email fails one or more of these tests, the message is bounced using a SendBounce call to SES; otherwise, the rule engine is instructed to continue processing the message.

Once you’ve selected the blueprint, you can customize it to fit your individual needs. First, you’ll need to fill in your domain as the bounce sender:

var sendBounceParams = {
  BounceSender: 'mailer-daemon@<MYDOMAIN>.com',
  OriginalMessageId: messageId,
  MessageDsn: {
    ReportingMta: 'dns; <MYDOMAIN>.com',
    ArrivalDate: new Date(),
    ExtensionFields: []
  },
  BouncedRecipientInfoList: []
};

The heart of the filtering logic, however, lies in the following if statement:

if (receipt.spfVerdict.status === 'FAIL' ||
        receipt.dkimVerdict.status === 'FAIL' ||
        receipt.spamVerdict.status === 'FAIL' ||
        receipt.virusVerdict.status === 'FAIL')

You can modify this conditional statement to fit your use case. Perhaps there’s a particular sender you want to block, or you noticed a pattern of spam with a particular header. For this example, we’ll bounce all emails with the subject line “Buy stuff!”

var mailMetadata = sesNotification.mail;

if (receipt.spfVerdict.status === 'FAIL' ||
        receipt.dkimVerdict.status === 'FAIL' ||
        receipt.spamVerdict.status === 'FAIL' ||
        receipt.virusVerdict.status === 'FAIL' ||
        mailMetadata.commonHeaders.subject === 'Buy stuff!')

The filter function will be invoked synchronously to ensure that it can return a disposition value back to SES, which will then use this value to decide whether to continue processing the message. This particular filter function returns “stop_rule_set” as the disposition after bouncing the message, effectively instructing SES to stop processing the message once the Lambda function returns.

context.succeed({
  disposition: 'stop_rule_set'
});

Check out our documentation for more information about disposition values.

Because the filter function uses SES’s SendBounce API, you need to make sure that your Lambda function’s execution IAM role is allowed to call SendBounce. You can use the following example policy document, which allows your filter function to call SendBounce for any verified domain:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ses:SendBounce"
      ],
      "Resource": "*"
    }
  ]
}

Configuring your email filter

Now that you’ve defined your filter function, you need to configure it to process your inbound email stream. To do this, you can create an SES receipt rule that synchronously invokes an AWS Lambda function, which can be done either through the SES console or by using the CreateReceiptRule API (see our documentation to find out more about creating receipt rules). For this example, we’ll use the console’s receipt rule wizard to create our filter rule.

Lambda action

Note that the invocation type must be set to RequestResponse (synchronous) so that your filter function can decide whether or not the message should be filtered out.

In order to prevent your existing rules from being evaluated for unwanted emails, you need to make sure that your filter rule is processed before any of your processing rules. You can do this by setting the position of your new rule in Step 3 of the receipt rule wizard.

Rule position

Once you’ve created your filter rule, it will be shown at position 1 of your rule set:

Rule set

Note that the rule was configured without specific recipients, which causes the rule to be evaluated for all your inbound emails regardless of the intended recipient. You can always limit the scope of your filter rule by configuring specific recipients to match.

Testing your email filter

You can now test your filter function by sending an email to the address you’re listening in on. To simulate spam or virus, you can use GTUBE or EICAR, respectively, as the message content.

Receiving email programmatically has traditionally been a complex task, due to the open nature of email and the abuse vectors that come with it. We hope that you’ll find the new AWS Lambda blueprint for filtering inbound email to be a useful starting point to help you deal with unwanted mail and manage your operating costs.

Happy hacking!

Debugging SMTP Conversations Part 3: Analyzing TCP Packets

by Elton Pinto | on | in How To | | Comments

We’ve finally reached the conclusion of our deep dive into how you can capture SMTP conversations should you need to debug an issue that lies deeper than your application. Now that we’ve gone over SMTP conversation basics and getting the easiest to decipher bits of a TCP conversation with TCP Flow, let’s look at all the information contained in a TCP conversation using TCP Dump and Wireshark.

Using TCP Dump

TCP Dump is an open source network packet analyzer (licensed under a 3-clause BSD license) which, in conjunction with the libpcap library, can also be used for capturing network traffic. It is one of the most widely used packet analyzers around because it provides a raw level of detail that solutions like TCP Flow don’t provide. It’s essentially a fire hose of data, so it’s sometimes used to capture data that is then read in using Wireshark, which is licensed under GNU GPL v2 and provides you with a great GUI for filtering and analyzing packets.

Amazon EC2 instances running an Amazon Linux AMI come with TCP Dump (tcpdump) pre-installed, so you don’t need to do anything there. The TCP Dump manual is even more intimidating than the TCP Flow manual, so here’s a simple base command you can start and experiment with:

sudo tcpdump -i any -w ~/captures/capture_%Y-%m-%d-%H-%M-%S.cap -G 30 -n -X -Z $USER “port 25”

  • The -i option specifies what network interface to listen on, just as in TCP Flow. For most folks, “any” is going to work just fine.
  • The -w option writes the raw packets to the file instead of printing to the console, and it’s followed by the file path and format. You can specify the time in plain old strftime format – in this example a file would look like ~/captures/capture_2014-04-30-19-15-00.cap for the time 2014-04-30T19:15:00Z if your machine’s time zone is UTC.
  • The -G option is very useful if your application processes large amounts of data – it lets you specify, in seconds, how often the dump file is rotated. In this case, it’ll create a new capture file every 30 seconds (and the file naming will follow what you specified in the -w option).
  • The -n option will forego printing FQDNs of host names referenced in the dump. If your application logs print IP addresses instead of host names, this option will make your life much easier.
  • The -X option will print each packet in hex and ASCII, which may come in handy in Wireshark.
  • The -Z option drops privileges to the user name specified, which means that you’ll own the captures instead of root owning them.
  • The end of the command line is, once again, a filtering expression as defined in the pcap filter manual.

A TCP Dump and Wireshark Example: START TLS

Unlike TCP Flow output, your TCP Dump capture file(s) will probably be very hard to read. This is where Wireshark comes in handy. Wireshark actually comes with the command-line tool tshark, which you could use instead of TCP Dump (it’s built on top of TCP Dump), but it doesn’t provide a lot of added value for the general use case. If your own computer is Linux, you should be able to just install Wireshark with yum:

sudo yum -y install wireshark wireshark-gnome

There are Windows and OS X installers available from the Wireshark website, which also has detailed documentation on the suite of features that you can take advantage of. There’s a lot of documentation there, so before you browse that it’s not a bad idea to play with the program a bit to get your feet wet.

Once you have Wireshark installed, transfer your TCP Dump capture from your EC2 instance to your own computer, fire up Wireshark, and open your TCP Dump capture. On Linux, you can simply pass the capture file to Wireshark as a command-line argument (you may or may not need sudo privileges to run it):

sudo wireshark ~/capture_2014-04-16-23-52-29.cap

I recommend immediately going to “View” -> “Time Display Format” and then changing the date format from epoch time (the default) to something more readable. You’ll see a table in the center pane of the GUI that displays one row per packet of data, followed by deeper details of a selected packet, followed by a pane with a hex and ASCII view of the packet. Above all this, you’ll see a blank field that you can fill in with a filter expression, of which Wireshark has an impressive array. You can explore them with the “+ Expression” button (you should see some familiar filters under the “TCP” section from the TCP Flow filter expressions) and then choose one to slice your traffic to just what you’re interested in. There are also some default filter expressions to choose from in “Analyze” -> “Display Filters”. Let’s once again take a look at a STARTTLS conversation with Amazon SES:

TCP Dump / Wireshark

There are a few items of note here:

  • Everything is color coded as ingress (light blue) or egress (black). You also see grey for the ACK packet. Speaking of which…
  • Notice that you can see where the connection was established with the SMTP server – the first three packets. The first packet is the SYN packet from the SMTP client to the SMTP server to open a TCP connection. The second packet is the SYN ACK from the server to the client that it received the SYN packet. The third packet is the ACK from the client to the server that it received the SYN ACK and the connection is established. This is especially useful if you’re trying to determine if there are high latencies during connection establishment or between when the connection is established and receiving the SMTP greeting (the fourth packet) or latencies between all that and when your client starts sending an EHLO, etc.
  • In the pane below this packet table, you can select slices of the packet to highlight in the last pane where the packet is displayed.
  • Just as in the TCP Flow output, everything under “Ready to start TLS” is unreadable. Again, if all you care about is the timing of packets or if you’re a relay that receives email in plaintext and then transmits the messages to Amazon SES, what we’ve discussed up to now will work just fine for your use case. You can still add logging in your application to print out the decrypted packets.
  • The above screenshot shows just one conversation, but depending on your TCP Dump filtering expression, you can capture everything that’s going on to try and detect congestion or interference from other network traffic that your machine is dealing with.

A TCP Dump and Wireshark Example: TLS Wrapper

TLS wrapper mode encrypts everything right from the get-go, so your ability to peek into what’s happening is very limited. There is a way to get Wireshark to tell you which packets are used in the TLS negotiation, though. At a high level, the TLS setup process involves these steps:

  1. The client and server negotiate security capabilities to determine what to use.
  2. The server transmits digital certificates and public/private key information to the client so that the client can verify the identity of the server.
  3. The client exchanges public/private key information with the server (including a pre-master secret) and may send a digital certificate to show that the client is who it says it is (if the server asked for this).
  4. The server authenticates the client and then the client and server both turn the pre-master secret into master secrets that are used to generate the session key (the same key is generated independently on both sides). This session key is used to encrypt/decrypt communications from here on out.

If you’re deeply curious, you can read more in RFC 2246 (TLS 1.0), RFC 4346 (TLS 1.1), and RFC 5246 (TLS 1.2). Since both the client and server use a public/private key pair as part of this set-up process, you’ll need at least the client’s private key in order for Wireshark to understand the handshake. If you use Open SSL, you can generate and supply this pretty easily:

openssl genrsa -out ~/rsa_key.pem

openssl s_client -crlf -connect email-smtp.us-east-1.amazonaws.com:465 -key ~/rsa_key.pem

The first command line creates the key and the second command line is the one shown in the Amazon SES Developer Guide but with a private key supplied. With this connection set up, you can have your SMTP conversations and use the same TCP Dump command as before, only with port 465:

sudo tcpdump -i any -w ~/captures/capture_%Y-%m-%d-%H-%M-%S.cap -G 30 -n -X -Z $USER “port 465”

Then, just transfer the capture file and rsa_key file to your own computer and fire up Wireshark:

TLS Dump example

Note that all you see are SYNs and ACKs. We can fix this. Go to “Edit” -> “Preferences”, expand “Protocols” and then select “SSL”. For the above example, if the rsa_key file is in my home directory I’d put the following in “RSA keys list”:

23.21.252.142,465,smtp,/home/elton/rsa_key.pem

The first element is the server IP address (visible in the Wireshark GUI), the second element is the server port, the third element is the application protocol, and the last element is the location of the private key file. You’ll see something like this:

TLS Dump

Now you’ll notice in the “Info” column that you can see more information at a high level and the protocol is specified in the previous column as TCP, SSL, or TLSv1 (whereas before it was just TCP). Additionally, the next pane with the packet breakdown has bits like “Secure Socket Layer” or whatever the protocol is that highlights the part of the packet involved. The “Application Data” rows are just encrypted SMTP messages. The real value though is being able to debug any TLS wrapper issues you may have by comparing good negotiations with bad ones or timestamps in good negotiations versus bad ones and getting to the bottom of whatever is going wrong.

We hope that these posts have given you a better understanding of what’s happening behind the scenes when you interact with Amazon SES, and empowered you to better debug problems that you may experience at the transport or network layer. Thanks for being an Amazon SES customer! Happy debugging!

Debugging SMTP Conversations Part 2: Capturing a Live Conversation

by Elton Pinto | on | in How To | | Comments

If your email-sending application has problems communicating with the Amazon SES SMTP interface (or your customers are having problems connecting to your SMTP server that proxies requests to Amazon SES), you’ll first probably check your application logs to see what’s going on. If you’re not able to find a smoking gun in your application logs though, what else can you do? Last week, we went over the basics of SMTP conversations and today we’ll explore how you can debug deeper than the application logs.

You may consider setting up an application layer wire log that shows all of the messages you’re sending and receiving, but one unlucky day you may find yourself with a lower-level issue on your hands. It could be a problem in the link between you and your ISP, between your ISP and the next hop, between your application and your kernel, or any number of other things.

A great way to get more data to help you figure out what’s going on is to go lower in the networking stack to the transport layer. Two well-known, freely available tools that can help you with this are TCP Flow and TCP Dump. TCP Flow is a great next step when you just want to see plaintext data packets in a human-readable format, while TCP Dump is more adept at giving you the kitchen sink so to speak (i.e., all the TCP packets in a variety of formats). In today’s post we’ll talk about TCP Flow. Since many of our customers use EC2 Linux-backed instances, we’ll focus on how to use TCP Flow from Linux.

Installing TCP Flow

TCP Flow lets you get your feet wet in transport layer debugging without overwhelming you with data. You can get the latest version using git clone:

sudo yum -y install git

mkdir ~/tcpflow && cd ~/tcpflow

git clone –recursive git://github.com/simsong/tcpflow.git

Currently, the latest version is 1.3, and the steps in the README work on a standard EC2 with a 64-bit AMI (tested on ami-bba18dd2 and ami-2f726546), though you may also need to yum install openssl-devel. If you encounter any problems doing this you can also try downloading the latest version from the GitHub site, though the install instructions may be in the NEWS file instead of README.

If you can run sudo /usr/local/bin/tcpflow -h and see the usage information, then the install was a success and you’re ready to boogie. Otherwise, double check the console output to see if some step failed. You can get more detailed usage information from man tcpflow.

Using TCP Flow

As you can see in the TCP Flow usage information, there are a lot of options to help you toggle what you’re looking for; these can be overwhelming at first glance. Let’s look at a reasonable set of options to start you off on the right track:

sudo /usr/local/bin/tcpflow -i any -g -FT -c port 25 > ~/tcpflow_out

  • The -i option specifies what network interface to listen on (‘any’ is a reasonable default to start you off)
  • The -g option was renamed in a recent version (it used to be -J), but it’s just to give you information in different colors, which you’ll soon see is nice to have.
  • The -c option prints to the console instead of creating individual files. By default, TCP Flow creates two files for each TCP conversation – one file for the packets coming in and one for the packets being transmitted. The -c option can be a useful alternative because the console interleaves the input and output packets.
  • The -F option is all about the format of the output files, and the ‘T’ prepends each file name with an ISO-8601 timestamp. If you output to the console using the -c option, it will still prepend all the lines of your conversation with the timestamp to the millisecond even though you’re not creating any files.
  • The “port 25” bit is a filtering expression, as defined in the pcap filter manual. Depending on what your instance is up to, listening to all traffic can be overwhelming so it’s a good idea to filter on what you care about. You can filter on dozens of things including the source or destination host/port, port ranges, and protocol.

Once you have your TCP Flow output, you can look at it with the color coding preserved (there’s one color for packets sent and one for packets received) using less:

less -R ~/tcpflow_out

You can pipe grep, too, if you’re trying to isolate an incident via a specific source/destination port or address:

grep 1.2.3.4 ~/tcpflow_out | less –R

A TCP Flow Example

If you establish a STARTTLS connection with the Amazon SES SMTP endpoint on port 25 and you use the above TCP Flow command, the output from less might look something like this:

TCP Flow screenshot

You’ll notice that the output is actually readable – there’s a timestamp for each packet in ISO 8601 format followed by the source IP and port of the packet and then the destination IP and port of the packet. You don’t get TCP packet headers or SYN/ACK packets or any of those details, but maybe your problem doesn’t require that much information.

From this point on, however, the conversation will look like gibberish since it’s just a TLS handshake and then all the packets are encrypted. If you use TLS wrapper mode, all the packets will look like gibberish. The nature of TLS makes it tough to decrypt these packets, but TCP Dump and Wireshark will allow us to decrypt at least some of the handshake (we’ll go over these in the next blog post of this series). TCP Flow is still useful on its own, though, if you’re receiving plaintext SMTP conversations from your customers and then proxying messages to Amazon SES for final delivery.

One last thing to note on TCP Flow – you can use the -r option to read in a TCP Dump capture and make it look readable for you.

We hope that you’ve found these tips handy, but the best is yet to come – in the next post of this series we’ll show you how to milk your TCP connections for all the data they’ve got. Thanks again for being a customer!

Debugging SMTP Conversations Part 1: How to Speak SMTP

by Elton Pinto | on | in How To | | Comments

Amazon SES strives to make your email sending as simple and quick as possible, which means that users of our HTTP API don’t even have to worry about what an SMTP conversation is or how to capture one. Even a lot of our SMTP interface users outsource the problem to software like Microsoft Outlook or PHP Mailer that takes care of these details for them. But if you’re experiencing an issue sending mail that can’t be explained by a recent code change or an error message from SES, understanding how an SMTP conversation works can be helpful. Or maybe you’re just curious as to what those bits flying around look like. In today’s blog post, the first in a series to help you debug these issues, we’ll go over the basics of an SMTP conversation.

Conversational SMTP

The Simple Mail Transfer Protocol (SMTP) was first officially put into writing in 1982 in RFC 821 as a way to “transfer mail reliably and efficiently”, but the protocol that the majority of ISPs use today is described in RFC 5321. The protocol includes a basic handshake, supported commands and responses at each step of the conversation, and finally transmission of message content. Here is a summary of the various steps of a typical conversation, without any extensions involved:

  1. An SMTP client opens a connection with an SMTP server. Generally, this is on port 25 or 587.
  2. The SMTP server responds with a 220 code and may follow that with a header that describes the server. It could also respond with a 554 status code to reject the connection, and then the client’s only option would be the QUIT command.
  3. The SMTP client sends either an EHLO or HELO command to the server. Either command must be followed by a space and then the domain of the client. Contemporary clients and servers should support EHLO, so EHLO is what we’ll use in this example.
  4. The SMTP server should respond to the EHLO with the 250 status code, its domain name, and a server greeting, and one line for every SMTP extension it supports.
  5. Now the SMTP client is in business and can start defining the mail to be sent, starting with what’s commonly referred to as the “envelope from” header. The client sends “MAIL FROM:” followed by a reverse-path address, which defines where bounce messages are sent if the message can’t be delivered after it’s accepted (receiving MTAs add this to incoming mail with the Return-Path header). If the mail being sent is a bounce message, this address should be empty (i.e., “<>”). The reverse-path address can optionally be followed by mail parameters defined by a supported SMTP extension (as advertised in the EHLO reply). The conversation cannot proceed until this MAIL FROM command is sent and accepted.
  6. The SMTP server must accept the MAIL FROM with a “250 OK” reply if the format is good and the address is deemed acceptable. Otherwise, the server typically responds with a 550 or 553 error response to indicate whether the failure is temporary or permanent. Other acceptable error status codes are 552, 451, 452, 503, 455, and 555 (see RFC 5321 for their definitions).
  7. The SMTP client can now define whom the email is for using the RCPT TO command. The syntax is very similar to MAIL FROM: it must be the literal “RCPT TO:” followed by the forward-path address surrounded by angle brackets. This can also optionally be followed by any parameters necessary to use an SMTP extension advertised in the EHLO reply. The RCPT TO command can only define a single recipient. If there are multiple recipients, the command can be issued multiple times, but the client needs to wait for a response from the server each time before supplying another destination.
  8. The SMTP server usually validates that the address is deliverable and responds with “250 OK” if it is. Otherwise, it typically returns a 550 reply. Other acceptable error status codes are 551, 552, 553, 450, 451, 452, 503, 455, and 555 (see RFC 5321 for their definitions). Some servers will accept all mail and only validate the destination after the SMTP conversation has completed.
  9. Finally, the SMTP client can initiate sending the body of the email by issuing the DATA command with no other text after it.
  10. The SMTP server responds with a 354 reply if it is ready to accept the message, or else a 503 or 554 if there was no valid MAIL FROM or RCPT TO command sent.
  11. If the SMTP server responded with a 354 reply, the client submits the message text, followed by the end of mail data indicator, which is a line containing only a period. The message generally starts with headers (one per line, e.g., “Header-name: header-value”) and then is followed by the body of the message.
  12. If the message is accepted, the SMTP server replies with a “250 OK” reply.
  13. The client can now initiate a new conversation with the server or send the “QUIT” command to politely close out the connection.
  14. If the SMTP server receives a QUIT, it is supposed to send a “221 OK” reply and then close the connection.

For deeper details of SMTP, please refer to RFC 5321. You can communicate with SES via SMTP over a number of clients including Microsoft Outlook – see the developer guide for more details. If you’d like to see the conversation yourself, you can also use telnet. There are a few additional things worth noting:

  • The server must not intentionally close the connection unless it sees a QUIT command except in the case of a timeout or if the server has to go down.
  • At any time during the conversation, the SMTP client can send the RSET command (just “RSET”, no additional parameters) to abort the current mail transaction so that the conversation can start fresh.
  • Other supported commands are VRFY to verify that a string represents a valid user or mailbox, EXPN to confirm that a string identifies a mailing list and returns membership of that list, NOOP to get a “250 OK” reply from the server, and HELP to get help information.
  • One notable extension that Amazon SES supports for secure communication on ports 25, 587, and 2587 is STARTTLS, as detailed in RFC 3207. After the EHLO, the client sends the command “STARTTLS”, the server replies with “220 Ready to start TLS” (or 501 or 454 error codes), and then TLS negotiation occurs to set up encryption keys. The conversation is then reset and must start with EHLO all over again with all transactions from here on out encrypted. Amazon SES also supports TLS wrapper mode on ports 465 and 2465.
  • Another notable extension that Amazon SES requires for authentication is AUTH PLAIN LOGIN, which is where you would enter your SMTP credentials. This is explained in some detail in the Developer’s Guide, but a recent blog post also goes into authentication in general.

Here is a sample SMTP conversation with SES in TLS wrapper mode with the conversation contents decrypted. The blue text comes from Amazon SES and the red text comes from a sample client:

PROXY TCP4 74.120.248.95 10.44.15.76 14659 465

220 email-smtp.amazonaws.com ESMTP SimpleEmailService-793939519 iqAfLvOj6BjiiCjSnD6S

EHLO email-smtp.us-east-1.amazonaws.com

250-email-smtp.amazonaws.com

250-8BITMIME

250-SIZE 10485760

250-AUTH PLAIN LOGIN

250 Ok

AUTH LOGIN

334 VXNlcm5hbWU6

bXlfZmFrZV91c2VybmFtZQ==

334 UGFzc3dvcmQ6

bXlfZmFrZV9wYXNzd29yZA==

235 Authentication successful.

MAIL FROM:<test@ses-example.com>

250 Ok

RCPT TO:<success@simulator.amazonses.com>

250 Ok

DATA

354 End data with <CR><LF>.<CR><LF>

MIME-Version: 1.0

Subject: Test message

From: Senior Tester <test@ses-example.com>

Content-Type: text/html; charset=”UTF-8″

Content-Transfer-Encoding: quoted-printable

To: success@simulator.amazonses.com>

<b>Cool email body</b>

.

250 Ok 0000012345678e09-123a4cdc-b56c-78dd-b90e-d123be456789-000000

QUIT

221 Bye

We hope that you feel better informed now on how email works at a high level! In the next post of this series we’ll go over how you can easily capture a live SMTP conversation. Thanks for being a customer of Amazon SES!

All about SES daily quota

by Samuel Minter | on | in How To | | Comments

Every SES account has a quota limiting the number of messages that can be sent daily.  Here are some of the most common questions we hear about this quota:
 

Why do you have a daily quota?

Quite simply, the daily quota is a mechanism to limit the potential damage if an account starts sending spam.  Even if you have a long-established history of permitted sending, it is not impossible for something to happen (say, a compromised account) that would result in unpermitted sending from your account.  So we want to make sure each customer has enough room to send the email they want to send, yet cap it to prevent sending problems from getting out of control. 
 

How do I tell what my quota is?

Brand-new accounts in the sandbox have a quota of 200 messages per day.  Once you have asked for and been granted production access, the default quota is 10,000 messages per day, which is more than enough for most of our customers.  After you have been successfully sending awhile, or if you ask for and are granted an increase, your quota may be higher.  To check your quota at any time you can check the SES console or use our GetSendQuota API.
 

Is my daily quota related to my TPS limit?

Yes.  While there is no specific mathematical relationship between a daily quota and TPS limit, we raise these limits in tandem, because a higher daily quota generally needs to be supported by a higher TPS.  There are some use cases that require a high TPS that do not need a high daily quota, but if we raise one, we will raise the other as well.
 

When does my daily quota reset?

It doesn’t.  "Daily" in this case does not mean midnight to midnight in your time zone, or any time zone.  It is a rolling 24-hour period.  We’re always looking at the 24 hours immediately prior to the current moment.  If during that time you have sent less than your quota, you can send more, up to whatever you have left of your quota.  If you have already sent your full allowed amount, you can’t send more until some of that sending "rolls off" of the 24-hour window. 
 

What happens if I keep sending after I’ve used up my quota?

If you have exceeded your quota, any additional attempts to send will result in an error.  You should be watching for these errors and, if appropriate, retrying your send after some of your sending has rolled off the 24-hour window and you have quota available for more sending.  Details of the errors that are returned can be found here.
 

If I copy lots of people in one message, does this count as one or many?

The quota limit is based on the number of recipients, not the number of emails.  So if you send a single email to 50 addresses, it counts as 50 toward your quota, not as one.
 

Are there specific quota levels?

Generally we set quotas at nice even numbers like 10,000 or 25,000 or 50,000.  You might occasionally see a quota that isn’t a round number, but it is nothing to worry about, and is usually just a transitional state as your quota is adjusting upward.
 

How do I increase my quota?

After you have been sending awhile, under certain circumstances our system will detect that you may need more quota and will start increasing your quota automatically.  If however your current quota is less than you need or anticipate needing shortly, then you should not wait for an automatic adjustment, you should submit a request for an increased quota here.  Your request will be reviewed and we will get back to you within one business day…  usually less.  Make sure to include as much information as possible about your use case, what you intend to do with the quota, and why you need the specific amount requested.  (Don’t just ask for a huge number because you don’t want to deal with limits.)  You are asking us to trust you to responsibly use the higher quota, so the more detail you give us, the better we can understand what your needs are as we evaluate your request.  If you don’t already have a long history of good sending with us, or you are asking for a really large increase over your existing quota, this becomes especially important. Because each user is different we will evaluate each request on a case-by-case basis.
 

How to check your domain verification settings

by Valentin Munteanu | on | in How To | | Comments

So you performed the domain verification steps listed in the Developer Guide but the domain status in the SES Console still says Pending? To validate that the domain verification steps were executed properly, you have to check that the validation token was published to DNS under the correct record. Let’s say that you’re validating the domain ses-example.com and that the verification token (TXT record) value you obtained from the SES Console is fmxqxT/icOYx4aA/bEUrDPMeax9/s3frblS+niixmqk=. Here are the steps you need to follow.

Finding the Name Servers for the Domain

Each domain is served by one or more name servers, which map the domain name to the IP address of the host computer. To find the name servers that serve our domain, we are going to use the nslookup tool, which you run from a command line. You can use nslookup on both Windows and Linux. The commands in this post were executed on Windows 7. To get to the command line on Windows 7, for example, click Start and then type cmd.  On Linux-based operating systems, open a terminal window. The output of nslookup might be a little bit different on your system.

Note that for troubleshooting purposes, it is best to query the DNS servers that are serving your domain because they contain the most up-to-date information. Sometimes, even if the DNS records look fine in your DNS server, it might still take time for that information to propagate to the SES DNS servers.

At the command prompt, type nslookup -type=NS ses-example.com to list all of the name servers that are serving ses-example.com. In the command’s output, we see that there are 6 dns servers, pdns2.ultradns.net, pdns3.ultradns.org and so on . Let’s pick pdns2.ultradns.net.

name server list

Verify TXT record publishing

The next step is to check that the verification token is published properly. We are looking for a TXT record under _amazonses.ses-example.com.  Note that the underscore before amazonses is optional. See the Developer Guide for more information.

At the command prompt, type nslookup -type=TXT  _amazonses.ses-example.com pdns2.ultradns.net, as shown in the screenshot below.

nslookup output

From the output of the command, you can see that the value published under _amazonses.ses-example.com is our token, fmxqxT/icOYx4aA/bEUrDPMeax9/s3frblS+niixmqk=, which means that the TXT record is published correctly.

Using dig

On Linux you can also use the dig command. To use dig, in a terminal window, type dig NS ses-example.com +short to list all of the name servers for your domain.

dig output

To check that the verification token is published properly, remember that we’re looking for a TXT record under _amazonses.ses-example.com.  As you can see in the output below, the verification token we are looking for is indeed present.

Published token

Now, let’s see what the output looks like for the most common mistakes.

Common mistakes

Always make sure that the published verification token is exactly the same as the one listed in the SES console.

Common mistakes:

  • The token is published directly under the domain (ses-example.com in our case)
  • The value being published is _amazonses.ses-example.com= fmxqxT/icOYx4aA/bEUrDPMeax9/s3frblS+niixmqk=.

Example nslookup output of these mistakes is below.

Common mistakes

  • Another common mistake is that the verification token is published under a record that contains the domain twice.  In this case the token would be published under _amazonses.ses-example.com.ses-example.com instead of _amazonses.ses-example.com.

I hope that you found this blog post helpful. Please feel free to browse or post on the SES forum if you have questions about domain verification.

 

Amazon SES IP addresses

by Adrian Hamciuc | on | in How To | | Comments

One of the questions we get from time to time here at Amazon SES is “What IP address is my email going out through?” In this blog post we will see how to find the outgoing IP addresses that SES is using by querying the Amazon SES SPF records in DNS.

What is an outgoing IP address and why should I care?

When SES connects to an ISP to deliver an email using the standard email sending protocol (SMTP), a connection is established between the SES mail transfer agent (MTA) that sends out the email and the ISP’s MTA that will receive it. Each MTA has an IP address associated with it. What we are interested in is the SES MTA’s IP address (outgoing IP address). Knowing this information could be useful in a number of situations. For example, you might want to whitelist SES IP addresses with your company’s receiving email servers to let emails sent through SES go through your firewall while still protecting yourself from attacks coming from the rest of the Internet.

How do I figure out the addresses Amazon SES uses for sending my emails?

SES maintains a number of IP addresses from which your email can be sent, and you can figure out those addresses by querying SES’s SPF record (in the amazonses.com domain).

Here’s how to do the query in a Linux terminal window (including a filter to include only the SPF record):

$ dig TXT amazonses.com +short| grep 'v=spf1'

At the time of this post’s writing, this is the DNS response returned for that query:

"v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ~all" 

Here’s the equivalent query (and result) using the Windows command prompt:

C:>nslookup -type=TXT amazonses.com | find "v=spf1"

"v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ~all"

You can also use a number of online tools to perform a similar DNS query for you, but it’s up to you to determine how reliable and trustworthy they are.

Any email sent through SES will be sent through one of the IP addresses listed in the record. In this example, which is valid at the time of the writing of this blog post, we can see that SES is using three blocks of IPV4 (32-bit) addresses: 199.255.192.0/22, 199.127.232.0/22 and 54.240.0.0/18. The blocks are defined using CIDR notation: each block is specified as an IP address followed by the ‘/’ sign and then the number of bits in the prefix. The 199.255.192.0/22 block, for instance, designates all IP addresses that have the first 22 bits equal to the first 22 bits of 199.255.192.0, with the remaining bits taking all possible value combinations. Therefore, that block designates all addresses in the interval 199.255.192.0 – 199.255.195.255. The block 199.127.232.0/22 designates all IP addresses in the interval 199.127.232.0 – 199.127.235.255 and, finally, the 54.240.0.0/18 block designates all IP addresses in the interval 54.240.0.0 – 54.240.63.255.

Keep in mind that those IP addresses are subject to change. If SES adds or removes any outgoing IP address, we will update the SPF record, so you need to check back from time to time, if you want to make sure you have the latest list of IP address ranges. Another thing to note is that there is no guarantee with regard to which particular SES IP address of the list of IP addresses, your email will be sent through. If you need to perform a whitelisting process for the emails you’re sending through SES you will have to whitelist all SES IP address blocks.

I hope the information in this blog post is helpful to you. If you have any questions, feel free to browse or post in the SES forum.

Three places where your email could get delayed when sending through SES

by Adrian Hamciuc | on | in How To | | Comments

Email sending is one of those activities you may not pay much attention to when all goes well, but you really notice when it’s not working. In today’s fast moving world, the speed of information transfer is critical, so slow email delivery is a particularly painful problem. In this blog post, we will explore three places where your emails could be delayed before your recipients get a chance to read them.

The typical path of an email message delivered via Amazon SES is pretty simple: your application generates the email and then passes it to SES in one way or another. After a brief period spent inside the SES pipeline, SES attempts to contact the receiving ISP and deliver the message. The ISP, after successfully enqueuing the message, displays it to the intended recipient. Let’s see now what could go wrong at each of these steps, and how we can identify the source of the problem.

Scenario 1: Emails are delayed before arriving at SES

The first step in the process is for your application to contact SES to pass it the message. This is one place where problems could occur.

Remember that for each message it successfully enqueues, SES returns a message ID that you can log on your side. To verify that your calls are actually successful, you can track these message IDs. If SES never accepted your message, you can be sure it won’t be delivered! If you are able to log a message ID, it means that your email safely arrived in SES. You can then match the timestamp of when the message ID was logged with the timestamp of when the call was made inside your application. If the difference is too big, it means something happened between the moment your application wanted to send the email and until SES acknowledged its receipt.

Typical problems we’ve seen include custom software (such as Postfix) sitting between the customer’s application and SES, which were buffering or otherwise delaying messages. If you have such a setup, make sure you examine the logs of the software you’re using for evidence of any delays there.  

In order to test that the problem lies between your application and SES, and not inside SES or between SES and the ISP, you can try to send a test email from the Amazon SES console to that specific ISP. If that test email arrives on time, it’s a pretty good indicator that the delay is occurring before your email even reaches SES.

Scenario 2: Emails are delayed inside SES

Once SES accepts the message, it will process it as quickly as possible, before handing it over to the ISP. We take latency very seriously, and, in the event of significant service delays, we will update the public status dashboard.

Soft bounces and Amazon SES

SES service delays are, however, not the only reason why emails might not be delivered to the receiving end. For various reasons, the ISP might be temporarily refusing your email. This is referred to as a soft bounce. Typical reasons why ISPs might soft bounce your email are if the destination address has a full inbox, the email is of a larger size than they support, or they have a service issue and cannot accept email. ISPs can also soft bounce emails as a form of throttling – if they see too many emails delivered too quickly they sometimes interpret that as a spammer trying to attack their customers.

When a soft bounce event happens, SES continuously retries to deliver your email for 12 hours before giving up. There is no limit of retries during this 12-hour interval, and, as long as the ISP recovers, your email will be delivered.

To verify whether soft bouncing is the issue, you can try sending to different ISPs. If you only see delays with one of them, that’s a pretty good indicator that that particular ISP has a problem receiving your email.

Scenario 3: Emails are delayed after leaving SES

One particularly thorny problem we sometimes encounter is when we successfully deliver the email to an ISP quickly and in the first attempt, only to have it take a very long time to appear in the recipient’s inbox. This can happen for a variety of reasons, for example the ISP is encountering a technical problem, or it is delaying making the email available to its users because it does not yet trust it.

To narrow down the problem, try sending to a different ISP (or to a different address in the same ISP), using an email with different content, different (or lack of) attachments, different “From” address, etc. and see if that makes a difference. Remember that as long as you haven’t received a bounce notification from Amazon SES, it means your message is either somewhere in the SES pipeline, with its delivery being continuously retried, or it has already been enqueued by the ISP, and it being processed on their end.

We hope this entry sheds a bit more light over email latencies and SES and will help you identify the source of the problem. Feel free to post any comments or questions for us in the AWS Forum.

DKIM Troubleshooting Series: Deliverability Considerations

by Adrian Hamciuc | on | in Best Practices, How To | | Comments

Hello and welcome to the last entry in the Amazon SES DKIM troubleshooting blog series. So far, we have seen various technical problems that appeared between us and properly signed emails, and we also saw how DKIM helps protect our domain from various impersonators. We will now see whether DKIM can also improve our deliverability (the likelihood that our emails will arrive in our recipients’ Inboxes as opposed to their Spam or Junk folders).

Why are my emails still getting to the spam folder, even if I am properly using DKIM?

Ok, we have set up proper authentication, but now we need to focus on a new problem. Our emails are still arriving in the Spam folders of our customers for a major ISP. We cannot understand what’s wrong. Wasn’t DKIM supposed to prevent that?

DKIM is a great way to show ISPs that the emails we send actually belong to us. It also helps ISPs differentiate our traffic from other emails originating from the same IP address, by associating our email with our domain reputation. This way, we’re giving ISPs the opportunity to decide whether to place our emails in the Spam or Inbox based on our domain reputation regardless of the actual originating IP address.

What DKIM doesn’t do, however, is influence our domain’s actual standing with ISPs. By signing the email, our domain took responsibility for its content. If that content is unsolicited, for example, and the recipient clicks the "Report Spam" button, then we will be in trouble, and so will emails originating from our domain. Alternatively, if we are sending a high-quality email which is gladly accepted by our recipients, our good (domain) reputation will follow across outgoing IP addresses.

We start reading the Amazon SES Email Sending Best Practices whitepaper and discover the different building blocks of developing a rock-solid sending reputation. For example, we can start by analyzing our high complaint rate and investigating what we can do to reduce it. This is no longer a matter for this blog post though…

This seven part blog post series covered the most common issues we notice our customers having when they set up EasyDKIM with SES. We hope to have shed some light on details of this process that are more obscure on first sight. If you haven’t found the answer to your problem here, if the suggestions in these posts don’t work for you or if you just want to say hi, we invite you to post on the Amazon SES Forum and ask any questions you may have.

Happy sending and thank you for using SES!

DKIM Troubleshooting Series: Authentication Considerations

by Adrian Hamciuc | on | in How To | | Comments

Hi, and welcome to another entry in the Amazon SES DKIM troubleshooting series. So far we have focused on technical issues, but now it’s time to take a step back and look at the bigger picture. Exactly why did we go to all this trouble in setting up DKIM for our domain?

My emails are signed and the signature validates. Does this mean I’m safe from spoofing attacks?

Even if all our email is DKIM signed and validated by all ISPs, this is no time to rest. The world is full of ethically dubious people who want to impersonate us and steal our customers, and DKIM is only step one in protecting ourselves.

So what exactly is DKIM doing to help us with this? The way this standard ensures the integrity of our emails is to calculate a hash of the message’s body and some of its headers, and then to digitally sign the hash. As long as the private key is kept secure, and the public key is accessible through DNS, the DKIM signature gives ISPs confidence that the domain that signs the message acknowledges its content through the signature. Let’s have another look at the DKIM signature to see exactly what we’re talking about:

DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
s=xtk53kxcy4p3t6ztbrffs6d54rsrrhh6; d=ses-example.com;
t=1366720445;
h=From:To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Date:Message-ID;
bh=lcj/Sl5qKl6K6zwFUwb7Flgnngl892pW574kmS1hrS0=;
b=nhVMQLmSh7/DM5PW7xPV4K/PN4iVY0a5OF4YYk2L7jgUq9hHQlckopxe82TaAr64
eVTcBhHHj9Bwtzkmuk88g4G5UUN8J+AAsd/JUNGoZOBS1OofSkuAQ6cGfRGanF68Ag7
nmmEjEi+JL5JQh//u+EKTH4TVb4zdEWlBuMlrdTg=

The "d" tag represents the domain that signs the email. In this case that would be our own domain, ses-example.com. This domain takes responsibility for the email, guaranteeing its integrity through the signature. The "h" tag contains a list of all of the headers of this email that are signed (and thus have their integrity guaranteed) by the signing domain. The "bh" tag is the result of a hash function applied to the email’s body (very useful to ensure that the content itself hasn’t been tampered with). Finally, the "b" tag is the actual signature itself, applied to the body hash and headers. If that checks out, then both the message body and all of the headers included in the "h" tag are considered to be authenticated by the signing domain.

That’s all very nice, but how does it stop an attacker?

The hashing and key encryption algorithms we use are well known, so surely anyone can generate their own key pair and calculate the signature themselves, right? In fact… this is actually true. Anyone can sign any email with a DKIM signature, the problem is getting ISPs to trust that that signature is ours and not the attacker’s. The key here is in the "s" and "d" tags of the signature. The "d" tag is our domain, and the "s" (or "selector") tag indicates which of our domain’s signing keys has been used for this particular email. It’s these two tags that help build the location of the public key in DNS and form the obstacle in the attacker’s path. In this signature’s case, any ISP that wants to validate our email will try to retrieve the public key from a record xtk53kxcy4p3t6ztbrffs6d54rsrrhh6._domainkey.ses-example.com ( <selector>._domainkey.<domain> ). An attacker can calculate hashes and signatures all they want but they won’t have permission to publish any key they control into our DNS, so they will never be able to compute a signature that has our domain as the "d" tag (and thus has its integrity guaranteed by us). DNS spoofing, it turns out, is not so easy to perform (although it has happened)! With only the public key visible in DNS, it is not computationally easy for any attacker to deduce the private part. Just in case, SES regularly rotates the keys, to thwart any such attempt.

Setting up DMARC can also signal ISPs to drop all emails that aren’t authenticated (via DKIM or SPF). If an attacker wants to impersonate our domain they can either put in a broken signature or no signature at all. In both cases, DMARC signals ISPs to quarantine (put in the Spam folder) or drop those emails. That is definitely a good step forward in dealing with phishing spam!

SES also offers other forms of authentication such as SPF, which are highly recommended and will further improve our security.

Next steps

In the next blog entry, we will have a look at another email-related concept that is indirectly influenced by DKIM: deliverability.