Inside SendGrid’s Expanded Relationship with AWS

In today’s world of technology dependencies, ensuring your application’s messages reach the intended recipients is critical. That’s why some of the largest and most successful software-defined organizations such as Airbnb, Spotify, Uber, and Yelp rely on SendGrid as their customer communication platform for their most valuable email.

Unlike most high-growth tech start-ups, SendGrid was born in the data center. Following the company’s launch in 2009, it built a high-performing and highly scalable private, internal cloud, achieving cost-efficient price-to-performance and four-hour server deploy times. Despite this, SendGrid chose to aggressively expand their relationship with AWS as their single, strategic cloud partner. Important factors in this decision included the business and marketing advantages of the AWS Partner Network (APN), AWS Marketplace, and the technology enablement advantages derived from elastic capabilities and managed services. So we are excited to officially announce SendGrid is joining the AWS family as an APN Partner and customer.

Inside SendGrid’s Expanded Relationship with AWS

By JR Jasperson, Chief Architect at SendGrid, an APN Technology Partner

I am excited to unveil two related blockbuster projects we’ve been working on for a long time. The first is a robust, expanded relationship with AWS. At SendGrid, we have been running many critical workloads with many cloud providers – including AWS – for many years. However, through our relationship with AWS, we will be deepening and broadening our collaboration and aggressively expanding our footprint with AWS. Below, we’ll share additional details into why, and what this will mean for our customers.

The second announcement is that we’ve been quietly working on a complete re-­architecture of our mail processing pipeline for more than a year, and will roll these changes out throughout the remainder of 2017. While worthy of its own blog post, the new architecture was created and intended from day one to lay the groundwork for the SendGrid-AWS relationship. Our new pipeline is truly cloud-­native and designed from the ground up to run optimally on AWS. In so doing, we’ve avoided several pitfalls that have beset other mail providers, so it’s helpful to tell both stories to highlight the advantages of AWS that we will be passing along to our customers.

Some Background

When SendGrid was founded in 2009, we created the notion of a transactional Email Service Provider (ESP)—sometimes referred to as a Cloud Mail Transfer Agent (MTA). Over the years, we have developed unique expertise at solving for the interesting and unique challenges that arise from serving an incredibly high volume (we surpassed the 1 trillion emails sent mark in March 2017), multi-tenant SaaS email delivery platform. Many of these challenges are unique to this sector, when running in a cloud environment such as the AWS Cloud. Experience with single-tenant and/or on-premises solutions is only a fraction of what it takes to be successful as a true Cloud MTA.

At SendGrid we strive to live by our 4 H values: Humble, Honest, Hungry and Happy and to be unwavering champions for our customers. So we’re proud that our customers put more value on our differentiators: ease of use, simple integration, deliverability, proven availability, scalability, and performance—we feel that these numbers speak for themselves.

Why Now?

The adage “success lives at the intersection of preparation and opportunity” immediately comes to mind while considering the question of “why become an APN Partner now”? In our case, preparation meant re-­architecting to a cloud-native platform. Two examples immediately stand out to illustrate why we viewed this as a prerequisite.

• Email is delivered through a store-­and-­forward mechanism which naturally implies statefulness in the system. As other “Cloud MTAs” have discovered, stateful compute is a poor fit for the cloud as it erodes many of its advantages. This is why we have decoupled state from our MTAs in the new architecture which allows us to take full advantage of AWS’ compute elasticity.
• In regards to IPs, deliverability is a measure of how many emails dispatched by a sender actually arrive in the recipient’s inbox as intended. It is perhaps the most important metric for an Email Service Provider. Surprisingly, deliverability is arduous to maintain, and one area in which SendGrid especially excels. Deliverability is not only highly dependent on the reputation of the specific IP address from which a given email is sent, but can also be negatively impacted due to bleed­-over effects from dubious activity or poor reputation from other IPs in the same subnet. This is why SendGrid controls the entirety of the subnets from which we send and have ensured this will always remain the case as we transition additional workloads into AWS. We’ve spent years cultivating our IPs, and the 50,000 plus paying customers who use these IPs are a testament to their viability. Our move to AWS takes into account that reputations are hard to build and easy to destroy, hence why we will retain full control of all of our IPs during and after our move to AWS. Surprisingly, this is not the case with all “Cloud MTAs.”

Customer Advantages

With much of the prep work now behind us, we are well positioned to turbocharge our mission to become the world’s most trusted communication platform—our relationship with AWS has been born not of necessity, but opportunity. While the most commonly-­cited benefits of AWS certainly apply in our case, I’ll skip these well­-covered facets and instead highlight several that are more nuanced to a Cloud MTA and most importantly how we expect our customers to benefit from these advantages.

• For years, SendGrid has maintained distributed Points of Presence (PoPs) around the globe to quickly respond to API and SMTP requests. The performance and responsiveness of these PoPs is a function of their proximity to our customers. We will be leveraging AWS’ immense global reach to create additional PoPs in many AWS regions, further reducing latency and increasing throughput for our customers.
• Running a high­-scale Cloud MTA requires near real-­time analysis of a massive flow of data behind the scenes—challenges that require stream processing and machine learning technologies. We manage the infrastructure for these systems today, which naturally require overhead to build and maintain. Leveraging AWS capabilities such as Amazon Kinesis Streams, Amazon EMR, and Amazon Machine Learning will enable us to do more faster, allowing us to spend more time developing features for our customers.
• The ability to visualize or deeply analyze facets of messaging campaigns (such as deliverability by recipient domain or click/open rates of A/B tests) is critical to maximize the value that SendGrid provides to our customers. Supporting and continuously enhancing these capabilities represents a non­trivial engineering challenge for SendGrid 
due both to our substantial volume (finding and associating discrete events in an enormous sea of data) and providing the flexibility necessary to meet diverse requirements. AWS offers a number of analytics capabilities such as Amazon Athena, Redshift, and AWS Data Pipeline that we have already begun to explore as more efficient and advanced ways for our customers to use SendGrid to optimize their messaging.

We will be leveraging AWS to allow us to focus on delivering even more value to our customers at an accelerated rate. While we’ve already been working on these projects for quite some time, we know these are just the first steps in an exciting journey— if you haven’t already, come join us and help shape the future of messaging!