Edmunds.com Cuts Costs and Boosts Availability by Using AWS Fargate Spot

Many car shoppers turn to Edmunds.com when they’re researching vehicles and looking for ways to simplify the car-buying process. The company runs 68 applications on its site and sees 30 million monthly requests. Because the applications have to be highly available any time customers use them, Edmunds.com is always searching for ways to optimize both performance and cost in its infrastructure.

Already operating in the Amazon Web Services (AWS) Cloud, Edmunds.com took advantage of the ability to test other AWS services that could continue to simplify and evolve its infrastructure: it used Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances, which let users access unused Amazon EC2 capacity for up to a 90 percent discount compared to Amazon EC2 On-Demand Instance prices, as well as AWS Fargate, a serverless compute engine for containers that removes the need to provision and manage servers. Using those services, Edmunds.com increased availability and reliability for its end users by providing fault-tolerant applications and cut costs on compute power, all while saving its team time by not having to manage the instances.

Edmunds has been in business since 1966, and it launched its web offering, Edmunds.com, in 1995. The website has been helping people get the best possible deal when they purchase a vehicle, now providing vehicle information, pricing, and rankings at the click of a button. The company works with more than 20,000 dealers and has close partnerships with manufacturers such as General Motors, Ford, and Toyota. In 2016, Edmunds.com migrated from an on-premises infrastructure to the AWS Cloud. It then experimented with various AWS services to highly automate its system. Most recently, Edmunds.com has focused on cost and performance optimization, which led it to the powerful combination of Amazon EC2 Spot Instances and AWS Fargate Spot.

The Edmunds.com system is built on four environments spread across multiple AWS Regions. It includes two quality assurance environments for testing, one in the US East and the other in the US West, to enable fault tolerance. Its 68 applications deploy across multiple AWS Regions and between two and four Availability Zones. Initially, the system ran on Reserved Instances. But seeking cost optimizations, the company decided in 2019 to test Amazon EC2 Spot Instances in its quality assurance environments, since potential interruptions wouldn’t directly affect customers. “Resiliency is an important factor in everything that we do. While we do want to reduce costs, we didn’t want to lose sight of the fact that uptime is absolutely critical,” says Emil Ndreu, executive director of cloud and security operations at Edmunds.com. The company started using Amazon EC2 instances with a single instance type and then later diversified its instance pools to increase resiliency. After a year of operating its quality assurance environments on Amazon EC2 Spot Instances, Edmunds.com found that one of its production runways can operate on Amazon EC2 Spot Instances as well.

Originally, Edmunds.com ran its containers on Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service, powered by Amazon EC2 Spot Instances and Reserved Instances. That pairing gave the company granular control over the instance types it used for specific workloads. But later, Edmunds.com opted to replace Amazon ECS with the fully managed AWS Fargate, which would save the lean Edmunds.com team a lot of time. Though the company must keep Amazon ECS running on Amazon EC2 until its commitment to the Reserved Instances is expired, today about 80 percent of the company’s customer-facing applications run on AWS Fargate Spot.

Transitioning to Amazon EC2 Spot Instances was a no-brainer, according to Ndreu: “It’s a testament to our AWS infrastructure. It’s imperative that our applications are designed to be fault tolerant when they’re deployed. On AWS, everything was multi–Availability Zone from day one, and every application is deployed in such a fashion that it can handle an outage regardless of instance type or service.”

Using Amazon ECS, Edmunds.com self-managed Amazon EC2 Spot Instances, from patching and autoscaling policies to distributing the instance pools across workloads and selecting the specific Amazon EC2 instance for each pool. But Amazon ECS on AWS Fargate enables a more hands-off approach. “Developing efficiencies around the uptime availability and optimization is a core competency that my team is generally tasked with,” says Ndreu. “AWS Fargate Spot handles it automatically. It’s so overly simplified that the amount of workload you’re extracting and allocating to other efficiencies your team can work toward is incredible.”

Using Amazon ECS with AWS Fargate Spot, Edmunds.com reaps cost and performance benefits. “AWS Fargate makes you run more optimally, and therefore you save more in terms of costs associated with running the services within that solution,” says Ndreu. Edmunds.com users see availability of 99.999 percent. Compared to using AWS Fargate, which is on demand, AWS Fargate Spot saves Edmunds.com 25–30 percent on costs. In January 2020, Edmunds.com also started using Savings Plans, a flexible pricing model for AWS compute usage. “The main motivation there is that I am no longer bound to a particular instance type or Region,” says Ndreu. From October 2019 to August 2020, Edmunds.com saved $100,000 as it optimized on AWS, partly due to Amazon EC2 Spot Instances and Savings Plans. Further, Edmunds.com now can autoscale in seconds rather than minutes, and compute costs have gone down by 30 percent, showing that the company was overprovisioned before handing the reins over to the managed service. “Our autoscaling capabilities, our multi-Region deployment, and our active capabilities have worked so well for us that we haven’t made any hefty preparations to accommodate high-traffic events,” says Ndreu.

Now Edmunds.com can test what-if scenarios before it sends new services to market, evaluating total capacity to prevent a potential outage from impacting customers. In writing these scenarios, Edmunds.com can then see how to mitigate them, whether that means adjusting autoscaling, increasing the number of containers, or otherwise. “We’re putting a huge emphasis on how we release new services to consumers in a more efficient and faster way,” says Ndreu. “Instead of doing perhaps 10–20 releases a day, we maybe increase to 20–30 releases a day.”

With the serverless compute of AWS Fargate and the cost efficiency of Spot pricing and Compute Savings Plans, Edmunds.com saves on compute costs while conserving time and effort for its lean team and improving performance for its end users. The company looks forward to working toward running 100 percent of its customer-facing applications on AWS Fargate Spot as well as taking advantage of the managed service to focus on additional ways to increase efficiencies in areas it couldn’t before—such as achieving operational efficiency by not managing the underlying Amazon EC2 clusters and creating multiaccount strategies and fine-grained cost management. Edmunds.com also plans to use continuous integration and continuous delivery workloads to facilitate rapid delivery of new software features and fixes.

Overall, having the flexibility to continually experiment with using new AWS services to optimize and simplify its infrastructure within the AWS Cloud has been integral to Edmunds.com operations. “We can find the right tool for the right job,” says Ndreu. “Rather than a one-size-fits-all approach, we have extreme flexibility that has worked very well for us.”