ITV Saves $150,000 in Compute Costs by Using Amazon EKS on Amazon EC2 Spot Instances

ITV’s product development teams build application services on a standardized architectural pattern of AWS services, open-source tools, and DevOps principles, known internally as the Common Platform, which hosts applications that support broadcasting, advertising sales, video on demand, BritBox, and the ITV Hub. The Common Platform is managed by a team of platform engineers with expertise in using AWS for DevOps and infrastructure provisioning.

In the first half of 2020, ITV’s total viewing was up 4 percent, with online viewing up 13 percent. Yet it faced severe headwinds from advertisers conserving cash. The company realized it had to adjust for the COVID-19 pandemic and future changes in user behavior. “We explored how we could run the same AWS services and meet the same or greater level of demand at a lower cost while hitting the same goals for reliability, security, and agility,” says Jonathan Harvey, head of the Common Platform at ITV. ITV also had to find a simple way to cost-effectively scale its services up and down to meet fluctuating traffic. “We see spikes in daily demand as thousands of people jump on the same video stream for a popular episode. But at 5:00 a.m., few people are online. We need an environment that adapts to both.”

So ITV explored how to gain cost efficiencies from its Amazon EKS clusters. The company began an 18-month migration of its Common Platform workloads to Amazon EKS, beginning in the summer of 2019. ITV’s Common Platform engineers regularly met AWS Technical Account Manager teams and AWS Solutions Architects to review AWS services, explore ideas, and revise optimizations on existing solutions. Now 75 percent of the Common Platform’s workloads run on Amazon EKS. “As our understanding of Kubernetes and Amazon EKS has grown,” says Harvey, “we’ve taken advantage of more features, such as Spot Instances.”

“By gaining a better understanding of what happens behind the scenes for Spot Instance terminations, we’ve built our confidence in running Spot Instances in production,” says Harvey. Using Spot Instances for Amazon EKS saved ITV up to 60 percent in costs compared to Amazon EC2 On-Demand Instances and shaved $150,000 off the company’s Amazon EC2 bill in just 1 year. Using Spot Instances also helped ITV significantly cut the cost of its systems management tooling, which monitors, logs, and observes its primary systems. Now 24 percent of ITV’s compute runs on Spot Instances, up from 9 percent the previous year. “If we can put more of our production workloads on Spot Instances, we might take some seasonality out of our costs,” explains Harvey. “I want a strong relationship between service demands and the number of workloads. Our tech choices on AWS are helping us get there.”

AWS briefed the Common Platform team on using Spot Fleets for Amazon EKS–hosted workloads. This feature helps maintain application availability and enables the team to automatically add or remove Amazon EC2 instances according to the conditions it defines. Using this feature, the team can scale instances across purchase options, Availability Zones, and instance families in a single application to optimize performance and cost. “If we run a multinode Amazon EKS cluster,” says Harvey, “we can safely put two on Spot Instances, knowing the last in the cluster will remain on an On-Demand Instance and stay up even if there are Spot Instance terminations.”

ITV hosts containerized microservices on Amazon EKS for cost-effective scalability during volume spikes. Previously, to prepare for a sharp jump in viewers, ITV had to manually choose to increase the size and number of Amazon EC2 instances and manually scale down afterward. The scaled environment might run 24 hours a day even if the show was on for 1 hour. Now ITV can automatically scale the number of instances to meet any demand level and then scale back down afterward without administrator-level users taking advance action. “Amazon EKS gives us enough of Kubernetes that we can use things like the Kubernetes Horizontal Pod Autoscaler feature,” says Harvey. “It gives us the flexibility to optimize scaling and takes much of the pain out of cluster management.”

Previously, ITV managed its microservices on individual Amazon EC2 instances using open-source configuration management tools. Deployments took 40 minutes from code commit to running in production. By using containers and Amazon EKS, deployments now take as little time as 4 minutes, and ITV runs hundreds every week. As a result, the company increased the number of hosted microservices by about 30 percent in 1 year. For more efficiency, the company is now migrating to Amazon EKS namespaces to reduce the number of production clusters.

ITV has an AWS sandbox environment where engineers can explore how AWS services and new features can enhance ITV’s environment and enable continuous innovation. ITV is focusing on developing the capacity of its on-demand streaming services: it has extended the distribution of BritBox and is planning a rollout of BritBox to 25 countries worldwide. The company is also exploring initiatives using various AWS Media Services, which would replace the hardware transcode previously on premises.

Using Amazon EKS and Amazon EC2 Spot Instances, ITV is prepared to meet user behavior changes through the COVID-19 pandemic and beyond. “We care about the reliability, security, cost efficiency, and speed at which we deploy changes on AWS,” says Harvey. “Those priorities match up fantastically well with how our workloads run on Amazon EKS and Spot Instances: they are reliable enough, highly available, and quick to deploy. Now we have a service that’s awesome to run and comes with a big discount on the cost of running it.”