Second Spectrum Reduces Kubernetes Hosting Costs by 90% Using AWS Load Balancer Controller and Amazon EKS
Second Spectrum, a company that provides artificial intelligence–driven tracking technology for sports broadcasts, is modernizing the way people engage with sports. Broadcasters and commentators use the company’s tools to provide quick insights accompanied by helpful visuals. To improve these tools and to quickly develop new ones, Second Spectrum relies on the efficient use of engineering resources from Amazon Web Services (AWS).
Second Spectrum began using Application Load Balancer for containerized workloads that it initially ran on self-managed Kubernetes clusters. The company soon recognized that it could run its Kubernetes clusters more efficiently by migrating to Amazon Elastic Kubernetes Service (Amazon EKS), a managed Kubernetes service that gives companies the flexibility to start, run, and scale Kubernetes applications on AWS or on premises. Second Spectrum also saw an opportunity to streamline workload routing using AWS Load Balancer Controller, which would enable the company to simplify operations by sharing an Application Load Balancer across multiple applications that were deployed in an Amazon EKS–managed Kubernetes cluster. This integrated AWS solution helped Second Spectrum reduce Kubernetes hosting costs by 90 percent, reduce errors by up to 75 percent, improve application response time by 5 ms per request, increase staff productivity, and create a scalable, replicable means of streamlining workload provisioning throughout the company.
The direct-pod internet protocol target registration from AWS Load Balancer Controller enabled us to clean up and consolidate our configuration into a simple-to-use Kubernetes input.”
Senior Director of Infrastructure, Second Spectrum
Seeking an Efficient Load Balancer Solution for Kubernetes
Growing out of a research group at the University of Southern California, Second Spectrum was founded in Los Angeles in 2013. TV viewers of National Basketball Association, Major League Soccer, or Premier League games or matches have seen Second Spectrum technology in action—for example, the use of computer vision to track players’ movements. As a startup, Second Spectrum seeks to quickly experiment with new tool offerings and push the most viable ones into production. To do that, the company needs efficient routing so that it can provision certain parts of its architecture in a replicable way.
For application deployment, Second Spectrum has used containerization since 2014. “Kubernetes is a beast to wrangle,” says Sheldon Kwok, senior director of infrastructure at Second Spectrum. “Because there are so many moving parts, it is extremely time consuming to understand and manage Kubernetes internals in house.
”Within this containerized environment, Second Spectrum has relied on Application Load Balancer since its release in 2016. Application Load Balancer operates at the request level (layer 7), routing traffic to targets like containers or internet protocol addresses. However, the team still faced an inefficiency challenge. “There was a whole extra network step: you had to route through the Kubernetes proxy on a NodePort that wouldn’t necessarily be in the same Availability Zone as your final service,” says Kwok. The Second Spectrum team was keen on eliminating this extra step to improve performance.
Using AWS Load Balancer Controller and Amazon EKS for Efficient Resource Use
The engineers at Second Spectrum were intrigued when AWS released AWS Load Balancer Controller in 2020, which supported pods, or small groups of application containers that can be deployed and managed in concert. “The direct-pod internet protocol target registration from AWS Load Balancer Controller enabled us to clean up and consolidate our configuration into a simple-to-use Kubernetes input,” says Kwok.
Second Spectrum received further support from Amazon EKS, which handled the Kubernetes API server for the company and served as a more efficient alternative to the self-managed Kubernetes cluster. “Using Amazon EKS takes away all the headache that comes with managing Kubernetes,” says Kwok. “It lets us work on more product-related features.” While improving productivity, the streamlined managed environment of Amazon EKS also enables the company to save on costs: “We previously spent about $1,000 per month just running one Kubernetes cluster for the API servers,” says Kwok. “Amazon EKS costs us around $100 per month.”
AWS Load Balancer Controller and Amazon EKS enabled Second Spectrum to monitor the Kubernetes pods directly and skip the extra step of first going through the instances. With this new configuration, the team noticed a performance improvement of around 5 ms per request—a small but important win when it comes to delivering near-real-time tracking and visualizations for an enhanced viewing experience. This AWS-automated process also greatly reduces the potential for human error in the configuration chain. “The rate at which we experience problems with load balancing and availability should go down by about 75 percent,” says Eric Lui, vice president of engineering operations at Second Spectrum.
The Second Spectrum team estimates that the simpler configuration saves at least 2 weeks of staff time per year because engineers are better equipped to self-serve rather than wait for the infrastructure team. Factor in the use of Amazon EKS—which removes time spent training new hires on operating self-managed Kubernetes clusters—and the expected time savings adds up to approximately 3 full-time equivalent months per year. The company can use just one Application Load Balancer for its entire namespace, and AWS Load Balancer Controller can dynamically scale to handle an influx of incoming requests and automatically stop routing to proxy servers that go down.
Using the Latest AWS Offerings to Maximize Value
The Second Spectrum team sees this new configuration as an important step in its move toward a DevOps approach to software engineering. “We’re looking to put these tools in the hands of developers so that they can take responsibility for their own load balancing configuration: getting it deployed, testing it, and setting up health checks,” says Lui. “That unlocks the capability for people to be more autonomous and more productive. In terms of impact to the organization, that is a key piece.”
Now that the Second Spectrum engineers don’t spend as much time managing Kubernetes clusters, they have more time to explore other opportunities. Going forward, the team plans to take advantage of price savings for elastic workloads by migrating those workloads to AWS Fargate, a serverless compute engine for containers that works alongside Amazon EKS. Also, the new routing configuration is scalable and replicable, so the company can stand up a similar configuration in Europe.
As Second Spectrum continues advancing sports analysis, it expects AWS offerings will play a key role. “AWS is very open to customer requests. We often ask for new features, and our Technical Account Manager has mentioned that other customers have asked for something similar,” says Kwok. “We try to stay on top of all the new functionality AWS provides to make use of AWS as much as possible.”
Second Spectrum's Reference Architecture
Original architecture, using Application Load Balancer and self-managed Kubernetes.
New architecture, now using Amazon EKS and AWS Load Balancer Controller.
About Second Spectrum
Second Spectrum was founded in Los Angeles in 2013. As the official tracking provider for the National Basketball Association, Major League Soccer, and the Premier League, Second Spectrum provides an advanced machine understanding of games that unlocks fast technology-driven insights.
Benefits of AWS
- Reduced Kubernetes hosting costs by 90%
- Expects to reduce error rate by 75%
- Expects to save 3 full-time equivalent months per year in engineering and new-hire training time
- Improved application response time by 5 ms per request
AWS Services Used
Amazon Elastic Kubernetes Service (Amazon EKS) gives you the flexibility to start, run, and scale Kubernetes applications in the AWS Cloud or on-premises. Amazon EKS helps you provide highly-available and secure clusters and automates key tasks such as patching, node provisioning, and updates.
Application Load Balancer
Ideal for advanced load balancing of HTTP and HTTPS traffic, Application Load Balancer provides advanced request routing targeted at delivery of modern application architectures, including microservices and container-based applications.
AWS Load Balancer Controller
The AWS Load Balancer Controller manages AWS Elastic Load Balancers for a Kubernetes cluster.
AWS Fargate is a serverless compute engine for containers that works with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS). Fargate removes the need to provision and manage servers, lets you specify and pay for resources per application, and improves security through application isolation by design.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.