Delivery Hero Saves 70% Running Kubernetes on Amazon EC2 Spot Instances

Running Kubernetes workloads on Amazon EC2 Spot Instances also prompted Delivery Hero to discover ways to make its applications more operationally resilient. It did so by finding techniques that add stability to applications. “Delivery Hero is experiencing extremely high growth, and with that comes big concerns about uptime and application availability,” says Vondra. During its migration to Amazon EC2 Spot Instances, the company reviewed its applications and saw repeated issues tied to availability and termination that were fairly easy to solve. To assure high availability, the company added redundancy by running multiple instances so that if one instance terminated, another one started immediately. The company also employed scripts so that terminations occurred gracefully, without leaving any open or uncompleted transactions.

“The migration process prompted us to create a production-ready checklist and an Amazon EC2 Spot Instance checklist,” says Vondra. “Following these checklists increased our application resilience. Using these best practices made our migration easier.” Best practices included utilizing termination notice handlers, relying on a de-scheduler to prevent a pileup of Kubernetes nodes, and taking advantage of AWS Auto Scaling so the company can maintain sufficient capacity at all times.

AWS Auto Scaling plays a critical role for Delivery Hero. The company uses a microservices architecture to support the applications it operates in 39 countries. For legal and compliance reasons, the company’s services are deployed separately. Each country deploys about 10 services, totaling 390 different applications and deployments that must scale individually. Each country has peak times tied to common lunch and dinner hours, during which there can be a four- to fivefold increase in traffic and services must scale to match those peaks. With each application scaling flexibly within the cluster, the cluster itself must also be able to scale elastically.

The company also uses various Amazon EC2 instance types. Depending on the demand, the company’s needs can range from only a few instances to hundreds or thousands. The pods that run within Kubernetes are configured to run on specific instance types, based on the needs of each pod. Some pods contain CPU-intensive algorithms, such as the algorithm that assigns orders to delivery riders. Others contain applications that are more memory-heavy, so those run on memory-optimized instances.

By using AWS Auto Scaling with Kubernetes clusters that utilize Spot Instances, Delivery Hero can run Kubernetes in the most simple, scalable, and cost-effective way, effortlessly expanding and contracting to match demand. These groups contain a collection of Amazon EC2 instances treated as logical groupings, launch instances as needed, and manage scaling policies.

Today, all the company’s logistics apps run on Amazon EKS, as do most of its consumer-facing websites. Approximately 90 percent of the company’s applications that use Kubernetes deploy those applications on Amazon EKS. “We prefer managed services because we don’t have to manage infrastructure and can instead allocate our resources to developing and improving our applications. That’s what made Amazon EKS so appealing to Delivery Hero,” says Vondra.

Delivery Hero collects telemetry data from its delivery staff as well as personalized data from the customers who use its apps. As the company looks ahead, it is focused on training machine-learning models to improve its predictions for when food will be ready for pickup, for delivery times between two endpoints, and for staffing needs to ensure optimal delivery-staffing levels. That innovation continues to be powered through the cloud. “Delivery Hero has always relied on the elasticity of the cloud,” says Vondra. “We’ve never considered operating pre-provisioned hardware. By relying on AWS infrastructure, we can concentrate on continuously improving our products and better serving customers instead of thinking about how we deploy and run our applications.”