Omi scales AI powered 3D rendering solution, reducing costs by 70% on Amazon EKS

Omi was founded in 2020 to meet a business need: producing high-quality visuals in less time and at lower cost. Omi provides brands with AI-powered 3D modeling of photos and video. It is breaking traditional production constraints, reducing costs, and supporting the effortless creation of tailored content for all marketing channels using 3D technologies and generative AI. The company operates in more than 17 countries, serves more than 600 customers and 1,000 brands, and has more than 5,000 daily active users who create content for ecommerce, social media, and ads. Omi reduces the time and cost associated with creating marketing imagery for its customers.

From the start, Omi set out to create a rapid and efficient solution for generating social media photos, 3D animated images, and videos to share on social media, ecommerce websites, and other channels.

Initially, Omi handled creation locally and achieved photorealistic rendering through remote servers. However, the growing demand for quick and low-cost services necessitated integrating generative AI capabilities. This integration imposed significant demands on compute and GPU resources, requiring new approaches to scalability management and maintaining high availability while controlling costs. “It’s very important to us to always build up our infrastructure to get excellent performance without sacrificing costs too much,” says Paul Borensztein, cofounder of Omi.

Omi’s staff initially dedicated 60–70 percent of their infrastructure team time to optimizing GPU scaling time and costs. The company faced operational challenges such as slow container startup times caused by large Docker images and scalability complexities involving multiple auto scaling groups and instance types. The traditional cluster auto scaler struggled with managing availability zones and instance types for generative AI workloads, leading to capacity issues and significant customer wait times.

To address these needs, Omi adopted Karpenter, a Kubernetes node lifecycle manager built at AWS that balances application availability, performance, and cost by dynamically adjusting Kubernetes clusters. “Using Karpenter, we’ve lowered instance launch time; that has helped our application reduce the average render time for a photo from 5 minutes to 1.5 minutes, which is quite significant,” says Elliot Maincourt, senior backend developer and site reliability engineer at Omi.

After facing challenges with self-managing Kubernetes clusters in the past, Omi’s team decided to begin its journey by running clusters on Amazon Elastic Kubernetes Service (Amazon EKS), a managed service to start, run, and scale Kubernetes. To further enhance its infrastructure and address operational challenges, Omi adopted Karpenter, which quickly and automatically adapts to changes in application load and resource requirements, intelligently selecting cheaper instances across availability zones. “Amazon EKS and Karpenter have been amazing at handling the scale of our workloads, sometimes growing from a single GPU instance to more than 250 in a matter of minutes and keeping costs under control,” says Borensztein. Omi’s solution can scale rapidly up to 1,000 GPU instances and, including CPU instances, can run over 1,500 machines concurrently on Amazon EKS. (See figures 1 and 2 below.)

By using Karpenter node lifecycle management in combination with cost-optimized instances, Omi reduced its overall costs by 70 percent. Omi uses Karpenter to manage its Amazon EKS node group, scaling to more than 3,500 pods within a 24-hour period. With this rapid scaling capability, Omi keeps up with customer demands. “Using Karpenter on AWS, we saw huge improvements, including in deployment and tuning, in less than 2 months,” says Borensztein.

Additionally, to serve its customers efficiently, Omi uses Bottlerocket, a Linux-based open-source operating system that is purpose built by AWS for running containers. Omi uses the Bottlerocket immutable OS to prefetch container images from an external data store before launching Kubernetes pods on the read-only operating system. By enhancing its solution using AWS, Omi aims to accelerate performance for customers. Omi operates clusters with more than 1,000 nodes at peak scale. As a result of these improvements, Omi has built a robust solution that reduces maintenance time for its team. “Now, we can focus on scaling the business instead of managing our infrastructure,” says Maincourt. “I can trust our clusters on Amazon EKS; they have never broken, and we push them quite hard.”

Omi’s customers are also reaping the benefits of these improvements. With the introduction of new per-product pricing options, those customers can now perform thousands of renders or more for a single product. “With all the optimization we’ve done over the years, including using AWS for this big project, there is almost no limit for our customers,” says Borensztein.

In less than 2 months, Omi achieved significant improvements using Karpenter and Amazon EKS, solutions that facilitated a focus on scaling the business rather than managing infrastructure. The company spends significantly less time improving its rendering capacity and reassessing its infrastructure needs.

Omi’s growth has accelerated in the UK and across Europe. Omi continues to enhance its offerings using AWS and now has plans to expand its services to new regions, including the United States. It has developed a solution at the convergence of two mega-trends, namely generative AI and 3D, in the booming market of content creation.

“The key benefit of using AWS is reducing render time for our customers,” says Maincourt. “But at the same time, we have lowered our costs by 70 percent, which is a great business goal to achieve.”