Kiwify reduces video transcoding costs by 70% with AWS infrastructure

According to Kiwify's DevOps engineer, Fernando Silva, until recently, this process was carried out on another streaming platform. However, the company went offline on two occasions, taking down the courses held on the platform. “We have around eight thousand videos running daily. That’s why we started to explore how to solve this,” says Fernando Silvia. At the time, the company was already an AWS customer and contacted the vendor to seek alternatives.

After carrying out several proofs of concept, Kiwify decided to develop its own encoder, which would be processed on Amazon EC2 spot instances using Intel CPUs. “We spent about a month carrying out tests before making this choice. We tested with spot and without spot to check the level of breakage and analyzed all of the numbers”, she recalls.

Although the use of CPU initially solved the problem, the joint work between AWS and Kiwify demonstrated that migrating processing to NVIDIA GPU-accelerated instances could bring even more performance and further cost reduction. Using NVIDIA L4 instances specifically, working with Amazon Elastic Kubernetes Service (Amazon EKS) and AWS Karpenter, Silva’s team was able to increase parallel processing leveraging GPU-accelerated EC2 spot instances, resulting in performance 12 times faster than that obtained with EC2 instances without acceleration.

With this result, in February the company decided to add GPU-accelerated EC2 spot instances to its processing, a project that was supported by NVIDIA. Today, Kiwify uses around 400 Spot GPU instances monthly, which, together with EC2 Spot instances without acceleration, process a volume of videos that can vary from 6 thousand to 12 thousand per day. In addition to its coding operations, Kiwify has 12 PB of data awaiting migration from the old platform to AWS.

With the implementation of GPU spot instances, Kiwify has reported a series of operational gains. Regarding content creators, the conversion of videos - which previously could take 30 minutes to 2 hours - is now done in a maximum of 20 minutes, drastically reducing new customer onboarding times.

“The AWS infrastructure also brought us stability. We started using spot instances in December and so far we haven't had any downtime,” Silva points out. The executive also points out that CDN (Content Delivery Network) output has grown 162% since January, with no increase in company costs. “Our issues ended after migrating 100% to AWS. For the customer, what matters is that the CDN is on the air and we've never had problems at this point,” he says.

Silva points out that all these benefits were obtained alongside a 70% reduction in costs compared to the model they adopted previously. “Since January, our client base has doubled and we have maintained video encoding costs. This result came with the use of spot, which allowed us to optimize processing and costs,” he says.