AWS Partner Network (APN) Blog
Armakuni uses Karpenter for resilient workload scaling and cost efficiencies
By Justin Mitchell, CTO North America – Armakuni
By Abhinav Dahagam, Head of AWS – Armakuni
By Akshat Trivedi, Sr. Solution Consultant – Armakuni
By Venkata Narasimha Venugopal Gomatham, Sr. Partner Solutions Architect – AWS
 |
| Armakuni |
 |
Many customers use Kubernetes clusters to host their critical workloads. Traditional autoscalers are often slow and are not flexible. Customers are seeking flexible options to allocate right-sized compute resources and faster auto-scaling to meet their application demands and reduce operational expenses.
In this blog post, we’ll share how we increased the performance and decreased operational costs for a business-critical container-based workload for the customer. The solution is to use Karpenter, an open-source autoscaler on AWS Elastic Kubernetes Services (EKS).
Armakuni, an AWS Premier Tier Consulting Partner, provides wide variety of cloud services. Major services include Cloud Advisory, Cloud Deployment, Cloud Integration, CloudOps, Data Engineering, Migration and Modernization services. Armakuni differentiates with strong capabilities in Cloud Native Development, DevOps, IoT, AI, GenAI, Big Data and ML.
Challenges — Inefficient cluster management and slow autoscaling
Customer has a codeless AI platform that leverages machine learning and optimization algorithms to analyze audience, media platforms, budget allocation, and other marketing data. The platform is designed to help customer brands to scale their marketing through data infrastructure and modeling. It enables data-driven decision-making in the face of complex consumer behavior.
The client was struggling to optimize their cloud infrastructure, particularly in terms of managing workloads efficiently. One of the key issues was the cost associated with maintaining their Kubernetes cluster. The traditional autoscaler setup, although dependable, strained the client’s resources. The need for a solution that balances the performance with cost-effectiveness became apparent.
The application experienced slow autoscaling processes. It would often take up to 120 seconds to dynamically adjust to varying workloads. The long wait time for resources to scale up or down led to operational bottlenecks, limiting the company’s ability to adapt quickly to fluctuating workloads.
This led to an impact on the customer’s clientele experience. The inability to promptly manage spikes in user traffic undermined the quality of their service. Due to in slower auto-scaling and inefficient resource management, excessive resources are allocated to accommodate potential peaks in the workload.
Karpenter on Amazon EKS for Autoscaling
To address the customer’s challenges, Armakuni implemented the solution using Amazon EKS, a managed Kubernetes service offered by AWS to simplify the deployment, management, and scaling of containerized applications using Kubernetes. It is a robust and highly available service that takes care of the underlying Kubernetes complexity. They used Karpenter, an open-source auto-scaling tool built by AWS, as an autoscaler. Karpenter works on any Kubernetes environment anywhere.
Figure 1 – Auto-scaling with Karpenter
When Karpenter is installed in a Kubernetes environment, it monitors events within Kubernetes cluster. Based on the events monitored, it makes decisions and signal the underlying cloud infrastructure to provision required compute resources for auto-scaling. There is an option to add constraints on what types of resources it should provision. This allowed Armakuni to focus on application development rather than infrastructure management. Karpenter is installed and configured in the EKS cluster to manage auto-scaling of pods in the EKS cluster. Find the details on how to install Karpenter here.
Following are the reasons for selecting Karpenter:
1. Efficient instance launches
Karpenter’s ability to launch instances in a matter of seconds is vital to our use-case. Suppose the company experiences a sudden influx of users, resulting in a surge in traffic to their analytics platform. When the Kubernetes cluster detects this increase in demand, Karpenter automatically provisions additional instances, ensuring there is capacity to handle the load. This swift instance launch capability minimizes latency and ensures a smooth user experience during peak periods. Karpenter reduced cluster provisioning times from up to 120 seconds to a remarkable 60 – 70 seconds.
2. Right-sized instance provisioning
Karpenter is an intelligent Kubernetes autoscaler that goes beyond faster EC2 instance launches by intelligently provisioning the right size resources for the workload. Instead of opting for a one-size-fits-all approach, Karpenter right-sizes instances based on workload requirements. Suppose the company experiences a surge in traffic for a specific analytics workload that requires substantial compute power but only for a limited duration. Karpenter recognizes this spike and intelligently selects instance types and sizes that match the workload requirements. Karpenter automatically scales down resources when traffic subsides. This efficient resource management helps businesses reduce costs.
Business Outcomes
By using the automated scaling capabilities of Karpenter on EKS, the company was able to realize improvements in two key areas:
1. Substantial cost-savings
Karpenter allowed access to a wide range of spot instances and select the optimal instance types for the workloads. Initially, the client’s monthly EC2 costs ranged from $4,200 to $4,400. After implementing Karpenter, the monthly EC2 expenses decreased to $2,400-$2,600, which amounts to 57% cost savings.
Figure 2 – Cost Comparison before and after Karpenter implementation
2. Resilient infrastructure
Karpenter opened new possibilities for the company’s infrastructure resilience. It enables the utilization of various spot instance families within a Kubernetes cluster. This ensures reliable node allocation with minimized downtime risks.
If spot instances become unavailable, Karpenter quickly allocates alternatives, maintaining a seamless and uninterrupted operation of their Kubernetes cluster. This translates to a highly dependable and resilient system capable of handling fluctuating workloads without disruptions.
To integrate Karpenter effectively and harnessing its full potential, we should consider the following prerequisites.
Prerequisites for adopting Karpenter
- Kubernetes Cluster: An up and running, stable, and well-configured Kubernetes cluster, such as Amazon EKS.
- Monitoring and Metrics: Implement a robust monitoring system to collect performance metrics from your cluster. Tools such as Prometheus and Grafana are invaluable for this purpose.
- Resource Awareness: Understand your application’s resource requirements and usage patterns to set appropriate scaling policies.
- Security Measures: Ensure proper security measures are in place to protect your cluster and autoscaler configurations.
- Testing Environment: Thoroughly test the autoscaler in a non-production environment to validate its behavior under various conditions like sudden spikes in load, sustained high demand, and unexpected failures.
These prerequisites lay the foundation for a successful auto-scaling implementation, which helps you to optimize resource utilization and cost-effectiveness in production environments.
Conclusion
Armakuni’s solution using Amazon EKS and Karpenter overcame major technical and operational overheads. It not only improved application performance due to swift autoscaling, but also reduced EC2 costs of the workload by about ~57%. The integration of Karpenter, coupled with a resilient AWS infrastructure fortified customer’s business critical application against downtime.
.
Armakuni – AWS Partner Spotlight
Armakuni is an AWS Premier Tier Consulting Partner that provides wide variety of cloud services. Major services include Cloud Advisory, Cloud Deployment, Cloud Integration, CloudOps, Data Engineering, Migration and Modernization services. Armakuni differentiates with strong capabilities in Cloud Native Development, DevOps, IoT, AI, GenAI, Big Data and ML.