Overview
The Friendli Container cuts LLM inference serving costs by 40-80% while providing low latency and high throughput during LLM serving. Friendli Container employs various specialized optimizations including our groundbreaking iteration batching (a.k.a. continous batching) technology, protected by patents in the United States and Korea. Experience the extraordinary performance of LLM serving on the Friendli Container with EKS add-on!
Highlights
- Innovation for high speed and low cost
- Effortless deployment
- Automatic management
Details
Features and programs
Financing for AWS Marketplace purchases
Quick Launch
Pricing
Dimension | Description | Cost/month |
|---|---|---|
A100 | License to run Friendli Container per one NVIDIA A100. | $548.00 |
H100 | License to run Friendli Container per one NVIDIA H100. | $730.00 |
A10G | License to run Friendli Container per one NVIDIA A10G. | $110.00 |
L4 | License to run Friendli Container per one NVIDIA L4. | $100.00 |
Vendor refund policy
We do not support any refunds currently.
Legal
Vendor terms and conditions
Content disclaimer
Delivery details
Friendli Container Operator via Helm Chart 0.1.1
- Amazon EKS
Helm chart
Helm charts are Kubernetes YAML manifests combined into a single package that can be installed on Kubernetes clusters. The containerized application is deployed on a cluster by running a single Helm install command to install the seller-provided Helm chart.
Version release notes
Friendli Container Operator 0.1.2
Additional details
Usage instructions
Detailed instructions: https://github.com/friendliai/examples/tree/main/aws/eks
Support
Vendor support
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
