Sold by: Waltsoft Inc.
Deployed on AWS
Self-hosted LLM proxy with virtual API keys, cost tracking, rate limits, and load balancing across Claude, GPT-4, Gemini, and 100+ models.
Overview
LiteLLM is the enterprise LLM proxy used by thousands of teams to centralize and control AI API spend. It presents a single OpenAI-compatible endpoint to your applications while routing requests to Claude, GPT-4, Gemini, Azure OpenAI, or any of 100+ supported providers. Virtual API keys let you issue scoped credentials per team or application with individual budget limits and rate controls. A built-in management dashboard tracks spend, latency, and usage by key, model, and team. All existing OpenAI SDK integrations work without code changes.
Highlights
- Single OpenAI-compatible endpoint routes to Claude, GPT-4, Gemini, and 100+ providers with zero SDK changes
- Virtual API keys with per-team budget limits and rate controls -- central cost governance across all AI spend
- Built-in dashboard tracks spend, latency, and usage per key, model, and team for full observability
Details
Sold by
Categories
Delivery method
Supported services
Delivery option
ECS Container
Latest version
Operating system
Linux
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Dimension | Description | Cost/hour |
|---|---|---|
LiteLLM Gateway - Enterprise LLM Proxy for Claude and OpenAI | Hourly software fee per running container task | $0.01 |
Vendor refund policy
Contact support@waltsoft.net within 30 days.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
ECS Container
Supported services: Learn more
- Amazon ECS
Container image
Containers are lightweight, portable execution environments that wrap server application software in a filesystem that includes everything it needs to run. Container applications run on supported container runtimes and orchestration services, such as Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS). Both eliminate the need for you to install and operate your own container orchestration software by managing and scheduling containers on a scalable cluster of virtual machines.
Version release notes
Initial container release v1.0.0.
Additional details
Usage instructions
Quick Start
Pull the container image
Run with Docker
Deploy on Amazon ECS
- Create an ECS cluster (Fargate or EC2 launch type)
- Register a task definition using the image above
- Map container port 4000 to the host or load balancer
- Attach a persistent volume at /data for configuration
- Run the service and note the assigned public IP or ALB DNS
Access the application
- Open http://YOUR-HOST:4000 in your browser
- Complete the first-run setup wizard if prompted
- Create your administrator account
Port Mappings
- 4000: Primary web interface
Persistent Data
Mount a volume at /data to preserve configuration across restarts.
Support
Contact support@waltsoft.net for deployment assistance. Response within 24 hours.
Support
Vendor support
For support, contact support@waltsoft.net or visit
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
REQUIRES PRIVATE OFFER
To purchase LiteLLM Enterprise Self-Hosted, please reach out to sales@berri.ai for a Private Offer.
LiteLLM is an OpenAI compatible Proxy Server (LLM Gateway) to call 2,000+ LLM APIs using the OpenAI format Bedrock, Huggingface, VertexAI, TogetherAI, Azure OpenAI, OpenAI, etc. Get started with Opensource LiteLLM here: https://github.com/BerriAI/litellm (40,000+ Github Stars)
An AI Gateway acts as the central control layer for all your LLM traffic - managing logging, retries, cost tracking, and robust routing. In this project, we deploy an open-source LiteLLM Server on AWS Fargate (with private ALB), configured for auto-scaling, metrics, and graceful shutdowns. We set it up as your unified API endpoint to models like Amazon Bedrock, OpenAI, Anthropic, and more. You get a fully functional gateway you can run in production, with deep visibility, centralized governance, and predictable billing. It’s perfect for teams that want reliable, consistent LLM access without engineering complexity.
Launch a ready to use Laravel AI development server with Laravel 13, MySQL, Redis, Open WebUI, LiteLLM, Adminer, and Mailpit on Ubuntu 24.04. This product has charges associated with it for the provision and deployment of the application and AMI support.
Private AI inference gateway with Open WebUI, Ollama, LiteLLM, local CPU model serving, and an OpenAI compatible API endpoint on Ubuntu 24.04. This product has charges associated with it for the provision and deployment of the application and AMI support.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.