Sold by: cloudimg
Deployed on AWS
Free Trial
AWS Free Tier
This product has charges associated with it for seller support. LiteLLM, the open source OpenAI-compatible gateway for 100+ LLM providers, preinstalled behind an nginx reverse proxy on port 80 and gated by a unique master key generated on first boot. Backed by 24/7 cloudimg support.
Overview
This is a repackaged open source software product wherein additional charges apply for cloudimg support services.
Overview LiteLLM is the popular open source gateway that puts a single OpenAI-compatible API in front of more than 100 large language model providers, including OpenAI, Anthropic, Amazon Bedrock, Azure OpenAI, Google, and self hosted models. It adds load balancing, automatic fallbacks, rate limiting, virtual API keys and spend tracking, so your applications integrate once and switch or combine providers freely. This image delivers the LiteLLM proxy fully installed and configured as a system service, so a production ready LLM gateway is running within minutes of launch. The current release available is LiteLLM 1.88.
Application Stack LiteLLM is installed into a dedicated Python virtual environment under /opt/litellm and run by an unprivileged service account on Python 3.12. It listens on the loopback address and an nginx reverse proxy fronts it on port 80, exposing the unauthenticated liveliness and readiness probes and the OpenAI-compatible API. A systemd service starts the proxy on boot and restarts it on failure.
Secure By Default The proxy is gated by a master key. This image generates a fresh master key, unique to your instance, on its first boot and writes it to a root only file. No shared or default key ships in the image. Callers authenticate with the standard Authorization Bearer header, and the master key also mints scoped virtual keys for your teams and applications.
Ready To Use Add your provider credentials to /etc/litellm/config.yaml, point your application's OpenAI base URL at the instance on port 80, and call any configured model through one API. Because LiteLLM calls out to external model endpoints, the image is CPU only and ships no model weights. Set a database URL to enable virtual keys, the admin UI and spend persistence.
cloudimg Support 24/7 technical support by email and chat. Help with LiteLLM deployment, configuring providers and model routing, load balancing and fallbacks, virtual keys and budgets, TLS termination and scaling.
Use Cases A single unified API for many LLM providers. Centralised key management, rate limiting and spend tracking for AI applications. A self hosted, in your own VPC LLM gateway for teams with data residency or compliance requirements. Provider failover and cost optimisation across models.
All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
Highlights
- LiteLLM, the open source OpenAI-compatible gateway for 100+ LLM providers, preinstalled as a systemd service in a Python virtual environment behind an nginx reverse proxy on port 80, ready to route requests with no manual setup
- Secure by default: the proxy is gated by a master key generated fresh for every instance on first boot and stored in a root only file, which also mints scoped virtual keys for teams and applications
- 24/7 technical support from cloudimg, with expert help configuring providers and model routing, load balancing and fallbacks, virtual keys and budgets, TLS and scaling
Details
Sold by
Categories
Delivery method
Delivery option
64-bit (x86) Amazon Machine Image (AMI)
Latest version
Operating system
Ubuntu 24.04
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Free trial
Try this product free for 7 days according to the free trial terms set by the vendor. Usage-based pricing is in effect for usage beyond the free trial terms. Your free trial gets automatically converted to a paid subscription when the trial ends, but may be canceled any time before that.
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time. Alternatively, you can pay upfront for a contract, which typically covers your anticipated usage for the contract duration. Any usage beyond contract will incur additional usage-based costs.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
If you are an AWS Free Tier customer with a free plan, you are eligible to subscribe to this offer. You can use free credits to cover the cost of eligible AWS infrastructure. See AWS Free Tier for more details. If you created an AWS account before July 15th, 2025, and qualify for the Legacy AWS Free Tier, Amazon EC2 charges for Micro instances are free for up to 750 hours per month. See Legacy AWS Free Tier for more details.
Dimension | Description | Cost/hour |
|---|---|---|
m5.large Recommended | m5.large | $0.08 |
t3.micro | t3.micro instance type | $0.04 |
t2.micro | t2.micro instance type | $0.04 |
g4ad.4xlarge | g4ad.4xlarge instance type | $0.24 |
m7a.32xlarge | m7a.32xlarge instance type | $0.24 |
x1e.8xlarge | x1e.8xlarge instance type | $0.24 |
c5d.2xlarge | c5d.2xlarge instance type | $0.24 |
r6a.16xlarge | r6a.16xlarge instance type | $0.24 |
m6a.xlarge | m6a.xlarge instance type | $0.12 |
c8a.large | c8a.large instance type | $0.08 |
Vendor refund policy
Refunds available on request.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
64-bit (x86) Amazon Machine Image (AMI)
Amazon Machine Image (AMI)
An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.
Version release notes
Initial release of the LiteLLM 1.88 OpenAI-compatible LLM gateway.
Additional details
Usage instructions
Connect via SSH on port 22 as the default login user for your operating system variant (the user guide lists it per variant; on Ubuntu it is 'ubuntu'). The LiteLLM proxy is served by nginx on port 80. Retrieve the generated master key with: sudo cat /root/litellm-credentials.txt. Health checks are open at http://<instance-public-ip>/health/liveliness; the OpenAI-compatible API requires the master key as 'Authorization: Bearer <master-key>'. Add your provider credentials and models to /etc/litellm/config.yaml, then restart with 'sudo systemctl restart litellm.service'. Point your application's OpenAI base URL at http://<instance-public-ip>/ and call any configured model. The proxy runs on loopback port 4000 and is managed with systemctl (litellm.service, nginx.service). Set DATABASE_URL in /etc/litellm/litellm.env to enable virtual keys, the admin UI and spend persistence. The user guide covers configuring providers, minting virtual keys, and enabling HTTPS.
Resources
Vendor resources
Support
Vendor support
cloudimg provides 24/7 technical support for this product by email and live chat. Our engineers help with deployment, configuration, updates, performance tuning and troubleshooting; critical issues receive a one hour average response. Contact support@cloudimg.co.uk .
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
REQUIRES PRIVATE OFFER
To purchase LiteLLM Enterprise Self-Hosted, please reach out to sales@berri.ai for a Private Offer.
LiteLLM is an OpenAI compatible Proxy Server (LLM Gateway) to call 2,000+ LLM APIs using the OpenAI format Bedrock, Huggingface, VertexAI, TogetherAI, Azure OpenAI, OpenAI, etc. Get started with Opensource LiteLLM here: https://github.com/BerriAI/litellm (40,000+ Github Stars)
An AI Gateway acts as the central control layer for all your LLM traffic - managing logging, retries, cost tracking, and robust routing. In this project, we deploy an open-source LiteLLM Server on AWS Fargate (with private ALB), configured for auto-scaling, metrics, and graceful shutdowns. We set it up as your unified API endpoint to models like Amazon Bedrock, OpenAI, Anthropic, and more. You get a fully functional gateway you can run in production, with deep visibility, centralized governance, and predictable billing. It’s perfect for teams that want reliable, consistent LLM access without engineering complexity.
Self-hosted proxy for 100+ LLM providers. Unified OpenAI-format API with load balancing, fallbacks, spend tracking, and rate limiting across all models.
Self-hosted proxy for 100+ LLM providers in OpenAI format. Load balancing, fallbacks, spend tracking, and rate limiting across all models.
Self-hosted AI gateway that unifies 100+ LLM APIs behind one OpenAI-compatible interface. Cost tracking, rate limiting, load balancing, and fallbacks for production AI.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.