Sold by: Software Sushi
Deployed on AWS
Free Trial
LiteLLM Proxy gives all your applications one OpenAI-compatible API in front of the LLM providers you use, so you can switch or combine models and keep provider keys in one place without changing application code. This Ubuntu 24.04 image starts the open source LiteLLM 1.104.2 gateway on first boot, secured with HTTPS and a private master key.
Overview
LiteLLM is an open source LLM gateway. Your applications call one OpenAI-compatible endpoint (/v1/chat/completions, /v1/embeddings and related routes) and the proxy forwards each request to the provider and model you configure. Platform and application teams use it to stop hard-coding vendors: change models, add fallbacks or route different apps to different providers in one config file, while client code stays the same.
This image runs the gateway inside your own account, close to your workloads, with provider keys held on an instance you control instead of in every application.
What is included: the open source LiteLLM 1.104.2 proxy installed from hash-pinned Python packages and started automatically on 127.0.0.1:4000 over HTTPS. On first boot each instance generates its own certificate for localhost and a private master key (/var/lib/litellm-proxy/master.key, readable by root only). A built-in mock route lets you send an authenticated chat completion right away, before you add a paid provider. The proprietary litellm-enterprise package is not included and no provider credentials are baked in.
Getting started takes a few minutes: launch t3.medium (2 vCPU, 4 GiB RAM) with a 20 GiB or larger encrypted gp3 root volume and IMDSv2, attach an instance profile with AmazonSSMManagedInstanceCore, allow outbound HTTPS and keep inbound rules closed. Connect with Session Manager and run sudo -iu ubuntu bash /opt/sushi/docs/demo.sh; it checks startup, HTTPS, health, rejection of bad keys and a mock chat completion. Forward port 4000 to your workstation, trust the instance certificate and point your OpenAI-compatible client at https://localhost:4000/v1 with the master key. Then add your providers in /etc/litellm-proxy using the bundled guide and the linked LiteLLM documentation.
Pricing: a software fee of USD 0.02 per running t3.medium instance-hour. An optional five-day free trial for one concurrent instance changes software billing only and then converts to the hourly fee; the same full software is installed before and after it. This is not a separate trial or evaluation edition, and there is no product-imposed time, user or usage limit. Additional concurrent instances incur software fees. EC2, EBS, data transfer and any model-provider charges are billed separately, including during the free trial.
This is a repackaged open source software product wherein additional charges apply for independent AWS packaging, validation, maintenance and setup documentation. LiteLLM is authored by BerriAI and published under the MIT license; Software Sushi provides the image, pinned packages, setup guide and packaging support and does not claim BerriAI endorsement. Back up /etc/litellm-proxy/ and /var/lib/litellm-proxy/ together; new image versions replace instances rather than migrating them.
Highlights
- One OpenAI-compatible endpoint for all your apps. Switch, combine or fail over between LLM providers without code changes.
- Secure by default: HTTPS on localhost, with a certificate and master key generated for each instance on first boot.
- Working in minutes: a built-in mock route answers an authenticated chat completion before you add any paid provider.
Details
Sold by
Categories
Delivery method
Delivery option
64-bit (x86) Amazon Machine Image (AMI)
Latest version
Operating system
Ubuntu 24.04
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Free trial
Try this product free for 5 days according to the free trial terms set by the vendor. Usage-based pricing is in effect for usage beyond the free trial terms. Your free trial gets automatically converted to a paid subscription when the trial ends, but may be canceled any time before that.
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Dimension | Cost/hour |
|---|---|
t3.medium Recommended | $0.02 |
Vendor refund policy
We offer a full refund of software charges for the first 48 hours billing starts if the product does not perform as described. To request a refund, email info@softwaresushi.com with your EC2 instance ID and a description of the issue. AWS infrastructure costs (EC2, EBS, data transfer) are billed by AWS and are not eligible for refund by us.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
64-bit (x86) Amazon Machine Image (AMI)
Amazon Machine Image (AMI)
An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.
Version release notes
LiteLLM 1.104.2 (MIT, by BerriAI) on Ubuntu 24.04 LTS, packaged by Software Sushi from hash-pinned Python wheels. The proxy starts automatically on 127.0.0.1:4000 over HTTPS with a per-instance self-signed certificate and master key generated at first boot. Includes an offline mock model and a guided walkthrough (bash /opt/sushi/docs/demo.sh). Proprietary litellm-enterprise is not included and no provider credentials are baked in.
Additional details
Usage instructions
- Launch t3.medium (4 GiB RAM, 20 GiB or larger encrypted gp3, IMDSv2) with AmazonSSMManagedInstanceCore and outbound HTTPS. No inbound rules are needed.
- Connect with aws ssm start-session --target <instance-id>. After cloud-init finishes, run: sudo -iu ubuntu bash /opt/sushi/docs/demo.sh (expect SUSHI_CUSTOMER_DEMO_PASSED).
- The master key is at /var/lib/litellm-proxy/master.key (read with sudo) and the certificate at /var/lib/litellm-proxy/tls/cert.pem.
- From your workstation: aws ssm start-session --target <instance-id> --document-name AWS-StartPortForwardingSession --parameters '{"portNumber":["4000"],"localPortNumber":["4000"]}', trust the instance certificate and call https://localhost:4000/v1/chat/completions with Authorization: Bearer <master key>.
- Add real providers in /etc/litellm-proxy following /opt/sushi/docs/README.md. Back up /etc/litellm-proxy/ and /var/lib/litellm-proxy/ together. Documentation: https://docs.litellm.ai/docs/simple_proxy Support: info@softwaresushi.com
Resources
Support
Vendor support
Software Sushi provides installation and configuration guidance for this image. Contact info@softwaresushi.com with your product version and the steps you took; never send keys or private configuration. Guaranteed response times require a separate agreement. This offer does not include BerriAI support or proprietary enterprise features.
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
Production-ready Hermes AI messaging agent on hardened Ubuntu 24.04. Add your model-provider and Telegram credentials and run it in your AWS account.
REQUIRES PRIVATE OFFER
To purchase LiteLLM Enterprise Self-Hosted, please reach out to sales@berri.ai for a Private Offer.
LiteLLM is an OpenAI compatible Proxy Server (LLM Gateway) to call 2,000+ LLM APIs using the OpenAI format Bedrock, Huggingface, VertexAI, TogetherAI, Azure OpenAI, OpenAI, etc. Get started with Opensource LiteLLM here: https://github.com/BerriAI/litellm (40,000+ Github Stars)
Nanobot with localhost WebUI, native Python tools and a private Telegram channel. Customer cloud-model credentials are required after launch.
This is a repackaged open source software product wherein additional charges apply for AMI packaging, first-boot configuration and setup documentation. Run restic v0.19.1 on Ubuntu 24.04 with a daily backup timer, local encrypted repository, check and restore scripts, and optional customer S3 via IAM.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.