Sold by: Xinference Platform
Open-source LLM inference engine. Production-grade serving for 100+ enterprises on AWS GPU infrastructure.
Overview
Image
1
Image
3
Xinference -- Production-Grade LLM Inference
Xinference is a fast-growing open-source inference engine in the AI ecosystem -- 9,300+ GitHub stars, 140+ contributors, and 8.3 M downloads. The Xinference enterprise platform builds on this foundation to deliver production-ready LLM serving on AWS GPU infrastructure.
Key Features
- Unified OpenAI-compatible API -- REST and gRPC; swap commercial LLMs for any open-source model with a one-line config change.
- Multi-backend optimisation -- vLLM, SGLang, llama.cpp, and TensorRT-LLM backends with automatic selection per workload.
- Continuous batching & prefix caching -- high-throughput serving with automatic optimisation for shared system prompts and RAG context.
- Multi-modal model support -- LLMs, embedding, reranking, image generation, and speech models under a single platform.
- Distributed deployment -- scale across multiple GPUs and nodes with built-in cluster management.
Why Xinference on AWS Marketplace
- One-click deployment on Amazon EC2 GPU instances (G5, G6, P4d, P5).
- Enterprise-grade security -- private VPC deployment, encryption at rest and in transit, SOC 2-aligned controls.
- Trusted by 100+ large enterprise customers** across finance, healthcare, and public sector.
- 24/7 support with dedicated solutions engineering for Marketplace subscribers.
Common Use Cases
- Internal LLM serving for enterprise RAG and chatbot applications.
- Fine-tuned model deployment (LoRA / adapter-level).
- High-throughput batch inference for document processing.
- Real-time multi-modal AI applications.
Highlights
- Production-ready LLM serving -- deploy any open-source model in minutes with OpenAI-compatible APIs.
- Multi-backend optimisation -- automatic vLLM / SGLang / TensorRT-LLM selection for best throughput per workload.
- Enterprise-trusted -- 9,300+ GitHub stars, 100+ large enterprise deployments globally.
Details
Sold by
Delivery method
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Dimension | Description | Cost/unit |
|---|---|---|
GPU-hours T4 | accumulate hours for T4 16GB instances | $1.20 |
GPU-hours L4 | accumulate hours for L4 24GB instances | $1.35 |
GPU-hours A10G | accumulate hours for A10G 24GB instances | $1.624 |
GPU-hours A100(80GB)*8 | accumulate hours for A100 80GB instances | $42.82 |
GPU-hours H100(80GB)*8 | accumulate hours for H100 80GB instances | $93.02 |
GPU-hours H200(141GB)*8 | accumulate hours for H200 141GB instances | $98.75 |
GPU-hours B200(180GB)*8 | accumulate hours for B200 180GB instances | $148.11 |
N/A | N/A | $100.00 |
Dimensions summary
You pay by GPU-hours, so cost accumulates based on how long you run models on each GPU type. Each dimension maps to a specific NVIDIA GPU class and memory size: T4 16GB, L4 24GB, A10G 24GB, A100 80GB, H100 80GB, H200 141GB, and B200 180GB. The larger multi-GPU dimensions bill for eight-GPU configurations. You choose the GPU that fits your workload, and your total scales with the hours you use across the selected types. An additional N/A unit also appears in the table.
Top-of-mind questions for buyers
One GPU-hour is one hour a GPU runs your models. Cost accrues per hour of active runtime on the GPU class you select. The multi-GPU dimensions bill eight-GPU configurations, so each hour reflects the full eight-card node running together, not a single card.
Charges accumulate based on GPU-hours of runtime. The platform auto-scales replicas to track traffic, scaling out during spikes and back down afterward. Running fewer replicas reduces accrued hours. Underlying AWS infrastructure fees may still apply separately when instances remain provisioned.
Each GPU dimension bills independently by its own accumulated hours. Your invoice adds the hours from every GPU class you run. Larger multi-GPU nodes drive more cost per hour because they meter eight-GPU configurations. You choose the mix that fits your workload.
xinference.co
Helpful?
Vendor refund policy
NA
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
Software as a Service (SaaS)
SaaS delivers cloud-based software applications directly to customers over the internet. You can access these applications through a subscription model. You will pay recurring monthly usage fees through your AWS bill, while AWS handles deployment and infrastructure management, ensuring scalability, reliability, and seamless integration with other AWS services.
Support
Vendor support
info@xinference.co -- monitored 24/7. Enterprise-tier subscribers receive a dedicated solutions engineering contact.
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
HAMi is an open-source Kubernetes middleware providing GPU compute and memory isolation, flexible GPU slicing, and topology-aware scheduling maximizing utilization for AI inference and training workloads across NVIDIA GPUs and AWS Neuron devices; part of the CNCF ecosystem. Works with NVIDIA GPU Operator, vLLM Production Stack, and Xinference.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.