Sold by: Ordinary Experts
Deployed on AWS
Free Trial
Deploy Open WebUI with GPU-accelerated LLM inference on your own AWS account. Run open-source models like Qwen, Phi-4, and Nemotron with vLLM on G6/G6e instances. OpenAI-compatible API included.
Overview
Open WebUI on AWS provides a production-ready deployment of Open WebUI with vLLM for GPU-accelerated large language model inference.
KEY FEATURES
- Open WebUI Interface - Modern, ChatGPT-style web interface for interacting with LLMs
- vLLM Backend - High-performance inference server with OpenAI-compatible API
- Flexible Model Support - Pre-tested models including Qwen 2.5 Coder (7B/14B/32B), Microsoft Phi-4, and NVIDIA Nemotron reasoning models
- GPU-Optimized - Runs on NVIDIA L4 (G6) and L40S (G6e) instances for cost-optimized or performance-optimized deployments
- Secure by Default - HTTPS via ALB, private VPC networking, and AWS Secrets Manager for credential management
- Persistent Storage - EBS-backed data volume survives instance replacements
- Developer Ready - OpenAI-compatible API works with tools like aider, LangChain, and any OpenAI SDK
SUPPORTED MODELS
- Qwen/Qwen2.5-Coder-7B-Instruct (coding, recommended for development)
- Qwen/Qwen2.5-Coder-14B-Instruct, Qwen/Qwen2.5-Coder-32B-Instruct (larger coding models)
- microsoft/phi-4 (general purpose, 14B)
- microsoft/Phi-4-mini-reasoning (fast reasoning, 3.8B)
- nvidia/OpenReasoning-Nemotron-7B/14B/32B (reasoning models)
- Custom models via ModelOverride parameter
INFRASTRUCTURE
CloudFormation deploys: VPC (or uses existing), Application Load Balancer with ACM certificate, Auto Scaling Group with GPU instance, Route53 DNS record, CloudWatch Logs, EBS persistent storage with automated backups, and AWS Secrets Manager integration.
IAM PERMISSIONS
EC2 Instance Role permissions:
- AmazonSSMManagedInstanceCore - Systems Manager Session Manager access
- CloudWatch Logs/Metrics - Stream logs and publish custom metrics
- EC2 Volume Operations - Attach persistent EBS data volume
- Secrets Manager - Manage application secrets (WEBUI_SECRET_KEY)
- SSM Parameter Store - Read optional custom configuration
Lambda Role (subnet-to-AZ mapping):
- AWSLambdaBasicExecutionRole - CloudWatch Logs access
- ec2:DescribeSubnets - Determine AZ for EBS volume placement
All IAM roles follow least-privilege principles.
EXTERNAL CONNECTIONS
This product connects to external services during deployment and operation:
- Hugging Face (huggingface.co): Downloads the selected LLM model on first instance boot. Model sizes range from 4GB to 60GB depending on selection.
- No customer data is transmitted to external services.
- All inference is performed locally on the EC2 instance.
GETTING STARTED
- Launch the CloudFormation stack with your DNS and certificate details
- Wait for deployment (model download takes 10-60 minutes depending on size)
- Access Open WebUI at your configured hostname
- Create your admin account on first login
- Generate API keys in Settings > Account for external tool integration
Highlights
- GPU-Accelerated Inference - Run LLMs on NVIDIA L4/L40S GPUs with vLLM for production-grade throughput and low latency
- Open WebUI Interface - Modern chat interface with conversation history, model switching, user management, and API key generation
- OpenAI-Compatible API - Integrate with existing tools like aider, LangChain, Continue.dev, or any OpenAI SDK client
Details
Sold by
Categories
Delivery method
Delivery option
CloudFormation Template v1.1.0
Open WebUI with vLLM on GPU
Latest version
Operating system
Ubuntu 24.04
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Free trial
Try this product free for 5 days according to the free trial terms set by the vendor. Usage-based pricing is in effect for usage beyond the free trial terms. Your free trial gets automatically converted to a paid subscription when the trial ends, but may be canceled any time before that.
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time. Alternatively, you can pay upfront for a contract, which typically covers your anticipated usage for the contract duration. Any usage beyond contract will incur additional usage-based costs.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Dimension | Cost/hour |
|---|---|
g6.24xlarge | $0.02 |
g6.48xlarge | $0.02 |
g6.16xlarge | $0.02 |
g6e.16xlarge | $0.02 |
g6.xlarge | $0.02 |
g6e.8xlarge | $0.02 |
g6.12xlarge | $0.02 |
g6.2xlarge | $0.02 |
g6.8xlarge | $0.02 |
g6e.12xlarge | $0.02 |
Vendor refund policy
We do not currently support refunds, but you can cancel at any time.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
CloudFormation Template v1.1.0
This CloudFormation template deploys a production-ready Open WebUI environment with vLLM backend for GPU-accelerated large language model inference.
WHAT'S DEPLOYED
- VPC with public and private subnets (or use your existing VPC)
- Application Load Balancer with HTTPS (requires ACM certificate)
- Auto Scaling Group with GPU-enabled EC2 instance (G6/G6e family)
- Route 53 DNS record pointing to the load balancer
- Persistent EBS data volume with automated daily backups via AWS Backup
- CloudWatch Log Groups for application and system logs
- AWS Secrets Manager secret for secure credential storage
- IAM roles with least-privilege permissions
FEATURES
- Pre-tested open-source models: Qwen3-8B, Qwen3-Coder-30B-A3B, OpenAI gpt-oss-20b, Microsoft Phi-4, NVIDIA OpenReasoning Nemotron, Zhipu GLM-4-9B
- OpenAI-compatible API for integration with agentic coding tools (opencode, aider) plus LangChain or any OpenAI SDK
- Custom model support via ModelOverride parameter
- Optional custom configuration via SSM Parameter Store
- Systems Manager Session Manager access (no SSH keys required)
REQUIREMENTS
- Route 53 hosted zone for DNS
- ACM certificate for HTTPS
- Sufficient service quota for GPU instances in your region
INSTANCE SIZING
- g6.xlarge (default): 24GB VRAM - Qwen3-8B, Phi-4-mini-reasoning
- g6e.xlarge / g6.2xlarge: 48GB VRAM - Phi-4, gpt-oss-20b, GLM-4-9B, Qwen3-Coder-30B-A3B (FP8)
- g6e.4xlarge / g6.4xlarge: 96GB VRAM - Qwen3-Coder-30B-A3B (FP16), OpenReasoning-Nemotron-32B
EXTERNAL CONNECTIONS
This product connects to Hugging Face (huggingface.co) to download the selected LLM model on first instance boot. Model sizes range from 4GB to 60GB. No customer data is transmitted. All inference is performed locally.
Deployment takes 15-60 minutes depending on model size due to model download and loading.
CloudFormation Template (CFT)
AWS CloudFormation templates are JSON or YAML-formatted text files that simplify provisioning and management on AWS. The templates describe the service or application architecture you want to deploy, and AWS CloudFormation uses those templates to provision and configure the required services (such as Amazon EC2 instances or Amazon RDS DB instances). The deployed application and associated resources are called a "stack."
Version release notes
- Upgrading to Open WebUI 0.10.2 (from 0.6.36)
- Includes Open WebUI 0.9.0 database schema migration (runs automatically on first boot; persistent /data volume preserved)
- Bumped CUDA toolkit and NVIDIA driver:
- CUDA 12.8.1 -> 13.0.2
- NVIDIA driver 570.124.06 -> 580.95.05
- cuDNN 9.8.0 -> 9.13.0 (cudnn-cuda-13 meta-package)
- Required for vLLM 0.20.0 (PyTorch built against newer CUDA)
- Pinned vLLM to 0.20.0 in packer install (was unpinned)
- Refreshed model dropdown:
- Added: Qwen/Qwen3-Coder-30B-A3B-Instruct (Apache 2.0, MoE 30B/3B-active, replaces Qwen2.5-Coder-14B/32B)
- Added: openai/gpt-oss-20b (Apache 2.0, general-purpose)
- Added: zai-org/GLM-4-9B-0414 (MIT, 9B general-purpose, 32K context; verified loading on g6e.4xlarge). No validated vLLM tool-call parser for the GLM-4-0414 dense series yet, so it uses the default config (tool calling off, 32K max-model-len).
- Removed: Qwen/Qwen2.5-Coder-14B-Instruct, Qwen/Qwen2.5-Coder-32B-Instruct, nvidia/OpenReasoning-Nemotron-7B, nvidia/OpenReasoning-Nemotron-14B
- Replaced default: Qwen/Qwen2.5-Coder-7B-Instruct -> Qwen/Qwen3-8B (Qwen3 generation, better tool-calling support, fits g6.xlarge)
- Fixed Open WebUI custom config SSM lookup hanging in systemd context: added explicit --region and timeout 30 wrapper to aws ssm get-parameter call in start-open-webui.sh
- Added model-aware vLLM tool-calling parser defaults so OpenAI-compatible clients (opencode, aider, etc.) work out of the box:
- Qwen / nvidia OpenReasoning Nemotron -> hermes parser
- microsoft/Phi-4-mini-reasoning -> phi4_mini_json parser
- openai/gpt-oss-* -> gpt_oss parser
- Custom overrides via CustomVllmConfigParameterArn still take precedence
- Added default --max-model-len + --kv-cache-dtype fp8 in start-vllm.sh:
- 32K context + FP8 KV cache for most models (16K for microsoft/phi-4 - that's its native limit)
- Required for agentic clients whose tool-definition system prompts exceed 8-12K tokens
- Fits comfortably on g6.xlarge / g6e.xlarge (24/48 GB VRAM) with FP8 quantization
- Refactored secret handling to use Secret construct's built-in WEBUI_SECRET_KEY generation (removed bespoke check-secrets.py)
- Updated oe-patterns-cdk-common to 4.5.1
- Bumped CloudFormation parameter name AsgAmiIdv101 -> AsgAmiIdv110
- Bumped devenv image to 2.8.6 (Node 22, EOL 2027-04-30)
Additional details
Usage instructions
See the GitHub repository for documentation: https://github.com/ordinaryexperts/aws-marketplace-oe-patterns-open-webui
Support
Vendor support
Support provided by Ordinary Experts
FREE ONBOARDING: We offer a complimentary one-hour onboarding session via Zoom to help you get started with your deployment, configure your environment, and answer any questions.
ONGOING SUPPORT: After onboarding, support is available via email. We respond within 2 business days. For urgent production issues, include "URGENT" in your email subject.
Documentation: https://github.com/ordinaryexperts/aws-marketplace-oe-patterns-open-webui Issues & Bug Reports: https://github.com/ordinaryexperts/aws-marketplace-oe-patterns-open-webui/issues Email: support@ordinaryexperts.com
Troubleshooting resources:
- CloudWatch Logs for application and system logs
- /var/log/vllm.log and /var/log/open-webui.log on the EC2 instance
- AWS Systems Manager Session Manager for secure shell access
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
The open source, Docker-based Open edX distribution designed for peace of mind
Sysdig is a simple tool for deep system visibility with native support for containers.
This product has charges associated with it for support. Ollama is a cutting-edge AI tool that empowers users to set up and run large language models, such as Llama 2 and 3, directly on their local machines. This innovative solution caters to a wide range of users, from experienced AI professionals to enthusiasts, enabling them to explore natural language processing without depending on cloud-based services.
The open version of ESM3; a frontier multimodal generative model that reasons over the sequences, structures, and functions of proteins
This product has charges associated with it, which includes support & maintenance. Production-ready Magento 2 Open Source eCommerce store AMI on Ubuntu 22.04 with Apache, MySQL, and ElasticSearch.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.