Listing Thumbnail

    LocalAI - Hardened Self-Hosted OpenAI-Compatible API Server

     Info
    Sold by: Lynxroute 
    Deployed on AWS
    Free Trial
    This is a repackaged open source software product wherein additional charges apply for hardening, security configuration, and setup support. LocalAI is an open-source, drop-in OpenAI-compatible API server delivered as a single statically linked Go binary - chat, embeddings, images, and audio served from your own VM with no GPU and no cloud egress. This Lynxroute build is hardened and ready out of the box: per-instance LOCALAI_API_KEY at first boot, host Nginx with TLS in front of the API, the LocalAI process bound to loopback, UFW firewall pre-configured, and a CIS Level 1 hardened Ubuntu 24.04 LTS base. MIT license - fully auditable, no vendor lock-in.

    Overview

    This is a repackaged open source software product wherein additional charges apply for hardening, security configuration, and setup support.

    WHAT IS LOCALAI

    LocalAI is an open-source, OpenAI-compatible inference server delivered as a single statically linked Go binary. It exposes the same REST surface as OpenAI - /v1/chat/completions, /v1/embeddings, /v1/audio/transcriptions, /v1/audio/speech, /v1/images/generations, /v1/models, plus a built-in /models gallery API - so any client written for OpenAI (openai-python, openai-node, LangChain OpenAI provider, LlamaIndex, IDE plugins) talks to it unchanged with one base URL swap. Inference runs entirely on CPU using llama.cpp and Whisper.cpp backends; GGUF model files load from local disk under /var/lib/localai/models and persist across restarts - no external database, no Redis, no cloud egress to OpenAI or Anthropic. Bring your own GGUF model (LLaMA family, Mistral, Qwen, Phi, Gemma) or install one in two clicks from the bundled gallery (mudler/LocalAI gallery index). MIT license, no vendor lock-in.

    WHAT THIS AMI ADDS

    Security hardening:

    • Per-instance LOCALAI_API_KEY (24-char random) generated at first boot, enforced by LocalAI for every /v1/, /models, /embeddings, /audio, /images call - never baked into the AMI
    • The same API key authenticates OpenAI-SDK REST clients (Authorization: Bearer) and the Web UI ("Login with API Token") - one credential, one rotation surface
    • LocalAI bound to 127.0.0.1:8080 - the API process never listens on a public interface
    • Host Nginx fronts port 443 with a self-signed certificate; HTTP redirects to HTTPS; security headers (X-Content-Type-Options, X-Frame-Options, Referrer-Policy) applied
    • UFW firewall pre-configured - only TCP 22, 80, 443 are exposed
    • fail2ban, AppArmor
    • CVE scan - every image is scanned for vulnerabilities before release

    OS hardening (CIS Level 1):

    • CIS Ubuntu 24.04 LTS Level 1 benchmark applied via ansible-lockdown
    • auditd, SSH hardening, kernel hardening, IMDSv2 enforced

    Compliance artifacts:

    • SBOM - CycloneDX 1.6 at /etc/lynxroute/sbom.json
    • CIS Conformance Report at /etc/lynxroute/cis-report.html
    • CIS Tailored Profile at /usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md

    Highlights

    • LocalAI security baked in: per-instance LOCALAI_API_KEY at first boot, LocalAI bound to 127.0.0.1, host Nginx with TLS on :443, all auth enforced by LocalAI for /v1/, /models, /embeddings, /audio, /images.
    • CIS Level 1 hardened Ubuntu 24.04 LTS: auditd, fail2ban, AppArmor, SSH key-only, IMDSv2 enforced. CVE-scanned before every release. SBOM (CycloneDX) and CIS Conformance Report included.
    • Drop-in OpenAI replacement on your own VM: /v1/chat, /v1/embeddings, /v1/images, /v1/audio routed by a single Go binary; openai-python, LangChain, and LlamaIndex work unchanged. CPU-only inference - no GPU, no NVIDIA stack, no per-call billing. MIT license - no vendor lock-in, ever.

    Details

    Delivery method

    Delivery option
    64-bit (x86) Amazon Machine Image (AMI)

    Latest version

    Operating system
    Ubuntu 24.04

    Deployed on AWS

    Features and programs

    Financing for AWS Marketplace purchases

    AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
    Financing for AWS Marketplace purchases

    Pricing

    Free trial

    Try this product free for 5 days according to the free trial terms set by the vendor. Usage-based pricing is in effect for usage beyond the free trial terms. Your free trial gets automatically converted to a paid subscription when the trial ends, but may be canceled any time before that.

    LocalAI - Hardened Self-Hosted OpenAI-Compatible API Server

     Info
    Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.
    Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator  to estimate your infrastructure costs.

    Usage costs (5)

     Info
    Dimension
    Cost/hour
    t3.large
    Recommended
    $0.05
    m6i.2xlarge
    $0.05
    m6i.large
    $0.05
    m6i.xlarge
    $0.05
    t3.medium
    $0.03

    AI Insights

     Info

    Dimensions summary

    You pay by the hour for the EC2 instance size you run. The five dimensions map to specific instance types, so your cost scales with the compute you choose. The t3.medium and t3.large options run on burstable general-purpose instances. The m6i.large, m6i.xlarge, and m6i.2xlarge options step up through general-purpose instances with more vCPU and memory. All run the same hardened image, so you select a size based on workload needs. Billing runs through the marketplace, and the hourly rate applies as long as the instance stays running.

    Top-of-mind questions for buyers

    The hourly software charge meters running time only. A fully stopped instance does not accrue the per-hour software fee. Underlying AWS storage for the instance volume may still apply while it is stopped. You pay the hourly rate as long as the instance stays running.
    Each dimension maps to one running EC2 instance of that size. You get a security-hardened Ubuntu 24.04 LTS image with the LocalAI server, CIS Level 1 hardening, a CycloneDX SBOM, and a CIS Conformance Report bundled inside. The larger m6i sizes add more vCPU and memory for heavier workloads.
    The hourly rate covers the software image and expert support with a 24-hour response target. It does not cover custom development, third-party integrations, or operation of your wider cloud environment. All billing and refunds are handled by the marketplace, not the vendor.
    lynxroute.com
    Helpful?

    Vendor refund policy

    We do not offer refunds for this product. AWS infrastructure charges (EC2, EBS, data transfer) are billed separately by AWS and are not refundable by us.

    How can we make this page better?

    Tell us how we can improve this page, or report an issue with this product.
    Tell us how we can improve this page, or report an issue with this product.

    Legal

    Vendor terms and conditions

    Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .

    Content disclaimer

    Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

    Usage information

     Info

    Delivery details

    64-bit (x86) Amazon Machine Image (AMI)

    Amazon Machine Image (AMI)

    An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.

    Version release notes

    LocalAI 4.12.1 r2

    • Security: fixes CVE-2026-111497, a critical (CVSS 9.8) buffer overflow in the Brotli decompression library that lets a crafted compressed stream crash a process or execute code. Ships Ubuntu brotli 1.1.0-2ubuntu0.1 (USN-8956-1). Every earlier image of this product carries the vulnerable library; replace running instances or run: sudo apt-get update && sudo apt-get install --only-upgrade libbrotli1 brotli python3-brotli
    • The application version is unchanged (LocalAI 4.12.1); this is a security rebuild
    • Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base, picking up every operating-system security update published since the previous build

    Additional details

    Usage instructions

    1. Launch instance (t3.large recommended; t3.medium minimum for the smallest quantized models)
    2. Open Security Group - allow TCP 443 from your IP only
    3. SSH: ssh -i key.pem ubuntu@<PUBLIC_IP>
    4. Read credentials: sudo cat /root/localai-credentials.txt
    5. Open https://<PUBLIC_IP>/ in your browser - accept the self-signed certificate warning
    6. Click "Login with API Token", paste the API key from the credentials file
    7. Web UI: Models tab -> pick a model from the gallery -> Install. Or drop a GGUF file into /var/lib/localai/models and run sudo systemctl restart local-ai
    8. REST API (OpenAI-compatible) with curl: curl -k https://<PUBLIC_IP>/v1/chat/completions
      -H "Authorization: Bearer <api-key>"
      -H "Content-Type: application/json"
      -d '{"model":"<your-model>","messages":[{"role":"user","content":"hi"}]}'

    The same API key authenticates Web UI sessions and OpenAI-SDK REST clients (Authorization: Bearer). Credentials are saved to /root/localai-credentials.txt at first boot. Models are NOT pre-loaded - install on demand from the gallery or upload your own GGUF files. Replace the self-signed TLS certificate with a CA-signed certificate for production use: sudo certbot --nginx -d YOUR_DOMAIN

    Resources

    Vendor resources

    Support

    Vendor support

    Lynxroute is not affiliated with the LocalAI project this AMI packages the MIT-licensed open-source LocalAI binary as a self-hosted EC2 service.

    Visit us online: https://lynxroute.com 

    For LocalAI documentation: https://localai.io/basics/getting_started/  For LocalAI upstream issues:

    AWS infrastructure support

    AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.

    Similar products

    Customer reviews

    Ratings and reviews

     Info
    0 ratings
    5 star
    4 star
    3 star
    2 star
    1 star
    0%
    0%
    0%
    0%
    0%
    0 reviews
    No customer reviews yet
    Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.