RouterBench - Private LLM Gateway Deployment on AWS AMI

BEFORE YOU LAUNCH

1. A domain you control. Three hostnames are served under a base domain you choose, e.g. example.com: api.example.com - the API (/v1/...) app.example.com - the console admin.example.com - the admin app
2. A VPC and a public subnet, so Let's Encrypt can reach the instance on port 80 to issue certificates. If the instance cannot be public, set TlsMode = internal (browsers will show a certificate warning).
3. An administrator email address. This becomes the first account.

LAUNCH

1. Launch the CloudFormation template from this listing.

2. Fill in Base domain and Admin email. If the domain is hosted in Route 53, supply its hosted zone ID and the DNS records are created for you.

3. Wait for the stack to reach CREATE_COMPLETE (about 5-10 minutes).

4. If you did not supply a hosted zone, create the DNS records shown in the DnsInstructions stack output: three A records, or one wildcard *.example.com, pointing at the PublicIp output. Certificates are issued automatically within a few minutes of those records resolving.

5. Open the ConsoleUrl stack output and sign in with your admin email. A strong password is generated on the instance at first boot; it is not stored in the AMI and differs on every instance. Read it once over Session Manager (EC2 console, Connect, Session Manager):

   sudo cat /var/lib/routerbench/data/config/initial-admin-password

   RouterBench then requires you to set a new password before any other page or API call will work. Delete that file once you have signed in.

   All other credentials (database password, cache password, session signing key, encryption key) are generated randomly on the instance at first boot and are never baked into the AMI.

FIRST STEPS

• Add provider API keys in the admin app, then enable models in the catalogue.

• Create an API key in the console and call the gateway:

  curl https://api .<your-domain>/v1/chat/completions
  -H "Authorization: Bearer <your-key>"
  -H "Content-Type: application/json"
  -d '{"model":"auto","messages":[{"role":"user","content":"hello"}]}'

• Invite your team from the console, or configure single sign-on.

ACCESSING THE INSTANCE

Use Session Manager: EC2 console -> Connect -> Session Manager. The instance role permits it. SSH is closed unless you set SshCidr and KeyName at launch, and password login is disabled.

sudo systemctl status routerbench cd /opt/routerbench && sudo docker compose ps sudo docker compose logs -f api gateway sudo cat /var/log/routerbench-firstboot.log

CONFIGURATION

Settings live in /opt/routerbench/.env on the data volume. Edit it as root, then run: sudo systemctl restart routerbench

Email (invites, password resets): set SMTP_HOST, SMTP_PORT, SMTP_USER, SMTP_PASS and SMTP_FROM_EMAIL. Until these are set, no email is sent. Single sign-on (OIDC - Entra ID, Okta, Google Workspace): set OIDC_DISCOVERY_URL, OIDC_CLIENT_ID, OIDC_CLIENT_SECRET and SSO_ENABLED=true. Register this redirect URI with your provider: https://api .<your-domain>/auth/oauth2/callback/sso

Do not add PORT to the .env file.

BACKUP AND RESTORE

All state lives on the encrypted EBS data volume, which the stack snapshots daily. To restore, launch a new stack and pass the existing volume ID as ExistingDataVolumeId.