JEV-27B Decision Model

WHAT YOU GET: one Ubuntu 24.04 GPU instance with the JEV-27B weights (54 GB) and the vLLM engine already on the root volume. Nothing is downloaded at first boot. API: OpenAI-compatible routes plus POST /v1/decide on port 8080, protected by a bearer key.

LAUNCH (EC2 console or CLI)

1. Instance type: g6e.2xlarge (recommended, 1x L40S 48 GB). g6e.4xlarge/8xlarge = same GPU, more CPU/RAM. g6e.12xlarge = 4 GPUs (tensor parallel 4).
2. Key pair: not required. Port 22 is closed; use AWS Systems Manager Session Manager.
3. Network: a subnet with a route to AWS APIs (public subnet, or private subnet with NAT / VPC endpoints for ssm, ssmmessages, ec2messages). Required only for Session Manager and OS updates.
4. Security group: allow TCP 8080 (API) and 8081 (readiness) ONLY from your client IP range. Replace the suggested 0.0.0.0/0 rule. Do not open 22 or 8000.
5. Storage: root volume >= 300 GiB gp3 (the AMI default is 300 GiB at 1000 MB/s; keep it).
6. IAM instance profile: a role with the AmazonSSMManagedInstanceCore policy (needed to read the key).
7. User data (optional, JSON): {"api_key":"YOUR-KEY-32-TO-128-CHARS","max_model_len":32768}. Leave empty to have a key generated on first boot. User data is read on the first boot only.

FIRST BOOT (about 10-15 minutes) The service verifies the weights, starts the engine and self-tests the API. Readiness: curl -s -o /dev/null -w '%{http_code}\n' http://INSTANCE_IP:8081/readyz -> 200 when ready. Progress/logs: aws ssm start-session --target INSTANCE_ID ; then sudo journalctl -u inferenceops-standalone -f

GET THE API KEY (root-only file, never printed in logs) aws ssm start-session --target INSTANCE_ID sudo cat /etc/inferenceops/api-key -> {"api_key": "..."}

CALL THE API API=http://INSTANCE_IP:8080/v1 ; KEY=... curl -s $API/models -H "Authorization: Bearer $KEY" curl -s $API/chat/completions -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -d '{"model":"jev-27b","messages":[{"role":"user","content":"hello"}],"max_tokens":64,"chat_template_kwargs":{"enable_thinking":false}}' curl -s $API/decide -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -d '{"model":"jev-27b","kind":"choice","state":"Customer charged twice, wants a refund.","question":"Which team handles it?","options":["Billing","Technical","Account","General"]}' Any OpenAI SDK works with base_url=$API and api_key=$KEY. Wrong key -> 401. The full contract is in /opt/inferenceops/gateway/openapi.yaml on the instance.

SECURITY: the API is plain HTTP. For internet clients put an Application Load Balancer with an ACM certificate in front (target 8080, health check 8081 /readyz). Engine port 8000 is bound to localhost only. IMDSv2 is enforced. STOP CHARGES: stop or terminate the instance. Model and key are on the root volume; a stopped instance resumes with the same key.