vLLM on Ubuntu 24.04 LTS - Supported by Easycloud

Once launched, SSH into the instance. The default username is 'ubuntu'. You can switch to the root user environment by running: sudo su -

This product requires SSH for initial setup: you must log in once to choose and download a model before the API port starts serving.

• OS: Ubuntu 24.04 LTS (x86_64, Minimal Installation)

• vLLM: 0.24 (OpenAI-compatible inference server, installed at /opt/vllm)

• Runtime: Python virtual environment with CUDA GPU support

• Config File: '/etc/vllm/vllm.env' (written by the vllm-model helper; do not edit by hand)

• No model is pre-installed (bring your own model): models are downloaded to /opt/models/<model-name> after you choose one

1. Launch on a GPU instance. Recommended: g4dn.xlarge (16 GB GPU memory, runs models up to about 4B parameters) or larger g5/g6 instances for bigger models. Match your model choice to the GPU memory of your instance - a model that is too large will fail to start.

2. The instance boots in under 3 minutes. At this point the vllm service is intentionally inactive and port 8000 is not yet listening - the login banner shows NO MODEL CONFIGURED YET. This is the normal factory state, not a fault.

3. SSH in and run one command to download and activate your chosen model - time varies by model size: sudo vllm-model Qwen/Qwen3-4B-Instruct-2507 This example fits 16 GB GPUs; any Hugging Face model ID is supported. Run vllm-model with no arguments to see more examples per GPU memory tier.

4. Note on model downloads: the instance downloads model files from the Hugging Face Hub, which requires outbound internet access. Model sizes range from hundreds of MB to tens of GB depending on your choice. Without an HF_TOKEN the download is anonymous and subject to Hugging Face rate limits; the vllm-model help output explains how to configure HF_TOKEN.

5. Retrieve your API key: the key equals this instance's EC2 Instance ID (visible in the AWS console). You can also confirm it on the instance by running: cat /etc/vllm/api-key

6. Call the OpenAI-compatible API from your client with the header Authorization: Bearer <YOUR_INSTANCE_ID>, for example: curl http://<YOUR_IP>:8000/v1/models -H 'Authorization: Bearer <YOUR_INSTANCE_ID>' Ready-to-copy curl examples are shown in the login banner and in /home/ubuntu/readme-vllm.

7. Manage the service: sudo systemctl start|stop|restart|status vllm. View logs with: sudo journalctl -u vllm

8. After an instance stop/start, your downloaded model is preserved and the service resumes automatically, but the inference engine needs about 2-3 minutes to re-initialize before it serves requests again.

• SSH (Port 22): Required - this product needs SSH for the one-time initial model setup and ongoing administration.

• vLLM OpenAI-compatible API (Port 8000): Serves the inference API once a model is configured; restrict to trusted IPs.

• Security Recommendation: For production environments, strictly limit access to these ports to trusted IP addresses only via cloud Security Groups or the local firewall.