Ollama GPU on Ubuntu 24.04 with Support by Easycloud

SSH is not required to use this product. The API key equals this instance's EC2 Instance ID, which is visible in the AWS console, so the inference API can be called as soon as the instance is ready. If you do want shell access, SSH into the instance; the default username is 'ubuntu'. You can switch to the root user environment by running: sudo su -

• OS: Ubuntu 24.04 LTS (x86_64, Minimal Installation)
• Ollama: 0.31 (GPU-accelerated, main binary at /usr/local/bin/ollama)
• Pre-loaded Model: qwen3:4b-instruct, stored in /usr/share/ollama/.ollama on the root volume - nothing is downloaded at runtime
• Runtime: NVIDIA GPU driver and CUDA stack pre-installed. This product requires an NVIDIA GPU instance such as g4dn.xlarge or larger.
• Reverse Proxy: nginx, site config at '/etc/nginx/sites-available/ollama-api'
• Ports: 8000 serves the outward-facing OpenAI-compatible API through nginx; the native Ollama port 11434 is bound to 127.0.0.1 only and is not reachable from outside the instance
• Config File: '/etc/ollama/api-key' (holds the current API key, refreshed at every boot)
• README: '/home/ubuntu/readme-ollama'

1. Launch on an NVIDIA GPU instance. Recommended: g4dn.xlarge or larger. Open port 8000 in the Security Group so clients can reach the API.
2. Allow up to about 5 minutes on first boot before the API responds (measured: 280 seconds from instance start to the first working request). The instance prewarms the model into memory during startup. This prewarm runs on every boot, not only the first one - after each stop and start it takes roughly 20 seconds.
3. Retrieve your API key: the key equals this instance's EC2 Instance ID, visible in the AWS console, so no SSH login is needed to obtain it. On the instance you can also read it with: cat /etc/ollama/api-key
4. Send your first request to the OpenAI-compatible chat completions endpoint, substituting your instance IP and instance ID: curl http://<instance-ip>:8000/v1/chat/completions -H "Authorization: Bearer <instance-id>" -H 'Content-Type: application/json' -d '{"model":"qwen3:4b-instruct","messages":[{"role":"user","content":"Hello"}]}'
5. Chat from the instance shell (optional, requires SSH): ollama run qwen3:4b-instruct
6. Manage the service with: sudo systemctl start/stop/restart/status ollama, and the same commands for nginx. Two helper units also run at boot: ollama-apikey refreshes the API key, and ollama-prewarm loads the model into memory.
7. View logs: Ollama has no separate log file and writes to the systemd journal - sudo journalctl -u ollama
8. Full documentation, including ready-to-copy request examples, is available on the instance at /home/ubuntu/readme-ollama

• SSH (Port 22): Optional - needed only to read the full README on the instance or to run the local command-line chat; the inference API itself does not require SSH.
• Ollama API via nginx (Port 8000): Required - serves the outward-facing Chat and API traffic over the OpenAI-compatible protocol; restrict to trusted IPs.
• Security Recommendation: For production environments, strictly limit access to these ports to trusted IP addresses only via cloud Security Groups or the local firewall.