Vision OCR LLM

1. Chat Completion Example Payload {
"model": "/opt/ml/model",
"messages": [
{"role": "system", "content": "You are a helpful medical assistant."},
{"role": "user", "content": "What should I do if I have a fever and body aches?"}
],
"max_tokens": 1024,
"temperature": 0.6
}

For additional parameters:

ChatCompletionRequest  OpenAI Chat API 

2. Text Completion

Single Prompt Example {
"model": "/opt/ml/model",
"prompt": "How can I maintain good kidney health?",
"max_tokens": 512,
"temperature": 0.6
}

Multiple Prompts Example {
"model": "/opt/ml/model",
"prompt": [
"How can I maintain good kidney health?",
"What are the best practices for kidney care?"
],
"max_tokens": 512,
"temperature": 0.6
}

Reference:

CompletionRequest  OpenAI Completions API 

3. Image + Text Inference

The model supports both online (direct URL) and offline (base64-encoded) image inputs.

Online Image Example { "model": "/opt/ml/model", "messages": [ {"role": "system", "content": "You are a helpful medical assistant."}, { "role": "user", "content": [ {"type": "text", "text": "What does this medical image show?"}, {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg "}} ] } ], "max_tokens": 2048, "temperature": 0.1 }

Offline Image Example (Base64) { "model": "/opt/ml/model", "messages": [ {"role": "system", "content": "You are a helpful medical assistant."}, { "role": "user", "content": [ {"type": "text", "text": "What does this medical image show?"}, {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}} ] } ], "max_tokens": 2048, "temperature": 0.1 }

Reference:

Multimodal Inputs 

Important Notes:

• Streaming Responses: Add "stream": true to your request payload to enable streaming
• Model Path Requirement: Always set "model": "/opt/ml/model" (SageMaker's fixed model location)