Realtime inference

Input

Supported content type: application/json Sample input:

{ 
  "instance":"One of the main things that Tower of Guns has going for it is the fact that every experience is randomly generated."
}

Output

Content type: application/json Sample output:

{ 
  "ner":[ 
    { 
      "end_pos":41,
      "key":"Tower of Guns",
      "start_pos":28,
      "type":"NE_PRODUCT_VIDEOGAME"
    }
  ]
}

Invoking endpoint

AWS CLI Command

You can invoke endpoint using AWS CLI:

aws sagemaker-runtime invoke-endpoint --endpoint-name "endpoint-name" --body '{"instance": "text"}' --content-type application/json --accept application/json out.json

Substitute the following parameters:

• "endpoint-name" - name of the inference endpoint where the model is deployed
• "text" - text to analyze
• out.json - filename where the inference results are written to

Python

Real-time inference snippet (more detailed example can be found in sample notebook):

runtime = boto3.client('runtime.sagemaker')
response = runtime.invoke_endpoint(
    EndpointName='endpoint-name', 
    ContentType='application/json', 
    Body=json.dumps({'instance': 'text to analyze'}))
response = response['Body'].read()
results = json.loads(response)

Transform jobs

Use Transform jobs to transform many document instances, see example in sampee notebook. Please see sample input and output for batch transform Supported content type for input and output: application/jsonlines

Limits for realtime and transform jobs: each document instance must have no more than 10,000 characters.

Resources

• Sample Notebook
• jsonlines format