AWS Machine Learning Blog

Train and host Scikit-Learn models in Amazon SageMaker by building a Scikit Docker container

by Morgan Du and Thomas Hughes | on | in SageMaker | Permalink | Comments |  Share

Introduced at re:Invent 2017, Amazon SageMaker provides a serverless data science environment to build, train, and deploy machine learning models at scale. Customers also have the ability to work with frameworks they find most familiar, such as Scikit learn.

In this blog post, we’ll accomplish two goals: First, we’ll give you a high-level overview of how Amazon SageMaker uses containers for training and hosting models. Second, we’ll guide you through how to build a Docker container for training and hosting Scikit models in Amazon SageMaker.

In the overview, we’ll discuss how Amazon SageMaker runs Docker images that have been loaded from Amazon Elastic Container Service (ECS) for training and hosting models. We will also discuss the anatomy of a SageMaker Docker image, including the training code and inference code.

If you are only interested in building, training, and deploying Scikit models in Amazon SageMaker, you can skip the overview. Instead, go right to the hands-on demonstration of how to containerize Scikit models in SageMaker with the minimal amount of effort.

Table of Contents

Overview of containers for Amazon SageMaker

SageMaker makes extensive use of Docker containers to allow users to train and deploy algorithms. Containers allow developers and data scientists to package software into standardized units that run consistently on any platform that supports Docker. Containerization packages code, runtime, system tools, system libraries and settings all in the same place, isolating it from its surroundings, and insuring a consistent runtime regardless of where it is being run.

When you develop a model in Amazon SageMaker, you can provide separate Docker images for the training code and the inference code, or you can combine them into a single Docker image. In this blog post, we build a single image to support both training and hosting.

Anatomy of an Amazon SageMaker container

Using this powerful container environment, developers can deploy any kind of code in the Amazon SageMaker ecosystem. You can also create a logical division of labor by creating a deployment team, such as a DevOps, security, and infrastructure teams (who maintains the container) and a data scientists team (who focuses on producing models that are later added to the container).

Example container folder

We use an example container folder to explain how Amazon SageMaker runs Docker for training and hosting your own algorithm. The key files are shown in the following container.


The Dockerfile is used to build the container. In the Dockerfile, you can specify requirements and dependencies to be installed in the container (such as Python, NGINX, and Scikit). Then the installed container executes programs included in the program folder. To do that, you need to name the program folder in your container folder. The program folder in our example container folder is named after the Scikit model, that is decision_trees. Next, you need to have a line in the Dockerfile to copy the program folder to container’s WORKDIR, which is also defined in the Dockerfile. This allows the container to know where to look for the installed programs.

Container startup program

When you run a Docker image, the container either starts to execute the program defined in ENTRYPOINT, or it looks for a program whose name is passed in as an argument. If an ENTRYPOINT is defined, the following command will run the default startup program:

docker run <image>

Alternatively, a specified program can be executed with the following command:

docker run <image> <startup program>

For an image built for only training or hosting, but not both, there is no need to build unused functionalities in the program folder. If you are building an image for training only, you only need to have a train file in the program folder. The train can also be set as the ENTRYPOINT, and the image starts with the train program by default for training service.

Similarly, if you are building an image for hosting only, files to be included in the program folder are nginx.conf,, serve, and The serve can be set as the ENTRYPOINT, and the image would start the serve program by default.

In this blog post we are building a single image for both training and hosting purpose, so we don’t define a default startup program as an ENTRYPOINT. Instead, Amazon SageMaker runs the image by using one of the two following commands. Then the container starts programs for training or hosting service accordingly.

docker run <image> train

docker run <image> serve

Later in this blog post we’ll discuss how to test and debug these program functionalities locally.

How Amazon SageMaker runs Docker containers

In the following sections we explain the differences between training and serving containers. This lets you know which files can be reused or changed for building containers for your own algorithms in Amazon SageMaker.

Running a container for Amazon SageMaker training

Amazon SageMaker invokes the training code by running a version of the following command:

docker run <image> train

This means that your Docker image should have an executable file in it that is called train. You will modify this program to implement your training algorithm. This can be in any language that is capable of running inside of the Docker environment, but the most common language options for data scientists include Python, R, Scala, and Java. For our Scikit example, we use Python.

At runtime, Amazon SageMaker injects the training data from an Amazon S3 location into the container. The training program ideally should produce a model artifact. The artifact is written, inside of the container, then packaged into a compressed tar archive and pushed to an Amazon S3 location by Amazon SageMaker.

Running a container for SageMaker hosting

Amazon SageMaker invokes hosting service by running a version of the following command

docker run <image> serve

This launches a RESTful API to serve HTTP requests for inference. Again, this can be done in any language or framework that works within the Docker environment. In this example we use a Python microframework, Flask.

In most Amazon SageMaker containers, serve is simply a wrapper that starts the inference server. Furthermore, Amazon SageMaker injects the model artifact produced in training into the container and unarchives it automatically.

If you opt to run Flask for your RESTful microservice, you can use the following files from the Amazon SageMaker sample notebooks as-is:

The only file that you need to consider modifying for hosting service is the The inference code is in, and it implements the Flask app for serving predictions. We deep dive into this file later, in our Scikit example.

Where SageMaker Containers are stored

For SageMaker to run a container for training or hosting, it needs to be able to find the image hosted in the image repository, Amazon Elastic Container Registry (Amazon ECR). The three main steps to this process are building locally, tagging with the repository location, and pushing the image to the repository. We discuss how this happens at a high level here, and providing you a practical hands-on walkthrough in the “Build the container” section.

To build the local image, call the following command:

docker build <image name>

This takes instructions from the Dockerfile we discussed earlier to generate an image on your local instance. After the image is built, we need to let our local instance of Docker know where to store the image so that SageMaker can find it. We do this by tagging the image with the following command:

docker tag <image name> <repository name>

The repository name has the following structure:

<account number>.dkr.ecr.<region><image name>:<tag>

Without tagging the image with the repository name, Docker defaults to uploading to Docker Hub, and not Amazon ECR. Amazon SageMaker currently requires Docker images to reside in Amazon ECR. To push an image to ECR, and not the central Docker registry, you must tag it with the registry hostname.

Unlike Docker Hub, Amazon ECR images are private by default, which is a good practice with Amazon SageMaker. If you want to share your Amazon SageMaker images publicly, you can find more information in the Amazon ECR User Guide.

Finally, to upload the image to Amazon ECR, with the Region set in the repository name tag, call the following command:

docker push <repository name>

We’ll give you more details about how this works in practice during our hands on discussion of Scikit. Right now we just want to establish how Amazon SageMaker works with Docker.

One final note on Amazon SageMaker Docker containers. We have already shown you that you have the option to build one Docker container serving both training and hosting, or you can build one for each. While building two Docker images can increase storage requirements and cost due to duplicated common libraries, you might get a benefit from building a significantly smaller inference container, allowing the hosting service to scale more quickly when reacting to traffic increases. This is especially common when you use GPUs for training, but your inference code is optimized for CPUs. You need to consider the tradeoffs when you decide if you want to build a single container or two.

The registry path of your own training image

After the image is built and pushed to Amazon ECR, Amazon SageMaker invokes the training service for your algorithm by running the Docker command we introduced earlier. You can use the Amazon SageMaker console to tell SageMaker the registry path where your own training image is stored in Amazon ECR. Later, we will show how to efficiently launch training jobs and endpoints by using the AWS command line interface. It’s useful to see how these pieces correspond to the console.

The registry path of your own inference image

Similarly, by indicating the location of your inference code image stored in Amazon ECR and a trained model artifact in Amazon S3, Amazon SageMaker invokes the hosting service for your algorithm through a SageMaker Runtime HTTPS endpoint, where inference requests are sent to to get prediction from the model, as shown in the following screenshot.

We will cover more details, step-by-step, later. Right now let’s focus on understanding how all the pieces are connected together.


At runtime, Amazon SageMaker injects external model artifacts, training data, and other configuration information available to Docker containers in /opt/ml/. This is also where Amazon SageMaker processes model artifacts, and where program output you wish to access outside of the Docker container is located. Knowing this will be helpful when you write your training and inferencing code.


It is especially important to know the correct training input location (/opt/ml/input/data/) and model location (/opt/ml/model/) for your program because it aids in local testing and debugging. Detailed mounting location mapping can be found in Using Your Own Training Algorithms, and Using Your Own Inference Code.

Building in practice with Scikit-Learn

“Build” is the first step of any Amazon SageMaker workflow. For Scikit in particular, this involves building three things: 1) the model, 2) the training code, and 3) the inference code. This work will need to be containerized within a Docker image.

Build the training code for Docker

When bringing your own model, like we are with our Scikit model, we need to build our model inside of a Python file, instead of inside of a Jupyter notebook. You can still experiment with your model pipeline in the notebook instance, but once you’re ready to move to the training cluster, we need the code in its own file. This is so that we can containerize it with Docker.

Let’s look at the sample training file that comes with the example notebooks, called train. This is the code that is launched when Amazon SageMaker launches a training job with a container. This file may be a bit overwhelming, so let us look at the important sections.

First, lines 21-31 define all the paths to where SageMaker mounts interesting things below in container’s /opt/ml/:

  • training data, including channels
  • configuration information,
  • model
  • output

Then, define a function to execute the training. If you are primarily concerned with building a Scikit model on Amazon SageMaker, the first important lines are 52-53:

train_y = train_data.ix[:,0]
train_X = train_data.ix[:,1:]

These lines specify what columns in your data are your label columns. In this sample, we use the Iris dataset. The first column of the CSV file is our target label, so we’re assigning that to the train_y variable. The remaining columns are then assigned to train_X. It’s also important to note that the Iris data doesn’t have a header column. For your data, you either have to edit these lines of code in train, or modify your incoming data file to look similar to the Iris data.

The next block of code has to do with obtaining any hyperparameters that may have been passed as part of the training job.  This is in lines 57-59:

max_leaf_nodes = trainingParams.get('max_leaf_nodes', None)
if max_leaf_nodes is not None:
    max_leaf_nodes = int(max_leaf_nodes)

In the example train file, there is only a single parameter passed, but it shows how to get the raw string that’s passed, and how to convert the string to the proper format. In this case, we want the max_leaf_nodes parameter as an integer. This can be expanded for the number of hyperparameters that you want to set for your own model.

Next, and probably most importantly, we define our Scikit model and call fit. This happens on lines 62 and 63:

clf = tree.DecisionTreeClassifier(max_leaf_nodes=max_leaf_nodes)
clf =, train_y)

This code should look very familiar if you’ve worked with Scikit before. Aside from how we’re setting up the hyperparameters, the model definition and the model fit is identical to how this would be done in a notebook. For the hyperparameters, notice that we’re pulling the max_leaf_nodes variable obtained above in the hyperparameter section. You can set this model definition to whatever Scikit model is most appropriate for your data.

Finally, see how the model is saved by the train file. This can be seen in lines 66-67:

with open(os.path.join(model_path, 'decision-tree-model.pkl'), 'w') as out:
    pickle.dump(clf, out)

We are using pickle for our object serialization, to save this model. It will be placed in the output folder of the docker container, and eventually pushed to an Amazon S3 location. These lines are the only lines in train that you really need to be concerned with if you want to build your own Scikit model in Amazon SageMaker.

Build the Inference Code for Docker

In addition to the train code that is run during a training job, we also want to add inference code to our container. This code is contained in This is the code that gets called when you try to connect to a SageMaker endpoint.

To start, we need to make sure we’re loading our model. We can see this on lines 30-31:

with open(os.path.join(model_path, 'decision-tree-model.pkl'), 'r') as inp:
    cls.model = pickle.load(inp)

The important thing here is to make sure that you have the right name for your pickle file. We saw just earlier that we named this file decision-tree-model.pkl in the train code, so we want to make sure that is the same here.

The next important lines are the actual predict lines, on 42-43:

        clf = cls.get_model()
        return clf.predict(input)

We don’t need to make any changes to this code, but it’s important to know this is how the actual prediction function gets called for your Scikit model.

Finally, we’ll want to take a look at how the model gets the input data. This happens in lines 67-69:

data ='utf-8')
s = StringIO.StringIO(data)
data = pd.read_csv(s, header=None)

These are probably the most important lines to learn because the loading of data is different than most data scientists are used to in their normal workflow. The data here is acquired from the Flask framework, which we treat as a file-like object in memory (StringIO), which we then load into Pandas using the read_csv function. The data itself is CSV data passed along to Flask by an http POST method.

We then get the prediction results from this line 76:

predictions = ScoringService.predict(data)

Finally, we return our prediction results by turning them back into a CSV file, using the following lines (79-83):

out = StringIO.StringIO()
pd.DataFrame({'results':predictions}).to_csv(out, header=False, index=False)
result = out.getvalue()

return flask.Response(response=result, status=200, mimetype='text/csv')

As you can see, you should not need to make many changes to to get your model to deploy effectively. Make sure the input data is formatted properly, and that the same pickle file is being loaded that you saved from training.

Build the container

Now that we have our code in train and, we can build our Docker container and push it to Amazon ECR. Create a directory with the following files:


This is the program folder we discussed in the overview. Again, we only have to pay attention to and train. The rest we can pull without editing from the SageMaker sample notebooks as-is. Details can be found here.


We also need a file to tell Docker how to build the image, known as Dockerfile. This goes in the parent container folder of the program folder we created earlier. You don’t need to make any changes to this file, except the name of the program folder, which is decision_trees in the example. You can download it from the sample SageMaker notebooks.

In addition, we provide a helper script for building the Docker image, called Now that we have all the pieces, we can run this script.

To practice the build and push process, we will reuse scripts in the sample-notebooks that come with Amazon SageMaker notebook instances, and we’ll run them on a notebook instance. Once you grasp the essentials of the building process, you can choose to replicate the process and build your own helper scripts from any terminal using the AWS Command Line Interface (CLI) installed and configured. Here is how you start a terminal on an opened notebook instance.

After you start the terminal, the example container folder can be located in the following container folder:

cd ~/SageMaker/sample-notebooks/advanced_functionality/ \
ls decision_trees/

The script will acquire the Region value from the current AWS CLI configuration. If no Region is currently set, it defaults to us-west-2. You can change the Region with the following command:

aws configure set region us-east-1

For training, you want your Amazon S3 bucket for training data to be in the same region as your Amazon SageMaker training job and your training container. For inference, you will need to push your model and your inference container to each Region in which you want to have it available in Amazon SageMaker. For example, if you are planning to deploy SageMaker endpoints in multiple Regions to reduce latency, it makes sense to deploy the container to multiple Regions as well.

To make sure files inside the program folder are set as executable, run the following commands before the build:

chmod +x decision_trees/train
chmod +x decision_trees/serve

The also needs to be set to executable, and requires an argument for a name of the image when running the script. Here we name the image decision_trees, and the Docker image will be created and pushed to Amazon ECR.

chmod +x
./ decision_trees

You will see output similar to this screenshot if your build and push is successful. The push refers to a repository name with the following structure:

<account number>.dkr.ecr.<region><image name>

You can also see our decision-trees image inside of Amazon ECR. Open the AWS Management Console, choose Services, and then Elastic Container Service under Compute. Select Repositories on the left, and you will see the decision-trees Repository name. Choose it, so you can see the Repository URI. This should match the terminal output of running Save this for later.

Testing and debugging

The build and push are combined in one step by running the script. But logically, before pushing the image to Amazon ECR, run some local tests to insure the Docker image functions the way you anticipate. Assume that you have the Docker image that you want to test built at this moment. You should see a repository decision_trees by listing locally built Docker image by running the following:

docker image ls

AWS provides sample scripts for running local tests of your container prior to creating any jobs or endpoints on Amazon SageMaker. They are:,, and, which can also be found in an example local_test folder. You can download the local_test folder anywhere with the same folder structure that follows as the example local_test folder. The test_dir/ will be mounted from your operating system to /opt/ml/ inside the Docker container.


To run the sample scripts for testing and debugging on the SageMaker notebook instance, let’s go back to the sample local_test folder.

cd ~/SageMaker/sample-notebooks/advanced_functionality/scikit_bring_your_own/container/local_test

Note, you might first need to set the executable flag for these files with the following command:

chmod +x *.sh

Testing the container for training will start a training job on your local built Docker image, using a sample data iris.csv. in ./test_dir/input/data/training/. You call it with the following command:

./ decision_trees

You can run local testing for training with an empty JSON for both havehyperparameters.json and resourceConfig.json, and you need to have a training channel under data/ that contains the training data. These are all aligned with the logic in the training code. The structure and contents under test_dir/input/ should be adjusted to your own training code.

We highly recommend that you use a very small sample of your data for local testing, and use the full data for the actual training job. If the script is successful, you should see a generated model file in the following directory ./test_dir/model/.

Now let’s try debugging with a simulation of a failed run, by deleting the hyperparameters.json file under ./test_dir/input/config/. The failed run will generate a output file failure under ./test_dir/output/ logging the details for your debugging and troubleshooting.

Testing the container for hosting

If training runs smoothly, you’ll also want to test the inference code. First, make sure there is a model file under ./test_dir/model/, then, start the RESTful service locally, by running the following script:

./ <image name> > output.log

Any output of our Flask application will now be sent to the ./output.log file.

Finally, we can test the endpoint code with the following command. You need to start another terminal:

./predict <payload file> [content-type]

The content-type is optional, and will default to text/csv. Now check the ./output.log file to make sure that the payload was accepted properly.

If all these scripts run well, then you’re all set to deploy a training job on Amazon SageMaker, and to later deploy the container in an endpoint! We have one last step to clean up our development environment. We need to stop our RESTful service. If this is the only Docker container you have running, you can stop it with the following command:

docker ps -a
docker kill <CONTAINER ID>

If you are running other Docker processes on your instance, you will want to locate each specific process and kill it.

Train and deploy using the Amazon SageMaker console

Create a training job

Now that we have our Docker image, and have set up our training and inference code, we can run a training job. Go to the AWS Management Console, choose Services, and then Amazon SageMaker under Machine Learning. Select Jobs on the left, and then choose the Create training job button.

You’ll see the Create training job window. Set a Job name like decision-trees-<datetime>. Adding the datetime is recommended to avoid ResourceInUse in job name. Under IAM role choose Create a new role, or Enter a custom IAM role ARN, which can be any role that has the AmazonSageMakerFullAccess, and AmazonS3FullAccess policy attached. Set your Algorithm to Custom. In our example, we use File as the Input mode. Next, in the Training image cell, put in your Docker Repository URI, and the Image Tag (the tag the Docker image as latest).

Scroll down to the Resource configuration. These configurations can be left alone for our example, but feel free to customize them for your own models. The main configuration here is the Instance type, as well as the number of instances under Instance count. Note, some algorithms, like Scikit algorithms, can only take advantage of a single instance. Setting this higher won’t automatically cause your algorithm to scale across multiple instances.

Your primary choices for instance type are from the m, c, and p families of instances. The m family is a balanced instance type, that is a good place to start if you are uncertain about where your model might be most intensive. The c family is for compute intensive workloads, while the p family provides GPU capacity, often used for deep learning. You can better see where your model is most intensive by monitoring CloudWatch for SageMaker.

Set any hyperparameters for this job in the Hyperparameters box. In our example, the only hyperparameter we can set is max_leaf_nodes. You may optionally set a value for this here. If you are interested in enabling more hyperparameter options for your algorithm, remember these need to be passed in your train file in the Docker image.

Next, set Input data configuration. The Channel name has to align with what you programmed in your train script. We recommend that the S3 location that you set up for your data input matches the channel name, especially when you have multiple channels. This way Amazon SageMaker can inject channel data to the corresponding container location (e.g. /opt/ml/input/data/<channel_name>), in the location where program train looks for data. In our example, we have a single channel called training. Our model is expecting a CSV file, with no header, where the first column is our target label. Feel free to upload the famous Iris data set to your own S3 location for this, as it is what is used in this example. Make sure to choose Done.

In our example, we will have the training data folder as follows.

Finally, you need to specify where the model artifact is going to be saved on Amazon S3 after the training job is done. Remember, this takes anything in the container that is output to /opt/ml/model/, and then it archives it with tar and gzip compresses it. Specify an Amazon S3 bucket in the S3 output path field.

Next, choose the Create training job button.

After the created training job is completed, you should be able to see the generated model in the S3 output path you provided. The full path of the model is

s3://<bucket name>/models/<job name>/output/model.tar.gz

Deploy an endpoint

After the training job has finished, we can deploy an endpoint, to take our model to production. We can do this from the SageMaker console. There are three steps: First, we have to create a model resource. Second, we create an endpoint configuration. Third, we deploy the actual endpoint.

The first step is to create the model resource. This contains the information Amazon SageMaker needs about the Docker image and the Amazon S3 location of the artifact.

In the Amazon SageMaker console, in the left navigation pane, under Resources select Models. Then choose the Create model button.

First we need to input a Model name, with a name like decision-trees, and then select or create an IAM role. You can use the same role that you used for your training job, as long as it has the AmazonSageMakerFullAccess IAM policy attached.

Next, we need to provide information about our container. We can use the same container we used for the training job. Provide the ECR URI that we saved earlier for our container in the Location of inference code image field. Make sure to include an image tag, like the latest tag. Next, in the Location of model artifacts cell, provide the S3 URI for the model artifact that you created in the training job. This was the S3 output path you set during training.

Choose the Create model button.

Next, we need to create an Endpoint configuration. We combine this with model resources to determine what kind of instances to provision. You can also use this to specify more than one model for doing A/B testing. For our example, we’ll only use a single model.

On the left navigation pane, under Resources, select Endpoint configuration. Choose the Create endpoint configuration button.

Give your endpoint configuration a name under Endpoint configuration name. Unless we anticipate having multiple versions of the same model, provide the same name as my model. Next, choose the Add model link in the lower left. Select the model you just created, and choose save. You can leave the additional values as their defaults for this example.

Choose the Create endpoint configuration button.

Now that we have a model and an endpoint configuration, we can deploy an actual endpoint. Activating an endpoint will provision an instance. Note that as long as the endpoint is active, charges will accrue, so it’s important to only activate an endpoint when you intend to direct traffic towards it.

In the left navigation pane, under Resources, select Endpoints. Choose the Create endpoint button.

Give your endpoint a name under Endpoint name. Use the same name as my model when we are only deploying a single model with no variants. In this case, use decision-trees. Leave Use an existing endpoint configuration selected.

Under Endpoint configuration, select the endpoint configuration we created in the last step. Then choose the Select endpoint configuration button. Finally, choose the Create endpoint button.

The result should look something like this:

Train and deploy using the AWS CLI

Create a training job

You have the option to launch a training job from the AWS CLI with the following command:

aws sagemaker create-training-job \
    --training-job-name <job name> \
    --algorithm-specification \
        TrainingImage=<Repository URI>:<Image Tag>,TrainingInputMode=File \
    --role-arn <Role ARN> \
    --input-data-config \
                                        "S3Uri":"s3://<bucket name>/data/training/",
          }' \
    --output-data-config \
        S3OutputPath=s3://<bucket name>/models/ \
    --resource-config \
        InstanceType=ml.m4.xlarge,InstanceCount=1,VolumeSizeInGB=1 \
    --stopping-condition \
        MaxRuntimeInSeconds=86400 \
    --hyper-parameters \

This command will create the same training job with same configuration as the one we created in the Amazon SageMaker console, except for the name of the training job because it has to be a unique identifier. The returned TrainingJobArn indicates that the creation of the training job is successful.  You can track the status of the training job in the Amazon SageMaker console, on the Jobs page.

The input-data-config allows an array of channel objects. For example, you can provide another channel if defined, by attaching it after the training channel, and using space to separate these two objects. The hyper-parameters allows a maximum of 100 hyperparameters, such as KeyName1=string,KeyName2=string. For example, if we have max_depth defined, we can attach max_depth=10 right after max_leaf_nodes=5. Please check aws.sagemaker.create-training-job and Using Shorthand Syntax with the AWS Command Line Interface for more detail when creating your own training job using AWS CLI.

Also, you can see that the S3OutputPath contains both training jobs now created from the Amazon SageMaker console and the AWS CLI.

Deploy an endpoint

You can use the following three AWS CLI commands to deploy a SageMaker endpoint. We also provide the expected output for the commands.

aws sagemaker create-model \
    --model-name <model name> \
    --primary-container \
        Image=<Repository URL>:<Image Tag>,ModelDataUrl=s3://<bucket name>/models/<job name>/output/model.tar.gz \
    --execution-role-arn <Role ARN>

aws sagemaker create-endpoint-config \
    --endpoint-config-name <endpoint config name> \
    --production-variants \
        VariantName=dev,ModelName=<model name>,InitialInstanceCount=1,InstanceType=ml.m4.xlarge,InitialVariantWeight=1.0

aws sagemaker create-endpoint \
    --endpoint-name <endpoint name> \
    --endpoint-config-name <endpoint config name>

You should see your endpoint InService after it is created successfully.

Test inferences

The easiest way to test our endpoint is from the SageMaker notebook instance. In a cell, run the following code:

import boto3
import io
import pandas as pd
import itertools

# Set below parameters
bucket = '<bucket name>'
key = 'data/training/iris.csv'
endpointName = 'decision-trees'

# Pull our data from S3
s3 = boto3.client('s3')
f = s3.get_object(Bucket=bucket, Key=key)

# Make a dataframe
shape = pd.read_csv(io.BytesIO(f['Body'].read()), header=None)

# Take a random sample
a = [50*i for i in range(3)]
b = [40+i for i in range(10)]
indices = [i+j for i,j in itertools.product(a,b)]

# Convert the dataframe to csv data
test_file = io.StringIO()
test_X.to_csv(test_file, header=None, index=None)

# Talk to SageMaker
client = boto3.client('sagemaker-runtime')
response = client.invoke_endpoint(


This will take a random sample of the data in your data file, and send it to the end point. It will also print out the result.


Congratulations! You have now built, trained, deployed, and tested your Scikit model in Amazon SageMaker. Feel free to modify any part of this workflow or code to better suit your own needs.

About the Authors

Morgan Du is a Machine Learning Consultant with AWS Professional Services. She is passionate about artificial intelligence, machine learning, and deep learning. She is dedicated in enabling innovative ML solutions that are impacting and powering our everyday life.




Thomas Hughes is a Data Scientist with AWS Professional Services. He has a PhD from UC Santa Barbara and has tackled problems in in the social sciences, education, and advertising. He is currently working to solve some of the trickiest problems that arise when machine learning meets big data.