Amazon SageMaker FAQs
What is Amazon SageMaker?
In which AWS Regions is SageMaker available?
What is the service availability of SageMaker?
How does SageMaker secure my code?
SageMaker stores code in ML storage volumes, secured by security groups and optionally encrypted at rest.
What security measures does SageMaker have?
SageMaker ensures that ML model artifacts and other system artifacts are encrypted in transit and at rest. Requests to the SageMaker API and console are made over a secure (SSL) connection. You pass AWS Identity and Access Management roles to SageMaker to provide permissions to access resources on your behalf for training and deployment. You can use encrypted Amazon Simple Storage Service (Amazon S3) buckets for model artifacts and data, as well as pass an AWS Key Management Service (AWS KMS) key to SageMaker notebooks, training jobs, and endpoints to encrypt the attached ML storage volume. SageMaker also supports Amazon Virtual Private Cloud (Amazon VPC) and AWS PrivateLink support.
How am I charged for SageMaker?
You pay for ML compute, storage, and data processing resources that you use for hosting the notebook, training the model, performing predictions, and logging the outputs. With SageMaker, you can select the number and type of instance used for the hosted notebook, training, and model hosting. You pay only for what you use, as you use it; there are no minimum fees and no upfront commitments. For more details, see Amazon SageMaker Pricing and the Amazon SageMaker Pricing Calculator.
How can I optimize my SageMaker costs, such as detecting and stopping idle resources to avoid unnecessary charges?
There are several best practices that you can adopt to optimize your SageMaker resource usage. Some approaches involve configuration optimizations; others involve programmatic solutions. A full guide on this concept, complete with visual tutorials and code samples, can be found in this blog post.
What if I have my own notebook, training, or hosting environment?
SageMaker provides a full and complete workflow, but you can continue using your existing tools with SageMaker. You can easily transfer the results of each stage in and out of SageMaker as your business requirements dictate.
Is R supported with SageMaker?
Yes. You can use R within SageMaker notebook instances, which include a preinstalled R kernel and the reticulate library. Reticulate offers an R interface for the Amazon SageMaker Python SDK, helping ML practitioners build, train, tune, and deploy R models. You can also launch RStudio, an integrated development environment (IDE) for R in Amazon SageMaker Studio.
What is Amazon SageMaker Studio?
Amazon SageMaker Studio provides a single, web-based visual interface where you can perform all ML development steps. SageMaker Studio gives you complete access, control, and visibility into each step required to prepare data and build, train, and deploy models. You can quickly upload data, create new notebooks, train and tune models, move back and forth between steps to adjust experiments, compare results, and deploy models to production all in one place, making you much more productive. All ML development activities including notebooks, experiment management, automatic model creation, debugging and profiling, and model drift detection can be performed within the unified SageMaker Studio visual interface..
How does SageMaker Studio pricing work?
There is no additional charge for using SageMaker Studio. You pay only for the underlying compute and storage charges on the services that you use within SageMaker Studio.
In which Regions is SageMaker Studio supported?
You can find the Regions where SageMaker Studio is supported in the Amazon SageMaker Developer Guide.
How can I check for imbalances in my model?
Amazon SageMaker Clarify helps improve model transparency by detecting statistical bias across the entire ML workflow. SageMaker Clarify checks for imbalances during data preparation, after training, and ongoing over time, and also includes tools to help explain ML models and their predictions. Findings can be shared through explainability reports.
What kind of bias does SageMaker Clarify detect?
What is RStudio on Amazon SageMaker?
RStudio on SageMaker is the first fully managed RStudio Workbench in the cloud. You can quickly launch the familiar RStudio integrated development environment (IDE) and dial up and down the underlying compute resources without interrupting your work, making it easier to build ML and analytics solutions in R at scale. You can seamlessly switch between the RStudio IDE and SageMaker Studio notebooks for R and Python development. All your work, including code, datasets, repositories, and other artifacts, is automatically synchronized between the two environments to reduce context switch and boost productivity.
How does SageMaker Clarify improve model explainability?
SageMaker Clarify is integrated with SageMaker Experiments to provide a feature importance graph detailing the importance of each input for your model’s overall decision-making process after the model has been trained. These details can help determine if a particular model input has more influence than it should on overall model behavior. SageMaker Clarify also makes explanations for individual predictions available through an API.
What ML governance tools does SageMaker provide?
SageMaker provides purpose-built ML governance tools across the ML lifecycle. With Amazon SageMaker Role Manager, administrators can define minimum permissions in minutes. Amazon SageMaker Model Cards makes it easier to capture, retrieve, and share essential model information from conception to deployment, and Amazon SageMaker Model Dashboard keeps you informed on production model behavior, all in one place. For
more information, see ML Governance with Amazon SageMaker.
What does SageMaker Role Manager do?
You can define minimum permissions in minutes with SageMaker Role Manager. It provides a baseline set of permissions for ML activities and personas with a catalog of pre-built IAM policies. You can keep the baseline permissions, or customize them further based on your specific needs. With a few self-guided prompts, you can quickly input common governance constructs such as network access boundaries and encryption keys. SageMaker Role Manager will then generate the IAM policy automatically. You can discover the generated role and associated policies through the AWS IAM console. To further tailor the permissions to your use case, attach your managed IAM policies to the IAM role that you create with SageMaker Role Manager. You can also add tags to help identify the role and organize across AWS services.
What does SageMaker Model Cards do?
SageMaker Model Cards helps you centralize and standardize model documentation throughout the ML lifecycle by creating a single source of truth for model information. SageMaker Model Cards auto-populates training details to accelerate the documentation process. You can also add details such as the purpose of the model and the performance goals. You can attach model evaluation results to your model card and provide visualizations to gain key insights into model performance. SageMaker Model Cards can easily be shared with others by exporting to a PDF format.
What does SageMaker Model Dashboard do?
SageMaker Model Dashboard gives you a comprehensive overview of deployed models and endpoints, letting you track resources and model behavior violations through one pane. It allows you to monitor model behavior in four dimensions, including data and model quality, and bias and feature attribution drift through its integration with SageMaker Model Monitor and SageMaker Clarify. SageMaker Model Dashboard also provides an integrated experience to set up and receive alerts for missing and inactive model monitoring jobs, and deviations in model behavior for model quality, data quality, bias drift, and feature attribution drift. You can further inspect individual models and analyze factors impacting model performance over time. Then, you can follow up with ML practitioners to take corrective measures.
How do I get started with SageMaker quickly?
SageMaker JumpStart helps you quickly and easily get started with ML. SageMaker JumpStart provides a set of solutions for the most common use cases that can be deployed readily in just a few steps. The solutions are fully customizable and showcase the use of AWS CloudFormation templates and reference architectures so you can accelerate your ML journey. SageMaker JumpStart also provides foundation models and supports one-step deployment and fine-tuning of more than 150 popular open-source models, such as transformer, object detection, and image classification models.
Which foundation models are available in SageMaker JumpStart?
SageMaker JumpStart provides proprietary and public models. For a list of available foundation models, see Getting Started with Amazon SageMaker JumpStart .
How do I start using the foundation models in SageMaker JumpStart?
You can access foundation models through SageMaker Studio, the SageMaker SDK, and the AWS Management Console. To get started with proprietary foundation models, you must accept terms of sale in the AWS Marketplace .
Can I see the model weights and scripts of proprietary models with SageMaker JumpStart?
No. Proprietary models do not allow customers to view model weights and scripts.
In which Regions are SageMaker JumpStart foundation models available?
How are SageMaker JumpStart foundation models priced?
For proprietary models, you are charged for software pricing determined by the model provider and SageMaker infrastructure charges based on the instance used. For publicly available models, you are charged SageMaker infrastructure charges based on the instance used. For more information, see Amazon SageMaker Pricing and the AWS Marketplace .
How does SageMaker JumpStart help protect and secure my data?
Security is the top priority at AWS, and SageMaker JumpStart is designed to be secure. That's why SageMaker gives you ownership and control over your content through simplified, powerful tools that help you determine where your content will be stored, secure your content in transit and at rest, and manage your access to AWS services and resources for your users.
- We do not share customer training and inference information with model sellers in the AWS Marketplace. Similarly, the seller’s model artifacts (for example, model weights) are not shared with the buyer.
- SageMaker JumpStart does not use customer models, training data, or algorithms to improve its service and does not share customer training and inference data with third parties.
- In SageMaker JumpStart, ML model artifacts are encrypted in transit and at rest.
- Under the AWS Shared Responsibility Model, AWS is responsible for protecting the global infrastructure that runs all of AWS. You are responsible for maintaining control over your content that is hosted on this infrastructure.
By using a model from AWS Marketplace or SageMaker JumpStart, users assume responsibility for the model output quality and acknowledge the capabilities and limitations described in the individual model description.
Which publicly available models are supported with SageMaker JumpStart?
SageMaker JumpStart includes over 150 pre-trained publicly available models from PyTorch Hub and TensorFlow Hub. For vision tasks such as image classification and object detection, you can use models like RESNET, MobileNet, and single-shot detector (SSD). For text tasks such as sentence classification, text classification, and question answering, you can use models like BERT, RoBERTa, and DistilBERT.
How can I evaluate and select the foundation models?
What is Amazon SageMaker Canvas?
SageMaker Canvas is a no-code service with an intuitive, point-and-click interface that lets you create highly accurate ML-based predictions from your data. SageMaker Canvas lets you access and combine data from a variety of sources using a drag-and-drop user interface, automatically cleaning and preparing data to minimize manual cleanup. SageMaker Canvas applies a variety of state-of-the-art ML algorithms to find highly accurate predictive models and provides an intuitive interface to make predictions. You can use SageMaker Canvas to make much more precise predictions in a variety of business applications and easily collaborate with data scientists and analysts in your enterprise by sharing your models, data, and reports. To learn more about SageMaker Canvas, see Amazon SageMaker Canvas FAQs .
What is Amazon SageMaker Autopilot?
SageMaker Autopilot is the industry’s first automated machine learning capability that gives you complete control and visibility into your ML models. SageMaker Autopilot automatically inspects raw data, applies feature processors, picks the best set of algorithms, trains and tunes multiple models, tracks their performance, and then ranks the models based on performance, all with just a few clicks. The result is the best-performing model that you can deploy at a fraction of the time normally required to train the model. You get full visibility into how the model was created and what’s in it, and SageMaker Autopilot integrates with SageMaker Studio. You can explore up to 50 different models generated by SageMaker Autopilot inside SageMaker Studio so it’s easy to pick the best model for your use case. SageMaker Autopilot can be used by people without ML experience to easily produce a model, or it can be used by experienced developers to quickly develop a baseline model on which teams can further iterate.
How does SageMaker Canvas pricing work?
With SageMaker Canvas, you pay based on usage. SageMaker Canvas lets you interactively ingest, explore, and prepare your data from multiple sources, train highly accurate ML models with your data, and generate predictions. There are two components that determine your bill: session charges based on the number of hours for which SageMaker Canvas is used or logged into, and charges for training the model based on the size of the dataset used to build the model. For more information, see Amazon SageMaker Canvas Pricing .
Can I stop a SageMaker Autopilot job manually?
Yes. You can stop a job at any time. When a SageMaker Autopilot job is stopped, all ongoing trials will be stopped and no new trial will be started.
How can I build a repeatable ML workflow in SageMaker?
Amazon SageMaker Pipelines helps you create fully automated ML workflows from data preparation through model deployment so you can scale to thousands of ML models in production. You can create Pipelines with the SageMaker Python SDK and view, execute, audit them from the visual interface of the SageMaker Studio. SageMaker Pipelines takes care of managing data between steps, packaging the code recipes, and orchestrating their execution, reducing months of coding to a few hours. Every time a workflow executes, a complete record of the data processed and actions taken is kept so data scientists and ML developers can quickly debug problems.
How do I view all my trained models to choose the best model to move to production?
Which components of SageMaker can be added to SageMaker Pipelines?
How do I track my model components across the entire ML workflow?
How does the pricing for SageMaker Pipelines work?
There is no additional charge for SageMaker Pipelines. You pay only for the underlying compute or any separate AWS services you use within SageMaker Pipelines.
Can I use Kubeflow with SageMaker?
How does SageMaker Components for Kubeflow Pipelines pricing work?
There is no additional charge for using SageMaker Components for Kubeflow Pipelines.
How can SageMaker prepare data for ML?
SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for ML. From a single interface in SageMaker Studio, you can browse and import data from Amazon S3, Amazon Athena, Amazon Redshift, AWS Lake Formation, Amazon EMR, Snowflake and Databricks in just a few steps. You can also query and import data that is transferred from over 50 data sources and registered in AWS Glue Data Catalog by Amazon AppFlow. SageMaker Data Wrangler will automatically load, aggregate, and display the raw data. After importing your data into SageMaker Data Wrangler, you can see automatically generated column summaries and histograms. You can then dig deeper to understand your data and identify potential errors with the SageMaker Data Wrangler Data Quality and Insights report, which provides summary statistics and data quality warnings. You can also run bias analysis supported by SageMaker Clarify directly from SageMaker Data Wrangler to detect potential bias during data preparation. From there, you can use SageMaker Data Wrangler’s pre-built transformations to prepare your data. Once your data is prepared, you can build fully automated ML workflows with Amazon SageMaker Pipelines or import that data into Amazon SageMaker Feature Store .
What data types does SageMaker Data Wrangler support?
How can I create model features with SageMaker Data Wrangler?
How can I visualize my data in SageMaker Data Wrangler?
How does the pricing for SageMaker Data Wrangler work?
How can I train ML models with data prepared in SageMaker Data Wrangler?
How does SageMaker Data Wrangler handle new data when I have prepared my features on historical data?
How does SageMaker Data Wrangler work with my CI/CD processes?
Which model does SageMaker Data Wrangler Quick Model use?
What size data does SageMaker Data Wrangler support?
Does SageMaker Data Wrangler work with SageMaker Feature Store?
What is SageMaker Feature Store?
SageMaker Feature Store is a fully managed, purpose-built platform to store, share, and manage features for machine learning (ML) models. Features can be discovered and shared for easy reuse across models and teams with secure access and control, including across AWS accounts. SageMaker Feature Store supports both online and offline features for real-time inference, batch inference and training. It also manages batch and streaming feature engineering pipelines to reduce duplication in feature creation and improve model accuracy.
What are offline features?
What are online features?
How do I maintain consistency between online and offline features?
How can I reproduce a feature from a given moment in time?
How does pricing work for SageMaker Feature Store?
What does SageMaker offer for data labeling?
SageMaker provides two data labeling offerings, Amazon SageMaker Ground Truth Plus and Amazon SageMaker Ground Truth. Both options allow you to identify raw data, such as images, text files, and videos, and add informative labels to create high-quality training datasets for your ML models. To learn more, see Amazon SageMaker Data Labeling .
What is geospatial data?
What are SageMaker geospatial capabilities?
Why should I use geospatial ML on SageMaker?
What are Amazon SageMaker Studio notebooks?
How do SageMaker Studio notebooks work?
SageMaker Studio notebooks are one-step Jupyter notebooks that can be spun quickly. The underlying compute resources are fully elastic, so you can easily dial up or down the available resources and the changes take place automatically in the background without interrupting your work. SageMaker also enables one-step sharing of notebooks. You can easily share notebooks with others and they’ll get the exact same notebook, saved in the same place.
With SageMaker Studio notebooks, you can sign in with your corporate credentials using IAM Identity Center. Sharing notebooks within and across teams is easy since the dependencies needed to run a notebook are automatically tracked in work images that are encapsulated with the notebook as it is shared.
How are SageMaker Studio notebooks different from the instance-based notebooks offering?
How do SageMaker Studio notebooks work with other AWS services?
How does SageMaker Studio notebooks pricing work?
Do I get charged separately for each notebook created and run in SageMaker Studio?
No. You can create and run multiple notebooks on the same compute instance. You pay only for the compute that you use, not for individual items. You can read more about this in our metering guide .
In addition to the notebooks, you can also start and run terminals and interactive shells in SageMaker Studio, all on the same compute instance. Each application runs within a container or image. SageMaker Studio provides several built-in images purpose-built and preconfigured for data science and ML.
How do I monitor and shut down the resources used by my notebooks?
I’m running a SageMaker Studio notebook. Will I still be charged if I close my browser, close the notebook tab, or just leave the browser open?
Do I get charged for creating and setting up an SageMaker Studio domain?
No, you don’t get charged for creating or configuring an SageMaker Studio domain, including adding, updating, and deleting user profiles.
How do I see the itemized charges for SageMaker Studio notebooks or other SageMaker services?
As an admin, you can view the list of itemized charges for SageMaker, including SageMaker Studio, in the AWS Billing console. From the AWS Management Console for SageMaker, choose Services on the top menu, type "billing" in the search box and select Billing from the dropdown, and then select Bills on the left panel. In the Details section, you can select SageMaker to expand the list of Regions and drill down to the itemized charges.
What is Amazon SageMaker Studio Lab?
Why should I use SageMaker Studio Lab?
How does SageMaker Studio Lab work with other AWS services?
What is SageMaker Canvas?
SageMaker Canvas is a visual drag-and-drop service that allows business analysts to build ML models and generate accurate predictions without writing any code or requiring ML expertise. SageMaker Canvas makes it easier to access and combine data from a variety of sources, automatically clean data and apply a variety of data adjustments, and build ML models to generate accurate predictions in a single step. You can also easily publish results, explain and interpret models, and share models with others within your organization to review.
What data sources does SageMaker Canvas support?
SageMaker Canvas helps you seamlessly discover AWS data sources that your account has access to, including Amazon S3 and Amazon Redshift. You can browse and import data using the SageMaker Canvas visual drag-and-drop interface. Additionally, you can drag and drop files from your local disk, and use pre-built connectors to import data from third-party sources such as Snowflake.
How do I build an ML model to generate accurate predictions in SageMaker Canvas?
Once you have connected sources, selected a dataset, and prepared your data, you can select the target column that you want to predict to initiate a model creation job. SageMaker Canvas will automatically identify the problem type, generate new relevant features, test a comprehensive set of prediction models using ML techniques such as linear regression, logistic regression, deep learning, time-series forecasting, and gradient boosting, and build the model that makes accurate predictions based on your dataset.
How long does it take to build a model in SageMaker Canvas? How can I monitor progress during model creation?
The time it takes to build a model depends on the size of your dataset. Small datasets can take less than 30 minutes, and large datasets can take a few hours. As the model creation job progresses, SageMaker Canvas provides detailed visual updates, including percent job complete and the amount of time left for job completion.
What is Amazon SageMaker HyperPod?
When should I use SageMaker HyperPod?
Does SageMaker support distributed training?
Yes. SageMaker can automatically distribute deep learning models and large training sets across AWS GPU instances in a fraction of the time it takes to build and optimize these distribution strategies manually. The two distributed training techniques that SageMaker applies are data parallelism and model parallelism. Data parallelism is applied to improve training speeds by dividing the data equally across multiple GPU instances, allowing each instance to train concurrently. Model parallelism is useful for models too large to be stored on a single GPU and require the model to be partitioned into smaller parts before distributing across multiple GPUs. With only a few lines of additional code in your PyTorch and TensorFlow training scripts, SageMaker will automatically apply data parallelism or model parallelism for you, allowing you to develop and deploy your models faster. SageMaker will determine the best approach to split your model by using graph partitioning algorithms to balance the computation of each GPU while minimizing the communication between GPU instances. SageMaker also optimizes your distributed training jobs through algorithms that fully utilize the AWS compute and network in order to achieve near-linear scaling efficiency, which allows you to complete training faster than manual open-source implementations.
What is Amazon SageMaker Experiments?
What is Amazon SageMaker Training Compiler?
SageMaker Training Compiler is a deep learning (DL) compiler that accelerates DL model training by up to 50 percent through graph- and kernel-level optimizations to use GPUs more efficiently. SageMaker Training Compiler is integrated with versions of TensorFlow and PyTorch in SageMaker, so you can speed up training in these popular frameworks with minimal code changes.
What is Amazon SageMaker Debugger?
How does SageMaker Training Compiler work?
SageMaker Training Compiler accelerates training jobs by converting DL models from their high-level language representation to hardware-optimized instructions that train faster than jobs with the native frameworks. More specifically, SageMaker Training Compiler uses graph-level optimization (operator fusion, memory planning, and algebraic simplification), data flow-level optimizations (layout transformation, common sub-expression elimination), and backend optimizations (memory latency hiding, loop oriented optimizations) to produce an optimized model training job that more efficiently uses hardware resources and, as a result, trains faster.
What is Managed Spot Training?
How can I use SageMaker Training Compiler?
SageMaker Training Compiler is built into the SageMaker Python SDK and SageMaker Hugging Face Deep Learning Containers. You don’t need to change your workflows to access its speedup benefits. You can run training jobs in the same way as you already do, using any of the SageMaker interfaces: SageMaker notebook instances, SageMaker Studio, AWS SDK for Python (Boto3), and AWS Command Line Interface (AWS CLI). You can enable SageMaker Training Compiler by adding a TrainingCompilerConfig class as a parameter when you create a framework estimator object. Practically, this means a couple of lines of code added to your existing training job script for a single GPU instance. Most up-to-date detailed documentation, sample notebooks, and examples are available in the documentation .
How do I use Managed Spot Training?
What is the pricing of SageMaker Training Compiler?
SageMaker Training Compiler is a SageMaker Training feature and is provided at no additional charge exclusively to SageMaker customers. Customers can actually reduce their costs with SageMaker Training Compiler as training times are reduced.
When should I use Managed Spot Training?
How does Managed Spot Training work?
Do I need to periodically checkpoint with Managed Spot Training?
How do you calculate the cost savings with Managed Spot Training jobs?
Which instances can I use with Managed Spot Training?
Which Regions are supported with Managed Spot Training?
Managed Spot Training is supported in all Regions where SageMaker is currently available .
Are there limits to the size of the dataset I can use for training?
There are no fixed limits to the size of the dataset you can use for training models with SageMaker.
What algorithms does SageMaker use to generate models?
What is Automatic Model Tuning?
What models can be tuned with Automatic Model Tuning?
Can I use Automatic Model Tuning outside SageMaker?
Not at this time. The best model tuning performance and experience is within SageMaker.
What is the underlying tuning algorithm for Automatic Model Tuning?
Currently, the algorithm for tuning hyperparameters is a customized implementation of Bayesian Optimization. It aims to optimize a customer-specified objective metric throughout the tuning process. Specifically, it checks the object metric of completed training jobs, and uses the knowledge to infer the hyperparameter combination for the next training job.
Does Automatic Model Tuning recommend specific hyperparameters for tuning?
No. How certain hyperparameters impact the model performance depends on various factors, and it is hard to definitively say one hyperparameter is more important than the others and thus needs to be tuned. For built-in algorithms within SageMaker, we do call out whether or not a hyperparameter is tunable.
How long does a hyperparameter tuning job take?
The length of time for a hyperparameter tuning job depends on multiple factors, including the size of the data, the underlying algorithm, and the values of the hyperparameters. Additionally, customers can choose the number of simultaneous training jobs and total number of training jobs. All these choices affect how long a hyperparameter tuning job can last.
Can I optimize multiple objectives simultaneously, such as optimizing a model to be both fast and accurate?
Not at this time. Currently, you need to specify a single objective metric to optimize or change your algorithm code to emit a new metric, which is a weighted average between two or more useful metrics, and have the tuning process optimize towards that objective metric.
How much does Automatic Model Tuning cost?
There is no charge for a hyperparameter tuning job itself. You will be charged by the training jobs that are launched by the hyperparameter tuning job, based on model training pricing .
How do I decide to use SageMaker Autopilot or Automatic Model Tuning?
SageMaker Autopilot automates everything in a typical ML workflow, including feature preprocessing, algorithm selection, and hyperparameter tuning, while specifically focusing on classification and regression use cases. Automatic Model Tuning, on the other hand, is designed to tune any model, no matter whether it is based on built-in algorithms, deep learning frameworks, or custom containers. In exchange for the flexibility, you have to manually pick the specific algorithm, hyperparameters to tune, and corresponding search ranges.
What is reinforcement learning?
Reinforcement learning is a ML technique that enables an agent to learn in an interactive environment by trial and error using feedback from its own actions and experiences.
Can I train reinforcement learning models in SageMaker?
Yes, you can train reinforcement learning models in SageMaker in addition to supervised and unsupervised learning models.
How is reinforcement learning different from supervised learning?
Though both supervised and reinforcement learning use mapping between input and output, unlike supervised learning where the feedback provided to the agent is the correct set of actions for performing a task, reinforcement learning uses a delayed feedback where reward signals are optimized to ensure a long-term goal through a sequence of actions.
When should I use reinforcement learning?
While the goal of supervised learning techniques is to find the right answer based on the patterns in the training data, the goal of unsupervised learning techniques is to find similarities and differences between data points. In contrast, the goal of reinforcement learning (RL) techniques is to learn how to achieve a desired outcome even when it is not clear how to accomplish that outcome. As a result, RL is more suited to enabling intelligent applications where an agent can make autonomous decisions such as robotics, autonomous vehicles, HVAC, industrial control, and more.
What type of environments can I use for training RL models?
Amazon SageMaker RL supports a number of different environments for training RL models. You can use AWS services such as AWS RoboMaker, open-source environments or custom environments developed using Open AI Gym interfaces, or commercial simulation environments such as MATLAB and SimuLink.
Do I need to write my own RL agent algorithms to train RL models?
No, SageMaker RL includes RL toolkits such as Coach and Ray RLLib that offer implementations of RL agent algorithms such as DQN, PPO, A3C, and many more.
Can I bring my own RL libraries and algorithm implementation and run them in SageMaker RL?
Yes, you can bring your own RL libraries and algorithm implementations in Docker Containers and run those in SageMaker RL.
Can I do distributed rollouts using SageMaker RL?
Yes. You can even select a heterogeneous cluster where the training can run on a GPU instance and the simulations can run on multiple CPU instances.
What deployment options does SageMaker provide?
What is Amazon SageMaker Asynchronous Inference?
How do I configure auto-scaling settings to scale down the instance count to zero when not actively processing requests?
You can scale down the SageMaker Asynchronous Inference endpoint instance count to zero in order to save on costs when you are not actively processing requests. You need to define a scaling policy that scales on the "ApproximateBacklogPerInstance" custom metric and set the "MinCapacity" value to zero. For step-by-step instructions, please visit the autoscale an asynchronous endpoint section of the developer guide.
What is Amazon SageMaker Serverless Inference?
SageMaker Serverless Inference is a purpose-built serverless model serving option that makes it easy to deploy and scale ML models. SageMaker Serverless Inference endpoints automatically start the compute resources and scale them in and out depending on traffic, eliminating the need for you to choose instance type, run provisioned capacity, or manage scaling. You can optionally specify the memory requirements for your serverless inference endpoint. You pay only for the duration of running the inference code and the amount of data processed, not for idle periods.
Why should I use SageMaker Serverless Inference?
What is Provisioned Concurrency for SageMaker Serverless Inference?
Why should I use Provisioned Concurrency?
With on-demand serverless endpoints, if your endpoint does not receive traffic for a while and then your endpoint suddenly receives new requests, it can take some time for your endpoint to spin up the compute resources to process the requests. This is called a cold start. A cold start can also occur if your concurrent requests exceed the current concurrent request usage. The cold start time depends on your model size, how long it takes to download your model, and the start-up time of your container.
To reduce variability in your latency profile, you can optionally enable Provisioned Concurrency for your serverless endpoints. With Provisioned Concurrency, your serverless endpoints are always ready and can instantaneously serve bursts in traffic, without any cold starts.
How will I be charged for Provisioned Concurrency?
As with on-demand Serverless Inference, when Provisioned Concurrency is enabled, you pay for the compute capacity used to process inference requests, billed by the millisecond, and the amount of data processed. You also pay for Provisioned Concurrency usage, based on the memory configured, duration provisioned, and amount of concurrency enabled. For more information, see Amazon SageMaker Pricing.
What is SageMaker shadow testing?
Why should I use SageMaker for shadow testing?
What is Amazon SageMaker Inference Recommender?
SageMaker Inference Recommender reduces the time required to get ML models in production by automating performance benchmarking and tuning model performance across SageMaker ML instances. You can now use SageMaker Inference Recommender to deploy your model to an endpoint that delivers the best performance and minimizes cost. You can get started with SageMaker Inference Recommender in minutes while selecting an instance type and get recommendations for optimal endpoint configurations within hours, eliminating weeks of manual testing and tuning time. With SageMaker Inference Recommender, you pay only for the SageMaker ML instances used during load testing, and there are no additional charges.
Why should I use SageMaker Inference Recommender?
How does SageMaker Inference Recommender work with other AWS services?
Can SageMaker Inference Recommender support multi-model endpoints or multi-container endpoints?
No, we currently support only a single model per endpoint.
What type of endpoints does SageMaker Inference Recommender support?
Currently we support only real-time endpoints.
Can I use SageMaker Inference Recommender in one Region and benchmark in different Regions?
We support all Regions supported by Amazon SageMaker, except the AWS China Regions.
Does SageMaker Inference Recommender support Amazon EC2 Inf1 instances?
Yes, we support all types of containers. Amazon EC2 Inf1, based on the AWS Inferentia chip, requires a compiled model artifact using either the Neuron compiler or Amazon SageMaker Neo. Once you have a compiled model for an Inferentia target and the associated container image URI, you can use SageMaker Inference Recommender to benchmark different Inferentia instance types.
What is Amazon SageMaker Model Monitor?
Can I access the infrastructure that SageMaker runs on?
No. SageMaker operates the compute infrastructure on your behalf, allowing it to perform health checks, apply security patches, and do other routine maintenance. You can also deploy the model artifacts from training with custom inference code in your own hosting environment.
How do I scale the size and performance of an SageMaker model once in production?
SageMaker hosting automatically scales to the performance needed for your application using Application Auto Scaling. In addition, you can manually change the instance number and type without incurring downtime by modifying the endpoint configuration.
How do I monitor my SageMaker production environment?
SageMaker emits performance metrics to Amazon CloudWatch Metrics so you can track metrics, set alarms, and automatically react to changes in production traffic. In addition, SageMaker writes logs to Amazon CloudWatch Logs to let you monitor and troubleshoot your production environment.
What kinds of models can be hosted with SageMaker?
SageMaker can host any model that adheres to the documented specification for inference Docker images. This includes models created from SageMaker model artifacts and inference code.
How many concurrent real-time API requests does SageMaker support?
SageMaker is designed to scale to a large number of transactions per second. The precise number varies based on the deployed model and the number and type of instances to which the model is deployed.
How does SageMaker support fully managed model hosting and management?
What is Batch Transform?
Batch Transform enables you to run predictions on large or small batch data. There is no need to break down the dataset into multiple chunks or manage real-time endpoints. With a simple API, you can request predictions for a large number of data records and transform the data quickly and easily.
What deployment endpoint options does SageMaker support?
What is autoscaling for elasticity?
What is Amazon SageMaker Edge Manager?
SageMaker Edge Manager makes it easier to optimize, secure, monitor, and maintain ML models on fleets of edge devices such as smart cameras, robots, personal computers, and mobile devices. SageMaker Edge Manager helps ML developers operate ML models on a variety of edge devices at scale.
How do I get started with SageMaker Edge Manager?
To get started with SageMaker Edge Manager, you need to compile and package your trained ML models in the cloud, register your devices, and prepare your devices with the SageMaker Edge Manager SDK. To prepare your model for deployment, SageMaker Edge Manager uses SageMaker Neo to compile your model for your target edge hardware. Once a model is compiled, SageMaker Edge Manager signs the model with an AWS generated key, then packages the model with its runtime and your necessary credentials to get it ready for deployment. On the device side, you register your device with SageMaker Edge Manager, download the SageMaker Edge Manager SDK, and then follow the instructions to install the SageMaker Edge Manager agent on your devices. The tutorial notebook provides a step-by-step example of how you can prepare the models and connect your models on edge devices with SageMaker Edge Manager.
What devices are supported by SageMaker Edge Manager?
SageMaker Edge Manager supports common CPU (ARM, x86) and GPU (ARM, Nvidia) based devices with Linux and Windows operating systems. Over time, SageMaker Edge Manager will expand to support more embedded processors and mobile platforms that are also supported by SageMaker Neo.
Do I need to use SageMaker to train my model in order to use SageMaker Edge Manager?
No, you do not. You can train your models elsewhere or use a pre-trained model from open source or from your model vendor.
Do I need to use SageMaker Neo to compile my model in order to use SageMaker Edge Manager?
Yes, you do. SageMaker Neo converts and compiles your models into an executable that you can then package and deploy on your edge devices. Once the model package is deployed, the SageMaker Edge Manager agent will unpack the model package and run the model on the device.
How do I deploy models to the edge devices?
SageMaker Edge Manager stores the model package in your specified Amazon S3 bucket. You can use the over-the-air (OTA) deployment feature provided by AWS IoT Greengrass or any other deployment mechanism of your choice to deploy the model package from your S3 bucket to the devices.
How is SageMaker Edge Manager SDK different from the SageMaker Neo runtime (dlr)?
Neo dlr is an open-source runtime that only runs models compiled by the SageMaker Neo service. Compared to the open source dlr, the SageMaker Edge Manager SDK includes an enterprise grade on-device agent with additional security, model management, and model serving features. The SageMaker Edge Manager SDK is suitable for production deployment at scale.
In which Regions is SageMaker Edge Manager available?
SageMaker Edge Manager is available in six Regions: US East (N. Virginia), US East (Ohio), US West (Oregon), EU (Ireland), EU (Frankfurt), and Asia Pacific (Tokyo). For details, see the AWS Regional Services list .
What is Amazon SageMaker Neo?
SageMaker Neo enables ML models to train once and run anywhere in the cloud and at the edge. SageMaker Neo automatically optimizes models built with popular DL frameworks that can be used to deploy on multiple hardware platforms. Optimized models run up to 25 times faster and consume less than a tenth of the resources of typical ML models.
How do I get started with SageMaker Neo?
To get started with SageMaker Neo, sign in to the SageMaker console, choose a trained model, follow the example to compile models, and deploy the resulting model onto your target hardware platform.
What are the major components of SageMaker Neo?
SageMaker Neo contains two major components: a compiler and a runtime. First, the SageMaker Neo compiler reads models exported by different frameworks. It then converts the framework-specific functions and operations into a framework-agnostic intermediate representation. Next, it performs a series of optimizations. Then, the compiler generates binary code for the optimized operations and writes them to a shared object library. The compiler also saves the model definition and parameters into separate files. During execution, the SageMaker Neo runtime loads the artifacts generated by the compiler—model definition, parameters, and the shared object library to run the model.
Do I need to use SageMaker to train my model in order to use SageMaker Neo to convert the model?
No. You can train models elsewhere and use SageMaker Neo to optimize them for SageMaker ML instances or AWS IoT Greengrass supported devices.
Which models does SageMaker Neo support?
Currently, SageMaker Neo supports the most popular DL models that power computer vision applications and the most popular decision tree models used in SageMaker today. SageMaker Neo optimizes the performance of AlexNet, ResNet, VGG, Inception, MobileNet, SqueezeNet, and DenseNet models trained in MXNet and TensorFlow, and classification and random cut forest models trained in XGBoost.
Which hardware platforms does SageMaker Neo support?
In which Regions is SageMaker Neo available?
To see a list of supported Regions, view the AWS Regional Services list .
Amazon SageMaker Savings Plans
What are Amazon SageMaker Savings Plans?
Why should I use SageMaker Savings Plans?
How can I get started with SageMaker Savings Plans?
How are Savings Plans for SageMaker different from Compute Savings Plans for Amazon EC2?
How do Savings Plans work with AWS Organizations/Consolidated Billing?
Savings Plans can be purchased in any account within an AWS Organization/Consolidated Billing family. By default, the benefit provided by Savings Plans is applicable to usage across all accounts within an AWS Organization/Consolidated Billing family. However, you can also choose to restrict the benefit of Savings Plans to only the account that purchased them.