Q: What is Amazon SageMaker?
Amazon SageMaker is a fully managed service that provides every developer and data scientist with the ability to build, train, and deploy machine learning (ML) models quickly. SageMaker removes the heavy lifting from each step of the machine learning process to make it easier to develop high quality models.
Q: In which regions is Amazon SageMaker available?
For a list of the supported Amazon SageMaker AWS regions, please visit the AWS region table for all AWS global infrastructure. Also, for more information, see regions and endpoints in the AWS general reference.
Q: What is the service availability of Amazon SageMaker?
Amazon SageMaker is designed for high availability. There are no maintenance windows or scheduled downtimes. SageMaker APIs run in Amazon’s proven, high-availability data centers, with service stack replication configured across three facilities in each AWS region to provide fault tolerance in the event of a server failure or Availability Zone outage.
Q: How does Amazon SageMaker secure my code?
Amazon SageMaker stores code in ML storage volumes, secured by security groups and optionally encrypted at rest.
Q: What security measures does Amazon SageMaker have?
Amazon SageMaker ensures that ML model artifacts and other system artifacts are encrypted in transit and at rest. Requests to the SageMaker API and console are made over a secure (SSL) connection. You pass AWS Identity and Access Management roles to SageMaker to provide permissions to access resources on your behalf for training and deployment. You can use encrypted Amazon S3 buckets for model artifacts and data, as well as pass a KMS key to SageMaker notebooks, training jobs, and endpoints, to encrypt the attached ML storage volume. Amazon SageMaker also supports Amazon Virtual Privacy Cloud (VPC) and AWS PrivateLink support.
Q: Does Amazon SageMaker use or share models, training data, or algorithms?
Amazon SageMaker does not use or share customer models, training data, or algorithms. We know that customers care deeply about privacy and data security. That's why AWS gives you ownership and control over your content through simple, powerful tools that allow you to determine where your content will be stored, secure your content in transit and at rest, and manage your access to AWS services and resources for your users. We also implement responsible and sophisticated technical and physical controls that are designed to prevent unauthorized access to or disclosure of your content. As a customer, you maintain ownership of your content, and you select which AWS services can process, store, and host your content. We do not access your content for any purpose without your consent.
Q: How am I charged for Amazon SageMaker?
You pay for ML compute, storage, and data processing resources you use for hosting the notebook, training the model, performing predictions, and logging the outputs. Amazon SageMaker allows you to select the number and type of instance used for the hosted notebook, training, and model hosting. You only pay for what you use, as you use it; there are no minimum fees and no upfront commitments. See the Amazon SageMaker pricing page for details.
Q: How can I optimize my Amazon SageMaker costs, such as detecting and stopping idle resources in order to avoid unnecessary charges?
There are several best practices you can adopt to optimize your Amazon SageMaker resource utilization. Some approaches involve configuration optimizations, others involve programmatic solutions. A full guide on this concept, complete with visual tutorials and code samples, can be found in this blog post.
Q: What if I have my own notebook, training, or hosting environment?
Amazon SageMaker provides a full end-to-end workflow, but you can continue to use your existing tools with SageMaker. You can easily transfer the results of each stage in and out of SageMaker as your business requirements dictate.
Q: Is R supported with Amazon SageMaker?
Yes, R is supported with Amazon SageMaker. You can use R within SageMaker Notebook instances, which include a pre-installed R kernel and the reticulate library. Reticulate offers an R interface for the Amazon SageMaker Python SDK, enabling machine learning practitioners to build, train, tune, and deploy R models.
Q. How can I check for imbalances in my model?
Amazon SageMaker Clarify helps improve model transparency by detecting statistical bias across the entire ML workflow. SageMaker Clarify checks for imbalances during data preparation, after training, and ongoing over time, and also includes tools to help explain ML models and their predictions. Findings can be shared through explainability reports.
Q. What kind of bias does Amazon SageMaker Clarify detect?
Measuring bias in ML models is a first step to mitigating bias. Bias may be measured before training and after training, as well as for inference for a deployed model. Each measure of bias corresponds to a different notion of fairness. Even considering simple notions of fairness leads to many different measures applicable in various contexts. You need to choose bias notions and metrics that are valid for the application and the situation under investigation. SageMaker currently supports the computation of different bias metrics for training data (as part of SageMaker data preparation), for the trained model (as part of SageMaker Experiments), and for inference for a deployed model (as part of SageMaker Model Monitor). For example, before training, we provide metrics for checking whether the training data is representative (that is, whether one group is under-represented) and whether there are differences in the label distribution across groups. After training or during deployment, metrics can be helpful to measure whether (and by how much) the performance of the model differs across groups. For example, start by comparing the error rates (how likely a model's prediction is to differ from the true label) or break further down into precision (how likely a positive prediction is to be correct) and recall (how likely the model will correctly label a positive example).
Q. How does Amazon SageMaker Clarify improve model explainablity?
Amazon SageMaker Clarify is integrated with SageMaker Experiments to provide a feature importance graph detailing the importance of each input for your model’s overall decision-making process after the model has been trained. These details can help determine if a particular model input has more influence than it should on overall model behavior. SageMaker Clarify also makes explanations for individual predictions available via an API.
Q. What is Amazon SageMaker Studio?
Amazon SageMaker Studio provides a single, web-based visual interface where you can perform all ML development steps. SageMaker Studio gives you complete access, control, and visibility into each step required to build, train, and deploy models. You can quickly upload data, create new notebooks, train and tune models, move back and forth between steps to adjust experiments, compare results, and deploy models to production all in one place, making you much more productive. All ML development activities including notebooks, experiment management, automatic model creation, debugging and profiling, and model drift detection can be performed within the unified SageMaker Studio visual interface.
Q. How does Amazon SageMaker Studio pricing work?
There is no additional charge for using Amazon SageMaker Studio. You only pay for the underlying compute and storage charges on the services you use within Amazon SageMaker Studio.
Q. In which regions is Amazon SageMaker Studio supported?
You can find the regions where Amazon SageMaker Studio is support in the documentation here.
Low Code Machine Learning
Q. What is Amazon SageMaker Autopilot?
Amazon SageMaker Autopilot is the industry’s first automated machine learning capability that gives you complete control and visibility into your ML models. SageMaker Autopilot automatically inspects raw data, applies feature processors, picks the best set of algorithms, trains and tunes multiple models, tracks their performance, and then ranks the models based on performance, all with just a few clicks. The result is the best performing model that you can deploy at a fraction of the time normally required to train the model. You get full visibility into how the model was created and what’s in it and SageMaker Autopilot integrates with Amazon SageMaker Studio. You can explore up to 50 different models generated by SageMaker Autopilot inside SageMaker Studio so it’s easy to pick the best model for your use case. SageMaker Autopilot can be used by people without machine learning experience to easily produce a model or it can be used by experienced developers to quickly develop a baseline model on which teams can further iterate.
Q: What built-in algorithms are supported in Amazon SageMaker Autopilot?
Amazon SageMaker Autopilot supports 2 built-in algorithms: XGBoost and Linear Learner.
Q: Can I stop an Amazon SageMaker Autopilot job manually?
Yes. You can stop a job at any time. When an Amazon SageMaker Autopilot job is stopped, all ongoing trials will be stopped and no new trial will be started.
Q: How do I get started with Amazon SageMaker quickly?
Amazon SageMaker JumpStart helps you quickly and easily get started with machine learning. SageMaker JumpStart provides a set of solutions for the most common use cases that can be deployed readily with just a few clicks. The solutions are fully customizable and showcase the use of AWS CloudFormation templates and reference architectures so you can accelerate your machine learning journey. SageMaker JumpStart also supports one-click deployment and fine-tuning of more than 150 popular open source models such as transformer, object detection, and image classification models.
Q: Which open source models are supported with Amazon SageMaker JumpStart?
Amazon SageMaker JumpStart includes 150+ pre-trained open source models from PyTorch Hub & TensorFlow Hub. For vision tasks such as image classification and object detection, you can leverage models such as ResNet, MobileNet, and Single-Shot Detector (SSD). For text tasks such as sentence classification, text classification, and question answering, you can use models such as BERT, RoBERTa, and DistilBERT.
Q. What solutions come pre-built with Amazon SageMaker JumpStart?
SageMaker JumpStart includes solutions that are preconfigured with all necessary AWS services to launch a solution into production. Solutions are fully customizable so you can easily modify them to fit your specific use case and dataset. You can use solutions for over 15 use cases including demand forecasting, fraud detection, and predictive maintenance, and readily deploy solutions with just a few clicks. For more information about all solutions available, visit the SageMaker getting started page.
Q: How does Amazon SageMaker JumpStart pricing work?
You are charged for the AWS services launched from SageMaker JumpStart, such as training jobs and endpoints, based on SageMaker pricing. There is no additional charge for using Amazon SageMaker JumpStart.
Machine Learning Workflows
Q. How can I build a CI/CD pipeline with Amazon SageMaker?
Amazon SageMaker Pipelines helps you create fully automated ML workflows from data preparation through model deployment so you can scale to thousands of ML models in production. SageMaker Pipelines comes with a Python SDK which connects to SageMaker Studio so you can take advantage of a visual interface to build each step of the workflow. Then using a single API, you can connect each step to create an end-to-end workflow. SageMaker Pipelines takes care of managing data between steps, packaging the code recipes, and orchestrating their execution, reducing months of coding to a few hours. Every time a workflow executes, a complete record of the data processed and actions taken is kept so data scientists and ML developers can quickly debug problems.
Q. How do I view all my trained models to choose the best model to move to production?
Amazon SageMaker Pipelines provides a central repository of trained models called a model registry. You can discover models and access the model registry visually through SageMaker Studio or programmatically through the Python SDK making it easy to choose your desired model to deploy into production.
Q. What components of Amazon SageMaker can be added to Amazon SageMaker Pipelines?
The components available through Amazon SageMaker Studio including SageMaker Clarify, SageMaker Data Wrangler, SageMaker Feature Store, SageMaker Experiments, SageMaker Debugger, and SageMaker Model Monitor can be added to SageMaker Pipelines.
Q. How do I track my model components across the entire ML workflow?
Amazon SageMaker Pipelines automatically keeps track of all model constituents and keeps an audit trail of all changes thereby eliminating manual tracking, and can help you achieve compliance goals. You can track data, code, trained models, and more with SageMaker Pipelines.
Q. How does the pricing for Amazon SageMaker Pipelines work?
There is no additional charge for Amazon SageMaker Pipelines. You only pay for the underlying compute or any separate AWS services you use within SageMaker Pipelines.
Q. Can I use Kubeflow with Amazon SageMaker?
Yes. Amazon SageMaker Components for Kubeflow Pipelines, are open-source plugins that allow you to use Kubeflow Pipelines to define your ML workflows and use SageMaker for the data labeling, training, and inference steps. Kubeflow Pipelines is an add-on to Kubeflow that let you build and deploy portable and scalable end-to-end ML pipelines. However, when using Kubeflow Pipelines, ML ops teams need to manage a Kubernetes cluster with CPU and GPU instances and keep its utilization high at all times to reduce operational costs. Maximizing the utilization of a cluster across data science teams is challenging and adds additional operational overhead to the ML ops teams. As an alternative to an ML optimized Kubernetes cluster, with Amazon SageMaker Components for Kubeflow Pipelines you can take advantage of powerful SageMaker features such as data labeling, fully managed large-scale hyperparameter tuning and distributed training jobs, one-click secure and scalable model deployment, and cost-effective training through Amazon EC2 Spot instances without needing to configure and manage Kubernetes clusters specifically to run the machine learning jobs.
Q: How does Amazon SageMaker Components for Kubeflow Pipelines pricing work?
There is no additional charge for using Amazon SageMaker Components for Kubeflow Pipelines.
Q. How can Amazon SageMaker prepare data for machine learning?
Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning. From a single interface in SageMaker Studio, you can import data from Amazon S3, Amazon Athena, Amazon Redshift, AWS Lake Formation, and Amazon SageMaker Feature Store , and in just a few clicks SageMaker Data Wrangler will automatically load, aggregate, and display the raw data. It will then make conversion recommendations based on the source data, transform the data into new features, validate the features, and provide visualizations with recommendations on how to remove common sources of error such as incorrect labels. Once your data is prepared, you can build fully automated machine learning workflows with Amazon SageMaker Pipelines or import that data into Amazon SageMaker Feature Store.
Q. How can I create model features with Amazon SageMaker Data Wrangler?
Without writing a single line of code, Amazon SageMaker Data Wrangler can automatically transform your data into new features. SageMaker Data Wrangler offers a selection of pre-configured data transforms, such as convert column type, one hot encoding, impute missing data with mean or median, rescale columns, and data/time embeddings. For example, you can convert a text field column into a numerical column with a single click, or author custom transforms in PySpark, SQL, and Pandas.
Q. How can I visualize my data in Amazon SageMaker Data Wrangler?
Amazon SageMaker Data Wrangler helps you understand your data and identify potential errors and extreme values with a set of robust pre-configured visualization templates. Histograms, scatter plots, and ML-specific visualizations, such as target leakage detection, are all available without writing a single line of code. You can also create and edit your own visualizations.
Q. How does the pricing for Amazon SageMaker Data Wrangler work?
You pay for all ML compute, storage, and data processing resources you use for Amazon SageMaker Data Wrangler. You can review all the details of Amazon SageMaker Data Wrangler pricing here. As part of the AWS Free Tier, you can also get started with SageMaker Data Wrangler for free.
Q. How do I store features for my ML models?
Amazon SageMaker Feature Store provides a central repository for data features with low latency (milliseconds) reads and writes. Features can be stored, retrieved, discovered, and shared through SageMaker Feature Store for easy re-use across models and teams with secure access and control. SageMaker Feature Store supports both online and offline features generated via batch or streaming pipelines. It supports backfilling the features and provides both online and offline stores to maintain parity between features used in model training and inference.
Q. How do I maintain consistency between online and offline features?
Amazon SageMaker Feature Store automatically maintains consistency between online and offline features without additional management or code. SageMaker Feature Store is fully managed and maintains consistency across training and inference environments.
Q. How can I reproduce a feature from a given moment in time?
Amazon SageMaker Feature Store maintains time stamps for all features at every instance of time. This helps you retrieve features at any period of time for business or compliance requirements. You can easily explain model features and their values from when they were first created to the present time, by reproducing the model from a given moment in time.
Q. What are offline features?
Offline features are used for training because you need access to very large volumes over a long period of time. These features are served from a high-throughput, high-bandwidth repository.
Q. What are online features?
Online features are used in applications required to make real-time predictions. Online features are served from a high-throughput repository with single digit millisecond latency for fast predictions.
Q. How does pricing work for Amazon SageMaker Feature Store?
You can get started with Amazon SageMaker Feature Store for free, as part of the AWS Free Tier. With SageMaker Feature Store, you pay for writing into the feature store, and reading and storage from the online feature store. The SageMaker Pricing Page has all details on how the pricing works for SageMaker Feature Store.
Q. What is Amazon SageMaker Ground Truth?
Amazon SageMaker Ground Truth provides automated data labeling using machine learning. SageMaker Ground Truth will first select a random sample of data and send it to Amazon Mechanical Turk to be labeled. The results are then used to train a labeling model that attempts to label a new sample of raw data automatically. The labels are committed when the model can label the data with a confidence score that meets or exceeds a threshold you set. Where the confidence score falls below your threshold, the data is sent to human labelers. Some of the data labeled by humans is used to generate a new training dataset for the labeling model, and the model is automatically retrained to improve its accuracy. This process repeats with each sample of raw data to be labeled. The labeling model becomes more capable of automatically labeling raw data with each iteration, and less data is routed to humans.
Q: What are Amazon SageMaker Studio Notebooks?
Amazon SageMaker Studio Notebooks are collaborative, flexible, managed Jupyter notebooks that are part of Amazon SageMaker Studio, a fully integrated development environment for machine learning.
Q: How are SageMaker Studio Notebooks different from the instance based notebooks offering?
SageMaker Studio Notebooks offer a few important features that differentiate them from the instance-based notebooks. With the Studio Notebooks, you can quickly launch notebooks without needing to manually provision an instance and waiting for it to be operational. The start-up time of launching the UI to read and execute a notebook is faster than the instance-based notebooks.
You also have the flexibility to choose from a large collection of instance types from within the UI at any time. You do not need to go to the AWS console to start new instances and port over your notebooks.
Each user has an isolated home directory independent of a particular instance. This directory is automatically mounted into all notebook servers and kernels as they’re started, so you can access your notebooks and other files even when you switch instances to view and run your notebooks.
SageMaker Studio Notebooks are integrated with AWS SSO, making it easy to use your organizational credentials to access the notebooks. Notebook sharing is an integrated feature in SageMaker Studio Notebooks. You can also share your notebooks with your peers using a single click.
Q. How do Amazon SageMaker Studio Notebooks work?
Amazon SageMaker Studio Notebooks are one-click Jupyter notebooks that can be spun quickly. The underlying compute resources are fully elastic, so you can easily dial up or down the available resources and the changes take place automatically in the background without interrupting your work. SageMaker also enables one-click sharing of notebooks. You can easily share notebooks with others and they’ll get the exact same notebook, saved in the same place.
With SageMaker Studio Notebooks you can sign in with your corporate credentials using AWS SSO. Sharing notebooks within and across teams is easy, since the dependencies needed to run a notebook are automatically tracked in work images that are encapsulated with the notebook as it is shared.
Q. How do Amazon SageMaker Studio Notebooks work with other AWS services?
Amazon SageMaker Studio Notebooks give you access to all SageMaker features, such as distributed training, batch transform, hosting, and experiment management. You can access other services such as datasets in Amazon S3, Amazon Redshift, AWS Glue, Amazon EMR, or AWS Lake Formation from SageMaker Notebooks.
Q: How does SageMaker Studio Notebooks pricing work?
You pay for both compute and storage when you use SageMaker Studio notebooks. See Amazon SageMaker Pricing for charges by compute instance type. Your notebooks and associated artifacts such as data files and scripts are persisted on Amazon EFS. See Amazon EFS Pricing for storage charges. As part of the AWS Free Tier, you can get started with Amazon SageMaker Studio Notebooks for free.
Q: Do I get charged separately for each notebook created and run in SageMaker Studio?
No. You can create and run multiple notebooks on the same compute instance. You pay only for the compute that you use, not for individual items. You can read more about this in our metering guide.
In addition to the notebooks, you can also start and run terminals and interactive shells in Studio – all on the same compute instance. Each application runs within a container or image. SageMaker Studio provides several built-in images purpose-built and pre-configured for data science and machine learning. You can read more about the Studio developer environment in the guide for using SageMaker Studio Notebooks.
Q: How do I monitor and shutdown the resources used by my notebooks?
You can monitor and shutdown the resources used by your SageMaker Studio notebooks through both SageMaker Studio visual interface and AWS Management Console. See documentation for more details.
Q: I’m running a SageMaker Studio Notebook. Will I still be charged if I close my browser, close the notebook tab, or just leave the browser open?
Yes, you will continue to be charged for the compute. This is similar to starting Amazon EC2 instances in the AWS management console and then closing the browser. The Amazon EC2 instances are still running and you still incur charges unless you explicitly shut down the instance.
Q: Do I get charged for creating and setting up a Studio domain?
No, you don’t get charged for creating or configuring a Studio domain, including adding, updating, and deleting user profiles.
Q: How do I see the itemized charges for Studio Notebooks or other SageMaker services?
As an admin, you can view the list of itemized charges for SageMaker, including Studio, in the AWS Billing console. From the AWS management console for SageMaker, choose Services on the top menu, type Billing on the search box and select Billing from the dropdown, then select Bills on the left panel. In the Details section, you can click on SageMaker to expand the list of regions and drill down to the itemized charges.
Q. What is Amazon SageMaker Experiments?
Amazon SageMaker Experiments helps you organize and track iterations to machine learning models. SageMaker Experiments helps you manage iterations by automatically capturing the input parameters, configurations, and results, and storing them as ‘experiments’. You can work within the visual interface of SageMaker Studio, where you can browse active experiments, search for previous experiments by their characteristics, review previous experiments with their results, and compare experiment results visually.
Q. What is Amazon SageMaker Debugger?
Amazon SageMaker Debugger automatically captures real-time metrics during training such as confusion matrices and learning gradients to help improve model accuracy. The metrics from SageMaker Debugger can be visualized in SageMaker Studio for easy understanding. SageMaker Debugger can also generate warnings and remediation advice when common training problems are detected. SageMaker Debugger also automatically monitors and profiles system resources such as CPUs, GPUs, network, and memory in real-time, and provides recommendations on re-allocation of these resources. This enables you to use your resources efficiently during training and helps reduce costs and resources.
Q. Does Amazon SageMaker support distributed training?
Yes. Amazon SageMaker can automatically distribute deep learning models and large training sets across AWS GPU instances in a fraction of the time it takes to build and optimize these distribution strategies manually. The two distributed training techniques that SageMaker applies are data parallelism and model parallelism. Data parallelism is applied to improve training speeds by dividing the data equally across multiple GPU instances, allowing each instance to train concurrently. Model parallelism is useful for models too large to be stored on a single GPU and require the model to be partitioned into smaller parts before distributing across multiple GPUs. With only a few lines of additional code in your PyTorch and TensorFlow training scripts, SageMaker will automatically apply data parallelism or model parallelism for you, allowing you to develop and deploy your models faster. SageMaker will determine the best approach to split your model by using graph partitioning algorithms to balance the computation of each GPU while minimizing the communication between GPU instances. SageMaker also optimizes your distributed training jobs through algorithms that fully utilize the AWS compute and network in order to achieve near-linear scaling efficiency, which allows you to complete training faster than manual open source implementations.
Q: What is Managed Spot Training?
Managed Spot Training with Amazon SageMaker lets you train your machine learning models using Amazon EC2 Spot instances, while reducing the cost of training your models by up to 90%.
Q: How do I use Managed Spot Training?
You enable the Managed Spot Training option when submitting your training jobs and you also specify how long you want to wait for Spot capacity. Amazon SageMaker will then use Amazon EC2 Spot instances to run your job and manages the Spot capacity. You have full visibility into the status of your training job, both while they are running and while they are waiting for capacity.
Q: When should I use Managed Spot Training?
Managed Spot Training is ideal when you have flexibility with your training runs and when you want to minimize the cost of your training jobs. With Managed Spot Training, you can reduce the cost of training your machine learning models by up to 90%.
Q: How does Managed Spot Training work?
Managed Spot Training uses Amazon EC2 Spot instances for training, and these instances can be pre-empted when AWS needs capacity. As a result, Managed Spot Training jobs can run in small increments as and when capacity becomes available. The training jobs need not be restarted from scratch when there is an interruption as Amazon SageMaker can resume the training jobs using the latest model checkpoint. The built-in frameworks and the built-in computer vision algorithms with SageMaker enable periodic checkpoints, and you can enable checkpoints with custom models.
Q: Do I need to periodically checkpoint with Managed Spot Training?
We recommend periodic checkpoints as a general best practice for long running training jobs. This prevents your Managed Spot Training jobs from restarting if capacity is pre-empted. When you enable checkpoints, Amazon SageMaker resumes your Managed Spot Training jobs from the last checkpoint.
Q: How do you calculate the cost savings with Managed Spot Training jobs?
Once a Managed Spot Training job is completed, you can see the savings in the AWS management console and also calculate the cost savings as the percentage difference between the duration for which the training job ran and the duration for which you were billed.
Regardless of how many times your Managed Spot Training jobs are interrupted, you are charged only once for the duration for which the data was downloaded.
Q: Which instances can I use with Managed Spot Training?
Managed Spot Training can be used with all instances supported in Amazon SageMaker.
Q: Which AWS regions are supported with Managed Spot Training?
Managed Spot Training is supported on all AWS regions where Amazon SageMaker is currently available.
Q: Are there limits to the size of the dataset I can use for training?
There are no fixed limits to the size of the dataset you can use for training models with Amazon SageMaker.
Q: What algorithms does Amazon SageMaker use to generate models?
Amazon SageMaker includes built-in algorithms for linear regression, logistic regression, k-means clustering, principal component analysis, factorization machines, neural topic modeling, latent dirichlet allocation, gradient boosted trees, sequence2sequence, time series forecasting, word2vec, and image classification. SageMaker also provides optimized Apache MXNet, Tensorflow, Chainer, PyTorch, Gluon, Keras, Horovod, Scikit-learn, and Deep Graph Library containers. In addition, Amazon SageMaker supports your custom training algorithms provided through a Docker image adhering to the documented specification.
Q: What is Automatic Model Tuning?
Most machine learning algorithms expose a variety of parameters that control how the underlying algorithm operates. Those parameters are generally referred to as hyperparameters and their values affect the quality of the trained models. Automatic model tuning is the process of finding a set of hyperparameters for an algorithm that can yield an optimal model.
Q: What models can be tuned with Automatic Model Tuning?
You can run automatic model tuning in Amazon SageMaker on top of any algorithm as long as it’s scientifically feasible, including built-in SageMaker algorithms, deep neural networks, or arbitrary algorithms you bring to SageMaker in the form of Docker images.
Q: Can I use Automatic Model Tuning outside of Amazon SageMaker?
Not at this time. The best model tuning performance and experience is within Amazon SageMaker.
Q: What is the underlying tuning algorithm?
Currently, our algorithm for tuning hyperparameters is a customized implementation of Bayesian Optimization. It aims to optimize a customer specified objective metric throughout the tuning process. Specifically, it checks the object metric of completed training jobs, and leverages the knowledge to infer the hyperparameter combination for the next training job.
Q: Will you recommend specific hyperparameters for tuning?
No. How certain hyperparameters impact the model performance depends on various factors and it is hard to definitively say one hyperparameter is more important than the others and thus needs to be tuned. For built-in algorithms within Amazon SageMaker, we do call out whether or not a hyperparameter is tunable.
Q: How long does a hyperparameter tuning job take?
The length of time for a hyperparameter tuning job depends on multiple factors including the size of the data, the underlying algorithm, and the values of the hyperparameters. Additionally, customers can choose the number of simultaneous training jobs and total number of training jobs. All these choices affect how long a hyperparameter tuning job can last.
Q: Can I optimize multiple objectives simultaneously like a model to be both fast and accurate?
Not at this time. Right now, you need to specify a single objective metric to optimize or change your algorithm code to emit a new metric, which is a weighted average between two or more useful metrics, and have the tuning process optimize towards that objective metric.
Q: How much does Automatic Model Tuning cost?
There is no charge for a hyperparameter tuning job itself. You will be charged by the training jobs that are launched by the hyperparameter tuning job, based on model training pricing.
Q: How do I decide to use Amazon SageMaker Autopilot or Automatic Model Tuning?
Amazon SageMaker Autopilot automates everything in a typical machine learning workflow, including feature preprocessing, algorithm selection, and hyperparameter tuning, while specifically focusing on classification and regression use cases. Automatic Model Tuning, on the other hand, is designed to tune any model, no matter whether it is based on built-in algorithms, deep learning frameworks, or custom containers. In exchange for the flexibility, you have to manually pick the specific algorithm, determine the hyperparameters to tune, and corresponding search ranges.
Q: What is reinforcement learning?
Reinforcement learning is a machine learning technique that enables an agent to learn in an interactive environment by trial and error using feedback from its own actions and experiences.
Q: Can I train reinforcement learning models in Amazon SageMaker?
Yes, you can train reinforcement learning models in Amazon SageMaker in addition to supervised and unsupervised learning models.
Q: How is reinforcement learning different from supervised learning?
Though both supervised and reinforcement learning use mapping between input and output, unlike supervised learning where the feedback provided to the agent is correct set of actions for performing a task, reinforcement learning uses a delayed feedback where reward signals are optimized to ensure a long-term goal through a sequence of actions.
Q: When should I use reinforcement learning?
While the goal of supervised learning techniques is to find the right answer based on the patterns in the training data, the goal of unsupervised learning techniques is to find similarities and differences between data points. In contrast, the goal of reinforcement learning (RL) techniques is to learn how to achieve a desired outcome even when it is not clear how to accomplish that outcome. As a result, RL is more suited to enabling intelligent applications where an agent can make autonomous decisions such as robotics, autonomous vehicles, HVAC, industrial control, and more.
Q: What type of environments can I use for training reinforcement learning models?
Amazon SageMaker RL supports a number of different environments for training reinforcement learning models. You can use AWS services such as AWS RoboMaker, open source environments or custom environments developed using Open AI Gym interfaces, or commercial simulation environments such as MATLAB and SimuLink.
Q: Do I need to write my own RL agent algorithms to train reinforcement learning models?
No, Amazon SageMaker RL includes RL toolkits such as Coach and Ray RLLib that offer implementations of RL agent algorithms such as DQN, PPO, A3C, and many more.
Q: Can I bring my own RL libraries and algorithm implementation and run in Amazon SageMaker RL?
Yes, you can bring your own RL libraries and algorithm implementations in Docker Containers and run those in Amazon SageMaker RL.
Q: Can I do distributed rollouts using Amazon SageMaker RL?
Yes. You can even select a heterogeneous cluster where the training can run on a GPU instance and the simulations can run on multiple CPU instances.
Q. What is Amazon SageMaker Model Monitor?
Amazon SageMaker Model Monitor allows developers to detect and remediate concept drift. SageMaker Model Monitor automatically detects concept drift in deployed models and provides detailed alerts that help identify the source of the problem. All models trained in SageMaker automatically emit key metrics that can be collected and viewed in SageMaker Studio. From inside SageMaker Studio you can configure data to be collected, how to view it, and when to receive alerts.
Q: Can I access the infrastructure that Amazon SageMaker runs on?
No. Amazon SageMaker operates the compute infrastructure on your behalf, allowing it to perform health checks, apply security patches, and do other routine maintenance. You can also deploy the model artifacts from training with custom inference code in your own hosting environment.
Q: How do I scale the size and performance of an Amazon SageMaker model once in production?
Amazon SageMaker hosting automatically scales to the performance needed for your application using Application Auto Scaling. In addition, you can manually change the instance number and type without incurring downtime through modifying the endpoint configuration.
Q: How do I monitor my Amazon SageMaker production environment?
Amazon SageMaker emits performance metrics to Amazon CloudWatch Metrics so you can track metrics, set alarms, and automatically react to changes in production traffic. In addition, Amazon SageMaker writes logs to Amazon CloudWatch Logs to let you monitor and troubleshoot your production environment.
Q: What kinds of models can be hosted with Amazon SageMaker?
Amazon SageMaker can host any model that adheres to the documented specification for inference Docker images. This includes models created from Amazon SageMaker model artifacts and inference code.
Q: How many concurrent real-time API requests does Amazon SageMaker support?
Amazon SageMaker is designed to scale to a large number of transactions per second. The precise number varies based on the deployed model and the number and type of instances to which the model is deployed.
Q: What is Batch Transform?
Batch Transform enables you to run predictions on large or small batch data. There is no need to break down the data set into multiple chunks or manage real-time endpoints. With a simple API, you can request predictions for a large number of data records and transform the data quickly and easily.
Q: What is Amazon SageMaker Edge Manager?
Amazon SageMaker Edge Manager is a capability in Amazon SageMaker that makes it easier to optimize, secure, monitor, and maintain machine learning models on fleets of edge devices such as smart cameras, robots, personal computers, and mobile devices. SageMaker Edge Manager helps ML developers operate ML models on a variety of edge devices at scale.
Q: How do I get started with SageMaker Edge Manager?
To get started with SageMaker Edge Manager, you need to compile and package your trained ML models in the cloud, register your devices, and prepare your devices with the SageMaker Edge Manager SDK. To prepare your model for deployment, SageMaker Edge Manager uses SageMaker Neo to compile your model for your target edge hardware. Once a model is compiled, SageMaker Edge Manager signs the model with an AWS generated key, then packages the model with its runtime and your necessary credentials to get it ready for deployment. On the device side, you register your device with SageMaker Edge Manager, download the SageMaker Edge Manager SDK and then follow the instructions to install the SageMaker Edge Manager agent on your devices. The tutorial notebook provides a step-by-step example of how you can prepare the models and connect your models on edge devices with SageMaker Edge Manager.
Q: What devices are supported by SageMaker Edge Manager?
Amazon SageMaker Edge Manager supports common CPU (ARM, x86) and GPU (ARM, Nvidia) based devices with Linux and Windows operating systems. Over time, SageMaker Edge Manager will expand to support more embedded processors and mobile platforms that are also supported by SageMaker Neo.
Q: Do I need to use Amazon SageMaker to train my model in order to use Amazon SageMaker Edge Manager?
No, you do not. You can train your models elsewhere or use a pre-trained model from open source or from your model vendor.
Q: Do I need to use Amazon SageMaker Neo to compile my model in order to use Amazon SageMaker Edge Manager?
Yes, you do. Amazon SageMaker Neo converts and compiles your models into an executable that you can then package and deploy on your edge devices. Once the model package is deployed, the Amazon SageMaker Edge Manager agent will unpack the model package and run the model on the device.
Q: How do I deploy models to the edge devices?
Amazon SageMaker Edge Manager stores the model package in your specified Amazon S3 bucket. You can use the over-the-air (OTA) deployment feature provided by AWS IoT Greengrass, or any other deployment mechanism of your choice to deploy the model package from your S3 bucket to the devices.
Q: How is Amazon SageMaker Edge Manager SDK different from the SageMaker Neo runtime (dlr)?
Neo dlr is an open source runtime that only execute models compiled by the Amazon SageMaker Neo service. Compared to the open source dlr, the SageMaker Edge Manager SDK includes an enterprise grade on-device agent with additional security, model management, and model serving features. The SageMaker Edge Manager SDK is suitable for production deployment at scale.
Q: How is Amazon SageMaker Edge Manager related to AWS IoT Greengrass?
Amazon SageMaker Edge Manager and AWS IoT Greengrass can work together in your IoT solution. Once your ML model is packaged with SageMaker Edge Manager, you can use AWS IoT Greengrass’s OTA update feature to deploy the model package to your device. AWS IoT Greengrass allows you to monitor your IoT devices remotely, while SageMaker Edge Manager helps you monitor and maintain the ML models on the devices.
Q: How is Amazon SageMaker Edge Manager related to AWS Panorama? When should you use Amazon SageMaker Edge Manager versus AWS Panorama?
AWS offers the most breadth and depth of capabilities for running models on edge devices. We have services to support a wide range of use cases including computer vision, voice recognition, and predictive maintenance.
For companies looking to run computer vision on edge devices such as cameras and appliances, you can use AWS Panorama. Panorama offers ready to deploy computer vision applications for edge devices. It’s easy to get started with AWS Panorama by logging into the cloud console, specifying the model you would like to use in Amazon S3 or in SageMaker, and then writing business logic as a python script. AWS Panorama compiles the model for the target device and creates an application package so it can be deployed to your devices with just a few clicks. In addition, ISVs who want to build their own custom applications can use the AWS Panorama SDK, and device manufacturers can use the Device SDK to certify their devices for AWS Panorama.
Customers who want to build their own models and have more granular control over model features, can use Amazon SageMaker Edge Manager. SageMaker Edge Manager is a managed service to prepare, run, monitor, and update machine learning (ML) models across fleets of edge devices such as smart cameras, smart speakers, and robots for any use case such as natural langue processing, fraud detection, and predictive maintenance. SageMaker Edge Manager is for ML edge developers who want control over their model including engineering different model features and monitor models for drift. Any ML edge developer can use SageMaker Edge Manager through the SageMaker console and the SageMaker APIs. SageMaker Edge Manager brings the capabilities of SageMaker to build, train, and deploy models in the cloud to edge devices.
Q: In which AWS regions is Amazon SageMaker Edge Manager available?
Amazon SageMaker Edge Manager is available in 6 AWS Regions US East (N. Virginia), US East (Ohio), US West (Oregon), EU (Ireland), EU (Frankfurt), and Asia Pacific (Tokyo) see details on the AWS Regions Table.
Q: What is Amazon SageMaker Neo?
Amazon SageMaker Neo enables machine learning models to train once and run anywhere in the cloud and at the edge. SageMaker Neo automatically optimizes models built with popular deep learning frameworks that can be used to deploy on multiple hardware platforms. Optimized models run up to 25 times faster and consume less than a tenth of the resources of typical machine learning models.
Q: How do I get started with Amazon SageMaker Neo?
To get started with Amazon SageMaker Neo, you log into the Amazon SageMaker console, choose a trained model, follow the example to compile models, and deploy the resulting model onto your target hardware platform.
Q: What are the major components of Amazon SageMaker Neo?
Amazon SageMaker Neo contains two major components – a compiler and a runtime. First, the Neo compiler reads models exported by different frameworks. It then converts the framework-specific functions and operations into a framework-agnostic intermediate representation. Next, it performs a series of optimizations. Then, the compiler generates binary code for the optimized operations and writes them to a shared object library. The compiler also saves the model definition and parameters into separate files. During execution, the Neo runtime loads the artifacts generated by the compiler -- model definition, parameters, and the shared object library to run the model.
Q: Do I need to use Amazon SageMaker to train my model in order to use Amazon SageMaker Neo to convert the model?
No. You can train models elsewhere and use Neo to optimize them for Amazon SageMaker ML instances or AWS IoT Greengrass supported devices.
Q: Which models does Amazon SageMaker Neo support?
Currently, Amazon SageMaker Neo supports the most popular deep learning models that power computer vision applications and the most popular decision tree models used in Amazon SageMaker today. Neo optimizes the performance of AlexNet, ResNet, VGG, Inception, MobileNet, SqueezeNet, and DenseNet models trained in MXNet and TensorFlow, and classification and random cut forest models trained in XGBoost.
Q: In which AWS regions is Amazon SageMaker Neo available?
To see a list of support regions, view the AWS region table.
Q. What Deployment options does Amazon SageMaker provide?
After you build and train models, Amazon SageMaker provides three options to deploy them so you can start making predictions. Real-time inference is suitable for workloads with millisecond latency requirements, payload sizes up to 6MB, and processing times of up to 60 seconds. Batch transform is ideal for offline predictions on large batches of data that are available upfront. Asynchronous inference is designed for workloads that do not have sub-second latency requirements, payload sizes up to 1GB, and processing times of up to 15 minutes.
Q: What is SageMaker Asynchronous Inference?
Amazon SageMaker asynchronous inference queues incoming requests and processes them asynchronously. This option is ideal for requests with large payload sizes and/or long processing times that need to be processed as they arrive. Optionally, you can configure autoscaling settings to scale down the instance count to zero when not actively processing requests to save on costs.
Q: How do I configure autoscaling settings to scale down the instance count to zero when not actively processing requests?
You can scale down the Amazon SageMaker asynchronous inference endpoint instance count to zero in order to save on costs when you are not actively processing requests. You need to define a scaling policy that scales on the ‘ApproximateBacklogPerInstance’ custom metric and set the ‘MinCapacity’ value to zero. For step by step instructions, please visit the autoscale an asynchronous endpoint section of the developer guide.
Amazon SageMaker Savings Plans
Q: What are Amazon SageMaker Savings Plans?
Amazon SageMaker Savings Plans offer a flexible usage-based pricing model for Amazon SageMaker in exchange for a commitment to a consistent amount of usage (measured in $/hour) for a one- or three-year term. Amazon SageMaker Savings Plans provide the most flexibility and help to reduce your costs by up to 64%. These plans automatically apply to eligible SageMaker ML instance usages including SageMaker Studio Notebooks, SageMaker On-Demand Notebooks, SageMaker Processing, SageMaker Data Wrangler, SageMaker Training, SageMaker Real-Time Inference, and SageMaker Batch Transform regardless of instance family, size, or region. For example, you can change usage from a CPU instance ml.c5.xlarge running in US East (Ohio) to a ml.Inf1 instance in US West (Oregon) for inference workloads at any time and automatically continue to pay the Savings Plans price.
Q: Why should I use Amazon SageMaker Savings Plans?
If you have a consistent amount of Amazon SageMaker instance usage (measured in $/hour), and use multiple SageMaker components or expect your technology configuration (e.g. instance family, region) to change over time, SageMaker Savings Plans make it simpler to maximize your savings while providing flexibility to change the underlying technology configuration based on application needs or new innovation. The Savings Plans rate apply automatically to all eligible ML instance usage with no manual modifications required.
Q: How can I get started with Amazon SageMaker Savings Plans?
You can get started with Savings Plans from AWS Cost Explorer in the management console or by using the API/CLI. You can easily make a commitment to Savings Plans by using the recommendations provided in AWS Cost Explorer, to realize the biggest savings. The recommended hourly commitment is based on your historical On Demand usage and your choice of plan type, term length, and payment option. Once you sign up for a Savings Plan, your compute usage will automatically be charged at the discounted Savings Plans prices and any usage beyond your commitment will be charged at regular On Demand rates.
Q: How are Savings Plans for Amazon SageMaker different from Compute Savings Plans for Amazon EC2?
The difference between Savings Plans for Amazon SageMaker and Savings Plans for EC2 is in the services they include. SageMaker Savings Plans is only applicable to SageMaker ML Instance usage.
Q: How do Savings Plans work with AWS Organizations/Consolidated Billing?
Savings Plans can be purchased in any account within an AWS Organization/Consolidated Billing family. By default, the benefit provided by Savings Plans is applicable to usage across all accounts within an AWS Organization/consolidated billing family. However, you can also choose to restrict the benefit of Savings Plans to only the account that purchased them.