How it works

ML governance with Amazon SageMaker uses SageMaker Role Manager, SageMaker Model Cards, and SageMaker Model Dashboard to help you simplify access control and enhance transparency over your ML projects.

Diagram shows how to use ML governance with Amazon SageMaker to define minimum permissions in minutes, centralize and standardize model documentation, and audit model performance through a unified view.

Key features

Define minimum permissions in minutes with SageMaker Role Manager

Simplify permissions for ML activities

SageMaker Role Manager provides a baseline set of permissions for ML activities and personas through a catalog of prebuilt AWS Identity and Access Management (IAM) policies. ML activities can include data prep and training, and personas can include ML engineers and data scientists. You can keep the baseline permissions or customize them further based on your specific needs.

Automate IAM policy generation

With a few self-guided prompts, you can quickly input common governance constructs such as network access boundaries and encryption keys. SageMaker Role Manager will then generate the IAM policy automatically. You can discover the generated role and associated policies through the AWS IAM console.

Attach your managed policies

To further tailor the permissions to your use case, attach your managed IAM policies to the IAM role that you create with SageMaker Role Manager. You can also add tags to help identify and organize the roles across AWS services.

Streamline model documentation with SageMaker Model Cards

Capture model information

SageMaker Model Cards is a repository for model information in the Amazon SageMaker Console and helps you centralize and standardize model documentation so you can implement ML responsibly. You can autopopulate training details such as input datasets, training environments, and training results to accelerate the documentation process. You can also add details such as the model purpose and performance goals.

Visualize evaluation results

You can attach model evaluation results such as bias and quality metrics to your model card and add visualizations such as charts to gain key insights into model performance.

Share model cards

You can export your model cards to a PDF format to more easily share them with business stakeholders, internal teams, or your customers.

Get unified model monitoring with SageMaker Model Dashboard

Track model behavior

SageMaker Model Dashboard gives you a comprehensive overview of deployed models and endpoints so that you can track resources and model behavior violations in one place. You can monitor model behavior in four dimensions: data quality, model quality, bias drift, and feature attribution drift. SageMaker Model Dashboard monitors behavior through its integration with Amazon SageMaker Model Monitor and Amazon SageMaker Clarify.

Get unified model monitoring with SageMaker Model Dashboard

The risk rating shown above is for illustrative purposes only and may vary based on input provided by you.

Automate alerts

SageMaker Model Dashboard provides an integrated experience to set up and receive alerts for missing and inactive model monitoring jobs and deviations in model behavior.

Automate alerts

The risk rating shown above is for illustrative purposes only and may vary based on input provided by you.

Troubleshoot model deviations

You can further inspect individual models and analyze factors impacting model performance over time. Then you can follow up with ML practitioners to take corrective measures.

Customers

united airlines logo

“At United Airlines, we use machine learning (ML) to improve customer experience by providing personalized offers, enabling customers to be ready using Travel Readiness Center. Our use of ML also extends to airport operations, network planning, flight scheduling. As we were coming out of the pandemic, Amazon SageMaker played a critical role in Travel Readiness Center allowing us to handle large volumes of COVID test certificates, vaccine cards using document-based model automation. With Amazon SageMaker’s new governance capabilities, we have increased control and visibility over our machine learning models. SageMaker Role Manager simplifies the user setup process dramatically by providing baseline permissions and ML activities for each persona linked to IAM roles. With SageMaker Model Cards, our teams can proactively capture and share model information for review, and using SageMaker Model Dashboard, we are able to search and view models deployed on MARS—our internal ML platform. With all these new governance capabilities, we are saving significant amount of time and able to scale up.”

Ashok Srinivas, Director of ML Engineering and Ops, United Airlines

Capitec

“At Capitec, we have a wide range of data scientists across our product lines, building different ML solutions. Our ML engineers manage a centralized modeling platform built on Amazon SageMaker to empower the development and deployment of all these ML solutions. Without any built-in tools, tracking modeling efforts tends towards disjointed documentation and a lack of model visibility. With SageMaker Model Cards, we can track plenty of model metadata in a unified environment, and SageMaker Model Dashboard affords us visibility into the performance of each model. In addition, SageMaker Role Manager simplifies the process of managing access for data scientists in our different product lines. Each of these contribute towards our model governance being sufficient to warrant the trust that our clients place in us as a financial service provider.”

Dean Matter, ML Engineer, Capitec Bank

Resources

What's new

Stay up to date with the latest SageMaker ML governance announcements.

Read more »
Code samples

Download code samples from the GitHub repository.

Experiment now »
Guide

View technical documentation to learn how to use the SageMaker ML governance features.

Read the guide »
GUIDE

Accelerate governance tools integration with ML workloads.

Learn more »
Blog Post

Learn more about the new ML governance tools for SageMaker.

Read the blog »
Video

"Improve ML governance w/deep control & transparency in SageMaker” session from AWS re:Invent 2022.

Watch now »
Blog Post

Deep dive into Amazon SageMaker ML governance tools.

Read the blog »
Blog Post

Define customized permissions in minutes with Amazon SageMaker Role Manager.

Read the blog »