Amazon SageMaker provides purpose-built governance tools to help you implement ML responsibly. With Amazon SageMaker Role Manager, administrators can define minimum permissions in minutes. Amazon SageMaker Model Cards makes it easier to capture, retrieve, and share essential model information, such as intended uses, risk ratings, and training details, from conception to deployment. Amazon SageMaker Model Dashboard keeps you informed on model behavior in production, all in one place.
Watch this video to learn how to improve visibility into your ML models with SageMaker.
How it works
ML governance with Amazon SageMaker uses SageMaker Role Manager, SageMaker Model Cards, and SageMaker Model Dashboard to help you simplify access control and enhance transparency over your ML projects.
Define minimum permissions in minutes with SageMaker Role Manager
Simplify permissions for ML activities
SageMaker Role Manager provides a baseline set of permissions for ML activities and personas through a catalog of prebuilt AWS Identity and Access Management (IAM) policies. ML activities can include data prep and training, and personas can include ML engineers and data scientists. You can keep the baseline permissions or customize them further based on your specific needs.
Automate IAM policy generation
With a few self-guided prompts, you can quickly input common governance constructs such as network access boundaries and encryption keys. SageMaker Role Manager will then generate the IAM policy automatically. You can discover the generated role and associated policies through the AWS IAM console.
Attach your managed policies
To further tailor the permissions to your use case, attach your managed IAM policies to the IAM role that you create with SageMaker Role Manager. You can also add tags to help identify and organize the roles across AWS services.
Streamline model documentation with SageMaker Model Cards
Capture model information
SageMaker Model Cards is a repository for model information in the Amazon SageMaker Console and helps you centralize and standardize model documentation so you can implement ML responsibly. You can autopopulate training details such as input datasets, training environments, and training results to accelerate the documentation process. You can also add details such as the model purpose and performance goals.
Visualize evaluation results
You can attach model evaluation results such as bias and quality metrics to your model card and add visualizations such as charts to gain key insights into model performance.
Share model cards
You can export your model cards to a PDF format to more easily share them with business stakeholders, internal teams, or your customers.
Get unified model monitoring with SageMaker Model Dashboard
Track model behavior
SageMaker Model Dashboard gives you a comprehensive overview of deployed models and endpoints so that you can track resources and model behavior violations in one place. You can monitor model behavior in four dimensions: data quality, model quality, bias drift, and feature attribution drift. SageMaker Model Dashboard monitors behavior through its integration with Amazon SageMaker Model Monitor and Amazon SageMaker Clarify.
SageMaker Model Dashboard provides an integrated experience to set up and receive alerts for missing and inactive model monitoring jobs and deviations in model behavior.
Troubleshoot model deviations
You can further inspect individual models and analyze factors impacting model performance over time. Then you can follow up with ML practitioners to take corrective measures.
“At United Airlines, we use machine learning (ML) to improve customer experience by providing personalized offers, enabling customers to be ready using Travel Readiness Center. Our use of ML also extends to airport operations, network planning, flight scheduling. As we were coming out of the pandemic, Amazon SageMaker played a critical role in Travel Readiness Center allowing us to handle large volumes of COVID test certificates, vaccine cards using document-based model automation. With Amazon SageMaker’s new governance capabilities, we have increased control and visibility over our machine learning models. SageMaker Role Manager simplifies the user setup process dramatically by providing baseline permissions and ML activities for each persona linked to IAM roles. With SageMaker Model Cards, our teams can proactively capture and share model information for review, and using SageMaker Model Dashboard, we are able to search and view models deployed on MARS—our internal ML platform. With all these new governance capabilities, we are saving significant amount of time and able to scale up.”
Ashok Srinivas, Director of ML Engineering and Ops, United Airlines
“At Capitec, we have a wide range of data scientists across our product lines, building different ML solutions. Our ML engineers manage a centralized modeling platform built on Amazon SageMaker to empower the development and deployment of all these ML solutions. Without any built-in tools, tracking modeling efforts tends towards disjointed documentation and a lack of model visibility. With SageMaker Model Cards, we can track plenty of model metadata in a unified environment, and SageMaker Model Dashboard affords us visibility into the performance of each model. In addition, SageMaker Role Manager simplifies the process of managing access for data scientists in our different product lines. Each of these contribute towards our model governance being sufficient to warrant the trust that our clients place in us as a financial service provider.”
Dean Matter, ML Engineer, Capitec Bank
Stay up to date with the latest SageMaker ML governance announcements.
Download code samples from the GitHub repository.
View technical documentation to learn how to use the SageMaker ML governance features.
Accelerate governance tools integration with ML workloads.
Learn more about the new ML governance tools for SageMaker.
"Improve ML governance w/deep control & transparency in SageMaker” session from AWS re:Invent 2022.
Deep dive into Amazon SageMaker ML governance tools.
Define customized permissions in minutes with Amazon SageMaker Role Manager.