Scale AI governance with Amazon SageMaker and IBM watsonx.governance

As organizations look to scale AI across their operations addressing issues of trust, transparency, and compliance is essential. While AI models offer significant opportunities, managing them effectively requires addressing challenges such as bias, fairness, security, and model efficiency. Additionally, key regulations like the EU AI Act, U.S. AI Bill of Rights, Singapore AI Governance Framework, UK AI Regulation Policy, Canada’s Digital Charter Implementation Act, NYC HR Law, and the upcoming Brazilian AI Act all emphasize the need for responsible AI deployment. These regulations are joined by industry-specific standards in sectors such as healthcare, finance, and government, underscoring the complexity of managing AI at scale.

In this post, we will explore how to set up AI use case and model governance using the integration of IBM watsonx.governance with Amazon SageMaker. This integration helps provide a streamlined approach to applying comprehensive compliance and governance controls throughout the AI lifecycle. Together these products can support risk assessment, mitigation strategies, transparency, human oversight, and model management, ensuring responsible AI deployment while maintaining accountability and trust. We will also cover the specific workflows, lifecycle stages, and stakeholder roles involved, helping you implement responsible, transparent, and compliant AI practices within your organization.

For additional information on the safe and responsible development of AI, refer to the AWS Responsible AI documentation.

Automating AI Governance Across Model Lifecycle

Amazon SageMaker is a fully managed machine learning (ML) service that helps you build, train, and deploy ML models. IBM watsonx. governance integrates with your existing systems to automate responsible AI workflows, saving time and reducing costs while ensuring compliance.

On May 21st 2024, AWS and IBM announced an integration solution for AI/ML governance with watsonx.governance and Amazon SageMaker. This integration provides customers with a simplified path to automate risk management and regulatory compliance for their AI/ML models and use cases. Today, we are excited to announce a deeper integration between IBM watsonx.governance and Amazon SageMaker Model Registry to help you apply compliance and governance to your AI use cases, while onboarding models to those use cases with the appropriate governance workflows.

Model governance has the goal of ensuring that ML models meet business goals, maintain accuracy and fairness, and follow applicable regulations. It includes monitoring performance, managing risks, ensuring data quality, and maintaining accountability throughout the model’s lifecycle. When addressing business problems through machine learning, this governance becomes essential for developing and using AI systems that align with organizational values and regulatory requirements.

AI systems often combine multiple ML models, both existing and new. This requires governance at two levels. Use case governance manages overall AI system decisions, while model governance tracks individual model performance and compliance. These levels work together to create comprehensive governance across the AI lifecycle. For more information about model governance, refer to this post.

IBM watsonx.governance simplifies compliance with AI regulations through customizable workflows and automated risk management. Businesses can design and adapt workflows, including approval processes and notifications, to align with laws and regulations.

IBM watsonx.governance dashboards and reporting tools provide visibility into compliance status and risks, enabling organizations to monitor and adjust governance controls as regulations evolve. By tracking AI models across their lifecycle – from development to deployment and monitoring – watsonx.governance ensures businesses remain aligned with regulatory standards while maintaining transparency and trust in their Generative AI and AI/ML systems.

Solution Overview

With IBM watsonx.governance, users can start their journey by creating a use case and providing key business details, such as the risk level, supporting documents, assigned owners, tags, and other relevant information. They can then use IBM watsonx.governance features such as risk assessment, regulation, and compliance management capabilities to approve their use case.

These features include questionnaires that are aligned with key compliance acts that apply to the specific application or situation where AI is used to achieve a goal or solve a problem. Upon approval of the AI use case in watsonx.governance, data scientists and ML engineers can perform model building, registration, validation and deployment workflows.

As part of this process, data scientists and ML engineers can seamlessly onboard their models’ technical and business metadata onto watsonx.governance for comprehensive compliance, risk management, and model lifecycle management. With this integrated offering, customers gain access to highly scalable governance, risk, and compliance capabilities built to monitor and manage risk and compliance at scale:

1. Use case risk governance: Map policies, metrics, and models using a centralized location to organize, document, and maintain an enterprise-wide view of your model inventory.
2. Operational risk management: Integrate risk and control assessments, internal and external loss events, key risk indicators, and issue/action plans within a single environment.
3. Regulatory change management: Combine software, process automation, data feeds, and expertise for a more complete, accurate, and timely view of your compliance risks.
4. Model lifecycle management: Manage, govern and transition of models through their lifecycle stages, incorporating model metrics, technical and business metadata, and the deployment of approved models to production environments.

The following diagram illustrates an integration architecture between IBM watsonx.governance and Amazon SageMaker, with steps explained in the following text:

Figure 1. IBM watsonx.governance integrates with Amazon SageMaker to provide governance across the AI/ML model lifecycle.

The architecture shows how stakeholders work together across watsonx.governance and Amazon SageMaker during the life-cycle of AI/ML use-case and models:

1. A business stakeholder defines an AI/ML use-case to solve a business problem and provides key details like name, description, risk, and so on.
2. The stakeholder starts an approval workflow in watsonx.governance based on compliance requirements. Once approved:
   • A new SageMaker model package group is automatically created.
   • The data science team receives automatic email notification with details of the new use-case and SageMaker Model Package Group to start developing the model.
3. A data scientist creates an ML project for the new use-case.
4. The new SageMaker model package group is used within SageMaker Pipelines to build your model and record qualitative information about it.
5. Prepare the data to build your model training pipeline. Refer to the Data Protection & Privacy at AWS documentation to manage data privacy controls, protect your data, control access, and ensure proper encryption.
6. Evaluate the training data for data quality, and update the model package version with relevant metrics.
7. Train your model and register the candidate model package version with training metrics. A new model object is automatically created in watsonx.governance and associated with the AI/ML use-case.
8. Evaluate your trained model for model bias and model drift, and update the model package version with relevant evaluation metrics.
9. The SageMaker model package version metadata is automatically synchronized bi-directionally with watsonx.governance.
10. The Governance Officer checks that your model candidate meets the governance requirements based on the use-case risk profile and compliance needs.
11. Once your candidate model is approved in watsonx.governance, its approval status will be synchronized in the SageMaker model package version.
12. Using the status update to invoke an automated test deployment pipeline to deploy your model to the Pre-Prod environment.
13. Run model validation tests in your test environment to make sure the model complies with your use-case requirements.
14. Approve the model for deployment into production after validating the model.
15. Deploy the model to your production environment.
16. The stakeholder monitors the model’s performance and compliance through quality and bias metrics using watsonx.governance dashboards.

How it works

Prerequistes

Prior to setting up the integration between watsonx.governance and Amazon SageMaker, the following services must be configured:

• AWS CloudTrail: establish where Amazon SageMaker events are published to CloudTrail.
• Amazon EventBridge: create a rule to route SageMaker model group and model package updates to a FIFO queue.
• Amazon Simple Queue Service (SQS): create a FIFO queue is defined to be the target for all model group and model package updates from the EventBridge rule.
• AWS Resource Access Manager (RAM): can be used to create a central model registry for many underlying accounts when multiple account governance is required.

Multiple Account Governance

A single instance of watsonx.governance can be used to govern up to 10 central model registries to provide the enterprise with centralized AI governance. For detailed instructions on setting up the integration between watsonx.governance and Amazon SageMaker, refer to the IBM documentation. Be sure to review the Security best practices in IAM documenatation when configuring AWS Identity and Access Management (IAM) for watsonx.governance integration.

There are a few reasons to use multi-account central model registry, such as:

1. To use separate accounts for model development, testing, and production deployment.
2. Some jurisdictions do not allow data used for model development to leave their borders. In this case, it is necessary to separate information about the models from the models themselves. The information about the models can be collected and updated in the central registries, while all the development activities for the model remain in accounts within those jurisdictions.

Figure 2 shows how watsonx.governance connects to a SageMaker central model registry account to collect information about the models. This follows the hub and spoke reference architecture described here.

Figure 2. Connecting watsonx.governance with SageMaker central model registry account.

Each central model registry is tied to a business entity in watsonx.governance. Any use case, and associated model groups and models, that are created under the business entity configured for the central model registry are automatically synchronized between AWS SageMaker and watsonx.governance for AI governance.

Governance Dashboard

After configuring the integration of watsonx.governance with SageMaker, the business stakeholders and compliance officer will use watsonx.governance to review and approve model use cases and models. Upon logging in with their watsonx.governance credentials, users will be presented with a dashboard as shown in the following screenshot (Figure 3).

Figure 3. IBM watsonx.governance dashboard shows key compliance metrics and risk insights across the AI lifecycle.

This dashboard offers a high-level overview of the status of various use cases and models. Business stakeholders and compliance officers can quickly view models by provider, their lifecycle stages, compliance statuses, and any urgent issues requiring immediate attention. The dashboard is fully configurable to meet the specific needs of different personas within the organization.

Model Use Case Creation

When a business owner identifies a potential need for an AI model, the first step is to create an AI use case in watsonx.governance. The business owner provides as much information as possible to articulate the need as well as potential constraints on the model’s use and the data available for training (Figure 4). To ensure integration with SageMaker for AI governance, one of the use case’s parent entities must be the business entity configured in the central registry.

Figure 4. Defining the business use case for an AI model in watsonx.governance.

Stakeholders can then assess the relevancy of the AI use case against mandates, such as laws, regulations, and standards, that their organization must comply with as seen in figure 5.

Figure 5. Add applicable mandates to the AI Use Case.

Model Use Case Approval Process

After the use case is created, it undergoes an approval process, which may include conducting additional assessments for risk using the AI Risk Identification questionnaires. Risks may be identified and retrieved from the IBM AI risk atlas into watsonx.governance. By default, watsonx.governance provides the use case approval process as shown in the following diagram.

Figure 6. Use Case Approval Process in watsonx.governance.

Model Group Creation

Once the use case is approved, a model group is automatically created in watsonx.governance and linked to the use case (Figure 7).

Figure 7. Model Group created under use case on approval.

The model group is also automatically created in SageMaker (Figure 8), and the data scientist is notified to begin development within the assigned model group.

Figure 8. Automatically created model group in SageMaker upon Use Case Approval.

Model Creation and Synchronization

The data scientist can now proceed with model development and create a model package version within the automatically generated model group (Figure 9).

The model version is then automatically synchronized with watsonx.governance. Governance users can access more details about the model in SageMaker via a presigned URL provided in the Model view (Figure 10).

Figure 10. Model automatically synchronized with watsonx.governance.

The model is also automatically linked to the associated model group and use case in watsonx.governance, as seen in the following screenshot.

Figure 11. Automatically linked SageMaker model to model group and use case in watsonx.governance.

Information from the use case, model group, and model in watsonx.governance is also synchronized back to the associated Model Card in the SageMaker model package as custom metadata. The links in the custom metadata section allow users to navigate back to watsonx.governance when needed (Figure 12).

Figure 12. IBM watsonx.governance information in SageMaker Model Package.

SageMaker Model Cards document important details about ML models, such as performance, intended use, risks, and business context. In the integration with IBM watsonx.governance, Model Cards help track and manage governance information for each model version. This makes it easier to ensure transparency and compliance throughout the model lifecycle, supporting responsible AI development and decision-making.

Model Lifecycle

As soon as the model is synchronized to watsonx.governance, it can go through the model governance lifecycle process. The following diagram (Figure 13) is provided by default in watsonx.governance, which can be customized as needed to fit the business’ AI governance process.

Figure 13. Model Lifecycle Process in watsonx.governance.

As the workflow progresses, the model status in watsonx.governance is updated. This status is linked to the SageMaker model package status as well as the model package’s lifecycle object. The stage and status of the model package lifecycle object can then be used to drive the various ML pipelines. All but three of the statuses are driven solely by the model lifecycle process in watsonx.governance. The lifecycle stages and statuses marked in red can also be updated directly from within SageMaker, which would then be reflected back in the governance console. More details about staging in the model lifecycle can be found in the SageMaker documentation on Staging Construct for your Model Lifecycle.

Once a model is approved for deployment through the watsonx.governance console, its approval status is reflected in the corresponding model package in SageMaker (Figure 14).

Figure 14. Governance approval reflected in the SageMaker model package.

Additionally, when a model is deleted or decommissioned in SageMaker, it will also be decommissioned in watsonx.governance.

Model Monitoring

IBM offers model management capabilities which includes documentation, evaluation, and monitoring. Those capabilities are planned to be available in watsonx.governance SaaS on AWS in 2025. Customers using Amazon SageMaker Model Monitor or third-party monitoring solutions can leverage watsonx.governance APIs to push monitoring data into the governance dashboard. This can be done by associating metric values with specific metrics and models.

The architecture diagram below illustrates how you can integrate a third-party monitoring solution to push metrics into the watsonx.governance dashboard.

Figure 15. Sending third-party monitoring metrics to watsonx.governance.

Summary

In this post, we explored the integration between Amazon SageMaker and IBM watsonx.governance to govern Generative AI and AI/ML use cases and models throughout their entire lifecycle within an organization. We walked through an end-to-end architecture for developing an AI use case with embedded governance controls, covering use case creation, model building, model registration, and model deployment.

The integration of IBM watsonx.governance with Amazon SageMaker is designed to support customers on their journey toward responsible AI. By combining AWS AI/ML services with IBM’s expertise in AI governance, this collaboration provides organizations with the tools needed to meet risk management and compliance requirements, while proactively mitigating risks tied to evolving AI regulations and industry standards.

Call to action

Reach out to your AWS or IBM representative today to learn how Amazon SageMaker and IBM watsonx.governance can help unlock the full potential of your AI initiatives while maintaining the highest standards of risk management and compliance.

Additional Content:

• Optimize AI governance with Amazon SageMaker and IBM watsonx.governance
• watsonx.governance: Monitor AI models with Amazon SageMaker
• What is watsonx.governance?
• AI Governance with watsonx.governance and Amazon SageMaker
• IBM on AWS Partner Page
• Centralize model governance with SageMaker Model Registry Resource Access Manager sharing
• Improve governance of models with Amazon SageMaker unified Model Cards and Model Registry

Visit the AWS Marketplace for IBM watsonx solutions on AWS:

• IBM watsonx.governance as a Service on AWS
• IBM watsonx.data as a Service on AWS
• IBM watsonx.ai
• IBM watsonx Assistant
• IBM watsonx Orchestrate