What Is AI Governance?

AI governance is the collection of rules, policies, and oversight mechanisms for AI development. AI applications use massive data sets for training and generate outputs in unpredictable ways. This introduces challenges around data privacy, security, copyright, and bias. Poorly regulated training data impacts AI output quality and creates risks for unauthorized data access. AI governance provides a structured approach to monitoring, updating, and evaluating AI systems in a way that mitigates AI security risks and prevents flawed decision-making. It ensures that AI development within an organization respects the human rights of all stakeholders.

Artificial intelligence (AI), particularly generative AI, is a complex software algorithm inspired by the design of the human brain. It is like a black box and creates challenges around analyzing, interpreting, and predicting the relationship between input and output. AI governance is essential to address these challenges. With AI governance in place, implementing AI systems that align with ethical considerations, societal values and legal requirements is more effortless. You can mitigate the following risks.

Input risks

Generative AI systems learn from massive datasets that might contain inaccurate, limited, or biased information. AI models are software algorithms that apply what they know to generate predictions. As such, some predictions might be flawed. For example, a credit scoring system trained with limited data might favor specific demographics when approving loan applications. That's why data governance is essential to ensure training datasets fairly represent the subjects as they are in the real world.

Process risks

AI systems are developed by human developers, who might be influenced by their preferences, values, and beliefs. These patterns can manifest in AI models and result in biased predictions. For example, AI developers might program the model to filter loan applications based on certain income thresholds they feel justified. To minimize risks stemming from AI algorithms, organizations can implement regular audits to ensure consistency and accuracy in AI predictions.

Output risks

Output risks are questionable practices resulting in misinformation and other misappropriation of AI systems. For example, bad actors might use generative AI systems to create fake news, malware, or other purposes conflicting with ethical principles. Even if used with good intentions, a lack of understanding can also produce results that undermine good faith.

Implementing AI systems safely at scale is only possible with consensus from governing bodies, the private sector, and civil society. These guiding principles help organizations adopt responsible AI development at scale.

Precautionary

Despite AI systems' promises to transform industries, organizations should mitigate the risks they pose with the best AI governance practices.

Agile

AI technologies are evolving rapidly, with significant breakthroughs challenging conventional governing frameworks. Industry stakeholders need frameworks that are responsive, adaptable, and scalable.

Inclusive

AI governance efforts should heavily involve the private sector, as the latter pioneered most AI innovations. Adopting a collaborative effort assures positive societal implications when implementing AI technologies.

Impermeable

AI governance should cover all possible use cases, models, development, and training processes. This prevents oversights that allow some AI systems to operate beyond the purview of established regulations.

Targeted

Rather than depending on a generic regulatory approach, AI governance is more effective when tailored to a specific industry or application or to mitigate certain risks.

AI governance aims to ensure AI technologies consistently make fair and unbiased decisions. It also strengthens efforts to improve AI safety and explainability. However, AI systems vary in complexity, applications, and purposes. To better understand, interpret, and govern AI applications, it helps to classify them into these categories.

Consumer AI usage

Consumer AI is software that the public can access. For example, ChatGPT is a consumer application powered by generative AI. Teams may use consumer AI to perform day-to-day tasks without informing anyone. Organizations have little control over how the AI collects, stores, or processes data their employees input to the AI. Often, employees must be made aware that the information they submit in the app is stored and used to train the AI.

Consumer AI requires strong governance policies and access restrictions. Educating employees and key stakeholders on the data exposure risks of using such public apps is crucial. For example, organizations can set up a corporate AI ethics board to review such AI applications' data and privacy policies. Then, they can create their own AI usage guidelines and policies for employees and users.

AI APIs

It is the intentional use of consumer AI systems in enterprise apps. In this case, the organization directly subscribes to a third-party AI service. Usually, they connect internal applications with the vendor's dedicated application programming interface (API). The organization exchanges data with the third-party AI service programmatically and frequently.

Regulating the type of data the external AI system has access to is important when using enterprise AI applications. If the AI provider uses your data to train their model, public users might be able to see similar data in their results. Organizations should also be mindful of data privacy and security risks if unauthorized parties access the API key.

Pre-trained model

Pre-trained models are AI models that you can deploy without further training. However, they often require fine-tuning to provide helpful, accurate, and safe responses. While doing so, you might need to provide additional data to the model. Therefore, it's important to consider the sensitivity of data used for fine-tuning and the risks of public exposure. Also, pre-trained models learn from data sets curated by external parties. There are also risks of inaccurate output resulting from biased training data sets.

Fine-tuned model

Fine-tuned models have undergone fine-tuning and further refinement. They are often optimized for specific use cases. If you use a fine-tuned model, consider the quality of the data it learns from. Like a pre-trained model, a fine-tuned model might be biased because of discrepancies and limitations in training or fine-tuning data.

Self-trained model

These are deep learning models that you train from scratch. The process involves data preparation, training, audit, and fine-tuning, which are prone to various risks impacting the model's prediction. In this case, AI governance includes the entire pipeline from development to training.

Artificial intelligence governance frameworks are principles and recommendations to align AI developments with ethical, responsible, and legal requirements. International, regional, and national authorities have devised frameworks for governance and responsible use of AI across industries. Trust in AI is crucial, and integrating standards is one way to help earn public trust and support a responsible use approach. We share some common frameworks below.

ISO 42001

ISO 42001 is an international management system standard that provides guidelines for managing AI systems within organizations. It establishes a framework for organizations to systematically address and control the risks related to the development and deployment of AI. ISO 42001 emphasizes a commitment to responsible AI practices, encouraging organizations to adopt controls specific to their AI systems. It supports global interoperability, and sets a foundation for the development and deployment of responsible AI.

AIGA

The AI Governance Alliance (AIGA) is a consortium of global industry leaders, academic institutions, governments, and civil society organizations. AIGA published an AI governance framework that helps organizations comply with the EU AI Act. The framework consists of guidelines supporting fair, safe, and ethical implementations of AI systems.

NIST

The National Institute of Standards and Technology (NIST) released a framework allowing AI developers to increase the trustworthiness of their AI systems. The framework helps mitigate risks organizations and consumers face amidst evolving AI technologies.

Applying these methods helps organizations turn responsible AI governance frameworks into practical implementation.

Human in the loop

Generative AI models learn unsupervised from unlabeled datasets. However, applying human-in-the-loop learning allows subject matter experts to guide the machine learning algorithm towards a specific outcome. For example, medical professionals provide feedback to fine-tune an AI medical system to adjust its behavior.

Explainable AI

Explainable AI aims to help stakeholders understand why and how AI models reach a specific decision. It applies machine learning techniques that help developers better understand complex AI models, particularly mission-critical applications. For example, doctors and patients want to know why medical AI software produces a particular diagnosis.

Data governance

Data governance comprises policies, frameworks, and technologies that secure data throughout the entire AI development lifecycle. When training and operating AI systems, massive amounts of data are moved between servers, which requires appropriate safeguards at all data points. Besides safeguarding privacy, data governance ensures that information curated for training meets appropriate quality requirements.

Developing AI solutions that comply with AI regulations and frameworks is challenging because of the complexity involved. We share best practices that help.

• Form an AI team comprising machine learning engineers, data scientists, ethicists, and legal experts.
• Prioritize education to raise awareness amongst users and the broader community on AI usage and its implications.
• Apply interpretable techniques to evaluate the AI system's performance. If necessary, apply mitigative measures to refine the model's response.
• Continuously monitor the AI system for abnormal behavior in real-life usage. Take prompt action to remediate bias and inaccuracies.
• Instead of using generative AI technologies for every workflow, choose AI systems tailored for specific use cases. It's easier to regulate and govern AI with a narrower scope.

AWS allows organizations to build and scale generative AI applications while conforming to AI governance best practices. AI developers can deploy foundational models on Amazon Bedrock and Amazon Sagemaker Jumpstart, knowing that we don't train our models with your data nor share them with third-party providers.

Amazon SageMaker is a fully managed service that combines a broad set of tools to enable high-performance, low-cost AI development for any use case. It provides purpose-built governance tools to help you implement AI responsibly. For example, you can use:

• Amazon SageMaker Role Manager to define minimum permissions in minutes.
• Amazon SageMaker Model Cards to capture, retrieve, and share essential model information.
• Amazon SageMaker Model Dashboard to remain informed on model behavior in production.

Curating, managing, and securing massive training data sets is challenging and tedious. With Amazon, you can use our deep set of over 300 security and database services to govern and manage your datasets throughout the AI lifecycle. For example, you can use:

• Amazon Macie to identify and protect sensitive data when training AI models.
• AWS Control Tower to authenticate and authorize user access to AI workloads on AWS.
• AWS Lake Formation to govern and secure data that AI models use from a single location.

Get started with AI governance on AWS by creating a free account today.