Building AI Governance
Module 4 · Governance and Risk

Building AI Governance

Build an AI governance system that scales: an AI inventory, risk tiers with proportionate controls, a review board with clear decision rights, and alignment with the NIST AI RMF and ISO/IEC 42001.

Governance as a Path, Not a Gate

Bad AI governance is a single committee that reviews everything and approves little. Good AI governance is proportionate: light and fast for low-risk uses, rigorous for high-risk ones, and clear enough that teams know what’s expected before they start.

The goal is not to minimize AI. It’s to make sure the organization can use AI confidently because it knows what’s running, what could go wrong, and who’s accountable.


Two Frameworks Worth Knowing

NIST AI Risk Management Framework

The NIST AI RMF (version 1.0, released January 2023) is voluntary and widely used. It organizes AI risk management into four functions:

Govern culture, roles, policy Map context and risks Measure assess and track Manage prioritize and act
  • Govern: policies, accountability, culture, and processes across the organization. It underpins the other three.
  • Map: understand the context of each AI system: its purpose, users, affected people, and potential harms.
  • Measure: assess risks with appropriate methods, including testing for accuracy, bias, robustness, and security.
  • Manage: prioritize risks, apply controls, monitor, and respond.

NIST has also published a Generative AI Profile (NIST AI 600-1, July 2024) that applies the framework to risks specific to generative AI, such as confabulation, data privacy, and information integrity.

ISO/IEC 42001

ISO/IEC 42001:2023 is an international standard for an AI management system, structured like ISO 27001 for information security. Unlike the NIST framework, it’s certifiable: an external auditor can certify that your organization runs AI governance according to the standard. It’s increasingly requested by enterprise customers in procurement.

Many organizations use NIST AI RMF to design their risk practices and ISO/IEC 42001 to structure and certify the management system around them.


The Building Blocks

1. An AI inventory

You can’t govern what you can’t see. Maintain a register of AI systems, including vendor-embedded AI, with owner, purpose, data used, affected people, risk tier, and status. The inventory you started in your first 90 days becomes a permanent, maintained asset.

2. Risk tiers

Classify each use into a small number of tiers with proportionate controls:

TierExamplesTypical controls
LowDrafting internal documents, summarizing public informationApproved tools, acceptable-use policy, user training
MediumInternal decision support, customer-service drafts with human reviewOwner sign-off, evaluation before launch, monitoring, documented human review
HighDecisions about people (hiring, credit, eligibility), safety-relevant uses, fully automated customer-facing actionsFormal impact assessment, bias and robustness testing, review board approval, ongoing monitoring, incident plan
ProhibitedUses your policy or law forbidsNot permitted

Align tiers with applicable law. Uses that the EU AI Act treats as high-risk, for example, should land in your high tier.

Number of AI uses in each tierLowMost usesControls: Approved tools, acceptable-use policy, trainingMediumSomeControls: Owner sign-off, evaluation, monitoring, human reviewHighFewControls: Impact assessment, bias testing, board approval, incident planProhibitedRareControls: Not permittedMore scrutiny

3. A review body with clear decision rights

An AI review board (or risk council) with representatives from the CAIO’s team, legal, privacy, security, risk, HR, and the business. Define:

  • Which tiers it reviews (usually high only; medium can be delegated).
  • What it can decide: approve, approve with conditions, or reject.
  • Its service level, so a review takes days or weeks, not months.

4. Policy

A short, readable AI policy covering acceptable use, data handling, approved tools, human oversight expectations, transparency to customers and employees, and how to report concerns. Supplement it with standards (detailed requirements) for teams that build AI.

⚠️ Common Pitfall: One Committee Reviews Everything

Common mistake: sending every AI use, including low-risk productivity tools, to a monthly committee.

The committee is swamped, reviews become superficial, and teams route around governance. Tier first, then send only high-risk uses to the board. Make the low-risk path easy and documented.


Agents Raise the Stakes

AI agents that take actions (sending emails, updating records, making purchases, running code) need governance beyond what’s needed for AI that only drafts text. Add controls for:

  • Scope of permissions: least privilege, as with any system account.
  • Human approval points for consequential or irreversible actions.
  • Logging of actions, not just outputs.
  • Kill switches and rate limits.
🧪 Try It: Draft Your Risk Tiers
  1. List ten AI uses from your inventory, or invent plausible ones.
  2. Assign each to low, medium, high, or prohibited.
  3. For the high tier, list the controls you’d require before launch.
  4. Decide which tiers your review board will see, and its target turnaround time.
🧠 Knowledge Check

1. What are the four functions of the NIST AI RMF?

2. What distinguishes ISO/IEC 42001 from the NIST AI RMF?

3. Why should low-risk AI uses not go to the AI review board?

Up Next

The next lesson covers the regulatory landscape your governance has to satisfy.