Building AI Governance
Build an AI governance system that scales: an AI inventory, risk tiers with proportionate controls, a review board with clear decision rights, and alignment with the NIST AI RMF and ISO/IEC 42001.
Governance as a Path, Not a Gate
Bad AI governance is a single committee that reviews everything and approves little. Good AI governance is proportionate: light and fast for low-risk uses, rigorous for high-risk ones, and clear enough that teams know what’s expected before they start.
The goal is not to minimize AI. It’s to make sure the organization can use AI confidently because it knows what’s running, what could go wrong, and who’s accountable.
Two Frameworks Worth Knowing
NIST AI Risk Management Framework
The NIST AI RMF (version 1.0, released January 2023) is voluntary and widely used. It organizes AI risk management into four functions:
- Govern: policies, accountability, culture, and processes across the organization. It underpins the other three.
- Map: understand the context of each AI system: its purpose, users, affected people, and potential harms.
- Measure: assess risks with appropriate methods, including testing for accuracy, bias, robustness, and security.
- Manage: prioritize risks, apply controls, monitor, and respond.
NIST has also published a Generative AI Profile (NIST AI 600-1, July 2024) that applies the framework to risks specific to generative AI, such as confabulation, data privacy, and information integrity.
ISO/IEC 42001
ISO/IEC 42001:2023 is an international standard for an AI management system, structured like ISO 27001 for information security. Unlike the NIST framework, it’s certifiable: an external auditor can certify that your organization runs AI governance according to the standard. It’s increasingly requested by enterprise customers in procurement.
Many organizations use NIST AI RMF to design their risk practices and ISO/IEC 42001 to structure and certify the management system around them.
The Building Blocks
1. An AI inventory
You can’t govern what you can’t see. Maintain a register of AI systems, including vendor-embedded AI, with owner, purpose, data used, affected people, risk tier, and status. The inventory you started in your first 90 days becomes a permanent, maintained asset.
2. Risk tiers
Classify each use into a small number of tiers with proportionate controls:
| Tier | Examples | Typical controls |
|---|---|---|
| Low | Drafting internal documents, summarizing public information | Approved tools, acceptable-use policy, user training |
| Medium | Internal decision support, customer-service drafts with human review | Owner sign-off, evaluation before launch, monitoring, documented human review |
| High | Decisions about people (hiring, credit, eligibility), safety-relevant uses, fully automated customer-facing actions | Formal impact assessment, bias and robustness testing, review board approval, ongoing monitoring, incident plan |
| Prohibited | Uses your policy or law forbids | Not permitted |
Align tiers with applicable law. Uses that the EU AI Act treats as high-risk, for example, should land in your high tier.
3. A review body with clear decision rights
An AI review board (or risk council) with representatives from the CAIO’s team, legal, privacy, security, risk, HR, and the business. Define:
- Which tiers it reviews (usually high only; medium can be delegated).
- What it can decide: approve, approve with conditions, or reject.
- Its service level, so a review takes days or weeks, not months.
4. Policy
A short, readable AI policy covering acceptable use, data handling, approved tools, human oversight expectations, transparency to customers and employees, and how to report concerns. Supplement it with standards (detailed requirements) for teams that build AI.
Common mistake: sending every AI use, including low-risk productivity tools, to a monthly committee.
The committee is swamped, reviews become superficial, and teams route around governance. Tier first, then send only high-risk uses to the board. Make the low-risk path easy and documented.
Agents Raise the Stakes
AI agents that take actions (sending emails, updating records, making purchases, running code) need governance beyond what’s needed for AI that only drafts text. Add controls for:
- Scope of permissions: least privilege, as with any system account.
- Human approval points for consequential or irreversible actions.
- Logging of actions, not just outputs.
- Kill switches and rate limits.
- List ten AI uses from your inventory, or invent plausible ones.
- Assign each to low, medium, high, or prohibited.
- For the high tier, list the controls you’d require before launch.
- Decide which tiers your review board will see, and its target turnaround time.
1. What are the four functions of the NIST AI RMF?
Govern underpins Map, Measure, and Manage.
2. What distinguishes ISO/IEC 42001 from the NIST AI RMF?
ISO/IEC 42001 defines a certifiable AI management system; the NIST AI RMF is voluntary guidance.
3. Why should low-risk AI uses not go to the AI review board?
Proportionate governance keeps the board focused on high-risk uses and makes low-risk adoption easy.
Up Next
The next lesson covers the regulatory landscape your governance has to satisfy.