Logan Foster

Posted on Jul 18

How to Build an AI Governance Program From Scratch

Most AI governance programs don't fail because nobody understood the regulations.

They fail because the program was built as a document — a policy PDF, a slide deck, a framework mapping exercise — instead of an operating model with owners, cadence, and consequences.

If you've been handed the job of standing up AI governance from nothing, here's the sequence that produces something that still runs six months after the kickoff meeting.

Step 1: Inventory Before You Govern

You cannot govern what you haven't found.

Before selecting a framework, drafting a policy, or buying a governance tool, your first deliverable should be a complete AI system inventory. Not an aspirational list of "AI capabilities we might use" — an actual registry of AI systems currently in production or in active development.

For each system, capture:

What the system does and what decision or output it produces
Who owns it (a named individual, not a team)
What data it uses and where that data comes from
Who the affected parties are (employees? customers? the public?)
Whether it was built in-house, purchased from a vendor, or uses a third-party model API

This exercise usually surfaces more systems than anyone expected. Organizations routinely discover AI deployed in HR tools, customer service platforms, fraud detection layers, and marketing automation that nobody in legal or compliance knew was there.

The inventory is not a one-time exercise. It needs a process: an intake gate that requires AI system registration before any new AI project gets approved, budget, or infrastructure access.

Step 2: Classify Before You Control

Once you have the inventory, classify each system by risk — but use a classification methodology that produces legally defensible categories, not just a color-coded spreadsheet.

If you operate in the EU or work with EU residents, the EU AI Act's risk tier structure is your starting framework. The Act distinguishes between:

Prohibited practices — you need a documented representation that none of your systems fall here
High-risk systems — Annex III lists eight specific use-case areas (employment, credit, education, healthcare, biometrics, law enforcement, migration, justice)
Limited risk — specific transparency obligations apply (chatbot disclosure, deepfake labeling)
Minimal risk — no mandatory requirements, but internal governance still applies

If you operate outside the EU or exclusively domestically, use the NIST AI Risk Management Framework's risk profiles or ISO/IEC 42001's risk criteria as your classification baseline.

The output of classification is a tiered inventory: which systems need full conformity assessments, which need transparency measures, and which need only light-touch internal governance.

Document the rationale for every classification. "We decided this wasn't high-risk" is not a defensible position if a regulator asks. "We assessed this system against Annex III criteria and concluded it doesn't fall within the scope of employment AI because it doesn't filter candidates or make promotion decisions — here's the evidence" is.

Step 3: Assign Owners, Not Teams

The most common structural failure in AI governance programs is assigning ownership to teams or committees rather than named individuals.

Every AI system in your registry needs a named System Owner who is accountable for:

Keeping the system's governance documentation current
Triggering reassessment when the system changes materially
Ensuring human oversight mechanisms are operational
Escalating incidents or anomalies

The System Owner doesn't have to be a governance expert. They should be whoever is operationally closest to the system — typically the product manager or engineering lead responsible for it. Governance expertise lives in the AI governance function; accountability for a specific system lives with the person who knows it best.

This design avoids the commons problem: when "the AI committee" is responsible for everything, nobody is responsible for anything specific.

Step 4: Build Policies That Point to Processes

AI governance policies are not the end goal. They're signposts that point to processes.

A policy that says "the company will conduct AI impact assessments for high-risk systems" is meaningless without a defined impact assessment process, a template, a review cadence, and a named function responsible for conducting them.

For each policy statement, you should be able to answer:

Who does this?
How? (what's the process?)
When? (what triggers this activity?)
Where does the output live?
Who reviews it?

If you can't answer all five, the policy isn't operational yet.

The minimum viable policy set for most organizations in 2026:

AI Use Policy (what AI is permitted, what isn't, what requires approval)
AI Risk Assessment Procedure (how you classify and document system risk)
AI Incident Response Procedure (what happens when something goes wrong)
Third-Party AI Vendor Policy (what contractual and due diligence requirements apply)
Human Oversight Standards (which systems require human review before acting on AI output)

Step 5: Attach Governance to Existing Processes

The biggest implementation mistake is building AI governance as a parallel track that runs separately from how your organization actually makes decisions.

If your organization has a Change Advisory Board for technology changes — the AI governance review should happen before CAB, not after. If you have a vendor procurement process — AI vendor assessments should be part of that workflow, not a separate AI team process that runs concurrently.

Governance that requires people to do extra steps outside their normal workflow gets bypassed. Governance that lives inside the workflow gets followed.

Identify where in your existing processes the AI governance checkpoints should live:

Procurement: AI vendor assessment before contract signature
Development: Risk classification before project approval
Deployment: Conformity check before production launch
Ongoing: Periodic system review on a cadence tied to risk tier

Step 6: Measure Something

A governance program with no metrics is a compliance theater exercise.

At minimum, track:

Coverage rate: What percentage of AI systems in the inventory have completed risk classification?
Assessment completion: Of systems requiring a formal impact assessment, what percentage have one that's current?
Incident rate: How many AI-related incidents (errors, bias complaints, unexpected outputs) were reported in the last quarter?
Third-party compliance: What percentage of AI vendor contracts include required governance representations?

These don't need to be perfect. They need to tell you whether the program is actually running or just documented as running.

What to Avoid

Don't start with framework selection. Choosing between NIST AI RMF, ISO 42001, and EU AI Act compliance before you know what systems you have is backwards. The framework should match your inventory and your regulatory context, not precede it.

Don't build a committee as a governance substitute. An "AI Ethics Committee" that meets quarterly and reviews submissions is not a governance program. It's a review board. Governance is the operating infrastructure that ensures the right things happen before anything reaches a review board.

Don't treat the policy launch as the finish line. The launch is month one. The measure of the program is whether it's still running operationally at month six, month twelve, and beyond.

The Honest Timeline

For most mid-sized organizations (1,000–10,000 employees, 10–50 AI systems in active use), expect:

Month 1–2: AI inventory and initial classification
Month 2–3: Policy drafting, owner assignment, process design
Month 3–4: Integration into existing workflows
Month 4–6: First cycle of formal assessments for high-risk systems
Month 6+: Ongoing operation, metrics tracking, iteration

Building it faster is possible. Building it to last requires the time to get the owner assignment and workflow integration right.

Further reading:

DEV Community