DEV Community

Cover image for Chapter 56 — Secure AI Governance, Risk Management, Compliance, Policies, Controls, Model Inventory & Security Accountability
Black Shadow Team ©
Black Shadow Team ©

Posted on

Chapter 56 — Secure AI Governance, Risk Management, Compliance, Policies, Controls, Model Inventory & Security Accountability

#ai

56.1 Introduction

Technical security controls alone are not enough to operate a trustworthy AI platform.

An AI system may have:

  • authentication
  • authorization
  • encryption
  • monitoring
  • secure storage
  • sandboxing
  • model safety controls
  • incident response

Yet the organization can still have serious problems if nobody knows:

  • which AI models are deployed
  • who owns each model
  • what each model is allowed to do
  • what data each model can process
  • which risks have been accepted
  • which controls are mandatory
  • who approves high-risk changes
  • how security decisions are documented

This is the purpose of AI governance.

A useful principle is:

Governance converts security intentions into accountable, repeatable, and auditable decisions.


56.2 What Is AI Governance?

AI governance is the framework used to control how AI systems are:

  • designed
  • developed
  • evaluated
  • deployed
  • monitored
  • changed
  • retired

Governance should cover the entire AI lifecycle.

Idea
 ↓
Design
 ↓
Development
 ↓
Testing
 ↓
Approval
 ↓
Deployment
 ↓
Monitoring
 ↓
Change
 ↓
Retirement
Enter fullscreen mode Exit fullscreen mode

Security should exist throughout this lifecycle rather than being added only immediately before production.


56.3 Governance vs Security

Security and governance are closely related but different.

Security asks:

How do we protect the system?
Enter fullscreen mode Exit fullscreen mode

Governance asks:

What are we allowed to build?
Who is responsible?
What risks are acceptable?
Which controls are mandatory?
Who approves the decision?
How do we prove that controls exist?
Enter fullscreen mode Exit fullscreen mode

A mature organization needs both.


56.4 AI Governance Structure

A practical structure may contain:

AI Governance
   |
   +-- Policies
   |
   +-- Risk Management
   |
   +-- Model Inventory
   |
   +-- Data Governance
   |
   +-- Security
   |
   +-- Privacy
   |
   +-- Compliance
   |
   +-- Evaluation
   |
   +-- Monitoring
   |
   +-- Incident Management
   |
   +-- Accountability
Enter fullscreen mode Exit fullscreen mode

56.5 AI System Inventory

Every production AI system should have an inventory.

A model registry can track:

model_id
model_name
version
provider
owner
purpose
risk_level
deployment_status
data_classification
approved_use_cases
restricted_use_cases
evaluation_status
security_status
created_at
updated_at
retirement_date
Enter fullscreen mode Exit fullscreen mode

This creates organizational visibility.


56.6 Why Model Inventory Matters

Without an inventory, an organization may unknowingly have:

Model A → Production
Model B → Experimental
Model C → Forgotten API
Model D → Old version
Model E → Third-party provider
Enter fullscreen mode Exit fullscreen mode

The security team cannot properly protect systems it does not know exist.

Therefore:

Unknown AI systems represent governance risk.


56.7 Model Ownership

Each important AI system should have a clearly identified owner.

Possible roles:

BUSINESS_OWNER
TECHNICAL_OWNER
SECURITY_OWNER
PRIVACY_OWNER
MODEL_OWNER
Enter fullscreen mode Exit fullscreen mode

One person may hold multiple roles in a small organization.

The important requirement is that responsibility is explicit.


56.8 AI Risk Classification

Not every AI feature has the same risk.

A useful internal classification might be:

LOW
MEDIUM
HIGH
CRITICAL
Enter fullscreen mode Exit fullscreen mode

Example:

Low

Formatting assistant
Non-sensitive brainstorming
Enter fullscreen mode Exit fullscreen mode

Medium

Internal document summarization
Productivity assistant
Enter fullscreen mode Exit fullscreen mode

High

Sensitive organizational data processing
Automated decisions affecting important workflows
Agentic operations
Enter fullscreen mode Exit fullscreen mode

Critical

AI connected to highly privileged systems
High-impact autonomous actions
Systems where failure could cause significant harm
Enter fullscreen mode Exit fullscreen mode

The exact classification criteria should be defined by the organization's risk framework.


56.9 Risk Management Lifecycle

A practical AI risk process is:

IDENTIFY
   ↓
ANALYZE
   ↓
EVALUATE
   ↓
MITIGATE
   ↓
MONITOR
   ↓
REVIEW
Enter fullscreen mode Exit fullscreen mode

This process should repeat as the system changes.


56.10 Risk Identification

Possible AI risks include:

privacy leakage
prompt injection
model misuse
unsafe outputs
hallucination
data poisoning
model theft
credential compromise
agent misuse
supply-chain compromise
availability failure
vendor dependency
regulatory exposure
Enter fullscreen mode Exit fullscreen mode

The risk register should document these systematically.


56.11 Risk Register

A simple risk record might contain:

risk_id
description
asset
threat
likelihood
impact
risk_score
existing_controls
planned_controls
owner
status
review_date
Enter fullscreen mode Exit fullscreen mode

Example:

Risk:
Unauthorized retrieval of tenant documents.

Likelihood:
Medium

Impact:
High

Risk:
High

Controls:
Tenant isolation
Authorization filtering
Audit logging
Security testing
Enter fullscreen mode Exit fullscreen mode

56.12 Likelihood and Impact

Risk is commonly evaluated using:

Risk ≈ Likelihood × Impact
Enter fullscreen mode Exit fullscreen mode

For example:

Likelihood = High
Impact = High
Enter fullscreen mode Exit fullscreen mode

produces a high-priority risk.

However, numerical scoring is only a decision aid.

A low-frequency event can still require strong controls if the potential impact is severe.


56.13 Risk Treatment

A risk can be handled through several strategies.

MITIGATE
AVOID
TRANSFER
ACCEPT
Enter fullscreen mode Exit fullscreen mode

Mitigate

Add controls to reduce the risk.

Avoid

Do not deploy the risky capability.

Transfer

Move part of the risk through contracts, insurance, or service arrangements where appropriate.

Accept

Explicitly accept residual risk through an authorized decision.

Risk acceptance should never be accidental.


56.14 Security Policies

AI platforms should have documented policies.

Examples:

AI Security Policy
AI Data Handling Policy
AI Model Approval Policy
AI Vendor Policy
AI Agent Policy
AI Logging Policy
AI Incident Response Policy
AI Retention Policy
AI Access Control Policy
Enter fullscreen mode Exit fullscreen mode

Policies define expected behavior.


56.15 Standards and Procedures

A policy may say:

Sensitive AI workloads require security review.
Enter fullscreen mode Exit fullscreen mode

A procedure explains:

How the review is performed.
Who performs it.
Which evidence is required.
What approval means.
Enter fullscreen mode Exit fullscreen mode

Therefore:

Policy
   ↓
Standard
   ↓
Procedure
   ↓
Implementation
   ↓
Evidence
Enter fullscreen mode Exit fullscreen mode

56.16 Security Controls

Controls translate policy into technical or organizational mechanisms.

Examples:

MFA
RBAC
encryption
network segmentation
sandboxing
logging
rate limiting
backup
dependency scanning
model evaluation
human approval
Enter fullscreen mode Exit fullscreen mode

A control should have:

control_id
description
owner
implementation
test_method
evidence
review_frequency
status
Enter fullscreen mode Exit fullscreen mode

56.17 Control Mapping

Organizations often map controls to multiple frameworks.

Conceptually:

Internal Control
       |
   +---+---+
   |       |
Security   Privacy
Framework  Framework
   |       |
   +---+---+
       |
   Compliance
Enter fullscreen mode Exit fullscreen mode

This avoids implementing completely separate control systems for every requirement.


56.18 Evidence Management

A control is stronger when there is evidence that it actually operates.

Evidence can include:

configuration
audit log
security test
approval record
deployment record
access review
incident report
scan result
evaluation report
Enter fullscreen mode Exit fullscreen mode

For example:

Control:
MFA required for administrators.

Evidence:
Authentication configuration
+
Access-control test
+
Audit record
Enter fullscreen mode Exit fullscreen mode

56.19 Model Approval Process

A new model should not automatically become production-ready simply because it works technically.

A possible workflow is:

Model Proposed
      ↓
Use-Case Review
      ↓
Data Review
      ↓
Security Review
      ↓
Privacy Review
      ↓
Safety Evaluation
      ↓
Performance Evaluation
      ↓
Approval
      ↓
Production Deployment
Enter fullscreen mode Exit fullscreen mode

56.20 Model Cards and Documentation

Each important model should have useful documentation.

Possible fields:

purpose
capabilities
limitations
training information
evaluation results
known risks
supported inputs
restricted uses
security considerations
privacy considerations
version
provider
owner
Enter fullscreen mode Exit fullscreen mode

The documentation should reflect the actual system rather than being generic marketing material.


56.21 Third-Party AI Providers

AI applications frequently depend on external providers.

Governance should track:

provider
model
region
data handling
retention
security controls
contractual terms
availability
subprocessors
incident process
Enter fullscreen mode Exit fullscreen mode

Before using a third-party AI service, organizations should understand what data leaves their environment.


56.22 Vendor Risk

A provider dependency can create:

availability risk
privacy risk
security risk
cost risk
vendor lock-in
model behavior changes
service discontinuation
Enter fullscreen mode Exit fullscreen mode

A vendor assessment can reduce surprises.


56.23 Model Version Changes

AI providers may change models.

For example:

Model v1
   ↓
Provider update
   ↓
Model v2
Enter fullscreen mode Exit fullscreen mode

Even if the API remains compatible, behavior may change.

Therefore production systems should monitor:

quality
latency
cost
safety
security
output behavior
Enter fullscreen mode Exit fullscreen mode

after significant model changes.


56.24 Change Management

AI systems require controlled changes.

A change may include:

new model
new prompt
new system policy
new tool
new retrieval source
new memory mechanism
new provider
new security control
Enter fullscreen mode Exit fullscreen mode

A production change workflow might be:

Change Request
      ↓
Risk Assessment
      ↓
Testing
      ↓
Approval
      ↓
Deployment
      ↓
Monitoring
      ↓
Rollback if necessary
Enter fullscreen mode Exit fullscreen mode

56.25 Prompt Changes Are Security Changes

A common mistake is treating prompts as harmless configuration.

A system prompt can influence:

  • model behavior
  • tool usage
  • safety behavior
  • data handling
  • workflow decisions

Therefore significant prompt changes should be version-controlled and reviewed.

Example:

prompt_version = 42
Enter fullscreen mode Exit fullscreen mode

should be traceable to the deployed application version.


56.26 Agent Governance

AI agents require stronger governance because they can interact with external systems.

An agent inventory should record:

agent_id
purpose
owner
model
tools
permissions
data_access
approval_requirements
risk_level
deployment_status
Enter fullscreen mode Exit fullscreen mode

A particularly important field is:

allowed_tools
Enter fullscreen mode Exit fullscreen mode

The agent should not receive unrestricted access merely because its underlying model is capable of using many tools.


56.27 Human Approval Policies

Certain operations should require human approval.

Examples:

high-impact financial actions
destructive operations
large data exports
privileged configuration changes
external communications
organization-wide changes
Enter fullscreen mode Exit fullscreen mode

A governance policy can define:

action risk
+
user role
+
resource scope
=
approval requirement
Enter fullscreen mode Exit fullscreen mode

56.28 AI Safety Governance

Safety evaluations should be integrated into governance.

Possible evaluations include:

prompt injection testing
jailbreak testing
privacy leakage testing
bias testing
robustness testing
tool-use testing
multilingual testing
multimodal testing
adversarial testing
Enter fullscreen mode Exit fullscreen mode

The evaluation record should include:

test_version
model_version
test_cases
results
failures
mitigations
approval_status
Enter fullscreen mode Exit fullscreen mode

56.29 Security Exceptions

Sometimes a system cannot immediately meet a control requirement.

Instead of silently ignoring the control, create an exception.

Example:

exception_id
control_id
reason
risk
compensating_control
owner
approval
expiration_date
Enter fullscreen mode Exit fullscreen mode

Exceptions should expire.

An exception that never expires becomes an undocumented permanent weakness.


56.30 Separation of Duties

Important decisions should not depend on one uncontrolled individual.

For example:

Developer
   ↓
Builds system

Security Reviewer
   ↓
Reviews security

Business Owner
   ↓
Approves business use

Administrator
   ↓
Deploys
Enter fullscreen mode Exit fullscreen mode

The exact organizational structure can vary, but critical responsibilities should have appropriate separation.


56.31 Security Accountability

Every major AI capability should have answers to:

Who owns it?
Who can change it?
Who can approve it?
Who can access its data?
Who monitors it?
Who responds to incidents?
Who can retire it?
Enter fullscreen mode Exit fullscreen mode

If the answer is unclear, governance is incomplete.


56.32 AI Governance Database

A governance subsystem could include:

ai_systems
ai_models
ai_agents
ai_risks
ai_controls
ai_policies
ai_assessments
ai_approvals
ai_exceptions
ai_vendors
ai_incidents
ai_changes
Enter fullscreen mode Exit fullscreen mode

Relationships might look like:

AI System
   |
   +-- Models
   +-- Agents
   +-- Risks
   +-- Controls
   +-- Assessments
   +-- Vendors
   +-- Changes
Enter fullscreen mode Exit fullscreen mode

56.33 Example Model Registry Interface

interface AIModelRecord {
  id: string;
  name: string;
  version: string;

  provider: string;

  ownerId: string;

  purpose: string;

  riskLevel:
    | "LOW"
    | "MEDIUM"
    | "HIGH"
    | "CRITICAL";

  status:
    | "EXPERIMENTAL"
    | "APPROVED"
    | "PRODUCTION"
    | "DEPRECATED"
    | "RETIRED";

  approvedUseCases: string[];
  restrictedUseCases: string[];

  createdAt: Date;
  updatedAt: Date;
}
Enter fullscreen mode Exit fullscreen mode

56.34 Example Risk Record

interface AIRiskRecord {
  id: string;
  systemId: string;

  description: string;

  likelihood:
    | "LOW"
    | "MEDIUM"
    | "HIGH";

  impact:
    | "LOW"
    | "MEDIUM"
    | "HIGH"
    | "CRITICAL";

  status:
    | "OPEN"
    | "MITIGATED"
    | "ACCEPTED"
    | "CLOSED";

  ownerId: string;

  reviewDate: Date;
}
Enter fullscreen mode Exit fullscreen mode

56.35 Governance API Boundaries

Governance operations should be protected.

Example:

POST /governance/models
POST /governance/risks
POST /governance/assessments
POST /governance/approvals

GET /governance/models/:id
GET /governance/risks/:id
GET /governance/audit
Enter fullscreen mode Exit fullscreen mode

Only authorized administrative roles should access these endpoints.


56.36 Governance Audit Trail

Important governance actions should generate audit events.

Examples:

MODEL_REGISTERED
MODEL_APPROVED
MODEL_REJECTED
MODEL_DEPLOYED
MODEL_RETIRED

RISK_CREATED
RISK_ACCEPTED
RISK_MITIGATED

CONTROL_CREATED
CONTROL_TESTED

EXCEPTION_CREATED
EXCEPTION_APPROVED

POLICY_UPDATED
Enter fullscreen mode Exit fullscreen mode

These events establish accountability.


56.37 Compliance Management

Compliance should not be treated as a collection of checkboxes.

A mature compliance process connects:

Requirement
   ↓
Policy
   ↓
Control
   ↓
Implementation
   ↓
Evidence
   ↓
Testing
   ↓
Review
Enter fullscreen mode Exit fullscreen mode

This creates traceability.


56.38 Continuous Compliance

AI systems change frequently.

Therefore compliance should be continuously monitored.

A change in:

model
data
provider
prompt
agent
tool
architecture
region
Enter fullscreen mode Exit fullscreen mode

may require a new assessment.


56.39 Governance Metrics

Useful metrics include:

approved AI systems
unregistered AI systems
open high-risk findings
overdue risk reviews
expired exceptions
models without owners
models without recent evaluations
agents without tool reviews
controls failing tests
vendor assessments overdue
Enter fullscreen mode Exit fullscreen mode

These metrics provide leadership visibility.


56.40 AI Governance Dashboard

A governance dashboard could display:

AI SYSTEMS
  42 total
  35 production
   7 experimental

HIGH-RISK SYSTEMS
   5

OPEN HIGH-RISK FINDINGS
   3

EXPIRED EXCEPTIONS
   0

MODELS WITHOUT OWNER
   0

OVERDUE REVIEWS
   2
Enter fullscreen mode Exit fullscreen mode

The dashboard should help people make decisions rather than merely display numbers.


56.41 Governance Threat Model

Governance itself can be attacked.

Potential threats include:

unauthorized model approval
fake risk acceptance
audit-log manipulation
unapproved model deployment
hidden AI systems
expired security exceptions
privilege abuse
vendor misrepresentation
tampered evaluation results
Enter fullscreen mode Exit fullscreen mode

Therefore governance systems require the same security principles as other sensitive applications.


56.42 Protecting Approval Workflows

An approval should identify:

who approved
what was approved
which version
when
under which conditions
Enter fullscreen mode Exit fullscreen mode

Example:

Approval:
Model X
Version 7
Use case: internal summarization
Risk level: Medium
Approved by: authorized reviewer
Timestamp: recorded
Enter fullscreen mode Exit fullscreen mode

A future model version should not automatically inherit the approval of a materially different version.


56.43 Governance and DevSecOps

Governance should integrate with development pipelines.

Example:

Developer creates model configuration
          ↓
Security checks
          ↓
Policy checks
          ↓
Evaluation
          ↓
Approval status
          ↓
CI/CD deployment gate
Enter fullscreen mode Exit fullscreen mode

A production deployment can be blocked if required governance conditions are not satisfied.


56.44 Example Deployment Gate

Conceptually:

IF
model_registered
AND security_evaluation_passed
AND required_approval_exists
AND no_blocking_risk
AND deployment_policy_allows
THEN
deploy
ELSE
block
Enter fullscreen mode Exit fullscreen mode

This makes governance enforceable rather than purely documentary.


56.45 Governance as Code

Some policies can be represented programmatically.

Example:

function canDeploy(model: AIModelRecord): boolean {
  return (
    model.status === "APPROVED" &&
    model.riskLevel !== "CRITICAL"
  );
}
Enter fullscreen mode Exit fullscreen mode

Production systems should implement richer policy evaluation than this simplified example.

The important idea is that policy can become machine-enforceable.


56.46 Governance and Incident Response

An incident should update governance.

For example:

Incident
   ↓
Root Cause Analysis
   ↓
Risk Register Update
   ↓
Control Improvement
   ↓
Policy Update
   ↓
New Evaluation
Enter fullscreen mode Exit fullscreen mode

This creates a continuous improvement cycle.


56.47 AI Lifecycle Governance

The complete lifecycle can be represented as:

IDEATE
  ↓
CLASSIFY
  ↓
ASSESS
  ↓
DESIGN
  ↓
BUILD
  ↓
TEST
  ↓
APPROVE
  ↓
DEPLOY
  ↓
MONITOR
  ↓
REASSESS
  ↓
UPDATE
  ↓
RETIRE
Enter fullscreen mode Exit fullscreen mode

Security and governance should follow the system through every stage.


56.48 Production Governance Checklist

Inventory

  • [ ] All AI systems are registered.
  • [ ] Models have owners.
  • [ ] Agents have owners.
  • [ ] Third-party providers are documented.

Risk

  • [ ] Risk classification exists.
  • [ ] High-risk systems receive additional review.
  • [ ] Risk owners are assigned.
  • [ ] Risk reviews have defined schedules.

Security

  • [ ] Security evaluation is required.
  • [ ] Access controls are documented.
  • [ ] Logging is enabled.
  • [ ] Incident response exists.

Privacy

  • [ ] Data classification exists.
  • [ ] Sensitive data handling is documented.
  • [ ] Retention requirements are defined.
  • [ ] Deletion procedures exist.

Change Management

  • [ ] Models are versioned.
  • [ ] Prompts are versioned.
  • [ ] Agent permissions are reviewed.
  • [ ] Production changes are controlled.

Accountability

  • [ ] Approvers are identified.
  • [ ] Governance actions are audited.
  • [ ] Exceptions have expiration dates.
  • [ ] Evidence is retained appropriately.

Continuous Improvement

  • [ ] Incidents update risk assessments.
  • [ ] Security findings are tracked.
  • [ ] Controls are periodically tested.
  • [ ] Models are periodically reevaluated.

56.49 Reference Governance Architecture

A complete architecture can be represented as:

                         AI GOVERNANCE
                              |
       +----------------------+----------------------+
       |                      |                      |
    Inventory              Risk                  Policy
       |                      |                      |
    Models                 Assessments             Controls
    Agents                 Risk Register           Standards
    Providers              Exceptions              Procedures
       |                      |                      |
       +----------------------+----------------------+
                              |
                       Security Review
                              |
                       Privacy Review
                              |
                       AI Evaluation
                              |
                           Approval
                              |
                         Deployment
                              |
                         Monitoring
                              |
                      Incident Response
                              |
                       Lessons Learned
                              |
                         Reassessment
Enter fullscreen mode Exit fullscreen mode

56.50 Final Architecture Principle

A secure AI platform needs more than secure code.

It needs a system of accountability that makes important decisions:

  • visible
  • documented
  • authorized
  • reviewable
  • measurable
  • auditable
  • enforceable

The central governance chain is:

IDENTIFY
   ↓
CLASSIFY
   ↓
ASSESS
   ↓
CONTROL
   ↓
APPROVE
   ↓
DEPLOY
   ↓
MONITOR
   ↓
REVIEW
   ↓
RETIRE
Enter fullscreen mode Exit fullscreen mode

The most important principle is:

Every important AI capability should have an owner, a defined purpose, a known risk profile, appropriate controls, documented approval, and continuous oversight.

Governance should not exist separately from engineering.

The strongest architecture connects:

Governance
    +
Security
    +
Privacy
    +
Engineering
    +
Operations
    +
Incident Response
Enter fullscreen mode Exit fullscreen mode

into one continuous lifecycle.

That is what transforms an AI application from a collection of models and APIs into an accountable production system.

Top comments (0)