📰 Originally published on Securityelites — AI Red Team Education — the canonical, fully-updated version of this article.
🤖 AI/LLM HACKING COURSE
FREE
Part of the AI/LLM Hacking Course — 90 Days
Day 39 of 90 · 43.3% complete
⚠️ Authorised Scope: When I run an AI governance assessment, I’m not just testing the technology. I may need to review internal policies, risk registers, model documentation, and other governance records to understand how the organisation actually manages AI risk. Before I start, I always make sure the engagement scope explicitly gives me permission to access and review these documents. A technical testing scope does not automatically mean I’m authorised to examine internal governance documentation.
Let me tell you about an AI governance assessment I worked on. The organisation had done a lot of things right on paper. There was a detailed risk register, a data governance policy, a human oversight procedure, and a properly maintained incident log. If I had stopped at the documentation, I would have marked the governance controls as looking pretty solid.
But I wanted to see what happened in practice.
So I asked the human oversight team a very simple question: “Show me the last time a reviewer actually overrode an AI decision.”
Silence.
Not the quick kind of silence where someone is searching their memory. This was the uncomfortable kind. Eventually, someone said, “We haven’t really needed to. The AI is very accurate.”
That answer changed the entire assessment.
The procedure said reviewers were supposed to examine AI decisions and intervene when something looked wrong. In reality, reviewers had become an approval step. They were signing off on AI outputs without properly challenging them because everyone had become comfortable with the assumption that the AI was usually right.
And then we found the problem: a mortgage AI had been systematically discriminating by postcode for months. The human oversight process hadn’t caught it. The reviewers had effectively rubber-stamped the decisions because the control existed on paper, not because it was working in practice.
This is one of the most important lessons I want you to take from today’s assessment: documentation is evidence that a control was designed. It is not evidence that the control works.
When I assess AI governance, I don’t stop at policies, risk registers, or beautifully formatted compliance documents. I ask people to show me the control operating in the real world. I interview the people responsible for it. I watch the workflow. And one question I keep coming back to is: “Show me the last time this control actually caught something.”
If nobody can show you an example, don’t automatically mark the control as effective just because the procedure exists. You may have discovered a control that has never actually been tested.
That’s what we’re covering in Day 39: how to assess AI governance and compliance as a security tester — not as a document-review exercise, but as a practical test of whether the organisation’s controls actually work when they matter.
🎯 What You’ll Master in Day 39
Classify AI systems under EU AI Act risk tiers and identify applicable requirements
Map NIST AI RMF functions to testable security controls
Test whether governance documentation reflects actual operational practice
Assess human oversight mechanisms for practical effectiveness
Build a compliance gap register that maps technical findings to regulatory obligations
Produce governance assessment deliverables for both technical and legal audiences
⏱️ Day 39 · 3 exercises · Think Like Hacker + Kali Terminal + Think Like Hacker ### ✅ Prerequisites - Day 37 — AI Privacy Attacks — GDPR mapping from Day 37 uses the same regulatory translation methodology; Day 39 extends it to AI-specific frameworks - Day 25 — AI Security Report Writing — governance findings use the Day 25 report structure with regulatory mapping added; the executive summary format is the same - Basic familiarity with NIST AI RMF and EU AI Act structure — reference links in the Further Reading section ### 📋 AI Governance and Compliance Testing — Day 39 Contents 1. The AI Regulatory Landscape in 2026 2. EU AI Act Risk Classification and Requirements 3. NIST AI RMF — Testing the Four Functions 4. Human Oversight Effectiveness Testing 5. Building the Compliance Gap Register 6. Governance Assessment Deliverables In Day 38, I took you inside the fine-tuning process and showed you where the security risks can hide. Today, in Day 39, we’re stepping back and looking at the governance layer around that technology — the policies, processes, risk controls, and human oversight that are supposed to keep everything accountable.
And there’s an important distinction here: having a policy doesn’t mean the organisation is actually following it. I’m going to show you how I test that gap between what the documentation says and what people actually do.
Then, in Day 40, we’ll move from prevention to response. I’ll walk you through how to detect, contain, and recover from AI security incidents — including the problems AI-enabled attacks create that traditional incident response processes aren’t always prepared for.
📖 Read the complete guide on Securityelites — AI Red Team Education
This article continues with deeper technical detail, screenshots, code samples, and an interactive lab walk-through. Read the full article on Securityelites — AI Red Team Education →
This article was originally written and published by the Securityelites — AI Red Team Education team. For more cybersecurity tutorials, ethical hacking guides, and CTF walk-throughs, visit Securityelites — AI Red Team Education.
Top comments (0)