DEV Community

Hyelzor
Hyelzor

Posted on

Building a K8s-Native AI Governance Platform with OPA and ML-BOM

#ai #kubernetes #security

Building a K8s-Native AI Governance Platform with OPA and ML-BOM

AI workloads on Kubernetes are exploding. But governance? That's a mess.

We have models running in production, shadow AI workloads we don't know about, and zero visibility into who deployed what. Compliance teams are panicking. Security teams are overwhelmed.

That's why I built Project-Aegis.

🎯 Project-Aegis is an open-source, Kubernetes-native AI governance platform that discovers, scans, and enforces policies on AI/ML workloads in real-time.

🔗 GitHub: github.com/Hyelzor/Project-Aegis

🔥 The Problem: AI Governance is Broken

Here's what I kept running into:

· Shadow AI — Models deployed without anyone knowing
· No audit trail — Who deployed what, and when?
· Manual policy enforcement — Good luck applying consistent rules across 50+ clusters
· Compliance nightmares — Finance, healthcare, and regulated industries need proof, not promises

Most existing solutions are either:

· Cloud-specific (lock-in)
· Too heavy (enterprise bloat)
· Not designed for Kubernetes-native workflows

I wanted something that works out of the box with Helm, integrates with OPA for policies, and gives me tamper-evident audit logs out of the box.

So I built it.

🏗️ Architecture Overview

Here's how Project-Aegis works:

┌─────────────────────────────────────────────────────────────┐
│                    Kubernetes Cluster                       │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │
│  │  AI Agent   │  │  AI Agent   │  │    k8s-aibom        │ │
│  │  (LangChain)│  │  (LangChain)│  │    Controller       │ │
│  └──────┬──────┘  └──────┬──────┘  └──────────┬──────────┘ │
│         │                │                     │            │
│         └────────────────┼─────────────────────┘            │
│                          │                                  │
│                    [WebSocket]                              │
│                          │                                  │
└──────────────────────────┼──────────────────────────────────┘
                           │
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                    Aegis Backend (FastAPI)                  │
│  ┌────────────────────────────────────────────────────────┐ │
│  │  • ML-BOM Ingestion  • Policy Engine (OPA Rego)       │ │
│  │  • OpenA2A Scanner   • Audit Logger (HMAC chain)      │ │
│  └────────────────────────────────────────────────────────┘ │
│                           │                                  │
│                    ┌──────▼──────┐                          │
│                    │ PostgreSQL  │                          │
│                    │   (16)      │                          │
│                    └─────────────┘                          │
└──────────────────────────────────────────────────────────────┘
                           │
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                    React Dashboard                          │
│              (Real-time policy visualization)               │
└──────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Tech stack:

· Backend: Python 3.11 + FastAPI + SQLAlchemy
· Database: PostgreSQL 16
· Policy Engine: OPA Rego
· Frontend: React 18 + Vite + Tailwind + Recharts
· Orchestration: Kubernetes + Helm
· Agent SDK: TypeScript 5.x + LangChain wrapper

🧠 Key Features

1️⃣ ML-BOM Discovery & Ingestion

A Kubernetes controller (k8s-aibom) automatically discovers AI models deployed on your clusters and generates ML-BOM (Machine Learning Bill of Materials) documents.

Every model gets tracked — no more shadow AI.

2️⃣ OpenA2A Vulnerability Scanning

Every valid ML-BOM triggers an automated OpenA2A scan via webhook. Vulnerabilities are detected before they reach production.

3️⃣ Real-Time OPA Policy Enforcement

Policies are written in Rego and enforced in real-time:

# Example: Block vLLM deployments
deny[msg] {
    input.kind == "Deployment"
    input.spec.template.spec.containers[_].image == "vllm/vllm"
    msg = "vLLM deployments are blocked by policy"
}
Enter fullscreen mode Exit fullscreen mode

The policy engine supports three actions:

· Block — Prevent the workload from running
· Allow — Permit the workload
· Mask — Redact sensitive data from logs

4️⃣ Tamper-Evident Audit Logs

This is one of my favorite features. Every audit log entry is hashed using HMAC-SHA256 in a chain:

Log 1 → Hash(Log 1 + Secret)
Log 2 → Hash(Log 2 + Hash(Log 1 + Secret))
Log 3 → Hash(Log 3 + Hash(Log 2 + ...))
Enter fullscreen mode Exit fullscreen mode

If someone modifies a log entry, the chain breaks — and you can prove it. Critical for finance, healthcare, and regulated industries.

5️⃣ React Dashboard

A clean, real-time dashboard shows:

· All ML-BOMs across clusters
· Policy violations
· Audit trail
· Scan results

🚀 Getting Started in 5 Minutes

Option 1: Docker Compose (Local Dev)

git clone https://github.com/Hyelzor/Project-Aegis.git
cd Project-Aegis
make docker-up
Enter fullscreen mode Exit fullscreen mode

Option 2: Helm Chart (Kubernetes)

helm repo add aegis https://hyelzor.github.io/Project-Aegis
helm install aegis aegis/aegis
Enter fullscreen mode Exit fullscreen mode

That's it. The dashboard is available at http://localhost:3000.

📊 Real-World Validation

We recently had users test Aegis on staging clusters. The shadow AI detection caught two orphaned workloads they didn't even know were running.

One user said:

"Helm install was shockingly painless and the OPA policies started blocking my test workload right away, exactly as configured. Wish more k8s tools had audit trails this clean out of the box."

Another added:

"The shadow AI detection caught two orphaned workloads in our staging cluster that I didn't even know were running — which was both reassuring and slightly terrifying."

🔮 What's Next?

The platform is production-ready, but here's what I'm thinking about:

· More policy templates — Pre-built Rego rules for common AI frameworks
· Slack/Email alerts — Real-time notifications for policy violations
· Multi-cluster support — Centralized governance across 100+ clusters
· CNCF Sandbox — Eventually, I'd love to donate this to the CNCF

💬 Let's Talk

I'd love your feedback on:

  1. The OPA Rego policies — Are they comprehensive enough?
  2. The HMAC audit chain — Any edge cases I missed?
  3. ML-BOM discovery — What models/frameworks should I support next?

Drop a comment below or open an issue on GitHub.

🔗 GitHub: github.com/Hyelzor/Project-Aegis
⭐ Star it if you find it useful!

Built with ❤️ for the Kubernetes and AI community.

Top comments (0)