k8s-aibom: Automated AI BOM for Kubernetes Workloads

If you run a shared Kubernetes cluster, you already have AI workloads you don't know about. Someone shipped a vLLM inference service last sprint, a data team stood up a RAG pipeline behind a plain Deployment, and a contractor left an Ollama pod running in a namespace nobody audits. k8s-aibom, the controller Google Cloud open-sourced this month, exists to find exactly those. It watches your live workloads and writes a CycloneDX 1.6 ML-BOM for every AI system it can identify, so the answer to "what AI is running in this cluster right now" stops being a guess.

That "right now" is the whole point. A build-time SBOM tells you what your CI pipeline thought it was shipping. It says nothing about the pod a teammate kubectl apply'd by hand at 2am, or the image that pulled a new model layer since the last scan. Shadow AI is a runtime problem, and k8s-aibom is a runtime tool.

Why a runtime AI BOM is different

An AI Bill of Materials answers the same question a software BOM does, scoped to AI: which models, frameworks, and inference engines are in play, and where did they come from. The category matters now because regulators started asking. The EU AI Act's logging and transparency obligations, the NIST AI Risk Management Framework's "know what you deployed" controls, and ISO/IEC 42001's inventory clauses all assume you can produce a current, accurate list of your AI systems. You cannot produce that list from a spreadsheet someone updates quarterly.

Build-time and runtime BOMs solve different halves of the problem. Your build pipeline can attest to what it produced, and tools that live there are a good idea. But the pipeline never sees the workload that skipped it. If your supply-chain story stops at the CI system, read our take on why that is not enough in Supply Chain Security Proxy: Move Beyond Vulnerability Scanning. k8s-aibom fills the runtime gap: it reconciles against what the API server actually reports, not what a pipeline claims it built.

What the controller actually watches

k8s-aibom is a standard Kubernetes controller, not a DaemonSet or a privileged agent. It reconciles a set of workload kinds and emits a BOM per workload. The kinds it tracks:

• Deployments
• StatefulSets
• DaemonSets
• Jobs and CronJobs
• KServe InferenceService resources

That list covers the shapes AI actually takes in a cluster. Inference services and agent stacks run as Deployments, batch training and evaluation runs as Jobs, and model servers packaged for KServe show up as InferenceService objects. Because it reconciles against the API server, a workload created outside your GitOps flow is just as visible as one that went through it. That is the property that makes it useful against shadow AI: you did not have to know the workload existed for the controller to catalog it.

Detection works by pattern-matching signals the workload already carries: container image references, command-line arguments, environment variables such as HF_MODEL_ID, mounted volumes, and workload annotations. From those signals it recognizes a broad set of AI software:

Category	Examples it identifies
Inference runtimes	vLLM, Hugging Face TGI, NVIDIA Triton, Ollama, Ray Serve, SGLang
Agent frameworks	LangChain, LangGraph, AutoGen, CrewAI, Langflow, Flowise
Vector databases	Milvus, Qdrant, Weaviate, Chroma, pgvector
Training frameworks	PyTorch, KubeRay, JAX, Hugging Face Accelerate
Evaluation tools	lm-evaluation-harness, Ragas, Trulens

If you are already running two inference engines and cannot decide whether that is a problem, our comparison Choosing an LLM Serving Engine: vLLM vs TGI covers the tradeoffs k8s-aibom will happily inventory for you.

What detection looks like on a real workload

Concretely, picture a RAG API someone shipped as a plain Deployment. Nothing about the object name says "AI", but the pod spec gives it away:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rag-api
  namespace: team-ml
spec:
  template:
    spec:
      containers:
        - name: server
          image: vllm/vllm-openai:v0.6.3
          args: ["--model", "meta-llama/Llama-3.1-8B-Instruct"]
          env:
            - name: HF_MODEL_ID
              value: meta-llama/Llama-3.1-8B-Instruct

The controller does not need a label saying this is AI. The vllm/vllm-openai image matches its inference-runtime catalog, the --model arg and HF_MODEL_ID env var name the model, and both get recorded on the BOM. The image tag lands as declared because it is read straight from the spec; the runtime identification lands as inferred because it came from a heuristic. A workload that went out of its way to hide, say a custom image with the model passed through a mounted config file, would still surface the pieces the controller can see and mark the rest unresolved. That gap is visible in the BOM, which is the honest behavior: you get told what the tool is unsure about instead of a confident lie.

Deploying it

Deployment is a Helm install into its own namespace. You build and push the image, then install the chart:

$ git clone https://github.com/GoogleCloudPlatform/k8s-aibom.git
$ cd k8s-aibom
$ export IMG=my-registry.example.com/k8s-aibom:v1.0.0
$ make image
$ make docker-push
$ helm install k8s-aibom ./charts/k8s-aibom \
    --namespace k8s-aibom-system \
    --create-namespace \
    --set image.repository=my-registry.example.com/k8s-aibom \
    --set image.tag=v1.0.0

The controller does not scan every namespace by default, which is the right call on a busy cluster. You opt a namespace in with a label:

$ kubectl label namespace team-ml aibom.k8saibom.dev/enabled=true

This opt-in model is deliberate. On a large platform you probably want to start with the namespaces where AI is likely, confirm the BOMs look right, then widen the net. Rolling it cluster-wide on day one buries you in output before you have tuned anything.

The custom resources live under the aibom.k8saibom.dev/v1alpha1 API group. There are two kinds: AIBOM, a namespace-scoped resource holding the BOM for one workload, and AIBOMControllerConfig, a cluster-scoped singleton that configures where BOMs are sent.

Reading a BOM

Once a namespace is enabled, the controller starts producing AIBOM resources. You read them like any other object:

$ kubectl get aibom -A
$ kubectl describe aibom -n team-ml deployment-rag-api

The output is a CycloneDX 1.6 ML-BOM. If the document is small it lives inline in the resource status; if it is large the status carries a reference to the externalized copy instead, so you are not stuffing megabytes into etcd.

The detail worth understanding is the confidence model. k8s-aibom does not pretend every field is a hard fact. Each attribute is tagged as one of three states:

• declared: taken straight from the workload spec or an explicit annotation, so it is authoritative.
• inferred: derived from a heuristic, such as recognizing an inference runtime from its image and args.
• unresolved: the controller saw a signal but could not pin it down with confidence.

That grading is what makes the output auditable rather than a pile of guesses. When a reviewer asks why a BOM claims a workload runs a particular model, the answer is a field-level provenance tag, not a shrug. If you have been burned by AI tools that state everything with false certainty, our writeup AI Agent Risks: Lessons from Snyk's 10,000 Environment Audit is a good reminder of why that provenance matters.

Sending BOMs somewhere durable

A BOM that only lives in a cluster resource disappears when the workload does, which is useless for an audit trail. AIBOMControllerConfig defines sinks that push each BOM out. Three sink types exist: the always-on CR status (no external egress), a Google Cloud Storage bucket, and a generic webhook. A config with both external sinks looks like this:

apiVersion: aibom.k8saibom.dev/v1alpha1
kind: AIBOMControllerConfig
metadata:
  name: default
spec:
  sinks:
    - name: audit-archive
      type: GCS
      gcs:
        bucket: my-aibom-archive
        pathTemplate: "aibom/{namespace}/{kind}-{name}/{timestamp}.json"
        workloadIdentity: [e-mail]
    - name: graph-ingest
      type: Webhook
      webhook:
        endpoint: https://guac.internal.example.com/ingest
        auth:
          bearerToken:
            secretRef:
              name: graph-ingest-creds
              key: token

The GCS sink has a property worth calling out: writes use a DoesNotExist precondition, so a stored BOM cannot be overwritten once created. That turns the bucket into an append-only historical record. For anyone who has ever tried to reconstruct "what was running when the incident happened" from mutable logs, an immutable, timestamped BOM per workload is a real upgrade. Pair the pathTemplate above with a bucket retention policy and you have a compliance artifact that survives the workload that produced it.

The webhook sink is how you feed a graph database or an SBOM platform. A common pattern is pushing into a supply-chain graph so AI components sit alongside your other software inventory instead of in a separate silo.

Where it fits, and where it doesn't

k8s-aibom is narrow on purpose, and that is a strength. It does not scan for vulnerabilities, enforce policy, or block anything. It builds an accurate inventory of AI workloads and gets it somewhere durable. Everything downstream, such as CVE correlation, policy gates, and drift alerts, is a separate tool consuming the BOM. Trying to make one controller do all of that is how you end up with a privileged agent nobody trusts.

Keep three limitations in mind before you lean on it:

1. Detection is pattern-based, so it has a coverage frontier. A homegrown inference server with no recognizable image, args, or environment signals may land as unresolved or be missed. The v1alpha1 API group is a fair signal that the detection catalog is still moving. Treat the BOM as a strong lead, not a guarantee of completeness, and watch what shows up unresolved.
2. It reports, it does not enforce. Finding a shadow workload and doing something about it are different jobs. You still need policy tooling, whether that is an admission controller or a governance layer, to act on what the BOM reveals. If you are building that layer, Governing AI Agents in CI/CD with OPA and MCP covers the policy side.
3. The GCS sink is Google Cloud native. The webhook sink is portable and works anywhere, but the tightest integration, immutable object writes via Workload Identity, assumes GKE. On other platforms you wire the webhook into your own durable store.

The bottom line

Shadow AI is not going away, and "we think we know what's running" is not an answer an auditor accepts. k8s-aibom gives you a runtime, provenance-tagged inventory of the AI workloads actually live in your cluster, written to an immutable store you can hand to a compliance review. It is early software with a moving detection catalog, so verify its output rather than trusting it blindly. But as a way to turn shadow AI from an unknown into a tracked list, it is a genuinely useful addition to a Kubernetes security stack.

Start small: install it, enable one namespace where you suspect unmanaged AI, and read the first few BOMs. The controller runs as a lightweight reconciler with no privileged access, so there is little downside to letting it watch. For the broader context on how Google frames this problem, their software supply chain security guidance and the how GKE powers AI innovation writeup are worth reading. For the Kubernetes primitives the controller builds on, the upstream controllers documentation covers the reconcile loop it uses.