Kubernetes Audit Policy: A Checklist Against Common Blind Spots in Your Rules

Hi everyone! We want to share our experience writing and reviewing a Kubernetes audit policy.

A Kubernetes audit policy is one of those things that gets written once during cluster setup and then lives "as is" for years, accumulating exceptions for new components and almost never being reviewed as a whole. At some point it's no longer a security policy — it's a layer of three-year-old comments.

We recently sat down to revisit our own 500+ line config. We had assembled it from several sources, primarily based on the Kubernetes Threat Matrix. We want to share not so much the specific findings (those are specific to our cluster) as the principles and common traps that surfaced during the review. If you're writing or reviewing an audit policy for your own cluster, this checklist should save you some time.

How Audit Policy Works

In short: it's a list of rules, each describing which users/groups/verbs/resources it matches and which logging level to apply — from None (don't log at all) to RequestResponse (log the full request and response, including the body).

A key detail that's easy to miss: rules are evaluated top to bottom, and the first match wins. If a broad exception rule sits above a narrow, security-critical rule, the second one will never fire for the same requests. Most of the problems actually worth hunting for in a review are ordering mistakes, not mistakes in individual rules.

Principle 1: Separate Noise from Signal

In any reasonably active cluster, 90% of API traffic is get/list/watch from system components: the kubelet polling node status, Prometheus scraping metrics, controllers watching their resources. If you log all of that at a meaningful level, the log drowns in noise and finding a real incident becomes impossible.

The right pattern is to silence these streams precisely:

- level: None
  users: ["system:serviceaccount:kube-system:coredns"]
  verbs: ["get", "list", "watch"]

Important: silence as narrowly as possible — by specific user, specific verbs, and ideally specific resources. This is where the first common mistake hides.

Trap #1: An Exception Without Verb/Resource Restrictions

This one shows up regularly:

- level: None
  users: ["system:serviceaccount:some-ns:some-controller"]

Without verbs and resources, this rule silences every action by that user or service account — not just the routine read traffic it was written for, but any mutations that service account is ever granted (accidentally through an RBAC mistake, or intentionally when functionality expands). If that service account's token is compromised, the attack goes entirely unnoticed by the audit log — not because anyone planned a hole, but because the exception was written "by eye" for the component's current behavior, without explicitly fixing its boundaries.

Practical rule: when adding a new exception, always explicitly list verbs, even if the component currently only does get/list/watch. This pins down the boundaries — it defines what we see and what matters to us.

Trap #2: Stale Comments

# Any actions with secrets and configs (except list/watch) - critical to monitor
# for potential data leaks or unauthorized configuration changes
- level: None
  verbs: ["get", "create", "update", "patch", "delete"]
  resources:
    - group: ""
      resources: ["secrets", "configmaps"]
  users: [...]

The comment says "critical to monitor," but the rule actually disables logging for the listed service accounts.

This doesn't affect how the policy works, but inconsistencies like this are a direct path to someone reading the comment a year from now, taking it at face value, and spending a day debugging "why aren't events being logged when it says they should be." Reviewing comments is as much a part of code review as reviewing the logic.

Trap #3: Leftover Rules from Past Migrations

The CNI was migrated from kube-router to Cilium six months ago, but the exception for system:kube-router is still in the file. On its own it's harmless — it simply never fires, because that service account no longer exists. But:

• it clutters the file and confuses reviewers;
• if someone ever creates a service account with the same name for a different component, it inherits someone else's stale exemptions.

Simple check: periodically (say, once a quarter) compare the list of users and service accounts in your exception rules against the service accounts that actually exist in the cluster — kubectl get sa -A — and clean up the mismatches.

Principle 2: For Sensitive Data, Cap at Metadata

A separate pattern from the reviewed config:

# Secrets, ConfigMaps, and TokenReviews can contain sensitive & binary data,
# so only log at the Metadata level.
- level: Metadata
  verbs: ["get", "create", "update", "patch", "delete"]
  resources:
    - group: ""
      resources: ["secrets", "configmaps"]
    - group: "authentication.k8s.io"
      resources: ["tokenreviews"]

The logic is simple: the fact that a secret was accessed must be recorded (who, when, which secret), but the contents of the secret must never end up in the audit log under any circumstances — otherwise the audit log itself becomes a secret store, just a less protected one.

Principle 3: Use RequestResponse Where It's Justified

On the other end of the spectrum are events where you need to see not just the fact, but the full request body:

- level: RequestResponse
  resources:
    - group: ""
      resources: ["pods/exec", "pods/attach", "pods/portforward"]
- level: RequestResponse
  verbs: ["create", "update", "patch", "delete"]
  resources:
    - group: "rbac.authorization.k8s.io"
      resources: ["clusterrolebindings", "clusterroles", "rolebindings", "roles"]

exec/attach/portforward is direct access inside a running container — a classic vector when investigating incidents. RBAC changes are potential privilege escalation. In both cases, skimping on the logging level isn't worth it: the difference in log volume is small, and the value during an investigation is enormous.

Principle 4: Use Rule Ordering Deliberately, Not Accidentally

A good example of using ordering intentionally is how events are handled:

# Defense evasion tactic - detect attempts to delete k8s events
- level: Metadata
  verbs: ["delete"]
  resources:
    - group: ""
      resources: ["events"]
    - group: "events.k8s.io"
      resources: ["events"]

# Don't log events requests.
- level: None
  resources:
    - group: ""
      resources: ["events"]
    - group: "events.k8s.io"
      resources: ["events"]

Events themselves are a very noisy resource, and logging every access to them isn't practical. But deleting events is a classic defense evasion technique (an attacker cleaning up traces of their activity). By placing the narrow delete rule above the general "silence everything" rule, we get exactly the behavior we want: normal event traffic doesn't clutter the log, but an attempt to delete events is recorded.

Keep this in mind as a general pattern: if the same resource needs different sensitivity depending on the verb, the specific rule must always come before the general one.

Mini-Checklist for Reviewing Your Audit Policy

• Is there a rule for system:unauthenticated, and does it sit above all exception rules?
• Is there any exception rule (level: None) without explicit verbs and/or resources? If so, can it be narrowed?
• Are secrets/configmaps/tokens never logged above Metadata?
• Are RBAC changes, exec/attach/portforward, and network policy deletions logged at a sufficient level (Request/RequestResponse)?
• Do any exception rules reference users/service accounts that no longer exist in the cluster (leftovers from migrations)?
• Do the comments match what the rules actually do?
• Is there a catch-all rule at the very end of the file (usually level: Metadata) so nothing slips past the log by default?
• Are there any obviously temporary rules without a date or ticket for their removal?

Conclusion

A Kubernetes audit policy is not a "set it and forget it" thing. It grows with the cluster, and every new exception added in a hurry to silence yet another noisy component is a potential blind spot if it isn't explicitly bounded. The difference between "convenient" and "secure" here usually comes down to a single line — an explicit list of verbs.

If you have your own findings or patterns from working with audit policies, share them in the comments — we'd love to compare approaches.