DEV Community

dasmat
dasmat

Posted on

I built kubectl-tripwire to find admission webhook problems before they break a cluster

One thing I've noticed while working with Kubernetes is that admission webhooks are easy to forget about... until something goes wrong.

A webhook might look completely fine, but if its Service has no healthy endpoints, or every backend pod is sitting on one node, or the TLS setup is broken, a fail-closed webhook can suddenly start rejecting API requests.

I wanted something that could look at the whole dependency chain instead of making me check everything manually.

So I built kubectl-tripwire.

It analyses admission webhooks and checks things like:

failurePolicy
Service and EndpointSlice health
Backend pods
Node and zone spread
PodDisruptionBudgets
TLS configuration
JSON output for automation

Example:

kubectl tripwire admission

Instead of only listing webhook configs, it tries to answer questions like:

"If this webhook goes down, what actually happens?"

It's still very early (v0.1.0), so there are definitely things I want to improve.

If you have ideas, find bugs, or think there's something useful to add, I'd love to hear it.

Sample Output

══════════════════════════════════════════════════════════════════
  KubeTripwire — Admission Failure-Chain Analysis
  Cluster: production-east · 8 webhooks analyzed (5 mutating, 3 validating)
══════════════════════════════════════════════════════════════════

Webhook: istio-sidecar-injector
────────────────────────────────────────────────────────────
  Type:            MutatingWebhookConfiguration
  failurePolicy:   Fail (fail-closed)
  timeoutSeconds:  10
  matchPolicy:     Equivalent
  sideEffects:     None

  Scope:
    - Ops:       CREATE
      Resources: pods
    - Namespaces: 31 of 34 match selector

  Backend:
    Service:       istio-system/istiod (port 443)
    Endpoints:     1 of 3 ready
    Pods:          3
    Owner:         Deployment/istiod
    Nodes:         1 unique nodes
    Zones:         1 unique zones (us-east-1a)

  Findings:
    [HIGH] Fail-closed webhook with degraded endpoints
      Only 1 of 3 endpoints are ready. If the remaining endpoint(s) become unavailable, matching admission requests will be rejected.
      Evidence:
        • Ready endpoints: 1 of 3
        • failurePolicy: Fail
        • Scope: pods/CREATE in 31 namespaces
      Consideration: Investigate why some endpoints are not ready. Check backend pod status and events.

    [MEDIUM] Fail-closed webhook with all backend pods on a single node
      All backend pods are scheduled on the same node. A node failure or drain will remove all backend pods simultaneously.
      Evidence:
        • Backend pods: 3
        • All on node: worker-2a
        • failurePolicy: Fail
      Consideration: Consider pod anti-affinity rules or topology spread constraints to distribute backend pods across nodes.

══════════════════════════════════════════════════════════════════
  Admission Compound Risks
══════════════════════════════════════════════════════════════════

  ⚠ Fail-closed webhook with endpoint issues affecting wide scope
    Webhook:  istio-sidecar-injector
    Scenario: Matching API requests in 31 namespaces are at risk of rejection. If all endpoints become unavailable, no matching resources can be created or modified.

──────────────────────────────────────────────────────────
Summary:
  Webhooks analyzed: 8
  Critical findings: 0
  High findings:     1
  Medium findings:   1
  Low findings:      0
  Info observations: 0
  Compound risks:    1
  Scan duration:     42ms
──────────────────────────────────────────────────────────
Enter fullscreen mode Exit fullscreen mode

Top comments (0)