DEV Community: Muhtalip Dede

Kubernetes OOMKilled — diagnose with a plan, not a wall of kubectl

Muhtalip Dede — Fri, 14 Aug 2026 20:29:14 +0000

OOMKilled is a classic day-2 rabbit hole: events, limits, restarts, then a risky scale or edit. kprompt turns the investigation prompt into a reviewable plan before apply.

Originally published at https://kprompt.ai/blog/kubernetes-oomkilled.

OOMKilled is one of the most common “the app is broken” signals in Kubernetes — and one of the easiest to misread. The Pod may still show Running. Restarts climb. Logs look fine until they stop mid-request. Someone raises the memory limit “a bit,” the Deployment rolls, and two hours later it happens again. Or worse: they remove the limit entirely and the node starts evicting neighbors.

This guide is the operator ladder for memory kills: how to confirm OOMKilled, how requests and limits differ, what kubectl shows, and how to apply a bounded fix with a reviewable plan. kprompt's explain path detects OOM findings and can propose a memory patch — still behind approval, because raising limits is a real cluster change.

What OOMKilled actually means

When a container exceeds its memory limit, the Linux OOM killer (via cgroup enforcement) terminates the process. Kubernetes records the termination reason as OOMKilled. Exit code is often 137 (128 + SIGKILL). That is not an application “bug code” — it is the kernel saying the cgroup ran out of memory.

Limit hit → container killed → kubelet may restart it (CrashLoopBackOff if it keeps dying)
No memory limit → the container can grow until the node is under pressure (evictions, not always a clean OOMKilled on that Pod)
Requests affect scheduling; limits affect kill behavior — confusing them is the most common ops mistake

Confirm it before you patch

Do not raise memory because “it feels like OOM.” Read the Pod status. The smoking gun is usually Last State / Last Termination State on the container: Reason OOMKilled, Exit Code 137.

Classic kubectl confirmation

kubectl get pods -n staging
kubectl describe pod -l app=api -n staging
# Look under Containers → Last State:
#   Reason: OOMKilled
#   Exit Code: 137

kubectl get pod -n staging -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .status.containerStatuses[*]}{.name}{"="}{.lastState.terminated.reason}{" "}{end}{"\n"}{end}'

Also check current limits on the Deployment template — describe Pod shows what ran; the Deployment owns what will run next:

See memory requests and limits

kubectl get deploy api -n staging -o jsonpath='{range .spec.template.spec.containers[*]}{.name}{" limits="}{.resources.limits.memory}{" requests="}{.resources.requests.memory}{"\n"}{end}'

Requests vs limits (the part people skip)

Field	What it does	OOM relevance
requests.memory	Scheduler places the Pod on a node with enough capacity	Too low → noisy neighbor risk; does not by itself OOMKill
limits.memory	Hard cgroup cap for the container	Exceed this → OOMKilled
No limit	Container can use free node memory	May avoid OOMKilled on that Pod; can hurt the node

A healthy fix usually raises the limit (and often the request toward a sensible fraction of that limit) based on observed usage — not deleting limits to “make it stop.” If you have Prometheus, compare working set / RSS to the current limit before you double everything.

kubectl explain ladder for memory kills

Scope — which Deployment / Pod, which namespace and context
Status — restarts, Ready, Last State reason
Resources — limits and requests on the crashing container
Events — Failed / OOM / eviction messages on Pod or node
Logs — --previous for the crashed instance (may be empty if killed hard)
Change — bump memory or roll back a bad image / leaky release

Investigation sequence

kubectl describe deploy api -n staging
kubectl describe pod -l app=api -n staging
kubectl logs deploy/api -n staging --previous --tail=100
kubectl get events -n staging --field-selector reason=OOMKilling --sort-by='.lastTimestamp'

Natural-language explain → suggested patch

kprompt's explain path walks live Deployment → Pod → Events → Logs style signals. When it finds OOMKilled on a container, it can propose a follow-up: raise the Deployment memory limit (typically doubling a known limit in the suggested plan) and show the plan for approval. Reads run immediately; the patch does not apply until you confirm — or you pass --approve in a context you trust.

Detect and review a memory fix

$ kprompt "explain why api is crashing" -n staging

# … findings include OOMKilled on container app …

Suggested fix (requires approval):
Plan
  1. patch Deployment/api memory limit (e.g. 64Mi → 128Mi)

Risk: medium
Apply? [y/N]

That is the intent-compiler shape: evidence from the apiserver, a concrete mutation plan, human gate. It is not “the model silently edited production.” If you reject the plan, nothing changes — dig into leaks, heap dumps, or a bad release instead.

Use explain first on non-production or a staging clone of the workload
Read the before→after memory numbers in the plan — doubling forever is not a strategy
Prefer fixing leaks for steady growth; raise limits for genuine under-provisioning
After apply, use --wait on related rollouts or watch the Deployment until restarts stabilize

Manual patch when you want exact numbers

Sometimes you already know the target (512Mi limit, 256Mi request). Use kubectl or a reviewed kprompt plan with an explicit change — do not approve a suggested bump you have not sanity-checked against metrics.

Explicit memory patch

kubectl set resources deploy/api -n staging \
  --limits=memory=512Mi --requests=memory=256Mi

# or edit the template
kubectl edit deploy api -n staging

When raising memory is the wrong fix

Memory leak — usage climbs until any limit dies; fix the app or roll back the release
Cache without bound — tune the process (JVM heap, Node heap, Go pacer) to fit the cgroup
Wrong container — sidecar OOMs while you patch the app container
Node pressure — Pod evicted or node NotReady; look at node allocatable and neighbors
Burst then idle — a higher limit may be fine; also consider HPA/VPA later, not blind doubles

Production habits

Confirm OOMKilled in Last State before changing resources
Change one variable at a time — memory patch or image rollback, not both blind
Keep limits; size them from data
Record the plan (kprompt history or -o json) for the incident timeline
Revisit after 24h of metrics — did working set settle under the new limit?

Try it on a sandbox Deployment

Spin a tiny limit on kind or staging, force an OOM, then run explain and decide whether to approve the suggested patch. Pair with the ImagePullBackOff guide when the Pod never starts, and with the CrashLoopBackOff guide when the memory kill is what keeps the container looping.

Quick start

curl -fsSL https://kprompt.ai/install | bash
export KPROMPT_GEMINI_API_KEY="..."

kprompt "explain why api is crashing" -n staging
# review Suggested fix → y or n

Try: kprompt.ai · GitHub · brew install kprompt/tap/kprompt

An intent compiler for Kubernetes — not another chat REPL

Muhtalip Dede — Fri, 14 Aug 2026 20:28:01 +0000

Chat CLIs optimize for turn-taking. Operators need a reviewable artifact before mutate.

kprompt is built as an intent compiler: NL → PlanResult → safety → approve → apply. Same day-2 surface (Helm, Prom, GitOps…) under one approval loop. Experimental.

Originally published at https://kprompt.ai/blog/intent-compiler-not-chat.

Most Kubernetes AI demos look the same in a screenshot: a prompt box, some English, something that resembles kubectl. Underneath, products diverge. Some scan the cluster. Some run agents inside it. Some host chat in a SaaS control plane. And in the local CLI lane — where kubectl-ai and kprompt both sit — the important question is not who has the slicker REPL. It is what the tool emits before anything hits the apiserver.

Our locked bet: kprompt is an intent compiler. Plain English compiles into a typed, reviewable PlanResult — actions, risk, hard denies — that a human or CI can gate, then apply. It is not a free-form agent chat optimized for “keep talking until the cluster moves.” That difference is the product.

Same lane, different contract

We do not claim a unique category against every Kubernetes AI tool. The map is simpler:

K8sGPT — analyzer-first diagnosis (scan → explain). We are not a fleet scanner.
Kagent — in-cluster agent framework. We ship an optional Observe-only agent, not a multi-agent platform.
Hosted chat — managed control planes. We are BYOK and local by default.
kubectl-ai — natural-language kubectl fluency. Same lane as us; different mutate contract.

Trying to out-chat kubectl-ai on agentic REPL features is a losing strategy. Google can ship conversation quality and tool-calling surface area faster than a small OSS project. Competing there means forever second place. Competing on a printable, policy-shaped plan artifact is a fight worth picking.

What “intent compiler” means in practice

A chat REPL optimizes for turn-taking: the model calls tools, narrates, maybe runs kubectl. An intent compiler optimizes for an artifact you can refuse:

Compile → review → apply (or abort)

$ kprompt "scale api to 3" -n staging

Plan
  1. kubectl scale deployment/api --replicas=3 -n staging

Risk: low
Apply? [y/N] n
Aborted.

LLM proposes intent; Go packages own planning, safety, and execution
Mutations default to plan-only until y/N or an explicit --approve
Wipe-class prompts hard-deny before a useful apply path exists
CI consumes the same PlanResult JSON humans see summarized in the terminal

Same prompt, machine-readable gate

kprompt "scale api to 3" -n staging -o json | \
  jq '{intent:.plan.intent, risk:.risk, denied:.risk.denied}'

Why the artifact matters more than the chat

Platform teams already distrust “AI applied something.” They trust diffs, PRs, admission policy, and change tickets. A scrollback of model narration does not fit that muscle memory. A PlanResult does: intent, ordered actions, risk level, denied flag, applied boolean — something you can jq, archive, and teach juniors to read before they type y.

Chat REPL instinct	Intent compiler instinct
Keep the session going until it works	Emit one plan; refuse or approve
Tool calls are the product	The gated plan is the product
Speed to first kubectl	Speed to a reviewable change
Hard to put in CI without scraping text	JSON PlanResult is a first-class gate

Neither instinct is “wrong.” If you want kubectl fluency in an interactive session, a chat-shaped CLI is rational. If you want NL day-2 ops that behave like a change you would put in a pipeline, compile to a plan.

One contract across tools

The compiler model only pays off if it stretches past kubectl scale. kprompt routes day-2 backends — Helm install/upgrade previews, Prometheus performance explains, trace adapters, Workflow generation — through the same plan → safety → approve loop. The LLM does not become a second control plane; it proposes steps against real CLIs and APIs you already run.

Different backends, same gate

kprompt "install redis" -n cache
kprompt "why is my api slow?" -n production
kprompt "explain why api is crashing" -n staging
# Mutating suggestions still show a plan before apply

Post-v1 originality we are building toward — not shipping as vapor demos — is cluster-level NL ops on that same contract: optimize my cluster style reports with optional approved fixes, and service dependency graphs grounded in Kubernetes (and traces when available). Still plan-before-apply. Never a silent controller.

What we are not selling today

Honesty is part of the positioning. The Apache-2.0 CLI is free, local, and BYOK. Org policy sync, shared audit, and Team enrollment are explored for later — there is nothing to buy on the site today, and this post is not a pricing page. When governance ships, it should attach to the same PlanResult artifact, not invent a parallel chatbot product.

Not a hosted agent in your cluster (OSS path)
Not “unique NL kubectl” — kubectl-ai shares that job
Not a replacement for RBAC, admission, or GitOps
Experimental — hard denies help; they are not a production certificate

How to evaluate us in one afternoon

Do not score kprompt on who tells a better joke in a 40-turn chat. Score the contract:

Same mutate prompt in kubectl-ai and kprompt — what prints before apply?
Wipe-class prompt — does it fail closed?
JSON gate — can CI reject denied/high-risk without scraping ANSI?
Wrong namespace / scale to zero — does the plan make the blast radius obvious?

Thirty-minute drill

curl -fsSL https://kprompt.ai/install | bash
export KPROMPT_GEMINI_API_KEY="..."

kprompt "delete all pods" -n staging
kprompt "scale api to 0" -n staging -o json | jq .risk
kprompt "scale api to 2" -n staging
# read plan → n or y

Design principle we will not trade away

Compile to PlanResult, not chat scroll. The LLM proposes; the product artifact is a structured plan humans and policy can gate. Feature parity with agentic REPLs is explicitly out of scope as a north star. If a future feature cannot show up in a reviewable plan (or a clear read-only report), it probably is not a kprompt feature.

For the peer map, read the AI tools comparison. For a direct head-to-head, see kprompt vs kubectl-ai. For the safety loop, read plan → approve. For CI schema, read PlanResult JSON deep dive and PlanResult gates. Talk to your cluster — but make the cluster change look like something you would sign.

Try: kprompt.ai · GitHub · brew install kprompt/tap/kprompt

kprompt vs kubectl-ai: plan-before-apply vs NL kubectl

Muhtalip Dede — Fri, 14 Aug 2026 20:26:45 +0000

If you are shopping for a “kubectl-ai alternative,” you are already in the right lane: local kubeconfig + BYOK + natural language. The fork in the road is the mutate contract.

kprompt compiles intent into a typed PlanResult (actions, risk, hard denies), then requires approve before apply — including CI JSON gates. kubectl-ai is often a faster path to kubectl-shaped execution.

Originally published at https://kprompt.ai/blog/kprompt-vs-kubectl-ai.

If you searched for a kubectl-ai alternative or “AI Kubernetes CLI,” you will land on Google’s kubectl-ai and a handful of peers. kprompt sits in the same lane: local binary, your kubeconfig, natural language in. The useful question is not who has the slicker chat — it is what happens before anything mutates the cluster. For a job-first alternatives map (keep kubectl-ai vs switch), see kubectl-ai alternatives.

Short answer: use kubectl-ai when you want an agentic REPL that is excellent at generating and running kubectl. Use kprompt when you want an intent compiler — a typed, reviewable plan with risk and hard denies, optional CI JSON, and day-2 backends (Helm, metrics, GitOps) under one approval loop. Same problem space; different contract.

Quick decision

You care about…	Prefer	Why
Fast kubectl fluency / interactive chat with tool calls	kubectl-ai	REPL-first; strong model + tool-calling surface from Google’s project
Reviewable plan before every mutate	kprompt	Plan → safety → y/N (or --approve); wipe-class hard denies
Gate plans in CI with stable JSON	kprompt	PlanResult on stdout; human UI on stderr
One NL layer across Helm / Prom / GitOps	kprompt	Multi-tool routes with aggregate plan + single approval
MCP server / IDE agent integration today	Either	kubectl-ai has broad MCP mode; kprompt mcp serve is read/plan-only — mutations return a PlanResult, never auto-apply
BYOK + Apache-2.0 laptop-local CLI	Either	Both keep kubeconfig local; pick by mutate contract

Side-by-side

Dimension	kubectl-ai	kprompt
Primary artifact	Conversation + generated kubectl / tool calls	PlanResult (actions, risk, denies)
Default mutate UX	Agent executes kubectl (modes vary)	Show plan → approve on TTY
Safety model	Tool / mode dependent	Risk scoring + hard denies (wipe-class)
CI / policy gate	Bring your own wrappers	First-class --output json PlanResult
Day-2 stack	kubectl (+ extensible tools / MCP)	Helm, Argo, Prom, OTel, Grafana, GitOps… via tools detect
Positioning	AI-powered kubectl assistant	Intent compiler → AI SRE direction

What a scale looks like in each

Illustrative shapes — versions and flags change; read current docs for each project.

kubectl-ai — NL → kubectl fluency

kubectl-ai "scale deployment api to 3 in staging"
# Typically proposes / runs the matching kubectl

kprompt — compile → review → apply

kprompt "scale api to 3" -n staging

Plan
  1. scale Deployment/api replicas → 3

Risk: low
Apply? [y/N]

If your team’s fear is “the model applied something I did not see,” kprompt’s default path is built around that fear. If your team’s fear is “I am slow at remembering kubectl under pressure,” kubectl-ai’s REPL is built around that fear. Both are legitimate.

When kprompt is the better fit

Shared clusters where every mutate needs a visible plan
CI pipelines that must jq on risk.denied / plan.actions
Prompts that span Helm install, Prom explain, then an approved scale
Hard deny for wipe jokes and unscoped deletes as product behavior
You want the long-term AI SRE path (investigate / blast-radius) without giving up approval

When kubectl-ai is the better fit

You want a chat REPL that stays close to raw kubectl
MCP / IDE agent workflows are the primary integration
You already standardize on Google’s kubectl-ai releases and models
You prefer maximum conversational flexibility over a fixed PlanResult schema

Honest limits (both sides)

Neither replaces RBAC, admission controllers, or GitOps as source of truth
Neither is production-hardened by slogan — try on kind / staging first
kprompt is experimental OSS; plans can be wrong — always read the plan
kubectl-ai is a fast-moving Google project — features and UX shift; check upstream README
We are not claiming to out-chat kubectl-ai on agentic REPL quality

Try kprompt in five minutes

Install + safe read + one mutate plan

brew install kprompt/tap/kprompt
export KPROMPT_GEMINI_API_KEY="..."

kprompt "list deployments" -n staging
kprompt "scale api to 2" -n staging   # review plan, then y/N

For the wider peer map (K8sGPT, Kagent, hosted chat), see the AI tools comparison. For where kprompt is headed beyond AI kubectl, see Beyond AI kubectl: why kprompt is aiming at AI SRE and Roadmap & vision. Configure BYOK models on Providers. Optional always-on alerts: Observe agent. Safety docs before shared-cluster --approve.

Try: kprompt.ai · GitHub · brew install kprompt/tap/kprompt