How I Stopped an LLM From Skipping the Investigation, Looping Forever, and Inventing Pods — Without LangChain

An agent loop is just a for loop. The hard part is the invariants inside it.

LLMs misbehave in predictable ways: they skip steps, invent tools, loop forever, and fabricate evidence.
Every failure can be caught by a small, deterministic guardrail that replies with feedback instead of crashing.

3 a.m., and payment-service is down again

You know the routine. The pager fires, you open a terminal, and you start the ritual:

kubectl get pods -A
kubectl describe pod payment-service-6f7b8d9c0-ghi56 -n production
kubectl logs payment-service-6f7b8d9c0-ghi56 -n production --tail=50
kubectl get events -n production

Four commands, three context switches, and a lot of squinting before you spot the line that matters:

config validation failed: missing required field 'DB_PASSWORD'

The investigation follows a predictable shape, but it still takes a senior engineer’s instincts to run it efficiently. That combination — a repeatable process, with judgment calls at every step — is exactly what LLM agents should be good at.

So I built one. You ask it a question in plain English:

python agent.py "Which pods in the production namespace are unhealthy, and why?"

It decides which tools to call, digs through several rounds of evidence on its own, and returns a report like this:

payment-service-6f7b8d9c0-ghi56 — CrashLoopBackOff, 14 restarts
Evidence: logs show missing required field 'DB_PASSWORD'
Fix: restore the missing Secret entry, then roll out the deployment

inventory-service-59d8f7c6a-mno90 — ImagePullBackOff
Evidence: events show image v0.9.9-beta — manifest unknown
Fix: push the missing image or correct the tag

Every conclusion cites the tool result it came from.

One question in, a structured health report out — every step logged.

Getting a demo to produce that output once was easy. Getting it to produce that output reliably, even when the model misbehaves, was the real work. That is what this article is about.

Why I didn’t use a framework

An agent loop is, at its core, embarrassingly simple:

1. Send messages + tools to the LLM
2. Did the LLM return tool_calls?
   • No → that text is the final answer, exit
   • Yes → execute each tool, append results as role="tool", go to 1
3. Still not done after MAX_STEPS → force a summary

That fits in a single for loop. The hard part is not the loop itself.

The hard part is the invariants that have to hold inside the loop: every tool_call gets exactly one matching tool message, the loop always terminates, and the model never reports a fact it didn't observe.

Frameworks hide those invariants behind abstractions. That’s great — until production breaks and you need to know exactly what the model saw at step 4, and why it called list_pods for the third time.

So I wrote the loop by hand, using only the low-level openai and mcp SDKs. The entire agent fits in about 350 lines you can read in one sitting.

Architecture: three small pieces

Three files, three jobs:

• agent.py — the loop and all of its guardrails.
• mcp_client.py — a thin adapter. It launches the MCP server as a subprocess, converts MCP tool definitions to OpenAI’s format, and flattens tool results into strings.
• mock_k8s_server.py — a FastMCP server simulating a 3-node, 12-pod cluster with planted failures: CrashLoopBackOff, ImagePullBackOff, and two Pending pods caused by resource pressure.

The key piece of glue is surprisingly small. The LLM only understands OpenAI’s tool format, and the server only speaks MCP:

{
    "type": "function",
    "function": {
        "name": tool.name,
        "description": tool.description or "",
        "parameters": tool.inputSchema,  # MCP JSON Schema -> OpenAI parameters
    },
}

Because the tool list is fetched from the server at runtime, switching from the mock to a real cluster is a single line in .env:

MCP_SERVER_COMMAND=npx -y mcp-server-kubernetes

agent.py doesn't change at all.

Seven ways the model misbehaves — and the guardrails that catch them

This is the heart of the project. Every guardrail below exists because a model actually did the thing it prevents.

1. Answering without investigating

Symptom: You ask for a health check, and the model confidently replies “All pods look healthy” — without calling a single tool. It has no idea what the cluster looks like. It is pattern-matching on what a health report usually says.

Fix: Force tool use on the first step only, then let the model decide freely after that:

tool_choice="required" if step == 1 else "auto",

Why only step 1? If you force tools on every step, the model can never stop and write the report. You’ve replaced “answers too early” with “never answers.”

The second layer: not every OpenAI-compatible provider honors tool_choice. So if the model still answers with zero tool calls, the harness pushes back:

if tool_calls_made == 0 and step < cfg["max_steps"]:
    messages.append({
        "role": "user",
        "content": "You have not called any tool yet. Investigate the "
                   "cluster with the provided tools before answering.",
    })
    continue

Lesson: don’t trust a single API parameter to enforce behavior across providers. Enforce it in your own code.

2. Hallucinating tools

Symptom: The model calls a tool that doesn’t exist, such as delete_pod.

That case is worth pausing on. A hallucinated tool name is not just a bug. In an ops context, it is a safety issue.

Fix: A whitelist built from what the server actually exposes. If the model asks for something else, the call is rejected — and the model is told what it can use:

if name not in known_tools:
    messages.append({
        "role": "tool",
        "tool_call_id": tc.id,
        "content": f"Unknown tool '{name}'. "
                   f"Available tools: {', '.join(sorted(known_tools))}.",
    })
    continue

3. Malformed arguments

Symptom: get_pod_logs({"name": "x", "tail": "fifty"}). A required namespace is missing, and tail is a string instead of an integer.

You could forward that to the server and let it fail. But that wastes a round trip, and server error messages are often vague.

Fix: Pre-flight validation. Every MCP tool already ships with a JSON Schema, so the harness compiles one Draft 2020–12 validator per tool at startup, and checks arguments before the call:

errors = sorted(
    validator.iter_errors(args),
    key=lambda e: [str(p) for p in e.absolute_path],
)
if errors:
    detail = "; ".join(
        f"{'.'.join(map(str, e.absolute_path)) or '<root>'}: {e.message}"
        for e in errors
    )
    # -> "Invalid arguments for get_pod_logs: <root>: 'namespace' is a
    #     required property; tail: 'fifty' is not of type 'integer'"

The model gets a precise, field-level correction it can act on in the next step.

One more detail: if a tool’s schema is itself invalid, the harness logs a warning and skips validation for that tool instead of crashing. A guardrail should never become a new single point of failure.

4. Looping forever

Symptom: The model calls list_pods(namespace="all"), gets the result, and calls list_pods(namespace="all") again. And again. Each loop burns tokens and moves nothing forward.

Fix: Signature-based duplicate detection. Each call is normalized to tool:json(args, sort_keys=True), so {"a": 1, "b": 2} and {"b": 2, "a": 1} count as the same call.

A duplicate is not executed again. The model gets the cached result plus a nudge:

[Duplicate call] You already called this tool with identical arguments;
the result has not changed. Base your next step on it, try a different
tool, or write the final report.

After MAX_REPEATS duplicates, the harness stops the investigation and forces a summary.

The protocol trap that bit me: my first instinct was to break as soon as the repeat limit was hit. That corrupts the conversation.

An assistant message can contain several tool_calls, and the API requires every one of them to have a matching role="tool" message. Break halfway through, and the next request is rejected outright.

So the harness sets a flag, finishes answering every call in the current round, and only then exits:

if repeat_count >= MAX_REPEATS:
    # Do not break mid-round: every tool_call in this assistant
    # message still needs a matching tool message, or the next
    # request would be rejected.
    force_summary = True

This is the kind of invariant a framework hides from you — until it doesn’t.

Break mid-round and the next request fails with a 400. Finish the round, then exit.

5. Inventing evidence

Symptom: The report is well written and confident — and it mentions ghost-app-12345-abcde, a pod that does not exist.

This is the most dangerous failure, because it looks correct. An on-call engineer may spend 20 minutes chasing a ghost.

Fix: A grounding check. Before accepting a final answer, the harness extracts every pod-like name from the report and verifies that it appeared in at least one tool result:

POD_NAME_RE = re.compile(r"\b[a-z0-9]+(?:-[a-z0-9]+){2,}\b")

def _invented_pods(answer: str, evidence: str) -> list[str]:
    return sorted(
        {n for n in POD_NAME_RE.findall(answer) if n not in evidence}
    )

If any are found, the model is told exactly which names to remove or correct.

The trade-offs, stated honestly:

• The regex requires three or more hyphen-separated segments, matching Deployment-style names like payment-service-6f7b8d9c0-ghi56. That deliberately skips node-03 and kube-system to avoid false positives — which means node names aren't checked.
• It only verifies names, not claims. “payment-service has 99 restarts” would pass.
• The pushback happens at most once. A second wrong answer is accepted, because an unbounded correction loop is just a slower infinite loop.

That’s a narrow check. But it targets the highest-cost hallucination: sending a human after something that doesn’t exist.

6. Never converging

Symptom: The model keeps finding “one more thing” to check.

Fix: A hard MAX_STEPS budget (10 by default). When it runs out, the harness appends a final instruction and makes one more call with no tools parameter at all:

resp = await llm.chat.completions.create(
    model=cfg["model"], messages=messages
)

The model physically can’t call another tool. It has to summarize what it has already gathered. The loop is guaranteed to terminate.

7. Tool failures crashing the run

Symptom: The MCP subprocess hiccups, the arguments aren’t valid JSON, or the server returns isError. Naive code raises an exception, and the whole investigation is lost.

Fix: None of these raise. Each one becomes a tool result string sent back to the model:

except Exception as exc:
    return f"Tool call failed ({name}): {exc}"

The model can then decide to retry, pick another tool, or explain the gap in its report. A human engineer does the same thing when kubectl logs fails: you note it, and move on.

The principles underneath

The LLM proposes, the harness disposes.

Step back from the individual fixes, and a few principles emerge. I think they apply to almost any production agent.

1. The LLM proposes, the harness disposes

The model is a reasoning engine, not an executor. Every action it proposes passes through deterministic checks — whitelist, schema, deduplication, grounding — before it touches anything real.

2. Every guardrail is a feedback message, not an exception

None of the guards above crash the run. Each one rejects the action and tells the model how to fix it. Errors become observations, and observations are what the model reasons over. This turns a fragile pipeline into a self-correcting one.

3. Bound every correction

Every pushback has a budget: one evidence pushback, MAX_REPEATS duplicates, MAX_STEPS steps, and a final tool-less summary. Self-correction without a bound is a liability. With a bound, termination is guaranteed by construction — not by hoping the model behaves.

4. Tool descriptions are prompts

This one is easy to overlook. The docstrings on the MCP tools steer the model’s strategy:

The response already includes this pod’s events, so do not call get_events for the same pod; use get_pod_logs next if it is CrashLoopBackOff.

Do not use this for ImagePullBackOff or Pending pods: their containers never started, so no logs exist.

Those two sentences teach the model to skip wasted calls. If you own the MCP server, the tool documentation is your highest-leverage prompt.

5. Observability is not optional

Every run writes the complete conversation — system prompt, every tool call, every result, the final answer — to a JSONL trace. When the agent reaches a strange conclusion, you replay exactly what it saw. Trace-writing itself never raises, because diagnostics must not break the thing they’re diagnosing.

Testing a non-deterministic system deterministically

How do you write reliable tests for something built on top of an unpredictable LLM?

You don’t test the model. You test the harness against a model that misbehaves on purpose.

The test suite spins up a real local HTTP server that impersonates an OpenAI-compatible API. Each test scripts a “bad model,” response by response. The MCP side stays fully real: calls go through the actual client to the actual mock server.

A scripted fake LLM over real HTTP, a real agent loop, and a real MCP server — no API key required.

def test_unknown_tool_is_rejected(fake_llm):
    def script(body, n):
        if n == 1:
            return _completion(tool_calls=[
                _tool_call("c1", "delete_pod", {"name": "x"})
            ])
        joined = " ".join(_tool_contents(body))
        assert "Unknown tool 'delete_pod'" in joined
        assert "list_pods" in joined  # the whitelist is offered back
        return _completion(
            content="That tool does not exist; sticking to the whitelist."
        )

There are six tests, one per failure mode:

1. forced first-step tool use
2. pushback on a zero-tool answer
3. schema rejection
4. duplicate-driven forced summary
5. invented-pod pushback
6. unknown-tool rejection

None of them needs an API key, and all of them run in CI in seconds.

This is the part I’d most encourage other agent builders to copy: each guardrail should have a test that proves it fires.

Why this matters beyond a demo

Faster incident response. A four-command investigation with manual correlation becomes one question. The agent follows the same path a senior SRE would: overview first, then the unhealthy pods, then logs or events depending on the failure type.

Reports you can trust and audit. Every conclusion cites its source tool, the grounding check filters invented resources, and the full trace is on disk. When the report says “missing DB_PASSWORD”, you can see the exact log line it read.

Safe by default. The agent only has read-only tools, and the whitelist means it cannot call an action it wasn’t given — however confidently it hallucinates one.

No model lock-in. Any OpenAI-compatible endpoint works: OpenAI, DeepSeek, Qwen, or a local Ollama model for teams that can’t send cluster data to a third party. No data leaves your network unless you want it to.

Small enough to own. About 350 lines of agent code and no framework. Your team can read it in an afternoon, audit it before a security review, and change it without fighting abstractions.

A pattern, not just a tool. Nothing in the guardrail layer is specific to Kubernetes. Swap the MCP server, and the same loop can diagnose databases, CI pipelines, or cloud billing anomalies. Any read-only investigation over structured tools fits.

What it doesn’t do (yet)

Being honest about limits is part of building something people can trust:

• The grounding check is narrow. It verifies pod names, not numbers, node names, or causal claims.
• The cluster is simulated by default. The mock makes the demo reproducible. A real cluster is one config line away, but expect messier data.
• Context grows with every step. For long investigations, history compression or summarization would be needed.
• It’s read-only. Remediation actions like restarts or rollbacks would require human-in-the-loop approval, and I deliberately haven’t built that yet.
• The tool list is simple. For example, if one container starts but fails to pass the health check (the process would be still running), the agent may not be able to detect it.

Next on the roadmap: schema-validated structured reports, an offline evaluation set built from saved traces, and approval-gated write actions.

Back to 3 a.m.

The pager still fires. But now the first thing you run is a single question — and in the time it takes to open your laptop, you have a report that names the broken pods, explains why each is broken, cites the evidence, and suggests a fix.

The LLM does the reasoning. A few hundred lines of deterministic code keep it honest.

rhuab / agent_loop_demo_for_kubernetes

agent_loop_demo_for_kubernetes

Diagnosing unhealthy Kubernetes Pods usually means manually chaining kubectl get pods, describe, logs, and get events until the root cause shows up. This project automates that investigation with a hand-written Agent Loop — no agent framework, just the low-level openai and mcp SDKs — so an LLM calls Kubernetes tools over MCP, digs in over multiple rounds on its own, and produces a structured cluster health report.

📦 Core Features

• Hand-written Agent Loop, zero framework — no LangChain / AutoGen. The core loop in run_agent() (agent.py) is a single for loop: send messages + tools to the LLM, execute any tool_calls via MCP, append results as role="tool" messages, and repeat until the model answers in plain text.
• Standard MCP tool calling — mcp_client.py launches the MCP Server as a stdio subprocess, converts MCP tool definitions into OpenAI's tools format (inputSchema → parameters)…

View on GitHub

Clone it, run python -m pytest with no API key, then point it at your own cluster. If it catches something interesting — or misbehaves in a way my guardrails don't cover — I'd love to hear about it.

Comments

[Agentic Affairs]

The invariant list is the real contribution — 'every tool_call gets exactly one matching tool message, the loop always terminates, never report a fact you didn't observe' is a spec most frameworks never state out loud. The MAX_REPEATS detail is the one I'd steal: stopping the investigation but forcing a summary instead of breaking keeps the conversation uncorrupted. Did the duplicate-call guardrail catch mostly genuine loops, or model retries after truncated tool output that actually deserved a second call?

[John Hu]

In this project, all results returned by the MCP are captured within the messages variable. Therefore, it is impossible for the results to be truncated.

[Agentic Affairs]

That settles it for this project — if truncation can't happen, a repeated call is a loop, not a retry that deserved a second chance. Is there any other legitimate reason for the same call twice here, or does that let the guardrail fire without exceptions?

[John Hu]

I agree with your assessment: within this loop, there is virtually no justification for a subsequent call with the same (tool, args) pair. The guardrail mechanism can therefore be designed under the assumption that such loops are never legitimate.
The full information is preserved within the model's context window. The model isn't "repeating" because its previous attempt was interrupted; rather, it’s stuck in a cycle.

Furthermore, other common rationales for "retrying" don't hold water here:

1. Retrying after an immediate tool failure: Since the failure state is already present in the context, a retry wouldn't even reach the MCP (Model Context Protocol) layer.
2. Forgetting the context: The result—associated with the same signature—is still stored in the context.

Consequently, these repeated calls are simply the model spinning its wheels, not genuine retries. MAX_REPEATS should be viewed merely as a tolerance threshold—providing the model with a few "nudges" before forcing a conclusion—rather than an acknowledgment that the repetition itself is valid.

[Agentic Affairs]

That reframes MAX_REPEATS exactly right — not a concession that repetition is valid, but a tolerance window before forcing the conclusion. If the repeats are never legitimate in this loop, though, what do the nudges buy: are they for the model, giving it a chance to self-correct, or for the reader, so the forced summary doesn't look abrupt? Either way, treating the threshold as a grace period rather than a retry budget is the cleaner mental model.