DEV Community

Cover image for The Outer Agent Loop: How Engineers Are Building Production Agentic AI Systems in 2026
Manoranjan Rajguru
Manoranjan Rajguru

Posted on

The Outer Agent Loop: How Engineers Are Building Production Agentic AI Systems in 2026

Meta Description: Learn how senior engineers are building production-grade outer agent loops — the harness-level orchestration layer that wraps LLM coding agents like Claude Code. Explore architecture patterns, Python code, failure modes, and best practices for agentic AI systems in 2026.


Table of Contents

  1. The Loop That's Eating Software Engineering
  2. Two Loops, One System: Inner vs. Outer Agent Loops
  3. Anatomy of a Production Harness
  4. Code: Building Your First Outer Loop in Python
  5. The Completion Signal Problem: LLM-as-Judge
  6. Where Loops Win: Use Cases With High ROI
  7. Where Loops Fail: The Architecture Debt Trap
  8. Claude Tag and the Multiplayer Agent Paradigm
  9. Observability, Cost Bounds and Human Oversight
  10. The Cognitive Dependency Problem
  11. Designing Loops That Retain Engineering Sanity
  12. Conclusion: The Loop Is Coming. Build It Right.

The Loop That's Eating Software Engineering

"I don't prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops."
— Boris Cherny

That quote is not from a sci-fi novel. It's from a developer post that went viral in June 2026 — the same week Armin Ronacher (creator of Flask, Jinja2, and the orchestration platform Pi.dev) published a meditation titled "The Coming Loop" that racked up 349 upvotes and 244 comments on Hacker News in under 24 hours.

Something has shifted. Not the gradual, incremental kind of shift that comes with a new framework release. This is the tectonic kind — the kind where the job description changes before anyone updates the title.

The shift is this: the most productive AI-augmented engineers are no longer manually prompting language models. They are designing systems that prompt language models on their behalf, evaluate the results, and decide whether to continue, retry, branch, or escalate. They are writing harnesses. They are building loops.

And if you are not thinking about this architecture yet, you will be soon — because the discourse around "agentic loops" has moved from research papers to production dashboards faster than almost any previous AI pattern. Anthropic just announced that 65% of their own product team's code is written by an internal version of their new agentic system, Claude Tag. The Qwen team shipped Qwen-AgentWorld-397B, a 397-billion-parameter language model trained specifically to simulate agentic environments across 7 domains using over 10 million interaction trajectories. Teams are reporting 5x productivity gains. Codebases are growing that neither their authors nor any single human can fully explain.

This post is a deep technical guide to the outer agent loop — what it is, how to build it, where it wins, where it breaks, and how to retain engineering sanity as you ship it.

Outer Agent Loop Architecture — two nested glowing loop rings showing the inner agent loop nested inside the outer harness loop with task queue, judge, and human checkpoint nodes


Two Loops, One System: Inner vs. Outer Agent Loops

Before you can build a production harness, you need to understand the architectural distinction that almost every "agentic AI" discussion conflates: there are two loops, not one.

The Inner Loop

The inner loop is the one you're already familiar with if you've used Claude Code, GPT-4o with tools, or any modern LLM with function calling. It looks like this:

LLM receives message
  → calls a tool (read_file, run_tests, search_web)
  → receives tool result
  → calls another tool
  → ... (N times)
  → produces a final answer
  → loop exits
Enter fullscreen mode Exit fullscreen mode

This loop lives inside the model's context window for a single conversation turn. The model orchestrates it. It knows when it's "done" because it stops calling tools and emits a final response. There's no external orchestration — just the model, its tools, and its self-termination logic.

The inner loop is powerful. It's also fundamentally bounded by context window limits, attention drift over long conversations, and the model's own willingness to declare itself finished.

The Outer Loop

The outer loop — what the agentic engineering community is now calling "the harness loop" — lives outside the model's context. It is the code you write that wraps one or more inner loops. It decides:

  • Whether the inner loop's output is acceptable
  • Whether to inject follow-up messages into the same session
  • Whether to start a fresh session with modified context
  • Whether to fan out to parallel agents and merge results
  • Whether to escalate to a human checkpoint
  • When the overall task is truly complete
[Harness]
  → Dequeues task from task queue
  → Builds initial context (system prompt + task description + constraints)
  → Runs inner loop (one full LLM session)
  → Evaluates result with completion signal
  → If signal = CONTINUE: inject follow-up, re-run inner loop
  → If signal = RETRY: modify context, fresh inner loop
  → If signal = ESCALATE: push to human review queue
  → If signal = DONE: commit output, mark task complete
  → Dequeues next task
Enter fullscreen mode Exit fullscreen mode

The outer loop is where real agentic engineering happens. And it is, as Ronacher notes, simultaneously the most powerful and the most dangerous pattern in the current AI tooling landscape.

Side-by-side architecture diagram comparing the inner loop (single LLM session with tool calls) vs. the outer loop (harness with task queue, session manager, judge, and branching logic)


Anatomy of a Production Harness

A production-grade outer loop is not a while True: with an API call inside. It has at least six components that must be designed deliberately:

3.1 Task Queue

Tasks are the unit of work. A production harness dequeues tasks from a durable queue (Redis, SQS, a Postgres table with FOR UPDATE SKIP LOCKED) rather than accepting them inline. This gives you:

  • Persistence: tasks survive crashes
  • Deduplication: no duplicate work on retries
  • Prioritization: hot tasks jump the queue
  • Rate limiting: control API spend per time window

3.2 Context Builder

Each task needs a well-crafted initial context: a system prompt defining the agent's role, the task description, any relevant code or files, constraints on scope, and a description of the expected output format. The context builder is responsible for serializing all of this into the first message the model sees.

Bad context in = bad iterations out. Garbage in, garbage still in, but now with five retry cycles on top.

3.3 Session Manager

Session management tracks the state of a running conversation. It persists the message history so you can continue a session across outer loop iterations without losing context. This is critical for the "inject follow-up" pattern.

3.4 Completion Signal Engine

The most nuanced component. Given the model's output, produce a signal from the set {DONE, CONTINUE, RETRY, ESCALATE}. This can be:

  • Mechanical: run the test suite, check exit code
  • Syntactic: parse a structured JSON response the model was instructed to produce
  • Semantic: ask another LLM to judge quality (LLM-as-judge)
  • Hybrid: mechanical first, semantic fallback

3.5 Iteration Budget

Every harness needs hard limits: maximum number of iterations, maximum tokens consumed, maximum wall-clock time. Without these, a stuck loop burns money and blocks the queue.

3.6 Output Committer

When a task reaches DONE, the committer takes the final artifact and does something with it: opens a pull request, writes a file, pushes a message to Slack, updates a database record. The committer is the harness's boundary with the outside world.


Code: Building Your First Outer Loop in Python

Let's build a concrete, working outer loop harness using the Anthropic Python SDK. This example manages a code-generation task with automated test-based completion signals.

import anthropic
import subprocess
import json
from dataclasses import dataclass, field
from enum import Enum
from typing import Optional
import time

# ── Types ─────────────────────────────────────────────────────────────────────

class CompletionSignal(Enum):
    DONE = "done"
    CONTINUE = "continue"
    RETRY = "retry"
    ESCALATE = "escalate"

@dataclass
class Task:
    id: str
    description: str
    test_command: str          # Shell command to verify completion
    max_iterations: int = 8
    max_tokens_total: int = 80_000
    files_context: dict = field(default_factory=dict)  # filename -> content

@dataclass
class HarnessState:
    task: Task
    messages: list = field(default_factory=list)
    iteration: int = 0
    tokens_used: int = 0
    signal: CompletionSignal = CompletionSignal.CONTINUE

# ── Context Builder ───────────────────────────────────────────────────────────

def build_system_prompt(task: Task) -> str:
    """
    Construct the system prompt for the agent. This is your most
    important lever — invest heavily in its design.
    """
    files_block = ""
    if task.files_context:
        files_block = "\n\nRelevant files:\n"
        for fname, content in task.files_context.items():
            files_block += f"\n<file name='{fname}'>\n{content}\n</file>"

    return f"""You are an expert software engineer working autonomously.

Your task: {task.description}

Constraints:
- Write idiomatic, production-quality code with no unnecessary defensive fallbacks.
- Prefer making invalid states unrepresentable over handling every edge case.
- Do NOT add TODO comments — complete every implementation fully.
- When you are finished, respond with a JSON block: {{"status": "done", "summary": "..."}}
- If you need to ask a clarifying question you cannot answer yourself, respond with:
  {{"status": "escalate", "reason": "..."}}
{files_block}"""

# ── Completion Signal Engine ──────────────────────────────────────────────────

def run_tests(test_command: str) -> tuple[bool, str]:
    """Run the test suite. Returns (passed, output)."""
    result = subprocess.run(
        test_command,
        shell=True,
        capture_output=True,
        text=True,
        timeout=60
    )
    passed = result.returncode == 0
    output = result.stdout + result.stderr
    return passed, output

def evaluate_completion(state: HarnessState, last_response: str) -> CompletionSignal:
    """
    Multi-stage completion evaluation:
    1. Check for explicit structured status from the model
    2. Run the test suite mechanically
    3. Iteration / token budget enforcement
    """
    # Stage 1: Parse explicit model signal
    try:
        start = last_response.rfind('{"status"')
        if start != -1:
            json_str = last_response[start:]
            end = json_str.find('}') + 1
            parsed = json.loads(json_str[:end])
            if parsed.get("status") == "done":
                # Verify with tests before trusting self-report
                tests_passed, _ = run_tests(state.task.test_command)
                return CompletionSignal.DONE if tests_passed else CompletionSignal.CONTINUE
            elif parsed.get("status") == "escalate":
                return CompletionSignal.ESCALATE
    except (json.JSONDecodeError, ValueError):
        pass

    # Stage 2: Mechanical test verification
    tests_passed, test_output = run_tests(state.task.test_command)
    if tests_passed:
        return CompletionSignal.DONE

    # Stage 3: Budget enforcement
    if state.iteration >= state.task.max_iterations:
        return CompletionSignal.ESCALATE
    if state.tokens_used >= state.task.max_tokens_total:
        return CompletionSignal.ESCALATE

    return CompletionSignal.CONTINUE

# ── Core Harness Loop ─────────────────────────────────────────────────────────

def run_harness(task: Task) -> HarnessState:
    """
    The outer agent loop.

    Drives a full task to completion (or escalation), managing
    iterations, context injection, and completion signals.
    """
    client = anthropic.Anthropic()
    state = HarnessState(task=task)

    # Seed the conversation with the task
    state.messages = [{"role": "user", "content": task.description}]

    print(f"[Harness] Starting task: {task.id}")

    while state.signal == CompletionSignal.CONTINUE:
        state.iteration += 1
        print(f"[Harness] Iteration {state.iteration}/{task.max_iterations}")

        # ── Inner loop: one full LLM session ──────────────────────────────
        response = client.messages.create(
            model="claude-opus-4-5",
            max_tokens=4096,
            system=build_system_prompt(task),
            messages=state.messages,
        )

        assistant_text = response.content[0].text
        tokens_this_turn = response.usage.input_tokens + response.usage.output_tokens
        state.tokens_used += tokens_this_turn

        # Append assistant response to conversation history
        state.messages.append({"role": "assistant", "content": assistant_text})

        print(f"[Harness] Got response ({tokens_this_turn} tokens). Evaluating...")

        # ── Completion signal evaluation ───────────────────────────────────
        state.signal = evaluate_completion(state, assistant_text)

        if state.signal == CompletionSignal.CONTINUE:
            # Run tests to get feedback for next iteration
            _, test_output = run_tests(task.test_command)
            follow_up = (
                f"Tests still failing. Here is the test output:\n\n"
                f"```
{% endraw %}
\n{test_output[:3000]}\n
{% raw %}
```\n\n"
                f"Please analyze the failures and fix the implementation. "
                f"Remember: make invalid states unrepresentable, don't add fallbacks."
            )
            state.messages.append({"role": "user", "content": follow_up})

        elif state.signal == CompletionSignal.ESCALATE:
            print(f"[Harness] ESCALATING task {task.id} after "
                  f"{state.iteration} iterations ({state.tokens_used} tokens)")

        elif state.signal == CompletionSignal.DONE:
            print(f"[Harness] DONE: task {task.id} completed in "
                  f"{state.iteration} iteration(s), {state.tokens_used} total tokens")

        time.sleep(0.5)  # Rate limit courtesy pause

    return state


# ── Example Usage ─────────────────────────────────────────────────────────────

if __name__ == "__main__":
    task = Task(
        id="task-001",
        description="""
            Implement a Python function `parse_semver(version: str) -> tuple[int, int, int]`
            in the file `semver.py`. It should parse a semantic version string like
            '1.2.3' and return a tuple (major, minor, patch). Raise ValueError for
            invalid inputs. Do not use regex — use only string operations.
        """,
        test_command="python -m pytest tests/test_semver.py -q",
        max_iterations=6,
        files_context={
            "tests/test_semver.py": """
import pytest
from semver import parse_semver

def test_basic(): assert parse_semver("1.2.3") == (1, 2, 3)
def test_zeros(): assert parse_semver("0.0.0") == (0, 0, 0)
def test_large(): assert parse_semver("10.20.30") == (10, 20, 30)
def test_invalid_raises():
    with pytest.raises(ValueError): parse_semver("not.a.version")
def test_missing_patch_raises():
    with pytest.raises(ValueError): parse_semver("1.2")
"""
        }
    )

    final_state = run_harness(task)
    print(f"\nFinal signal: {final_state.signal.value}")
    print(f"Total iterations: {final_state.iteration}")
    print(f"Total tokens: {final_state.tokens_used}")
Enter fullscreen mode Exit fullscreen mode

A few key design decisions worth noting:

Test-first completion verification. Even when the model self-reports {"status": "done"}, we run the test suite before accepting it. Models are optimistic about their own work. Don't trust self-reports without mechanical verification.

Context injection over session restarts. When tests fail, we inject the failure output back into the same conversation rather than starting fresh. This preserves the model's understanding of what it already tried, avoiding redundant re-exploration.

Explicit budget enforcement. max_iterations and max_tokens_total are hard stops. When either is hit, the task escalates to a human queue — it does not silently fail or loop forever.


The Completion Signal Problem: LLM-as-Judge

Mechanical test runners work beautifully for tasks with verifiable outputs: unit tests pass or they don't. But a large category of valuable agentic tasks produce outputs that are correct-ish — hard to verify programmatically, but easy for an intelligent observer to assess. Code refactoring, documentation generation, architectural analysis, and PR descriptions all fall into this category.

For these tasks, the outer loop needs a semantic completion signal — and that's where LLM-as-judge enters the picture.

LLM-as-Judge architecture: primary agent generates output, judge model evaluates against rubric and returns DONE, CONTINUE, or ESCALATE verdict with specific issues

def llm_judge(
    client: anthropic.Anthropic,
    task_description: str,
    agent_output: str,
    rubric: str
) -> tuple[CompletionSignal, str]:
    """
    Use a fast, cheap model to judge whether the agent's output
    satisfies the task requirements according to a rubric.

    Returns (signal, reasoning).
    """
    judge_prompt = f"""You are a strict but fair code reviewer.

TASK DESCRIPTION:
{task_description}

EVALUATION RUBRIC:
{rubric}

AGENT OUTPUT TO EVALUATE:
{agent_output}

---
Evaluate the output against the rubric. Respond ONLY with valid JSON:
{{
  "verdict": "DONE" | "CONTINUE" | "ESCALATE",
  "score": <integer 1-10>,
  "reasoning": "<one paragraph explaining your verdict>",
  "specific_issues": ["<issue 1>", "<issue 2>"]  // empty list if DONE
}}"""

    response = client.messages.create(
        model="claude-haiku-4-5",   # Use fast/cheap model for judging
        max_tokens=512,
        messages=[{"role": "user", "content": judge_prompt}]
    )

    raw = response.content[0].text.strip()

    try:
        result = json.loads(raw)
        verdict_map = {
            "DONE": CompletionSignal.DONE,
            "CONTINUE": CompletionSignal.CONTINUE,
            "ESCALATE": CompletionSignal.ESCALATE,
        }
        signal = verdict_map.get(result["verdict"], CompletionSignal.ESCALATE)
        reasoning = result.get("reasoning", "No reasoning provided.")

        # Build actionable follow-up if continuing
        if signal == CompletionSignal.CONTINUE and result.get("specific_issues"):
            issues = "\n".join(f"- {i}" for i in result["specific_issues"])
            reasoning = f"Score: {result['score']}/10\n\nIssues to fix:\n{issues}"

        return signal, reasoning

    except (json.JSONDecodeError, KeyError):
        # If judge fails to produce valid JSON, escalate to human
        return CompletionSignal.ESCALATE, f"Judge produced invalid output: {raw[:200]}"
Enter fullscreen mode Exit fullscreen mode

Key observations about LLM-as-judge in practice:

Use a smaller model for judging. Your primary agent might use claude-opus-4-5 for reasoning depth. Your judge only needs to evaluate structured criteria — claude-haiku-4-5 handles this well at a fraction of the cost. This is asymmetric compute allocation: expensive model for generation, cheap model for evaluation.

The rubric is everything. Vague rubrics produce vague judgments. "Code quality is good" is not a rubric. "Functions are fewer than 30 lines, no except Exception, no hardcoded strings, all public functions have docstrings" — that is a rubric.

Beware sycophantic judges. LLMs have a known bias toward rating other LLM outputs favorably. Counter this by explicitly instructing the judge to "be strict," to "assume the output needs improvement unless all rubric criteria are demonstrably met," and to "give CONTINUE unless you are certain the output is complete."


Where Loops Win: Use Cases With High ROI

Not all tasks benefit equally from the outer loop pattern. Community experience — including Ronacher's analysis and the HN discussion — is converging on a set of domains where agentic loops deliver reliably high ROI:

Code Porting and Translation

Porting code between languages or frameworks is ideal for loops. The source code is the complete specification. The target is verifiable (tests pass, behavior matches). A real-world example: the reported migration of Bun's hot path from Zig to Rust, and Ronacher's own documented port of MiniJinja to Go. Both used harness-level loops to drive incremental translation verified by test suites. The loop runs until every test passes. There's no ambiguity about "done."

Performance Optimization and Benchmarking

A loop that tries optimization strategies, benchmarks each, and keeps only improvements is a natural fit. The signal is numeric: latency down, throughput up, memory reduced. The loop doesn't need to understand why a change works — it just needs to observe that it did. Iteration can be surprisingly creative here, surfacing non-obvious optimizations (cache alignment, SIMD usage, algorithm swaps) that a human wouldn't think to try in a time-boxed session.

Security Scanning and Fuzzing

Defenders increasingly need to loop because attackers already are. A harness can: generate candidate exploit inputs, run them against an interface, observe responses, and iteratively refine the attack surface — all without human intervention per cycle. The same pattern works for static analysis triage: loop over vulnerability reports, attempt reproduction, and auto-classify as "confirmed," "likely noise," or "needs human eyes."

Research Exploration and Hypothesis Testing

Give a loop a hypothesis ("Is there a performance regression in commit range X?"), a set of benchmarks, and access to git history. Watch it bisect, profile, and produce a structured report. The artifact is read-only (the harness doesn't commit code), so the risk surface is low and the signal is high. Many teams report dropping their initial research time from days to hours with this pattern.

Documentation Generation at Scale

Large codebases with thin documentation are a tractable loop problem. The harness enqueues every undocumented public function, the agent reads the implementation and writes a docstring, a judge evaluates clarity, and the result is committed. The loop runs until the queue is empty. Quality is imperfect — but an imperfect docstring beats no docstring for most codebases.


Where Loops Fail: The Architecture Debt Trap

Loops are not a universal accelerant. Ronacher's essay and the 244-comment HN thread identified a consistent failure mode that deserves serious attention: the architecture debt trap.

The Defensive Code Accumulation Problem

LLMs, as Andrej Karpathy observed, are "mortally terrified of exceptions." Given a test failure involving an unexpected input, a model's default response is to add a try/except, a None check, or a fallback branch. This is locally rational behavior — the specific test passes — but it is architecturally corrosive.

When a loop runs six iterations and each iteration adds two defensive branches, you get a codebase that is superficially more robust but fundamentally harder to reason about. The invariants that made the original system comprehensible have been papered over with fallbacks that obscure the real invariant violations rather than fixing them.

Ronacher's preferred alternative: make invalid states unrepresentable. Fix the type signature, add a validation layer at the boundary, eliminate the class of invalid inputs at the source. Without explicit constraint in your system prompt ("do not add fallback handlers — fix the root cause"), loops will systematically push your codebase toward defensive accumulation.

The Comprehension Cliff

There is a threshold beyond which no single human can load a codebase into working memory for effective maintenance. Loop-written code can cross this threshold in a single overnight run.

Ronacher describes encountering codebases that are "simultaneously useful and messy" — they pass their tests, they ship features, but they are only navigable by prompting the same class of model that wrote them.

The Judgment Atrophy Effect

A subtler failure mode: if developers use loops for everything, they lose the ability to reason about code they haven't written. The skill of close reading — understanding not just what code does but why it was written this way and what would break if it changed — requires regular exercise to maintain. Teams that wholesale outsource coding to loops are reporting declining code literacy in junior engineers who are now merging code they cannot explain.


Claude Tag and the Multiplayer Agent Paradigm

The outer loop pattern isn't just for individual engineers running overnight harnesses. Anthropic's freshly launched Claude Tag represents the next architectural evolution: team-level, multiplayer agentic systems.

Claude Tag lets Claude join a Slack workspace as a genuine team member. The architectural properties that make it interesting for engineers:

Channel-scoped identity with tool isolation. Each channel gets its own Claude identity with its own tool permissions, memory scope, and spending limits. A Claude in the #engineering channel with code repository access does not share context with a Claude in the #sales channel with CRM access. This is least-privilege applied to AI agents — a pattern that should inform how you design internal multi-agent systems.

Persistent, accumulating context. Unlike a stateless API call, Claude Tag builds an ongoing model of the channel's work: active projects, team members' roles, recurring patterns, and task history. This is the difference between a context window and a working memory. The architectural lesson: persistent agent state is a first-class concern, not an afterthought.

Async, long-horizon task execution. Delegate a task, Claude works on it over hours or days, and reports back in a Slack thread. This is the outer loop made socially native. The harness is Slack itself. The completion signal is a thread reply. The human oversight mechanism is the team's ability to @-reply with feedback.

Ambient proactivity. When enabled, Claude Tag flags relevant information without being asked — surfacing a PR that's blocking a critical path, noting that a metric crossed a threshold, following up on a thread that went quiet.

The 65% figure — 65% of Anthropic's product code authored by an internal version of Claude Tag — is the most significant production statistic in the current AI discourse.

Multiplayer AI agent architecture: central Claude node connected to isolated Slack channels (engineering, sales, support) with per-channel tool scopes, persistent memory, and async task delegation


Observability, Cost Bounds and Human Oversight

Building an outer loop without observability is like flying without instruments. Here is what a production harness must instrument:

Structured Logging Per Iteration

Every iteration should emit a structured log event capturing: task ID, iteration number, tokens consumed this turn, cumulative tokens, completion signal, judge score (if applicable), and timestamp.

import logging
import json
from datetime import datetime, timezone

logging.basicConfig(format='%(message)s', level=logging.INFO)
logger = logging.getLogger("harness")

def log_iteration(
    state: HarnessState,
    signal: CompletionSignal,
    judge_score: Optional[int] = None
):
    """
    Emit a structured log event for each harness iteration.
    Ship these to your log aggregator (Datadog, Grafana Loki, CloudWatch).
    """
    event = {
        "ts": datetime.now(timezone.utc).isoformat(),
        "event": "harness_iteration",
        "task_id": state.task.id,
        "iteration": state.iteration,
        "signal": signal.value,
        "tokens_used": state.tokens_used,
        "judge_score": judge_score,
        "budget_remaining_pct": round(
            (1 - state.tokens_used / state.task.max_tokens_total) * 100, 1
        ),
    }
    logger.info(json.dumps(event))

def log_task_complete(state: HarnessState):
    """Emit a task-level completion event for dashboarding."""
    event = {
        "ts": datetime.now(timezone.utc).isoformat(),
        "event": "harness_task_complete",
        "task_id": state.task.id,
        "final_signal": state.signal.value,
        "total_iterations": state.iteration,
        "total_tokens": state.tokens_used,
        # verify current pricing before using in production
        "estimated_cost_usd": state.tokens_used * 0.000015,
    }
    logger.info(json.dumps(event))
Enter fullscreen mode Exit fullscreen mode

Token Budgets as First-Class Config

Token budgets should be configurable per task type, not hardcoded globally:

# config/task_budgets.yaml
code_port: 120000
perf_benchmark: 80000
docstring_gen: 5000
security_scan: 200000
default: 40000
Enter fullscreen mode Exit fullscreen mode

Human Escalation Queues

Every ESCALATE signal needs a destination. Build a simple escalation queue that:

  1. Captures the full conversation history
  2. Provides context on why escalation occurred (budget exceeded? judge rejected?)
  3. Routes to the right human reviewer based on task type
  4. Provides a one-click interface to inject a resolution and resume the loop

Diff-Based Output Review

For harnesses that write code, never commit directly to main. Always:

  1. Commit to a feature branch
  2. Generate a human-readable diff summary using an LLM (yes, use the LLM to explain what it did)
  3. Require explicit human approval before merge

This is the minimum viable human-in-the-loop for code-writing agents. It is not optional.


The Cognitive Dependency Problem

Here is the most uncomfortable truth in this entire space, articulated with precision by Ronacher:

"We may create codebases that are not merely hard to maintain by humans, but that assume machine participation as part of their maintenance model."

This is already happening. Teams are merging code they cannot explain, diagnosing production incidents by feeding logs to LLMs rather than reading them, and writing specification documents by asking Claude to summarize what the system does — from the codebase that Claude itself wrote.

The loop is recursive. And each recursive layer adds distance between the humans nominally responsible for a system and their ability to actually understand or modify it.

The security implication is stark. If an attacker compromises an AI provider or injects malicious outputs into a loop, the blast radius is proportional to how much autonomy you've delegated. A harness that auto-merges to main and auto-deploys is a fully automated vulnerability injection pipeline if any component in the chain is compromised.

The business continuity implication is equally stark. What happens to your codebase if the model that understands it best becomes unavailable — due to export controls, pricing changes, API deprecation, or an outage? What if your team loses the last remaining ability to navigate the code without machine assistance?

This is not an argument against agentic loops. It is an argument for building them with honest accounting of the dependencies you are creating.


Designing Loops That Retain Engineering Sanity

Given everything above, here is a concrete set of design principles for building outer loops that are powerful without being corrosive:

Principle 1: Invariant-First System Prompts

Before the model writes a single line of code, your system prompt should articulate the invariants the system must maintain:

Invariants that must never be violated:
1. All user IDs are validated UUIDs before reaching the database layer.
2. All monetary amounts are stored as integers (cents), never floats.
3. Functions that return errors do so via Result<T, E>, never by returning None.
4. The 'events' table is append-only — never UPDATE or DELETE rows.
Enter fullscreen mode Exit fullscreen mode

Giving the model explicit invariants dramatically reduces defensive accumulation. The model stops papering over invariant violations because it's been told the invariant exists and must be preserved.

Principle 2: Mechanical Transforms Before Creative Generation

Lean on loops most heavily for mechanical translation tasks (where source code is the spec and tests are the oracle) and most lightly for greenfield creative generation (where neither spec nor oracle exists). The quality/cost curve is dramatically more favorable for mechanical transforms.

Principle 3: Bounded Autonomy Zones

Define explicit zones of autonomy. Within a zone, the loop runs freely. At zone boundaries, a human must explicitly approve continuation:

  • Zone: test files — loop can write and modify freely, auto-commit
  • Zone: implementation files — loop can write, must diff-review before commit
  • Zone: schema files — loop can propose, human must approve before any change
  • Zone: infrastructure code — loop cannot touch without explicit per-task human authorization

Principle 4: Legibility Budget

Allocate a portion of the loop's token budget specifically for legibility: generating commit messages that explain why not just what, updating inline comments, and producing a task-level summary that a human can read in under 2 minutes to understand what changed and why.

Principle 5: Decay the Loop's Influence Over Time

Files last modified more than 6 months ago require an explicit --force-edit flag before the loop is permitted to touch them. This prevents "archaeological" rewrites of stable, battle-tested code.


Conclusion: The Loop Is Coming. Build It Right.

The outer agent loop is not a productivity hack. It is an architectural primitive that is reshaping what it means to write software. Engineers who understand its mechanics — the distinction between inner and outer loops, the design of completion signals, the production requirements for harnesses, the failure modes of unconstrained loops — will be able to leverage it responsibly and effectively.

Those who treat it as a black box will build fast and break slow. Their codebases will accumulate defensive machinery nobody understands, their production incidents will become undiagnosable without AI assistance, and their systems will develop invisible dependencies on external services that could be revoked, rate-limited, or compromised.

The question, as Ronacher concluded, is not whether we will loop. Clearly we will. The question is: in a future of loops, how do we retain engineering judgment, preserve architectural sanity, and ensure that a responsible human can always understand, supervise, and if necessary, shut down what the machines are building?

That question is the most important engineering design problem of 2026. The patterns in this post are early answers. They will evolve. The engineers who engage seriously with the question — rather than racing toward maximum loop autonomy — are the ones who will build systems that remain comprehensible, maintainable, and trustworthy five years from now.

Build your loops. Build them deliberately. Build them with humans still in the picture.


Spotted a bug in the harness code? Have a production loop pattern you'd like to share? Drop it in the comments — this is one of those topics where community experience compounds faster than any single author can keep up with.


Originally researched and written on June 24, 2026. Sources: The Coming Loop by Armin Ronacher, Anthropic Claude Tag Launch, Qwen-AgentWorld Paper

Top comments (0)