DEV Community: Pramoda Sahu

Understanding the Agent Loop: How Tool-Using LLM Systems Actually Work

Pramoda Sahu — Thu, 18 Jun 2026 10:21:25 +0000

If you are building with tool-calling models, the most important design decision is often not the prompt. It is the loop around the model.

An LLM can decide it wants to use a tool, but it cannot execute that tool by itself. The surrounding application or SDK has to assemble context, inspect the model response, run tools, append results, and continue until a final answer is produced. That runtime cycle is the agent loop.

This article explains what the agent loop actually is, where the model stops and the harness begins, how tool calling works step by step, and which engineering tradeoffs show up once you move beyond demos.

TL;DR

An agent loop is the execution cycle that lets a model inspect context, request tools, observe results, and continue until it reaches a final answer.
The model is only one part of the system. The harness or SDK owns orchestration: prompt assembly, tool execution, retries, approvals, and termination.
State management matters as much as prompting. If you lose prior tool outputs or conversation continuity, the agent will behave like it forgot what just happened.
Performance depends heavily on prompt growth control, stable prompt prefixes, caching, and bounded tool output.
Safe agent design requires validation, approval gates for side effects, and clear rules for concurrency and history propagation.

The Agent Loop Is the System, Not Just the Model

The core problem is simple: a one-shot model call cannot inspect the world, act on it, and adapt to the result unless something outside the model manages that cycle.

That is the harness's job.

OpenAI's Codex architecture describes a user interaction as a turn, but a single turn may contain multiple internal iterations of model inference and tool execution. The OpenAI Agents SDK describes the same idea directly: invoke the agent, check whether there is final output, handle handoffs if needed, otherwise execute tool calls and re-run.

A practical mental model looks like this:

Build the input state.
Call the model.
Inspect the response.
If the model requested tools, validate and execute them.
Append tool results back into context.
Call the model again.
Stop only when the model returns a final answer.

That means the harness, not the model alone, is responsible for:

Prompt assembly
Message history management
Tool schema registration
Tool execution
Validation and error handling
Retry logic
Approval workflows
State persistence
Loop termination

This is why two systems using the same model can behave very differently. Their harnesses may make different decisions about context, tool ordering, truncation, approvals, and continuation.

What Goes Into a Single Turn

Before the loop can run, the system needs to define what the model sees.

The input state

A typical turn includes:

System or developer instructions
Tool definitions or schemas
Previous messages
Previous tool-call results
The current user request
Sometimes environment state, session metadata, or hidden runtime instructions

This matters because follow-up reasoning depends on prior observations being present. If the model requested a tool in one iteration and the result is not added back correctly, the next iteration cannot build on that work.

Inner loop vs outer loop

There are really two loops to think about:

Inner loop: model inference and tool execution inside a single user turn
Outer loop: the broader multi-turn conversation across user follow-ups

This distinction shows up clearly in Codex-style architectures. A user asks for something once, but the agent may internally perform several tool steps before replying. Then the next user message arrives, and the entire conversation thread continues from that accumulated state.

That is why state continuity is not optional. Without it, the outer loop breaks and the inner loop starts reasoning from an incomplete view of reality.

How the Model Decides Between Text and a Tool Call

Once the harness provides the current turn state, the model has a decision boundary: answer directly, or request one or more tools.

Tool calling works because the model is given structured tool definitions. Instead of producing only natural language, it can emit a structured request indicating which tool it wants and which arguments it wants to pass.

At that point, the model is effectively yielding control back to the application.

With custom tools, the client harness must take over, run the tool, and return the result. With hosted tools, more of that orchestration can happen inside the API itself.

This is an important architectural choice:

Tool type	Who orchestrates execution?	Main tradeoff
Hosted tool	API/runtime handles more of the loop	Simpler orchestration, less direct control
Custom function tool	Client harness executes it	More flexibility, more operational responsibility
MCP tool	Depends on integration and discovery flow	Adds discovery and caching concerns

The advantage of client-side orchestration is control. The cost is that you now own the failure modes.

Tool Execution Mechanics in Practice

Once the model emits a tool request, the harness needs to do more than just run it.

Validate before execution

A safe harness should validate:

Tool name
Argument structure
Argument types
Permission rules
Whether the tool is read-only or mutating

This is not just a security concern. It is also a quality concern. If the model asks for a tool with invalid arguments, returning an explicit tool error often gives it enough signal to self-correct on the next loop iteration.

Return the observation in the right format

The model needs a structured observation that closes the action-observation cycle.

A minimal pattern looks like this:

response = client.responses.create(
    input=initial_question,
    **MODEL_DEFAULTS,
)

while True:
    function_responses = invoke_functions_from_response(response)

    if len(function_responses) == 0:
        print(response.output_text)
        break

    print("More reasoning required, continuing...")
    response = client.responses.create(
        input=function_responses,
        previous_response_id=response.id,
        **MODEL_DEFAULTS,
    )

The key detail is not just the loop itself. It is that the next request continues from the previous response and includes the tool outputs produced by the harness.

A more explicit observation payload looks like this:

context.append({
    "type": "function_call_output",
    "call_id": tool_call.call_id,
    "output": str(result),
})

response_2 = client.responses.create(
    model="o3",
    input=context,
    tools=tools,
    store=False,
    include=["reasoning.encrypted_content"],
)

print(response_2.output_text)

That function_call_output item is the observation that lets the model continue reasoning with the tool result now available in context.

State Management Patterns: Where Many Agents Fail

One of the easiest ways to break an agent is to lose state continuity.

Common state strategies

There are several patterns in current OpenAI tooling:

Full history replay managed by the client
previous_response_id for server-managed continuation
conversation_id for conversation continuity
SDK-managed session persistence

Each approach has tradeoffs.

Full replay vs server-managed continuation

With full replay, the client sends all prior messages and tool results every time. This is simple to reason about, but payload size grows quickly.

With server-managed continuation, the client can send the new input along with a continuation identifier such as previous_response_id. That reduces payload size and offloads some history management.

This example from the Agents SDK shows response chaining:

from agents import Agent, Runner

async def main():
    agent = Agent(name="Assistant", instructions="Reply very concisely.")
    previous_response_id = None

    while True:
        user_input = input("You: ")

        # Setting auto_previous_response_id=True enables response chaining
        # automatically for the first turn, even when there is no actual
        # previous response ID yet.
        result = await Runner.run(
            agent,
            user_input,
            previous_response_id=previous_response_id,
            auto_previous_response_id=True,
        )

        previous_response_id = result.last_response_id
        print(f"Assistant: {result.final_output}")

This is convenient, but you still need to choose a consistent state strategy.

Do not mix incompatible modes

The Agents SDK documentation explicitly warns against combining session persistence with conversation_id, previous_response_id, or auto_previous_response_id in the same run path.

That is a practical design rule: pick one continuity model per call flow.

If you mix them, debugging becomes much harder because it is no longer obvious which state the model is actually seeing.

Prompt Growth, Caching, and Why Stable Prefixes Matter

As the loop continues, context grows.

Every new model call may include prior instructions, tool schemas, user messages, and tool outputs. If you simply keep appending everything forever, the number of bytes sent over the lifetime of a conversation can grow quickly.

Why Codex emphasizes prompt prefixes

The Codex architecture discussion highlights a useful principle: keep old prompt content as an exact prefix of the new prompt whenever possible. That improves prompt-cache reuse.

In practical terms, stable ordering matters for:

System instructions
Tool definitions
Environment metadata
Prior messages

If these move around between calls, cacheability drops. The same issue affects reproducibility. Even tool-definition ordering bugs can introduce cache misses and inconsistent behavior.

Compaction strategies

A production harness usually needs some combination of:

Truncating verbose tool output
Summarizing old history
Keeping static instructions stable and early
Bounding shell or retrieval output
Preserving only the most relevant observations verbatim

This matters even more for shell, retrieval, or computer-use tasks, where output can become noisy very quickly.

The goal is not just lower cost. It is maintaining a usable reasoning substrate for the model.

Safety and Control in the Loop

The more powerful the tools, the more important the harness becomes.

Approval gates and side effects

Read-only tool calls are different from side-effectful operations.

For example:

Fetching documentation is relatively low risk
Sending an email, editing a file, or executing a deployment is high risk

Mutating actions should often be:

Serialized instead of run concurrently
Approval-gated
Sandboxed when possible
Logged with enough metadata for auditability

This is one reason agent frameworks expose concurrency settings and approval workflows.

Validate arguments, not intentions

You cannot safely assume that a tool request is correct just because it came from the model. Validate the arguments before execution, and return structured error feedback when something is wrong.

That gives the loop a chance to recover without silently doing the wrong thing.

Do not over-prompt reasoning models

OpenAI's function-calling guidance for reasoning models notes that you should not force extra "think more before every function call" prompting. Reasoning models already perform internal reasoning, and excessive prompting can degrade performance.

That is a useful reminder that harness quality is often more important than prompt verbosity.

Multi-Agent Extensions and Their Tradeoffs

Once a single-agent loop works, teams often add handoffs or agent-as-tool patterns.

Conceptually, the loop stays the same:

Invoke one agent.
Detect whether it produced final output, a tool request, or a handoff.
Route execution accordingly.
Continue until termination.

The Agents SDK summarizes the semantics clearly:

The agent will run in a loop until a final output is generated. The loop runs like so:

1. The agent is invoked with the given input.
2. If there is a final output (i.e. the agent produces something of type `agent.output_type`), the loop terminates.
3. If there's a handoff, we run the loop again, with the new agent.
4. Else, we run tool calls (if any), and re-run the loop.

The tricky part is not the idea of handoffs. It is history propagation.

Recent community discussions show that when one agent is exposed as a tool to another, developers are often unsure how much history is forwarded automatically. In practice, this means you should not assume that all relevant context follows the handoff unless your framework explicitly guarantees it.

For multi-agent systems, explicit context composition is often safer than implicit inheritance.

Common Failure Modes and Debugging Strategies

Most agent bugs look obvious in hindsight.

Failure mode 1: losing continuity

Symptoms:

The agent repeats itself
It forgets prior tool results
MCP tool discovery keeps happening again

Check whether you are correctly passing previous_response_id, conversation_id, or full message history.

Failure mode 2: context flooding

Symptoms:

Long, low-quality responses
Poor tool selection
The model misses relevant facts

Check whether tool output is too verbose. Cap output size, summarize logs, and keep only useful observations.

Failure mode 3: unstable prompt construction

Symptoms:

Cache misses
Inconsistent behavior across similar runs
Higher token usage than expected

Check the ordering of instructions, tool schemas, and environment metadata.

Failure mode 4: unsafe tool execution

Symptoms:

Invalid API calls
Accidental side effects
Hard-to-reproduce failures

Validate tool names and arguments before execution. Treat tool requests as proposals, not commands.

Failure mode 5: incorrect concurrency

Symptoms:

Race conditions
Conflicting writes
Non-deterministic outcomes

Run read-only operations concurrently only when safe. Serialize or approval-gate mutating operations.

Practical Architecture Takeaways

The recent OpenAI ecosystem changes make one thing clear: the important boundary is no longer just model prompting. It is orchestration design.

The Responses API, Agents SDK, MCP integrations, and Codex harness examples all point to the same execution model:

The model chooses actions
The harness controls reality
State continuity determines coherence
Prompt discipline determines scalability
Safety controls determine whether the system is usable in practice

If you are building an agent today, the fastest path to a better system is often not a new prompt. It is a better loop.

Key Takeaways

The agent loop is the action-observation cycle that makes tool-using LLM systems possible.
The harness owns orchestration: context assembly, tool execution, validation, retries, approvals, and termination.
State continuity is critical. Losing prior responses or tool outputs breaks reasoning quality quickly.
Server-managed continuation can simplify history handling, but you should choose one state strategy consistently.
Prompt growth is an engineering problem. Stable prefixes, truncation, compaction, and bounded tool output all matter.
Hosted tools and custom tools shift the orchestration boundary in different ways.
Multi-agent patterns introduce history propagation and control-flow complexity that should be designed explicitly.
Safe execution requires argument validation, side-effect controls, and careful concurrency handling.

Conclusion

An agent loop is not a small implementation detail. It is the core runtime pattern that turns a model into a working system.

Once you see the loop clearly, many design decisions make more sense: why history management matters, why tool output must be bounded, why prompt ordering affects cacheability, and why side effects need approval and validation.

If you are building with tool-calling models, make the loop explicit first. Define how state is carried forward, how tools are validated, how observations are appended, and how the run terminates. In practice, that foundation will usually improve reliability more than any prompt tweak.

Understanding Pi Coding Agent: A Minimal, Extensible Architecture for Terminal-First AI Coding Workflow

Pramoda Sahu — Thu, 18 Jun 2026 09:17:18 +0000

TL;DR

Pi Coding Agent is built as a layered TypeScript toolkit, not a sealed coding assistant product.
Its architecture separates provider access, agent runtime, coding workflow, and terminal UI into distinct packages.
Context engineering is a first-class feature through AGENTS.md, SYSTEM.md, APPEND_SYSTEM.md, skills, and extension hooks.
Pi can run interactively, headlessly over JSONL RPC, or be embedded through its SDK using the same underlying runtime.
The flexibility comes with tradeoffs: no built-in sandbox, strict RPC framing rules, and extension authors need to understand trust and compaction behavior.

Introduction

Most coding agents present themselves as finished products: you install them, learn their commands, and work within the boundaries the authors chose. That can be fine if the built-in workflow matches your needs. It becomes limiting when you want to change how prompts are assembled, how tools are registered, how sessions are summarized, or how the agent is embedded inside your own application.

Pi Coding Agent takes a different path.

Based on the official Pi homepage, documentation, and repository, Pi from Earendil Works is better understood as a minimal agent harness with a coding-oriented runtime than as a fixed end-user product. It ships with useful defaults, but its architecture assumes users may want to replace or extend large parts of the workflow. The project explicitly positions advanced behavior such as plan-like workflows, extra commands, and other higher-level capabilities as things that can live in extensions or packages instead of being hardcoded into the core.

That design choice matters for engineers building AI tooling. It affects maintainability, portability, and how easily the system can adapt to terminals, IDE wrappers, automation pipelines, or internal developer platforms.

In this article, we will look at how Pi is structured, why its layering matters, how its context pipeline works, and what tradeoffs appear once you start using extensions, RPC mode, or SDK embedding.

The problem Pi is trying to solve

A coding agent has to do several jobs at once:

Talk to one or more model providers.
Maintain an agent loop with tool calls and state.
Manage coding-specific concerns such as filesystem access, shell execution, session history, and context limits.
Provide a user interface or integration surface.

Many tools solve all four inside one tightly coupled application. That can make the initial experience simple, but it often makes customization expensive. If you want to change prompt composition or session summarization, you may end up forking the project or working against internal assumptions.

Pi’s architecture addresses this by splitting responsibilities into layers.

The Pi stack: four layers instead of one monolith

According to the repository README, Pi is organized as a monorepo with distinct packages:

@earendil-works/pi-ai
@earendil-works/pi-agent-core
@earendil-works/pi-coding-agent
@earendil-works/pi-tui

This package split is the clearest way to understand the system.

Layer 1: `pi-ai`

This is the provider abstraction layer. Its role is to present a unified interface across multiple model providers.

Why this layer exists:

The agent loop should not depend directly on one provider SDK.
Provider switching should not require rewriting the coding runtime.
Frontends and extension systems should remain provider-agnostic where possible.

This is a standard but important decision. If provider-specific details leak into higher layers, the whole system becomes harder to test and evolve.

Layer 2: `pi-agent-core`

This is the runtime layer for core agent behavior, including tool calling and state management.

Why this matters:

Tool execution is a runtime concern, not a terminal UI concern.
State transitions in the loop should be reusable in both CLI and embedded modes.
A headless integration should get the same agent behavior as the interactive one.

Architecturally, this is the part that keeps Pi from being “just a CLI.”

Layer 3: `pi-coding-agent`

This is where Pi becomes a coding agent rather than a generic agent harness.

This layer includes:

coding workflow behavior
sessions and persistence
built-in file and shell tools
compaction and summarization
extensions
skills
mode-specific runtime assembly

This package is the operational center of the project. It contains the logic that most users think of as “Pi,” while still remaining separable from the lower-level runtime and the higher-level UI.

Layer 4: `pi-tui`

This is the terminal UI layer.

Its presence as a distinct package is important because it suggests the user interface is not the agent itself. The same runtime can support different frontends.

That leads directly to one of Pi’s strongest architectural decisions: frontend/runtime separation.

One runtime, multiple modes

The official docs describe four major usage modes:

interactive
print/JSON
RPC
SDK embedding

That means Pi is not tied to its terminal interface, even if the terminal is the primary experience.

Interactive mode

This is the user-facing CLI workflow most people will start with. It combines the runtime with the terminal UI and built-in commands.

Print and JSON modes

These modes are useful for automation or simple scripting where you want structured output without a long-lived interactive session.

RPC mode

RPC mode exposes Pi through a JSONL protocol over stdin/stdout. This is the mode that makes IDE integrations, editor plugins, and service wrappers plausible without reimplementing the core runtime.

For example:

pi --mode rpc [options]

{"id": "req-1", "type": "prompt", "message": "Hello, world!"}

This is a strong design choice because subprocess embedding is often the easiest integration path for tools written in another language or running in another environment.

SDK mode

For Node.js and TypeScript applications, Pi can be embedded in-process through its SDK.

import {
  type CreateAgentSessionRuntimeFactory,
  createAgentSessionFromServices,
  createAgentSessionRuntime,
  createAgentSessionServices,
  getAgentDir,
  runRpcMode,
  SessionManager,
} from "@earendil-works/pi-coding-agent";

const createRuntime: CreateAgentSessionRuntimeFactory = async ({ cwd, sessionManager, sessionStartEvent }) => {
  const services = await createAgentSessionServices({ cwd });
  return {
    ...(await createAgentSessionFromServices({
      services,
      sessionManager,
      sessionStartEvent,
    })),
    services,
    diagnostics: services.diagnostics,
  };
};

const runtime = await createAgentSessionRuntime(createRuntime, {
  cwd: process.cwd(),
  agentDir: getAgentDir(),
  sessionManager: SessionManager.create(process.cwd()),
});

await runRpcMode(runtime);

This snippet shows the decomposition clearly: services, session manager, runtime creation, then a mode runner on top.

Core runtime flow: prompt, tools, persistence, compaction

For AI agents, architecture is really about workflow under constraints. Pi’s runtime appears to follow a loop like this:

Load startup context and trust-sensitive configuration.
Assemble the system prompt and working context.
Run extension hooks before the model call.
Send the provider request.
Receive model output, including possible tool calls.
Execute tool calls and attach results.
Repeat until the assistant completes.
Persist session entries.
Compact older context when token pressure increases.

The interesting part is that this pipeline is not fully hardcoded. The extension system lets you intercept multiple stages.

Extension hooks make the loop observable and adjustable

The extension docs describe lifecycle events around startup, provider requests, tool calls, compaction, tree navigation, and shutdown. Examples mentioned in the source material include:

session_start
before_agent_start
tool_call
before_provider_request
after_provider_response
session_before_compact
session_compact
session_before_tree
session_tree
session_shutdown

That event model suggests a publish/subscribe architecture around the core loop instead of a single monolithic pipeline. This is one of the biggest reasons Pi feels more like a toolkit than a product.

Context engineering is built into the architecture

A lot of agent systems treat prompt engineering as text pasted into a config file. Pi treats it as infrastructure.

According to the docs and homepage, Pi can load:

AGENTS.md and CLAUDE.md from user/global and project directories
SYSTEM.md to replace the default system prompt
APPEND_SYSTEM.md to append to it
skills loaded on demand
prompt templates
extension-provided prompt modifications
project trust state

This is not a minor convenience feature. It changes how the system is operated.

Why on-demand skills matter

Skills are loaded only when needed instead of always being included in the prompt. That helps avoid bloating context windows and prompt caches.

This is a practical tradeoff:

Always-loaded instructions are simpler.
On-demand loading is more efficient and gives finer control.

Pi chooses the second option, which fits its broader design: minimal default core, dynamic behavior at runtime.

Prompt customization through extensions

Pi also allows extensions to modify the assembled system prompt before model execution.

export default function promptCustomizer(pi: ExtensionAPI) {
  pi.on("before_agent_start", async (event) => {
    const { systemPrompt, systemPromptOptions } = event;
    const customPrompt = addToolGuidance(systemPromptOptions, systemPrompt);
    const appendSection = mergeWithUserAppend(systemPromptOptions);

    return {
      systemPrompt: `${customPrompt}${appendSection}`,
    };
  });
}

This is a strong example of Pi’s philosophy. Prompt composition is not just a file-loading step; it is part of the runtime and open to modification.

Sessions, JSONL persistence, and branching

Pi stores sessions in JSONL and supports commands such as /resume, /new, /tree, /fork, and /clone.

That combination implies that the session model is not a flat transcript. It supports branching workflows where a user can explore alternate paths.

Why JSONL is a sensible choice

JSONL is a practical format for agent session storage because it is:

append-friendly
easy to inspect
easy to process line by line
convenient for event-like histories

For terminal-first tools, that is often a better fit than requiring a heavier database.

Branching changes the context story

The source material notes that branch summarization is used when switching branches so that context from the abandoned branch can be injected into the new branch’s working context.

That matters because branching is not just a UI feature. It affects memory and continuity.

Pi also distinguishes between full history and in-memory working context. Compaction affects the latter, not the underlying stored session history. That is an important operational detail if you are debugging behavior or writing extensions that depend on prior entries.

Compaction is not just token trimming

Most agent systems eventually need summarization because context windows are finite. Pi exposes compaction as a visible architectural feature rather than hiding it as internal bookkeeping.

The docs describe two summarization mechanisms:

auto/manual compaction
branch summarization

They also define cut-point rules. For example, tool results must remain attached to their tool calls, so valid compaction boundaries are restricted.

That is exactly the kind of implementation detail extension authors need to know. If your extension assumes history can be split anywhere, you may break tool-call coherence.

Pi even allows custom compaction logic through hooks.

pi.on("session_before_compact", async (event, ctx) => {
  const { preparation, branchEntries, customInstructions, signal } = event;

  // Cancel:
  return { cancel: true };

  // Custom summary:
  return {
    compaction: {
      summary: "...",
      firstKeptEntryId: preparation.firstKeptEntryId,
      tokensBefore: preparation.tokensBefore,
    },
  };
});

This makes compaction a policy surface, not just an implementation detail.

Tradeoffs of customizable compaction

The flexibility is useful, but it increases the burden on extension authors.

You need to understand:

firstKeptEntryId
tokensBefore
serialized and truncated tool outputs
valid cut points
how repeated compactions relate to earlier kept boundaries

If you ignore those details, summaries may be technically valid but operationally misleading.

Extensions are the real center of Pi’s design

Pi’s homepage explicitly says it skips some built-in features and expects users to add them through extensions or packages. That is one of the most unusual and important aspects of the project.

Dynamic tool registration

Tools are not fixed at compile time. An extension can register them during session startup.

import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { Type } from "typebox";

const ECHO_PARAMS = Type.Object({
  message: Type.String({ description: "Message to echo" }),
});

export default function dynamicToolsExtension(pi: ExtensionAPI) {
  const registeredToolNames = new Set<string>();

  const registerEchoTool = (
    name: string,
    label: string,
    prefix: string,
  ): boolean => {
    if (registeredToolNames.has(name)) {
      return false;
    }

    registeredToolNames.add(name);

    pi.registerTool({
      name,
      label,
      description: `Echo a message with prefix: ${prefix}`,
      promptSnippet: `Echo back user-provided text with ${prefix.trim()} prefix`,
      promptGuidelines: [
        "Use echo_session when the user asks for exact echo output.",
      ],
      parameters: ECHO_PARAMS,
      async execute(_toolCallId, params) {
        return {
          content: [{ type: "text", text: `${prefix}${params.message}` }],
          details: { tool: name, prefix },
        };
      },
    });

    return true;
  };

  pi.on("session_start", (_event, ctx) => {
    registerEchoTool("echo_session", "Echo Session", "[session] ");
    ctx.ui.notify("Registered dynamic tool: echo_session", "info");
  });
}

This is a clear signal that Pi’s workflow surface is intended to be extended, not merely configured.

What extensions can change

Based on the provided material, extensions can influence:

commands
tools
provider request/response handling
prompt assembly
compaction behavior
tree navigation behavior
UI interactions
workflow logic around session lifecycle

That is unusually broad. It also explains why Pi can remain small at the core while still supporting highly specialized workflows.

Headless integrations: RPC mode and its sharp edges

RPC mode is one of Pi’s most practical features for teams building wrappers or custom frontends. But the protocol details matter.

The docs specify strict JSONL semantics with LF as the record delimiter.

The source material calls out a concrete gotcha: Node’s readline is not protocol-compliant for this use case because it can split on Unicode line separators such as U+2028 and U+2029, which are valid inside JSON strings.

That means a robust client should:

split records on \n only
accept optional \r\n by stripping the trailing \r
avoid generic line readers that reinterpret other Unicode characters as line boundaries

This is a good example of a small but important systems detail. If you are embedding Pi inside an editor extension or orchestrator, protocol correctness matters more than convenience.

Security and operational concerns

Pi’s flexibility does not remove operational risk.

No built-in sandbox

The repository README states that Pi does not provide a built-in permission system for filesystem, process, network, or credential access. It runs with the launching user’s permissions.

That has an obvious implication: if you need stronger isolation, you should containerize or otherwise sandbox it externally.

Trust model affects what loads

Before trust is granted, Pi loads only a subset of context and extension sources. According to the docs, project-local extensions, package-managed project extensions, and project settings are loaded only after trust resolution.

In non-interactive modes, trust prompts are not shown, so automation behavior depends on defaults or explicit CLI overrides.

If you are building tooling around Pi, document this clearly. Otherwise, a project may behave differently in interactive use versus CI-like or subprocess-driven environments.

Extension lifecycle resets on fork and clone

After /fork or /clone, Pi emits session_shutdown for the old extension instance, reloads and rebinds extensions, and then emits session_start for the new session.

That means in-memory extension state is not automatically preserved. If state matters, persist it into session entries or rebuild it during startup.

Why this architecture matters in practice

Pi’s design is especially useful when you need one of the following:

a terminal-first agent that is still scriptable
a reusable runtime for editor or service integration
custom prompt assembly without forking the core project
organization-specific commands, tools, or policies through extensions
session storage that is inspectable and easy to process

In other words, Pi is less about delivering one ideal workflow and more about providing a stable substrate for many workflows.

That is the real architectural difference.

Key takeaways

Pi is best understood as a layered toolkit for coding agents, not a fixed assistant product.
The package split separates providers, agent runtime, coding workflow, and terminal UI in a clean way.
Context engineering is deeply integrated through files, skills, prompt templates, and hooks.
Sessions are durable and branch-aware through JSONL persistence and summarization mechanisms.
Extensions are central to the design and can reshape tools, prompts, compaction, and workflow behavior.
RPC and SDK modes make the same runtime usable in terminals, subprocess integrations, and custom applications.
Operational safety is your responsibility: sandboxing, trust configuration, and extension-state handling all need deliberate design.

Conclusion

Pi Coding Agent stands out because it treats extensibility as the default architecture rather than an afterthought. The minimal core is not a limitation by accident; it is the mechanism that keeps the system adaptable.

That makes Pi especially interesting for engineers who want more than a terminal chatbot. If you need a coding agent that can be embedded, wrapped, or reshaped without forking the entire application, Pi’s layered design is worth studying.

The practical next step is to evaluate it in the mode closest to your real use case:

If you want a terminal workflow, start with interactive mode.
If you want editor or service integration, inspect RPC framing carefully.
If you want deep control over behavior, study the extension lifecycle and compaction hooks before writing custom logic.

In Pi, the architecture is the product.

DEV Community: Pramoda Sahu

Understanding the Agent Loop: How Tool-Using LLM Systems Actually Work

TL;DR

The Agent Loop Is the System, Not Just the Model

What Goes Into a Single Turn

The input state

Inner loop vs outer loop

How the Model Decides Between Text and a Tool Call

Tool Execution Mechanics in Practice

Validate before execution

Return the observation in the right format

State Management Patterns: Where Many Agents Fail

Common state strategies

Full replay vs server-managed continuation

Do not mix incompatible modes

Prompt Growth, Caching, and Why Stable Prefixes Matter

Why Codex emphasizes prompt prefixes

Compaction strategies

Safety and Control in the Loop

Approval gates and side effects

Validate arguments, not intentions

Do not over-prompt reasoning models

Multi-Agent Extensions and Their Tradeoffs

Common Failure Modes and Debugging Strategies

Failure mode 1: losing continuity

Failure mode 2: context flooding

Failure mode 3: unstable prompt construction

Failure mode 4: unsafe tool execution

Failure mode 5: incorrect concurrency

Practical Architecture Takeaways

Key Takeaways

Further Reading

Conclusion

Understanding Pi Coding Agent: A Minimal, Extensible Architecture for Terminal-First AI Coding Workflow

TL;DR

Introduction

The problem Pi is trying to solve

The Pi stack: four layers instead of one monolith

Layer 1: pi-ai

Layer 2: pi-agent-core

Layer 3: pi-coding-agent

Layer 4: pi-tui

One runtime, multiple modes

Interactive mode

Print and JSON modes

RPC mode

SDK mode

Core runtime flow: prompt, tools, persistence, compaction

Extension hooks make the loop observable and adjustable

Context engineering is built into the architecture

Why on-demand skills matter

Prompt customization through extensions

Sessions, JSONL persistence, and branching

Why JSONL is a sensible choice

Branching changes the context story

Compaction is not just token trimming

Tradeoffs of customizable compaction

Extensions are the real center of Pi’s design

Dynamic tool registration

What extensions can change

Headless integrations: RPC mode and its sharp edges

Security and operational concerns

No built-in sandbox

Trust model affects what loads

Extension lifecycle resets on fork and clone

Why this architecture matters in practice

Key takeaways

Conclusion

Layer 1: `pi-ai`

Layer 2: `pi-agent-core`

Layer 3: `pi-coding-agent`

Layer 4: `pi-tui`