Spend Intelligence Where the Work Is Uncertain

The context window is not an execution engine

An enterprise agent rarely fails because it cannot produce one good answer. It fails because the work around that answer is messy.

The agent has to search several systems, filter results, follow permissions, call another tool, retain intermediate data, compare documents, retry a failed request, and decide whether the result is complete. Each step adds tool output to the context. Some of that information is essential for reasoning. Much of it is only plumbing.

When all of this orchestration happens through repeated model calls, the context window becomes a temporary database, workflow engine, scratchpad, and decision log at the same time. That is an expensive and fragile design. The model spends attention restating state instead of interpreting it.

A better question is not “How do we make the model do more?” It is “Which parts of this task actually require intelligence?”

Use the model in proportion to the surprise

Large language models are useful when the path is unclear. They can infer intent, weigh tradeoffs, connect information that was not explicitly linked, and adapt when the original plan stops fitting the evidence.

They are less valuable when the next action is already known:

• iterate over a set of files;
• apply the same filter to every result;
• normalize data from different tools;
• preserve intermediate output;
• retry a request under a defined policy;
• branch on a deterministic condition;
• join records by an identifier;
• write a report to a known location.

Those operations do not disappear when an agent performs them. They move to a different execution layer. Code is usually a better place for repeatable mechanics because it is explicit, testable, inspectable, and able to operate on large outputs without replaying them through the model’s context.

The model should step in when interpretation is needed: which result is relevant, whether two findings describe the same underlying issue, what tradeoff a team should accept, or whether the evidence is sufficient to make a recommendation.

This division of labor is the core design principle: code handles predictable mechanics; the model handles judgment under uncertainty.

A coding harness changes the shape of the problem

The approach described in Glean’s coding harness follows this principle. Instead of asking the model to orchestrate every tool call directly, the model writes code in a sandbox. That code coordinates the workflow, while the filesystem stores complete outputs and intermediate state.

The distinction matters.

Imagine an agent investigating a production incident across a ticketing system, deployment history, service catalog, logs, and ownership data. A naive workflow might return every result to the model after each call. The model then has to remember what it has seen, decide what to keep, and generate the next request repeatedly.

With a coding harness, the agent can write a small program that:

1. queries each source;
2. saves raw responses to files;
3. extracts only fields needed for comparison;
4. retries failed calls according to policy;
5. produces an intermediate summary;
6. asks the model to interpret the compressed, high-signal result.

The raw evidence remains available. The model does not need to carry all of it in active context at every step.

This is not merely an optimization for token usage. It creates a clearer boundary between execution and reasoning. The workflow can be inspected independently from the model’s final interpretation. Intermediate artifacts can be reused, compared, or audited. A long-running task has somewhere to put its state besides a conversation transcript.

The post about Glean’s harness reports 24% fewer tokens per request with the same output quality. That figure belongs to that implementation and should not be treated as a universal benchmark. The more durable lesson is architectural: repeated orchestration has a cost, and moving deterministic work outside the context window can reduce unnecessary model involvement. The original post describes the approach and result.

Not every workflow should become an agent loop

There is a temptation to put a model in the center of every enterprise process. That usually creates a system that is difficult to reason about.

A workflow that can be represented as a deterministic pipeline should remain one. If the input, transformations, validation rules, and outputs are known, ordinary code is easier to operate. An agent becomes useful at the boundaries where the system encounters incomplete information or needs to choose among plausible paths.

For example, consider a repository assessment. The mechanical work includes enumerating files, parsing manifests, building dependency graphs, locating API routes, calculating churn, and identifying references between components. These tasks benefit from reproducible analyzers and explicit artifacts.

The judgment layer is different. Someone must determine whether a dependency boundary reflects a real domain boundary, whether a hotspot is strategically dangerous, whether a migration seam is safe enough, or whether an apparent test suite provides meaningful protection. Those questions require context and interpretation.

Putting both layers inside one opaque agent loop makes it harder to know what happened. Was a conclusion based on a measured property, a tool response, an assumption, or the model’s guess? Separating execution from judgment makes that distinction visible.

Files are more than temporary storage

The filesystem in a coding harness is useful because it gives the agent durable working memory with a simple interface. It can store large responses, generated indexes, transformed datasets, failed attempts, and summaries at different levels of detail.

That changes how context can be managed:

• raw material stays available without being repeatedly injected into prompts;
• derived artifacts can be regenerated from known inputs;
• intermediate state can be inspected after a failure;
• separate steps can communicate through explicit files;
• the final reasoning prompt can contain only the evidence relevant to the decision.

There is an important operational requirement here: state must be governed. A directory full of undocumented files is not automatically reliable. The workflow should define naming, ownership, retention, provenance, and the relationship between raw and derived artifacts.

The same applies to tool permissions. A sandbox that can search repositories, call internal services, and write files needs narrow credentials and clear boundaries. Moving orchestration into code does not remove security risk; it makes the execution layer more powerful. The harness must make network access, secrets, filesystem scope, and allowed tools explicit.

Design for evidence, not just completion

An agent can produce a plausible answer while hiding a weak process. For enterprise work, completion is not enough. A reviewer needs to know what supports the result and what remains uncertain.

A practical agent workflow should preserve at least four distinctions:

• observed facts: directly returned by a tool or found in a source;
• derived results: calculated from those facts by a deterministic step;
• human-provided context: information supplied by an owner or operator;
• model judgment: an interpretation, prioritization, or recommendation.

These categories should not be flattened into one narrative. If a workflow claims that a service owns a business capability, the repository may show code ownership, while the business meaning may still require confirmation. If a migration is described as low risk, the evidence may support the existence of a seam but not the operational impact of cutting it.

The agent should be able to say when the evidence is incomplete. That is a stronger design than forcing a confident answer for every question.

A practical architecture for agentic work

A robust enterprise agent often has five layers:

1. Connectors retrieve information from approved systems.
2. Deterministic processors filter, parse, normalize, calculate, and validate.
3. State and artifacts preserve raw inputs, intermediate results, and provenance.
4. Reasoning steps ask the model to interpret a constrained set of evidence.
5. Controls enforce permissions, budgets, retries, review gates, and audit trails.

The model may still write the orchestration code. But the surrounding system should treat that code as an executable artifact, not as an invisible side effect of a conversation.

Before introducing an agent loop, classify each step. Ask:

• Is the operation repeatable with the same inputs?
• Can it be expressed as a deterministic transformation?
• Does it need semantic interpretation, or only extraction?
• What evidence must survive for a reviewer?
• What happens when a tool returns incomplete or contradictory data?
• Can the step run without exposing all prior output to the model?

This classification usually reveals where intelligence is being overspent.

From execution to accountable decisions

The most useful enterprise agents will not be the ones that place a model in every step. They will be the ones that know where the model adds value and where it introduces unnecessary variability.

That means building workflows in which mechanical execution is explicit, state is durable, evidence is traceable, and uncertainty is allowed to remain visible. The model then receives a smaller but more meaningful slice of the problem: the part where context, tradeoffs, and judgment matter.

This principle also applies to software architecture work. ArchGenerator follows a related discipline in its architecture reports and SDD kits: repository findings are tied to concrete evidence, assumptions are labeled, and missing information is not presented as certainty. The point is not to replace engineering judgment, but to give that judgment a cleaner foundation before work begins.

Whether the execution layer is a coding harness, a build pipeline, or an architecture analysis workflow, the design question is the same: let predictable work run predictably, and reserve intelligence for the moments that contain real surprise.