The real estate operations team had been chasing the same bug for three days. Their lease extraction agent was silently skipping certain clause types. The fix, once found, took two hours. Finding it took three days.
A new paper from Tencent and Indiana University finally names this pattern: behavior localization - the process of identifying every code location that implements the behavior you need to change. It is the hidden bottleneck in every production agent system, and it scales in cost with the complexity of your harness.
The Core Problem
Production agent harnesses are behaviorally distributed. A single observable behavior - how the agent handles a timeout, or parses a specific document format - may be implemented across several functions, multiple files, and different execution stages, linked by shared state that is never visible in one place.
Repositories are organized by files and modules. Modification requests describe behaviors. The gap between those two views is where engineering time disappears.
The paper proposes Harness Handbook: a behavior-centric representation of the codebase, built automatically via static analysis and LLM-assisted structuring, that maps what the harness does to where it is implemented.
Three-Level Progressive Disclosure
The Handbook is organized as a hierarchy:
- L1 (System Overview) - architecture, execution model, major stages, global data flow
- L2 (Component Overview) - per-stage responsibilities, inputs, outputs, dependencies
- L3 (Unit Deep Dive) - source-grounded implementation entries, each linked to a validated live code location
Every L3 entry must resolve to a real location in the current source. If the code changes and a locator no longer points anywhere, it is frozen and excluded from localization results until refreshed. The repository is always authoritative.
Behavior-Guided Progressive Disclosure (BGPD)
The Handbook enables a workflow the paper calls BGPD: instead of grep-ing the repository and guessing, a developer or coding agent starts at L1, descends to the relevant stage at L2, follows links to L3 source entries, and verifies those entries against live code before planning any edit.
Evaluated on realistic modification requests across two open-source harnesses, Handbook-Assisted planning improved behavior localization accuracy and edit-plan quality - while using fewer tokens at the planner. The largest gains appeared on cross-module changes, rarely-executed paths, and behaviors distributed across nonadjacent locations.
Why This Matters for PropTech
Production real estate platforms run complex harnesses - document extraction, compliance enforcement, tenant communications, maintenance dispatch - coordinated across dozens of files and execution stages.
When a CRM API changes, a new lease format appears, or a regulatory requirement shifts, the first challenge is always finding every place in the harness that touches the affected behavior. Harness Handbook makes that navigation automatic and verifiable.
Model swaps. A cheaper model replaces the current one. The Handbook surfaces every location where prompts, context windows, or tool-call formats need to adapt.
API evolution. An external integration changes its response format. The behavior "call this system and parse its response" may span an adapter, retry policy, schema validator, and error handler - in four files. The Handbook finds all four before the first edit.
Regulatory changes. New compliance requirements touch verification flows, data retention, logging, and output formatting across different execution stages. Missing one is a compliance failure.
Coding agent delegation. As teams delegate harness edits to coding agents, the quality of behavior localization determines whether the agent's edit plan covers all affected locations. A Handbook-equipped coding agent produces more complete plans with fewer wasted searches.
The Honest Caveats
Construction has a real cost - static analysis plus LLM inference. The paper argues it is recovered through token savings at edit time, but it is upfront investment.
The Handbook must stay current. Stale L3 entries with frozen locators give incomplete results, which may be worse than no Handbook if developers trust it uncritically. Resynchronization triggers on every non-empty diff.
Behavior localization is necessary but not sufficient. Better planning does not fix a bad agent or a broken evaluation gate.
Connection to Self-Improving Harnesses
Harness Handbook is the navigation layer that makes harness evolution loops practical. Before you can improve a harness - whether through Self-Harness-style automated editing or manual review - you need to find where the target behavior actually lives. This paper solves that prerequisite.
The production development loop becomes: observe failures ? localize behavior via Handbook ? propose targeted edits ? validate against held-out benchmark before shipping.
The Takeaway
The quality of your harness documentation is not a nice-to-have. It is the bottleneck that decides how quickly your system can adapt and how safely your coding agents can evolve it.
Originally published at vsebude.it
Top comments (0)