This digest covers what's new for AI agent builders between October 13 and October 20, 2026: fresh papers on harness self-improvement and verification, production write-ups on agent memory and evals, and a couple of infrastructure stories about what happens when agents, not humans, become the primary users of your tooling.
🔥 Highlights
Redesigning source control at Anthropic — agents now out-commit humans 30,000 to one daily.
Context Engineering with Deep Agents: How To Remember Without Keeping Everything In Context — a concrete playbook for long-running agent memory.
The Most Common Questions About Agent Evals, Answered — the FAQ every agent team eventually needs.
Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents — where frontier labs actually draw the autonomy line.
Self-Evolving Coding Agents via Harness-Policy Optimization — the agent now rewrites its own scaffold.
arXiv
- Learning to Steer LLM Agents and Multi-Agent Systems — 2026-10-15 Introduces a "steering agent" that watches single- and multi-agent executions during long-horizon tasks and intervenes when it detects drift from the goal, splitting the job into a cheap component that decides when to intervene and a more expensive one that decides how, trained jointly via RL. It generalizes to out-of-domain tasks and unseen executors without retraining to tune intervention frequency — useful if you're running long agentic loops that suffer from goal drift.
Hugging Face Daily Papers
Evidence-First Agentic Verification — 2026-10-14
Presents Sauron, a verifier for long, semi-verifiable tasks that checks every claim — including an agent's own self-reported success — against source evidence before trusting it, using bounded queries and joint investigation of related criteria to save context. Across 2,586 human-labeled criteria it posts the best F1 among five verifiers while costing less than the strongest baseline — a real option for teams that need something sturdier than loose "LLM-as-judge" checks.Interlock: From Local Correctness to Task Completion in Long-Horizon Agents — 2026-10-15
Tackles the pattern of agents that satisfy every individual requirement yet lose the connection between evidence, decision, and downstream effect, producing output that's locally correct but incomplete. It represents the task as a typed completeness graph, compiles that into a resettable workspace plus evaluation rubric, and uses an agent-as-judge to filter trajectories; fine-tuning on the accepted trajectories improves pass rate on automation and legal/operations benchmarks.Co-Evolving Acting and Forecasting for Language Agents — 2026-10-16
Introduces CoPE: a single model that acts and also learns to predict the future action sequence from any point in successful trajectories, using that forecast to shape the policy through shared parameters — at no extra inference cost, since the agent still acts step by step. It beats GRPO and observation-prediction baselines across five benchmarks (ALFWorld, ScienceWorld, WebShop, τ²-bench, AppWorld), with up to ~2x better sample efficiency on ScienceWorld — relevant if you train agents with RL on sparse, end-of-trajectory reward.RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent — 2026-10-16
A recursive self-improvement framework where a meta-agent diagnoses the performance bottleneck in a task agent's trajectories and picks which improvement workflow to apply (model, harness, or artifacts), distilling the results into reusable skills that feed back into orchestration. It reports state-of-the-art results across six benchmarks (code generation, tool use, search QA) with Qwen3-4B/8B.Self-Evolving Coding Agents via Harness-Policy Optimization — 2026-10-19
Proposes Self-HPO: the coding agent alternates between proposing changes to its own harness (the scaffold controlling tools, context, and feedback) based on analysis of its own failures, and training the policy via RL under that chosen harness — tasks that still fail feed the next round of search. It lifts average resolve rate from 57.8% to 64.8% with Qwen3.5-35B-A3B, with gains that generalize to out-of-distribution benchmarks.HarnessPro: Recursive Harness Self-Improvement with Pareto Optimization for Performance-Cost Trade-offs — 2026-10-20
A label-free method for evolving an agent's harness using specialized agents that analyze past trajectories, propose edits, and score candidates with LLM-generated tests — producing three harness variants (lite/medium/heavy) for different cost budgets. In heavy mode, with GPT-5.5/Codex on Terminal-Bench 2, SWE-Bench Pro, and ProgramBench, it gains 2 percentage points of pass rate while cutting per-task cost by 26.1% versus the RHO baseline.OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners — 2026-10-20
An agent-evaluation benchmark built from 31 real consulting engagements (totaling $3.68M) in operations research, paired with a simulated client (ORSim) that only reveals business facts when asked and has no technical expertise — testing whether the agent can formulate the problem, not just solve it. The best configuration (Claude Code) hits 88.39% strict feasibility, but no model matches the human expert on overall quality — a good reference for evaluating agents on open-ended reasoning domains beyond coding.
Anthropic Engineering Blog
- Redesigning source control at Anthropic — 2026-10-13 Anthropic replaced the monorepo's git setup with an in-house system, Scion, built on the premise that most clients — agents and CI jobs — are always connected: commits land as server-side changesets, checkouts are sparse/virtualized, and a layered cache replaces hundreds of git mirrors. The practical point for agent builders is the programmatic version-control API — agents can query history, blame, and merge base without a local clone, enabling automated bisect/revert pipelines at scale (the stated trigger was growing from ~2,500 to 30,000+ merges/day).
OpenAI news
Nothing confirmed this week — openai.com/news and several openai.com/index/... URLs returned HTTP 403 on every fetch attempt during research, so no title/URL/date could be verified directly against the source, and per the no-guessing rule nothing is included.
LangChain / LangGraph blog
Context Engineering with Deep Agents: How To Remember Without Keeping Everything In Context — 2026-10-14
Explains how the open-source Deep Agents harness avoids losing precision on long tasks without flooding the context window: it offloads large tool results to files, summarizes old messages, loads skills on demand, isolates noisy work in subagents, and keeps long-term memory in plain files. Because memory lives in user-controlled storage, it survives a model or provider swap — useful if you're designing the memory layer for a production agent.The Most Common Questions About Agent Evals, Answered — 2026-10-15
A technical FAQ on evaluating LLM agents: building datasets from production traces, the difference between offline experiments and online evaluators on live traffic, when to use code-based evaluators vs. LLM-judges vs. human review vs. pairwise comparison, multi-turn thread evaluation, and containerized benchmarks via Harbor that score an agent's final state. A direct reference if you're building out an agent's eval pipeline.How Numan Built an Eval-First AI Health Coach with LangGraph and LangSmith — 2026-10-19
Numan's "Nu" health coach is a LangGraph graph where a supervisor dispatches to specialist subagents, each with its own prompt and tools; guard nodes validate every response against clinical safety rules and send unsafe drafts back for regeneration, with a safe fallback if that fails repeatedly. Evals run multi-turn simulations of the same multi-agent graph scored by category-specific LLM judges — a solid case study in multi-agent orchestration plus eval-first design for sensitive domains.How Orange Business Built Live Intelligence with LangChain and LangSmith — 2026-10-13
Orange Business built its "Live Intelligence Studio" on LangChain (for model neutrality and control over agent architecture) with LangSmith for tracing/eval. Notable pieces: an MCP Registry for governed, auditable agent access to enterprise tools (M365, CRMs, internal APIs), and an Agent Registry for versioning and reusing agents/prompts, with multi-agent flows inspectable via per-step execution trees (input/output/latency).Search agent traces by behavior with AI Search in LangSmith — 2026-10-13
A new LangSmith feature (AI Search) lets you search agent traces by describing behavior in natural language (e.g. "frustrated users" or "leaked PII") instead of relying on tags or full-text search. Under the hood, SmithDB condenses each session into a deduplicated "trajectory" within a fixed token budget, and a fast decision model filters candidates in parallel — relevant if you need to debug or evaluate agent behavior in production.
Latent Space
Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents — 2026-10-14
Recaps an AI Engineer NYC panel with OpenAI, Anthropic, Two Sigma, and Bloomberg on deploying agents in finance. The core argument: safe autonomy is about where you place controls and human judgment, not the model itself — scoped identities, human approval gates for consequential actions, classifiers for ambiguous cases, plus out-of-model safeguards like audit logs and kill switches. A concrete, layered-control checklist for anyone shipping agents into a high-stakes, regulated domain.Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — 2026-10-13
Interview with Jason Ma (Dyna Robotics) on Dyna-2, a world-action model trained on over a million hours of human video that predicts future video frames and derives robot actions from that prediction. The notable control-loop detail: an agentic harness handles high-level decision-making and planning so the low-level policy can run long multi-step workflows (like an hour-long laundry task) while still doing ~100-dimension continuous action control for smooth motion — a useful read on hierarchical planning loops outside the chat-agent context.
Simon Willison
A quote from Geoffrey Huntley — 2026-10-13
A short, concrete trick for enforcing tool-use policy on a coding agent: instead of relying on instructions in AGENTS.md (which drift), add a pre-commit hook that scans the diff for calls bypassing your approved library, fails the commit, and feeds a corrective prompt back to the agent. Reported as getting compliance to "ten out of ten" — a directly reusable pattern for fighting tool-call drift in coding agents.Redesigning source control at Anthropic — 2026-10-13
Simon's own writeup linking to Anthropic's Scion post, pulling out that Claude is Scion's heaviest user (which is why offline support got deprioritized) and that checkouts increasingly originate from agent sandboxes. He also flags the piece's description of adversarial review — one model hunting for flaws in another's work — as effective but prone to pushing toward over-engineering, a useful caution for anyone using agent-vs-agent review as a control pattern.A quote from Addy Osmani, Morgan Westlee Lunt, and Qing Wang — 2026-10-16
A pull-quote from "How the Claude Code team does code review," arguing that whether a human read a diff is a weak proxy for risk, and that review is shifting from a final PR gate to a continuous process where Claude reviews like "spellcheck while you write," passing most changes before a human ever looks. A practical framing for agent builders: decide per-check whether an automated agent or a human catches a given class of problem best, and save human attention for higher-level judgment calls.
Through-line
Three independent threads converge on the same question this week: how do you trust what an agent just did? The harness-self-improvement cluster (RSIFlow, Self-HPO, HarnessPro) treats the agent's own scaffold as a thing to be evolved based on failure analysis, not just the model inside it. The verification papers (Sauron, Interlock) push back against taking an agent's self-reported success at face value, insisting on evidence-first checks instead. And the production write-ups — LangChain's eval FAQ, Numan's guard-node health coach, LangSmith's behavior search — all describe evals and tracing as the thing standing between "the agent did something" and "the agent did the right thing." Anthropic's own infrastructure story, covered independently by both its engineering blog and Simon Willison, is the starkest illustration: once agents outnumber humans 30,000 to one as commit authors, the tooling question stops being about developer experience and becomes about verification at scale.
What's catching your attention in agent evals or harness design this week? Drop a comment below.
Top comments (0)