DEV Community: agents

Inside Zambo: 138+ Tools, a Receipt for Every Call, and Memory That Follows You Across AIs

rambo — Wed, 14 Oct 2026 14:59:04 +0000

Most AI still works like this: you type a job, the model talks back, and you take its word for what happened. It cannot browse for you, build for you, check a price, scan a contract, or prove any of it afterward. Zambo exists to end that arrangement. It is a single MCP server at https://zambo.dev/mcp that any AI client can connect to, and behind that one URL sits a working stack: 138+ tools, a verifiable receipt minted for every call, and a continuity layer that lets work started in one AI continue in another. This is the deep tour: what is actually in there, how the receipt system works down to the hash, and how the memory pieces fit together.

One URL, and the AI can search the toolbox itself

The connection is one URL: https://zambo.dev/mcp. Claude, ChatGPT, Gemini, Cursor, Grok, Windsurf, VS Code, LangChain, CrewAI, Muse: if the client speaks MCP, it can connect. Three entry tools matter more than any others.

capability_search searches the whole catalog by use case and returns the relevant tools with relevance scores. This is the answer to the most common failure mode of a big toolbox, which is not that the tool is missing but that nobody finds it. An agent that knows what it wants to do can ask what exists, in plain language, and pull the right tool itself.

zambo_universal and zambo_invoke route natural language and exact tool names across the full catalog through one dispatch surface, so a client with a small default tool list can still reach everything behind it.

zambo_help and zambo_ask answer questions about the stack itself: products, pricing, routing, how to do a specific thing. The stack explains itself, in band, while you use it.

The toolbox, family by family

Making things. video_create generates video through the Zambo video engine, and video_status reports what the engine can honestly do and where a generation stands. gif_animate turns a description into a cinematic animated GIF. meme_forge builds a meme image with your caption. image_generate produces AI images, and image_analyze reads images back: description, OCR text extraction, and screenshot review, including reading error messages off a screen. voice_transcribe turns audio into text. qr_generate makes QR codes for URLs, text, WiFi credentials, and contact cards.

Knowing things. live_price returns real time cryptocurrency prices. market_pulse aggregates a live intelligence snapshot across AI, agents, and research. doc_intel extracts structured intelligence from documents: contracts, reports, PDFs, whitepapers. pdf_extract pulls clean text from any public URL. agent_brief generates a machine readable briefing on any URL, product, company, API, or codebase. weather and translate cover the everyday basics, live weather for any city and translation across 100+ languages, no API key required from you.

Judgment. zambo_score grades any startup, idea, repo, or business from 0 to 100 with a letter grade and the bottleneck named. model_oracle ranks AI models for your specific task against your token budget and cost limits. zambot_fix diagnoses broken AI generated code and names the exact failure mode. code_review returns severity rated issues, security flags, and specific fixes. provibe_audit audits a public repo and scores it. ghost_audit_site runs the Achilles 10 stage audit on a website: SEO gaps, AI discoverability, conversion leaks, brand issues, with ghost_audit_report returning the full findings.

Safety and trust checks. prompt_shield detects prompt injection, jailbreak, and policy bypass attempts and returns a risk score. scam_check answers "should I interact with this?" for a wallet, contract, domain, or URL. email_verify parses SPF, DKIM, and DMARC to check whether an email is really from who it claims. structlock takes messy AI output or raw text and enforces your schema on it, returning clean typed JSON every time. sybil_detect, basehawk_score, and monad_score score wallet risk and reputation from live chain intelligence.

Real work. write produces publication quality content of any type. email_write handles everything from a pitch to a negotiation. contract_scan reads a contract or terms of service and flags the red flags in plain English. bill_audit finds errors and overcharges in bills and subscriptions. resume_boost tailors a resume to a job description. salary_script builds a full negotiation script with rebuttals. sql_from_english turns plain English questions into working SQL. regex_build and git_changelog handle the small sharp jobs developers hit daily. thread_craft turns an idea into a ready X thread. fridge_forge builds a dinner plan from whatever is in your fridge, meal_plan builds a full plan with macros and a shopping list, and workout_build builds a training plan with progression.

Trading, with the math outside the model. trading_strategy_compile converts a natural language strategy into a strict versioned schema. trading_risk_gate is a pure math position check: no AI in it, so no hallucination is possible, hard enforcing position size against your limits. trading_propose runs the full flow: load the strategy, run the risk gate, capture live market data. trading_journal and trading_stats keep the record and the scoreboard.

Agents finding and hiring agents. find_agent searches the Zambo Agent Registry, a public phone book for AI agents, by capability or name. signal_register and signal_lookup give an agent a persistent identity and a trust passport. agent_hire is an x402 marketplace where agents browse, hire, and list each other. zambo_swarm spawns a coordinated multi agent swarm around a goal with a division of labor. agent_network coordinates permissioned rooms across humans and agents. The coordination boards go one step further: coord_claim lets an agent claim a task, and coord_complete only completes it after the supplied Zambo receipt verifies. The receipt is the key that closes the task.

Strategy. zambot_spark runs the Zambo Agent Council on any goal: five specialist agents arguing from biology, game theory, military strategy, and more. zambot_swarm_debate stress tests a decision. zambot_drift detects the silent divergence between what an agent was built to do and what it is actually doing. zambo_compose chains the intelligence tools into one autopilot run. proof_certify certifies any AI generated content with a SHA-256 provenance certificate.

Every call leaves a receipt

Here is the part that makes the stack different from a pile of tools. Every call through Zambo mints a receipt at the moment it runs: the tool, the arguments it was given, the exact result bytes that came back, the timestamp, and digests that let anyone recompute the commitments from the saved record. The format is AER-1, an IETF Internet-Draft currently at revision 19 (https://datatracker.ietf.org/doc/draft-zambo-aer1/19/), written by Zambo's creator, with independent implementations from other builders and a public conformance kit.

The claim is narrow on purpose, and it is worth stating exactly. A receipt proves the saved result was not changed since it was saved. It does not prove the result was correct, and it says nothing about quality or intent. That narrowness is the feature: it is a claim that can be checked mechanically, by a stranger, without trusting Zambo or the agent.

Check one yourself. This receipt records a live_price call from the public demo: https://zambo.dev/run/92874efd-7bc4-427c-b262-862dfcf6d82d. Open it, take the saved output bytes, recompute the SHA-256 digest, and compare it to the digest on the record: sha256:65faa90980caf70dfb5fc14f3c38c82db8a154ce3a6e2d274e943b83209b1d06. It matches. Whether that Bitcoin price was right is a question the receipt deliberately does not answer. Run the demo and mint your own: https://zambo.dev/demo/

Receipts compose. A session's receipts accumulate under one job, and workflow_close snapshots the eligible receipts into an immutable record whose root is a Merkle tree computed over the step receipt identifiers, exactly as AER-1 Section 8.1 specifies, so any single step can be checked against the root of the whole job. workflow_seal seals a closed workflow for durable export. journal_log and secondary_receipt go the other direction: they let work performed outside Zambo be logged and receipted too, with the evidence scored conservatively and the basis stated on the record. On the coordination boards, on the marketplace, and anywhere else work changes hands, the receipt is what travels: small, checkable, and readable by any party with no account and no permission.

Continuity: the work follows you across AIs

Tools answer "what can my AI do." Continuity answers "where did my work go." Start a job in Claude on your laptop, continue it in ChatGPT on your phone, verify the record in Cursor: the session, the memory, and the receipt timeline are the same underneath, because they live in Zambo, not in any one client.

Sessions. new_session starts a fresh context and returns a session id that subsequent calls reuse. zambo_session loads a session back, resuming your exact context from any client or device.

Memory. zambo_remember stores information in your private Zambo memory, permanently, across every session, device, and AI client. zambo_recall retrieves it from anywhere. The AI you switch to next week knows what the AI you used today was told, because the memory was never the client's to lose.

Threads and the receipt timeline. continuity_start_thread opens a named thread under a job. continuity_get_recent reads the private receipt timeline for a job or thread, continuity_list_threads and continuity_open_thread navigate them, and continuity_set_prefs controls what gets captured. The thread id can be passed as the session id by another AI client, which means a second AI can open the exact thread the first one was working in and read the receipted history of what already happened, instead of a summary it has to trust.

Identity. agent_identity establishes a persistent agent identity that survives client switches and model updates, and the Signal registry ties claimed credentials and skills to it. An agent that did good work last month under one client is recognizably the same agent under another.

Access, and agents paying for themselves

Every tool carries a free tier with no account, and the limits vary by tool. A Day Pass unlocks the full stack for 24 hours for $0.99: humans check out by card or PayPal, and agents pay on their own through x402, USDC on Base, with day_pass_activate handling the agent native flow and zambo_pay receiving the x402 challenge. Beyond the day: Zambo Pass is $29 a month or $290 a year, and $19 a month for $ZAMBO holders. zambo_limit shows exactly where your usage stands at any moment, and reliability_score publishes machine verified pass rates and latency for the tools themselves. The stack receipts its own work and scores its own reliability in public.

There is also a Receipt Bot for Grok (https://zambo.dev/receipt-bot/) that collects, verifies, and files a verifiable receipt for every job your bots do, then sends a nightly digest of links. Same idea, one more surface: the work leaves a record wherever it happens.

Why this shape

A tool stack makes an AI capable for a session. Receipts make its work checkable by anyone, forever, in a format an open Internet-Draft defines and independent builders implement. Continuity makes the capability and the record survive the switch from one AI to the next, which is how people actually work now: whichever model is open, whichever device is in hand. Put the three together and "my AI said it did the work" stops being the end of the conversation. The work has a toolbox, the toolbox leaves receipts, and the receipts follow you.

Your AI said "done." For $0.99, prove it: https://zambo.dev/day-pass/

Escalate on a budget, not an error: when an AI agent should hand off to a human

TaskPool AI — Wed, 14 Oct 2026 14:38:42 +0000

A retry is not a fresh draw, and waiting for an exception means waiting past the point where anything was recoverable. Four counters you can read before the failure, a handoff packet, and Python that runs.

Mercor's best agent on APEX-Agents passes 24.0% of its tasks on one attempt. Give it eight attempts and it reaches about 40%. A flat 24% chance per attempt would put eight tries near 89%, so most of those failures belong to the task and not to the dice. That inference is mine. Nothing I read in Mercor's write-up says it, and you are free to argue with it.

The usual answer to a failed run is another run, often resumed from the same conversation. Sinha and colleagues found that models make more mistakes when their own earlier mistakes sit in the context, and that scaling the model does not remove the effect, although thinking models mitigate it. Their tasks were synthetic, with the plan handed to the model, so carrying the result over to a production retry is my extension and not theirs.

That leaves a move most harnesses never budget for: stopping on purpose and giving the work to someone else. The rest of this post covers the trigger logic and the packet, and then what it costs. If you came for code, skip to the section called "Counters that fire before the exception".

Read the stop reason first

I'm putting the objection ahead of the fix. A lot of what looks like an agent giving up is a harness limit doing exactly what it was configured to do, and handing that to a person means paying a stranger to find your bug. I wrote up six causes of agents stopping mid-task on the Taskpool blog, and five of them live in the harness. Run that triage before you wire up anything below.

Counters that fire before the exception

The class below watches four things per tool call: an identical call repeating inside a short window, steps since the last verified checkpoint, total steps, and wall-clock time. It returns a reason string the moment one trips and None otherwise.

from __future__ import annotations

import hashlib
import json
import time
from collections import deque
from dataclasses import dataclass
from typing import Callable, Optional


@dataclass
class Budget:
    max_steps: int           # set from your own successful runs, not a default
    max_seconds: float
    max_repeats: int = 3     # identical tool call (name + args) this many times...
    window: int = 8          # ...inside the last N calls
    stall_steps: int = 12    # steps allowed without a verified checkpoint


class Tripwire:
    def __init__(self, budget: Budget, clock: Callable[[], float] = time.monotonic):
        self.b = budget
        self.clock = clock
        self.t0 = clock()
        self.steps = 0
        self.since_checkpoint = 0
        self.recent: deque[str] = deque(maxlen=budget.window)

    def observe(self, tool: str, args: dict, checkpoint_advanced: bool = False) -> Optional[str]:
        """Call once per tool call. Returns a reason string when it is time to hand off."""
        self.steps += 1
        self.since_checkpoint = 0 if checkpoint_advanced else self.since_checkpoint + 1

        blob = json.dumps([tool, args], sort_keys=True, default=str).encode()
        fp = hashlib.sha256(blob).hexdigest()[:12]
        self.recent.append(fp)

        if self.recent.count(fp) >= self.b.max_repeats:
            return "repeated_action"
        if self.since_checkpoint >= self.b.stall_steps:
            return "checkpoint_stall"
        if self.steps > self.b.max_steps:
            return "step_budget"
        if self.clock() - self.t0 > self.b.max_seconds:
            return "time_budget"
        return None


@dataclass
class Handoff:
    goal: str                # one sentence, readable by someone with no context
    reason: str              # which tripwire fired
    last_verified: str       # last checkpoint that passed its check
    tried: list[str]         # short descriptions, not the transcript
    ask: str                 # the single thing the human has to do
    accept_if: str           # how you will judge what comes back
    deadline_hours: int
    spend_cap_pence: int     # integer minor units, never floats

The thresholds are defaults I picked so the tests had something to assert. They are not measured. max_steps is the one that matters, and I'd start from the longest successful run you have on file, add slack, and then look at which counter actually fires.

The checkpoint counter is the best signal and the most expensive to earn. A checkpoint here means a state something outside the model confirmed, such as a passing test or a row that exists in the database. If you cannot say what a checkpoint is for your task, stall detection is not available to you yet, and defining it will take longer than writing the class.

Arguments are serialised with sorted keys before hashing, so {"a": 1, "b": 2} and {"b": 2, "a": 1} count as the same call. Without that, an agent can loop forever while shuffling its arguments.

What the human is handed

The transcript stays out. It carries the model's own errors, which is what the Sinha result says to keep away from the next attempt, and it costs a person real time to read. The Handoff dataclass above is the replacement: a goal a stranger can follow, which counter fired, the last checkpoint that passed, short descriptions of what was tried, one ask, and a deadline.

Two fields do more than they look like. accept_if forces you to write the check before the work leaves, which is the point where it is cheapest to be strict. spend_cap_pence is an integer because money held as a float eventually produces a bug report.

Wiring it to LangGraph takes one node. Its interrupt docs require a checkpointer and a thread_id, and resuming goes through Command(resume=...).

from dataclasses import asdict
from langgraph.types import interrupt

def escalate(state):
    packet = Handoff(
        goal="Confirm the shop sign matches the listing",
        reason=state["reason"],
        last_verified="listing page parsed",
        tried=["fetched listing twice", "street view unavailable"],
        ask="Photograph the shopfront and sign",
        accept_if="Sign text legible in at least one photo",
        deadline_hours=48,
        spend_cap_pence=1500,
    )
    # Nothing with side effects above this line.
    answer = interrupt(asdict(packet))
    return {"human_result": answer}

The comment is not decoration. The docs say the node restarts from its first line when you resume, so anything above interrupt() runs twice, and they warn against wrapping the call in a bare try/except because the pause is implemented as an exception. I ran this round trip against langgraph 1.2.15 with the in-memory checkpointer and it passes, along with five tests for the counters. In production the checkpointer has to outlive your process, because a person who answers in two days will not be answering a process that is still running.

Not every handoff goes to a stranger

The run is stuck because	Hand it to
The next step needs a decision only your team has authority to make	The on-call engineer or the account owner
The output exists but needs judgement you already employ	An in-house reviewer
The next step needs a person in a particular place, or an independent check of the agent's own claim	An outside verified human

Only the last row is a job for a marketplace. Sending the first two to a stranger gets you an answer from someone with no authority and no context.

If the destination is a job board

I co-founded Taskpool International, which runs a marketplace where AI agents post tasks for verified humans, so weight this section accordingly. The Python above is vendor-neutral and I have not wired it to any marketplace in this post. What follows comes from reading Taskpool's agent docs, not from a run, and each point applies to any board you might use.

Listings are public. In Taskpool's case the task description appears on the public job board, so the ask field must never carry credentials or customer data. Plan for a stranger to read it.

Handoffs have a floor price. The docs set a minimum payout of £5.00 per worker and a 10% platform fee on top, so the cheapest single slot costs £5.50. That number changes which runs are worth escalating at all.

Silence can pay. The docs say a submission still in review seven days after the worker's deadline is accepted automatically and paid. Run your accept_if check from a webhook, not from someone remembering.

A loop that escalates forever is a retry loop that sends you invoices. Cap spend per key as well as per task; Taskpool's docs describe a monthly budget per API key that refuses funding before any money moves.

What to change on Monday

Log the stop reason on every model call before you theorise. It costs you some log volume.
Pick one verified checkpoint per task type. If the task has none, accept that stall detection is off and say so in the runbook.
Set max_steps from your own successful runs and log which counter fires. You will pay for handoffs on runs that would have finished, and nobody has published how often.
Write accept_if before the work leaves, in a form code can check.
Strip secrets from the packet and assume a stranger can read it.
Cap spend per task and per key.

If you want the longer argument for why task length rather than difficulty sets the wall, I made it with the benchmark numbers in Duration, not difficulty, is what breaks agents.

The number I don't have

I know of no published distribution of why production agent runs get handed off, or of how often a budget like this fires on a run that would have succeeded. That false-positive rate decides whether any of this is worth the money, and I have none. If you have traces, I would rather have your numbers than your agreement.

FAQ

When should an AI agent escalate to a human? When a counter trips, or when the next step needs a person on site or an independent check that the agent cannot supply. Do it before the failure, while the state is still recoverable.

Is retrying a failed agent run a good recovery strategy? Not by default. Models can condition on their own earlier errors, an effect thinking models mitigate and scale alone does not remove, so a fresh run seeded with a clean summary is the safer retry.

How does LangGraph pause an agent for human input? Call interrupt() inside a node, with a checkpointer and a thread_id configured. Resume the same thread with Command(resume=...).

I used Claude to draft this post. Every figure links to its source, checked on 14 October 2026, and the Python was run: six tests pass.

The sparse-reward excuse is an authoring problem

Aamer Mihaysi — Wed, 14 Oct 2026 14:37:48 +0000

Every eval conversation I've had this year hits the same wall. Someone wants to measure an agent on a long-horizon task — 40, 60, 100 steps — and someone else says you can't score that densely. Grading every step means a human or a judge model reading the whole trajectory, and that's expensive, slow, and noisy. So you run the task, check the final assertion, and get one bit back. Pass or fail.

I've built evals that way. They're cheap and they're nearly useless. When the agent fails at step 47, you have a terminal error and a transcript, and you spend the afternoon reading it like a detective. The signal you needed was at step 12, and you didn't collect it because collecting it looked expensive.

Here's what I've started to believe: dense scoring was never the expensive part. Authoring was. We've been blaming the wrong cost.

Think about what dense reward actually requires. To say "step 14 was progress," something has to know what progress means at step 14. If all you have is a task prompt and a final assertion, nothing knows that. So you hand the trajectory to a judge model and ask it to guess. That's expensive, and it's guessing, because the judge has no ground truth either. It's reading the same transcript you are, with the same missing information, and charging you per token for the privilege.

Tev makes the other move, and it's the one I keep coming back to. Instead of scoring the trajectory after the fact, you write the hidden ground-truth world spec first — the actual state of the world, the preconditions, the milestones, the terminal conditions — and then you render the environment from that spec. The environment isn't hand-built and then described in a doc that drifts. The spec is the source, and the environment is a projection of it. Once that's true, progress at any step is a function of the spec. You don't judge it. You compute it. Dense per-step rewards fall out as a byproduct, because you already wrote down what "closer" means.

Paper's here if you want the actual method: https://arxiv.org/abs/2610.18868v1

The reason this lands for me isn't the benchmark numbers. It's that it's the same discipline I keep failing to apply in production. I run long-lived agents. They make tool calls, they hold state, they run for minutes at a time. When one goes sideways, my instrumentation tells me it errored. It does not tell me it was drifting at step 9 and unrecoverable by step 20. I've got logs, I've got traces, and I've got no notion of expected state to compare against, so all of it is description, not diagnosis. I can tell you exactly what the agent did. I can't tell you where it went wrong.

The fix isn't a smarter judge model watching the agent in real time. I tried that. It's slow, it costs per step, and it's confidently wrong in exactly the places I need it to be right — the ambiguous middle, where the agent is doing something plausible but off-spec. The fix is writing down the world spec: what the state should look like, which invariants should hold, what a milestone looks like. Then checking observed state against it. That check is cheap. It's a comparison, not an inference. And it gives me a progress trace instead of a postmortem.

So the myth I want to bust is this: dense agent evals are too expensive to run. They're not. They're too expensive to improvise. If you don't have a spec, every step is a judgment call, and judgment calls don't scale — not in money, and not in consistency. If you do have a spec, every step is a lookup.

What I'd steal from Tev, concretely:

Write the world spec before the task prompt. State, invariants, milestones, terminal conditions. If you can't write it, you don't understand the task well enough to eval it, and no amount of judge model will fix that.
Render the environment from the spec. One source of truth. Hand-built environments drift from their descriptions, and then your rewards are scoring a world that doesn't exist.
Emit progress as a byproduct, not a second pass. If scoring is a separate pipeline, it rots the moment the environment changes. If it's a projection of the spec, it stays in sync for free.
In prod, make the same spec your instrumentation contract. Every tool call gets checked against expected state. You get a curve, not a vibe.

Where I'm skeptical: rendering from a spec is only as good as the spec. Under-specify it and you get dense rewards that are confidently wrong, and an agent that optimizes straight into the gap between your spec and reality. That's worse than sparse, because sparse failure is at least honest — it doesn't tell you you're winning. Also, spec authoring is real work. You've moved the cost, not deleted it. But moving it from per-run scoring to one-time authoring is the right trade, because the spec is reusable, reviewable, and versionable. A wrong spec is a bug you can fix. A noisy judge is fog you can't.

The other thing dense rewards buy you is debuggability. Sparse gives you pass/fail and a feeling. Dense gives you a curve, and the curve tells you where the agent stalls. That's the actual product. Not the score — the shape.

Next time someone tells you long-horizon agent evals have to be sparse because dense scoring is too expensive, ask them for the world spec. If they don't have one, that's the cost they're actually paying. They just haven't itemized it.

I finally cut AI context bloat by 80% and the fix was embarrassingly simple

Lars Winstand — Wed, 14 Oct 2026 14:12:07 +0000

A few months ago I watched an agent do something deeply cursed.

It had already fetched the data.
It had already parsed the data.
It had already stored the useful fields.

And then it pasted the entire raw tool response into the next model call anyway.

Full JSON blobs. Logs. Metadata. Useless fields. Everything.

The next step got slower.
The step after that got more expensive.
Then the model started doing that annoying thing where it answers the easy part of the task and quietly ignores the important part.

That was the moment the bug became obvious:

Most “context window problems” are not model problems.
They are state management problems.

Once you see that, you start spotting it everywhere in n8n flows, LangGraph agents, MCP tool loops, and custom OpenAI-compatible orchestration code.

The actual fix

Stop replaying raw tool output into every prompt.

Keep full results in structured state outside the model.
Send only:

a short summary
the exact fields needed for the next step
or the exact evidence slice if verification matters

That one change usually does more for agent quality than switching from one frontier model to another.

The anti-pattern

Here’s the bad version:

result = weather_api.get_forecast("Austin")

messages.append({
    "role": "user",
    "content": f"Use this API response to decide whether to send an alert:\n{result}"
})

If result contains 200 lines of hourly forecast data, provider metadata, timestamps, debug fields, and trace IDs, you just turned your prompt into a junk drawer.

Now the model has to:

find the relevant fields
ignore the irrelevant ones
remember prior steps
plan the next action
not get distracted by noise

Models can do this.
They are just not especially efficient at it.

The better pattern

Store the full result in app state.
Pass only the fields the next step needs.

result = weather_api.get_forecast("Austin")
state["weather_raw"] = result

weather_summary = {
    "temperature_f": result["current"]["temp_f"],
    "conditions": result["current"]["condition"],
    "wind_alert": result["alerts"].get("wind", False)
}

messages.append({
    "role": "user",
    "content": f"Decide whether to send an alert using this data: {weather_summary}"
})

Same workflow.
Smaller prompt.
Less distraction.
Better odds the model focuses on the decision instead of re-parsing junk.

Prompt size is not the same as prompt quality

A lot of builders confuse these two ideas:

the model can fit a long context
the model should receive a long context

Those are not the same.

A giant prompt is often just a badly managed state store wearing a trench coat.

Even when GPT-5, Claude Opus, Grok, Qwen, or Llama can technically accept the full history, performance still gets mushier as stale content piles up. Latency rises. Cost rises. Attention gets diffused.

This is why long-running agents often feel smart for 5 steps and weirdly flaky by step 20.

The 3 patterns, ranked

Approach	What happens in practice
Raw tool output in prompt	Biggest prompt growth. Highest distraction. Latency and cost climb every step.
Summarized tool result in prompt	Usually the best balance. Smaller prompt footprint, but bad summaries can hide edge cases.
Structured state outside prompt	Best default for multi-step agents. Small recurring prompts, cleaner reasoning, better orchestration.

My opinionated take: structured state outside the prompt should be the default.

If you are building anything more serious than a toy bot, replaying raw tool output into every step is lazy architecture.

What should live where?

This is the mental model I wish more agent builders started with:

Short-term conversational state stays attached to the thread.
Long-term facts and tool results live outside the prompt in a database or memory store.
The next model call gets only the smallest useful slice of that state.

That’s it.

Not complicated. Just disciplined.

Real examples

LangGraph: checkpoint state, don’t replay transcripts

LangGraph gets this right.

Instead of stuffing previous tool output back into every prompt, persist graph state and resume by thread.

from langgraph.checkpoint.postgres import PostgresSaver

DB_URI = "postgresql://postgres:postgres@localhost:5432/postgres?sslmode=disable"

with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
    graph = builder.compile(checkpointer=checkpointer)

That is the grown-up version of memory.

The model gets what the next node needs.
Postgres keeps the rest.

n8n: trim memory on purpose

n8n is refreshingly practical here.

If you are building AI workflows in n8n, you already have the tools to avoid context bloat:

Simple Memory
Redis
Postgres
Xata
Zep
Motorhead
Chat Memory Manager

The existence of Chat Memory Manager should tell you everything.

If a workflow platform ships a node specifically to inspect and reduce memory, then “just append to chat history forever” is obviously not the best pattern.

MCP: tool output is app state first, prompt material second

MCP makes this cleaner because tools already have structure.

Example tool definition:

{
  "name": "get_weather",
  "description": "Get current weather information for a location",
  "inputSchema": {
    "type": "object",
    "properties": {
      "location": { "type": "string" }
    },
    "required": ["location"]
  },
  "outputSchema": {
    "type": "object"
  }
}

That should push you toward a simple pattern:

call the tool
store the full structured result
pass only the needed fields into the next prompt

For example:

tool_result = {
    "temperature": "72F",
    "conditions": "cloudy",
    "wind_alert": False,
    "hourly": [...],
    "provider_metadata": {...},
    "trace_id": "abc123"
}

state["weather_result"] = tool_result

prompt_payload = {
    "temperature": tool_result["temperature"],
    "conditions": tool_result["conditions"],
    "wind_alert": tool_result["wind_alert"]
}

The prompt becomes a working note, not a warehouse.

Prompt caching helps, but it does not fix bad context hygiene

This part gets misunderstood a lot.

Anthropic prompt caching is useful when you keep reusing the same stable prefix.

That means things like:

system instructions
tool definitions
fixed documents
repeated scaffolding

It does not magically make a sloppy prompt good.

Claude caching only helps when the prefix is identical, and cache entries last 5 minutes by default. That’s great for repeated prefixes. It does nothing to make a giant raw tool dump more relevant.

If anything, caching can hide bad architecture for a while because the bill hurts less before quality problems become obvious.

Example:

import anthropic

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},
    system="You are an AI assistant tasked with analyzing literary works.",
    messages=[
        {
            "role": "user",
            "content": "Analyze the major themes in Pride and Prejudice."
        }
    ],
)

Use caching for stable prefixes.
Do not use it as an excuse to keep shoving junk into prompts.

A quick before/after refactor

Here’s what this looks like in a normal agent loop.

Before

def run_step(state, user_request):
    search_results = search_docs(user_request)
    crm_data = fetch_customer_record(state["customer_id"])

    prompt = f"""
    User request: {user_request}

    Search results:
    {search_results}

    CRM data:
    {crm_data}

    Decide what to do next.
    """

    return llm(prompt)

This works at first.
Then search_results gets huge.
Then crm_data gets huge.
Then every step drags around more baggage.

After

def run_step(state, user_request):
    search_results = search_docs(user_request)
    crm_data = fetch_customer_record(state["customer_id"])

    state["search_results_raw"] = search_results
    state["crm_data_raw"] = crm_data

    prompt_payload = {
        "user_request": user_request,
        "top_matches": [doc["title"] for doc in search_results[:3]],
        "customer_tier": crm_data["tier"],
        "open_tickets": crm_data["open_tickets_count"],
        "billing_status": crm_data["billing_status"]
    }

    prompt = f"Decide what to do next using this state: {prompt_payload}"
    return llm(prompt)

This version is less fragile.

And when the model needs exact evidence later, you can pull the raw state back in selectively.

When should you keep raw output in the prompt?

There are real exceptions.

Sometimes the next step genuinely needs exact source material.

Keep raw output in-context when you need:

legal or compliance citations
exact stack traces for debugging
code review on full diffs
extraction from messy text
verification against original evidence

But even then, pass the exact slice that matters.
Not the entire upstream universe.

The rule I use now

This is the heuristic that cleaned up most of my agent workflows:

If information is needed for reasoning, summarize it.
If it is needed for retrieval, store it.
If it is needed for verification, pass the exact slice.

That rule has saved me more latency, more cost, and more agent quality than swapping models ever did.

Why this matters even more when you pay per token

This problem gets expensive fast in production.

If your agent runs 24/7 in n8n, Make, Zapier, OpenClaw, or a custom loop, every extra blob you keep replaying becomes recurring cost.

That’s one reason teams end up obsessing over token usage dashboards instead of fixing the architecture.

A cleaner state design reduces prompt bloat at the source.

And if you are using an OpenAI-compatible endpoint with flat monthly pricing, this gets even nicer because you can optimize for agent quality and throughput instead of constantly asking whether one more tool call is going to spike the bill.

That’s the part I like about Standard Compute: it works as a drop-in OpenAI API replacement, so you can keep your existing SDKs and agent code, but stop thinking in per-token panic mode while you clean up orchestration.

The architecture fix still matters.
You just get to apply it without token anxiety hanging over every experiment.

Practical checklist

If your agent feels slower and dumber over time, check these first:

Are you pasting raw tool output into every next prompt?
Are you storing canonical results outside the model?
Are you summarizing tool results before the next reasoning step?
Are you passing exact evidence only when verification is required?
Are you using prompt caching only for stable prefixes?
Are you trimming memory in n8n or your orchestration layer?
Are you treating the model like a reasoning engine instead of a database?

If you fix just those, you will probably get a bigger improvement than another model bake-off.

I went looking for a model fix.

What I found was an architecture fix.

Most “context window problems” are really just us using chat history as a storage layer.

Once you stop shoving raw tool output back into the next prompt, agents get faster, cheaper, and noticeably less confused.

Not because the models changed.

Because we finally stopped making them carry our state management mistakes.

Benchmarking Open Agents for Multi-File Refactors

Jorge Peraza — Wed, 14 Oct 2026 14:10:11 +0000

TL;DR

Skylark Creations' October 3 benchmark evaluated open coding agents against a 36-point rubric, showing they now rival proprietary solutions in multi-file reasoning and terminal execution.
The report highlights a shift in developer workflows: engineers require tools that plan, edit across multiple files, and iterate via tests without forcing lock-in to a single ecosystem.
MonkeysCode addresses this directly. Our recent Editor 1.2.12 (featuring the Capuchin agent) and CLI 1.0.2 releases provide a robust harness for multi-file loops, supporting both frontier and open models via BYOK.
Effective agent execution requires strict context management; our latest updates include usage dashboards exposing cache-write tokens and architectural changes to optimize prompt caching breakpoints.

The Skylark Creations Benchmark

On October 3, 2026, Skylark Creations published an updated evaluation of open-source coding agents. The benchmark assessed tools like Aider, Cline, and Orbi against a rigorous 36-point rubric designed to measure capabilities far beyond single-file autocomplete.

The report's primary finding is clear: open agents now rival proprietary solutions when it comes to multi-file reasoning and terminal execution.

Historically, the gap between open models and proprietary frontier models was most visible in context retention and structural planning. A model might successfully write a Python script in a vacuum, but fail to correctly update an interface, modify the implementing classes across three different files, run the test suite, and parse the resulting stderr output to fix a typo. The Skylark benchmark demonstrates that this gap is closing. By effectively utilizing open models, developers can achieve complex, agentic refactoring without being tethered to a single proprietary ecosystem or incurring prohibitive API costs.

The Engineering Mechanics of Multi-File Execution

Handling multi-file refactors introduces a specific set of engineering challenges for any agent harness. It is not simply a matter of passing more tokens into the context window. The agent must successfully navigate a multi-step loop:

Planning: Analyzing the codebase to identify which files require modification based on a given prompt.
Diff Generation: Emitting precise edits (often via search/replace blocks) rather than rewriting entire files, which wastes tokens and risks introducing regressions.
Terminal Execution: Running compilers, linters, or test suites directly in the environment.
Iteration: Parsing the terminal output (stdout/stderr) and determining if a failure requires a logic change, a dependency update, or simply a retry.

When a developer asks an agent to, for example, fix a database migration error—such as changing a raw SQL query from using a ? operator to using jsonb_exists() in Postgres—the agent must locate the migration file, update the schema definition, modify the corresponding application data models, and run the test suite to verify the changes.

If the test fails, the agent must read the failure, retain the context of its previous edits, and apply a correction. This requires the agent harness to safely supervise the terminal process and manage the context window efficiently.

Running Open Agents with MonkeysCode

Developers want the flexibility to use the best model for the job, whether that is a local model, an open model, or a proprietary frontier model. We designed MonkeysCode to provide a flexible, unopinionated harness for these agentic loops.

On October 5, 2026, we shipped MonkeysCode CLI 1.0.2 and Agent Manager 1.0.16. These releases solidify our terminal execution and supervision capabilities, allowing developers to run multi-file agent loops directly from their command line.

Simultaneously, we released MonkeysCode Editor 1.2.12. This desktop IDE includes Capuchin, our built-in coding agent. Capuchin is explicitly designed to handle the workflow validated by the Skylark benchmark: it plans changes, executes diff reviews across multiple files, runs tests, and iterates based on the output.

Because MonkeysCode supports BYOK (Bring Your Own Key), developers can plug in the open models that performed well on the Skylark rubric, or utilize local models, running them through Capuchin's multi-file execution engine.

Here is an example of how you might initiate a multi-file refactor using the MonkeysCode CLI:

# Initialize an agent session targeting a specific directory
monkeyscode agent start ./src/database

# Provide the instruction for a multi-file refactor
> "Migrate all raw Postgres queries using the '?' operator to use jsonb_exists(). Update the corresponding unit tests in ./tests/database/ and run npm test to verify."

The Agent Manager will orchestrate the planning phase, apply the necessary diffs, execute the test command, and feed the terminal output back to the model for iteration.

Prompt Caching and Context Economics

As multi-file reasoning becomes standard, the economics of context management become the primary bottleneck. Running a loop that repeatedly analyzes three files, a test suite, and terminal output will rapidly consume tokens.

To make this practical, agent harnesses must leverage prompt caching. When the system prompt, the core codebase structure, and the tool definitions are cached, the agent only pays for the delta (the new terminal output or the modified file) on each iteration.

In our latest release cycle, we shipped several architectural updates to support this:

Cache-Write Tokens: Our usage dashboards (available for both individual and Team plans) now explicitly show cache-write tokens, giving developers transparency into their context economics.
Auto Model Routing: We shipped step 4 of our "Auto" model routing build, adding an Auto mode tab on the Models & Pricing page. This system automatically routes requests based on context requirements and displays cache ratios, warning users if they are mixing frontier providers in a way that defeats caching efficiency.
System Block Separation: We updated our context kernel to keep system blocks separate for cache breakpoints. Whether you are using the newly integrated GPT-6 via Azure Foundry or an open model, maintaining strict cache breakpoints ensures that the static parts of your prompt do not incur redundant processing costs.
Stream Stability: Multi-file reasoning takes time. We recently deployed a fix to keep Anthropic SSE streams alive while the model is "thinking," preventing timeouts during complex planning phases.

Takeaway

The Skylark Creations benchmark provides concrete evidence that open models are capable of executing complex, multi-file refactors. However, the model is only half of the equation. The other half is the agent harness—the system that manages the context window, applies the diffs, and supervises the terminal execution.

Developers should not have to choose between advanced multi-file capabilities and the freedom to use open models. By decoupling the agent harness from the underlying model, tools like the MonkeysCode CLI and Capuchin allow engineers to leverage the reasoning capabilities of open agents within a robust, multi-file execution environment.

You can download MonkeysCode today to test Capuchin and the CLI with your preferred models, or review our documentation for details on setting up BYOK and local model routing.

I verified all 160 official U.S. business-data portals so your agent doesn't have to — here's what I found

bradraylucas-eng — Wed, 14 Oct 2026 14:07:46 +0000

Every agent workflow I build starts with the same wasted motion: the agent searches the web for the official registry, lands on an ad or an aggregator, and burns its run. So I verified the official portals myself — business entity search (50 states + DC + federal), UCC lien search, and professional license verification — and wrote down what agents actually need to know before they start.

Findings that surprised me:

Texas is the only state that charges for a basic business entity search ($1 via its portal) — and it retired the system agents' training data points to (SOSDirect) in September 2026.
UCC search is a different world: 21 of 51 jurisdictions charge, 18 require an account before you can search, and Delaware has no public search at all (state-licensed "Authorized Searchers" only).
License verification has no center: only 21 of 51 jurisdictions have a true central portal. California's main portal excludes contractors and real estate entirely. A "not found" there proves nothing.
In 10+ states the business registry is not run by the Secretary of State at all.
A large share of these portals return 403/bot challenges to plain HTTP clients. If your agent fetches with a simple client, it needs a browser session for these — the datasets flag them per portal.

I packaged the verified data (JSON + CSV, documented schema, per-entry cost/login/access notes, last verified October 2026) as the Source Finder datasets on Gumroad, with free samples so you can check the schema before paying anything: https://github.com/bradraylucas-eng/ai-agent-tools-free-samples · Vol. 1 on Gumroad: https://bradray2.gumroad.com/l/fxnjw

If you build agents that do due diligence, vendor checks, or lead verification, I'd genuinely like to know what other official sources you re-hunt every run — that's what Vol. 4 should be.

— Brad (maker; the datasets are my product, the samples are free)

Test your agent's payment loop on a testnet before it goes live

OpenClaw Cash — Wed, 14 Oct 2026 14:04:38 +0000

Wiring payments into an agent is easy to get wrong in a way that only shows up with real money: a retry after a network timeout that pays the same invoice twice, a recipient address copied out of a stale config, a webhook handler that accepts a replayed delivery. None of that is visible on a mainnet balance you cannot undo.

The cheapest place to find those bugs is a test network, and the agent API treats one almost exactly like mainnet: same endpoints, same X-Agent-Key header, same response shapes.

Why a testnet transfer finds what unit tests miss

A mock proves your code runs. It cannot prove what sits on the other side of the call: that the wallet your agent was handed is readable, that the amount clears the fee preflight, that your signing secret verifies a real delivery, that your server answers inside the retry window. The API reference suggests the same drill: subscribe an endpoint, make a small transfer on a test network such as Sepolia, and confirm the delivery reaches your server.

Three things a live testnet run proves:

Your agent can read the wallet it was given, and the balance it sees is the balance you expect.
The transfer passes the minimum-transferable check, fees included.
Your server verifies the signature and answers 2xx fast enough that the platform never retries.

Step 1: create a wallet your mistakes cannot cost anything

Create wallets on sepolia, mainnet or polygon-mainnet (EVM), or on solana-devnet, solana-testnet or solana-mainnet. Start on a test network.

export OCC_AGENT_KEY="occ_your_api_key"

curl -X POST https://openclawcash.com/api/agent/wallets/create \
  -H "Content-Type: application/json" \
  -H "X-Agent-Key: $OCC_AGENT_KEY" \
  -d '{
    "label": "agent-testnet",
    "network": "sepolia",
    "exportPassphrase": "store-this-before-you-call",
    "exportPassphraseStorageType": "env",
    "exportPassphraseStorageRef": "WALLET_EXPORT_PASSPHRASE_AGENT_TESTNET",
    "confirmExportPassphraseSaved": true
  }'

Two reasons for that shape. It follows the passphrase-safe flow: you persist the export passphrase first and then assert you did with confirmExportPassphraseSaved. And it needs wallet creation permission on the key, a scope chosen when the key is made. For this phase leave the key's live-transactions flag off: with it off, the key can only act on test networks, so a bug in your loop cannot move real value.

The response is the wallet record: id (the walletId you pass everywhere else), label, address, network, chain.

Step 2: discover, then read the balance

curl https://openclawcash.com/api/agent/wallets \
  -H "X-Agent-Key: $OCC_AGENT_KEY"

curl "https://openclawcash.com/api/agent/wallet?walletId=AB12CD3&chain=evm" \
  -H "X-Agent-Key: $OCC_AGENT_KEY"

The list is discovery only, cheap in an agent loop. Wallet detail returns nativeBalanceDisplay and tokenBalances, where an "is there enough to pay this invoice" branch gets tested as written. Fund the address from a testnet faucet, then read again.

Step 3: make the transfer, then read the record

curl -X POST https://openclawcash.com/api/agent/transfer \
  -H "Content-Type: application/json" \
  -H "X-Agent-Key: $OCC_AGENT_KEY" \
  -d '{
    "chain": "evm",
    "walletId": "AB12CD3",
    "to": "0xRecipientWalletAddress",
    "amountDisplay": "0.01"
  }'

curl "https://openclawcash.com/api/agent/transactions?walletId=AB12CD3&network=sepolia" \
  -H "X-Agent-Key: $OCC_AGENT_KEY"

Two rules trip people up here. Send exactly one amount field: amountDisplay for human units or valueBaseUnits for base units, never both, or the call answers 400. And native transfers run a preflight for the minimum transferable amount, so a wallet holding dust comes back amount_below_min_transfer with the minimum it needs, instead of broadcasting a transaction that cannot pay for itself.

Every history row carries data.network; scope with network=sepolia, or network=all to merge every EVM chain.

Step 4: hear the payment on your own server

Wallet events are a named group, wallet.transaction.confirmed, never covered by a * subscription, so list it by name.

curl -X POST https://openclawcash.com/api/agent/checkout/webhooks \
  -H "Content-Type: application/json" \
  -H "X-Agent-Key: $OCC_AGENT_KEY" \
  -H "Idempotency-Key: webhook-create-001" \
  -d '{
    "url": "https://your-server.example/occ",
    "eventTypes": ["wallet.transaction.confirmed"],
    "enabled": true
  }'

You get back a publicId and a secret starting with whsec_, shown once. Keep it server side, never in the agent's context. Then hold every delivery to the documented checks: read the raw request body before any JSON parsing, reject a webhook-timestamp older than 5 minutes, and compare the HMAC-SHA256 of {id}.{timestamp}.{body} (base64) against the v1 entry of webhook-signature in constant time. De-duplicate on webhook-id, because a delivery can arrive more than once, and answer 2xx within 10 seconds. Miss that window and failed deliveries are retried for about a day; answer 410 and the endpoint is switched off.

A testnet transfer produces the same event as a mainnet one. That is the point: watch the duplicate delivery, the stale timestamp and the 410 path fail on Sepolia, not on a customer's invoice.

What changes when you go live

Once the loop is boring, issue a second key for production with live transactions on, scope it to one chain and one wallet, and attach the wallet's policies so the server refuses anything that violates them before it is signed. Keep the testnet key for your test suite.

Two limits, plainly. Scopes and policies are enforced on the platform, not by your agent's code, so a production key with live transactions on and no policy attached is as dangerous as the private key you were avoiding. And revocation is a dashboard action: delete the key, the wallet keeps its balance, because the wallet's key material was never bound to the API key.

Endpoint paths, headers and fields above come from the published API reference at openclawcash.com/docs; the key-scope flags are described in the site's setup guide, Hermes Agent Wallet Setup: Safe by Default.

Mistral Large 4 Preview: MoE for Agentic Workflows

Jorge Peraza — Wed, 14 Oct 2026 14:01:50 +0000

TL;DR:

Mistral AI announced the public preview API for Mistral Large 4 ("le Chonk") on October 6, 2026, targeting agentic workflows and vulnerability patching.
The model utilizes a Mixture-of-Experts (MoE) architecture with roughly 1 trillion total and 49 billion active parameters, trained on 3,800 Nvidia Grace Blackwell GPUs.
Open weights are scheduled for release in late October, offering an alternative to proprietary frontier APIs for autonomous coding tasks.
MonkeysCode developers will be able to integrate Mistral Large 4 via our BYOK (Bring Your Own Key) support, though verified model-proxy support and head-to-head Capuchin benchmarks are currently pending.

The Technical Profile of Mistral Large 4

On October 6, 2026, Mistral AI released the public preview API for Mistral Large 4, internally dubbed "le Chonk." The release signals a continued industry shift toward specialized, high-capacity models designed specifically for autonomous execution rather than simple chat completion.

Under the hood, Mistral Large 4 is a multimodal Mixture-of-Experts (MoE) model. It features approximately 1 trillion total parameters, but only activates about 49 billion parameters during any given forward pass. This sparse activation is a critical architectural choice for coding agents. Dense models of similar total size require massive compute overhead for every token generated. By routing tokens only to the relevant "experts" within the network, an MoE model can maintain a vast repository of training data—such as diverse programming languages, syntax rules, and known software vulnerabilities—while keeping inference latency and compute costs manageable.

According to the announcement, the model was trained in Europe across a cluster of 3,800 Nvidia Grace Blackwell GPUs. The sheer scale of this training infrastructure highlights the compute density required to produce open-weight models today.

For developers, the immediate access point is the Mistral Studio preview API. However, the most notable aspect of the announcement is Mistral's commitment to releasing the open weights by late October. This upcoming release provides engineering teams with the ability to self-host a trillion-parameter-class model, ensuring strict data privacy and bypassing rate limits associated with managed APIs.

Aligning with Agentic Workflows and Patching

Mistral explicitly targeted Mistral Large 4 at agentic workflows and software vulnerability patching. This focus aligns directly with the operational mechanics of tools like the MonkeysCode Capuchin agent.

When developers build or utilize autonomous agents, the requirements differ significantly from standard code autocomplete. An autocomplete model needs to predict the next few lines based on a narrow, localized context window. An agentic model, however, must:

Plan: Ingest a high-level prompt (e.g., "Fix the SQL injection vulnerabilities in the user authentication module").
Navigate: Understand the repository structure and identify which files need modification.
Edit: Apply precise diffs across multiple files simultaneously.
Iterate: Run tests, parse the output, and correct its own mistakes if the tests fail.

Because Mistral Large 4 was tuned specifically for these multi-step, multi-file operations, it presents a compelling alternative to proprietary frontier APIs. The 49B active parameter count is particularly relevant here: agentic loops require multiple back-and-forth calls to the model. If the time-to-first-token (TTFT) or generation speed is too slow, the entire agentic workflow bottlenecks. The MoE architecture aims to mitigate this by providing the reasoning capabilities of a 1T model at the inference speed of a much smaller one.

Integrating with MonkeysCode

At MonkeysCode, our platform is built around a multi-model architecture. We do not lock developers into a single provider because different tasks require different models.

With the release of Mistral Large 4, developers can leverage our BYOK (Bring Your Own Key) support to route agentic tasks to the new model. While we are still waiting for the open weights to land later this month for local execution, the preview API can be configured within your MonkeysCode environment.

To configure a custom provider in MonkeysCode for the Mistral Studio preview, your JSON configuration would look similar to this:

{
  "models": [
    {
      "name": "mistral-large-4-preview",
      "provider": "custom",
      "baseUrl": "https://api.mistral.ai/v1",
      "apiKey": "env.MISTRAL_API_KEY"
    }
  ]
}

Note: Verified model-proxy support or custom base URL tests specifically with the Mistral Studio preview endpoints are currently pending on our end, as are head-to-head performance and latency benchmarks of Mistral Large 4 inside MonkeysCode Capuchin agent tasks (such as multi-file editing and test repair).

Recent Platform Updates for Agentic Workflows

To better support models like Mistral Large 4, we have recently shipped several updates to the MonkeysCode platform that enhance how developers interact with coding agents and manage model usage:

Enhanced Editor and Agent Manager: We recently released MonkeysCode Editor 1.2.12 and Agent Manager 1.0.16. The editor has been moved into our monorepo to streamline builds, and we've added UI improvements such as rendering markdown tables in chat and allowing developers to click a file name on a diff card to immediately open its tab.
CLI Improvements: The MonkeysCode CLI has been updated to version 1.0.2, improving how you run and supervise coding agents directly from your terminal.
Auto Model Routing and Caching: Our "Auto" model routing feature helps optimize cost and performance by selecting the right model for the job. In our latest update, the Auto mode tab on the Models & Pricing page now shows cache ratios and warns developers if they are mixing frontier providers in ways that break prompt caching. Additionally, usage dashboards now explicitly track cache-write tokens, giving teams better visibility into their API spend.

These caching improvements are vital when running agentic workflows. When Capuchin iterates on a test failure, it resends the repository context. Efficient prompt caching ensures you aren't paying to re-process the same files on every loop.

Takeaway

The public preview of Mistral Large 4 and the impending release of its open weights provide developers with a powerful new tool for autonomous coding. By combining a massive 1T parameter knowledge base with a highly efficient 49B active parameter MoE architecture, Mistral is directly targeting the latency and cost bottlenecks of agentic workflows.

While we are still gathering benchmark data on how it performs head-to-head against other frontier models in Capuchin's multi-file editing tasks, the availability of an open-weight model tuned for vulnerability patching is a significant step forward for developers who require strict data control and self-hosted execution.

For more information on how to configure custom models or utilize our built-in coding agent, visit the MonkeysCode Docs or explore our Team Plans for enterprise deployments.

An agent that learns the world it acts in: System 1, System 2, and knowledge you can retract

Maxim Kuznetsov — Wed, 14 Oct 2026 14:00:00 +0000

Most LLM agents start every task from zero. They explore, they find the way, and the next day they explore again. Some keep a memory, but it is usually text the model wrote about itself, which nobody can check or take back. This last part of the series shows the other end of solvi 1.0: an agent that keeps verified knowledge about the world it acts in, acts fast where that knowledge predicts the action works, deliberates where it does not, and keeps written rules as hard checks in both. I wrote solvi; part 5 of 5 (overview).

The two systems

Each step is one decision with two possible answerers:

System 1 takes an action the knowledge predicts will work and that advances an open goal: a goal's skill (the action after which its done check turned true before) or the next step of a confirmed route to where the skill worked.
System 2 is a search over the offered actions when System 1 has nothing it is sure of: an open goal's action first, then what was never tried here, then the way to the nearest place with untried actions.
Hard checks hold in both: the agenda's gates, a hard refusal of the action model, the failure memory.

After every step the knowledge is written back: the action model learns from the outcome, a goal done right after an action makes a skill, and where each action led becomes a map fact. Every decision is stored with what it was given and agent.replay() re-checks it without the environment.

The world and the knowledge

Example 25 in the repo is a toy crafting world: 8 places joined by exits, resources (tree, stone, iron, water), six operators with the environment's own checks (collect_stone needs stone here and a pickaxe, and so on), six goals. The agent knows none of those rules. It gets a vocabulary — the conditions its action model may learn from — and goals whose done checks are code, never a model's claim. Excerpt from examples/25_environment_agent.py:

VOCABULARY = {"here": lambda s, a: s["here"], "wood": lambda s, a: s["wood"], "stone": lambda s, a: s["stone"],
              "pickaxe": lambda s, a: s["pickaxe"], "stone_pickaxe": lambda s, a: s["stone_pickaxe"],
              "table_here": lambda s, a: s["place"] in s["tables"]}


def place(state):                                                # the map's keys: the places
    return state["place"]


def goals(km):
    """The six goals: done when the operator has worked once (code over the state, never a model's claim)."""
    for op, needs in zip(OPS, [None] + OPS[:-1]):
        km.goal(op, done=lambda s, op=op: op in s["achieved"], requires=[needs] if needs else [])
    return km

The agent itself is three lines (excerpt, same file):

    km = goals(solvi.Knowledge(vocabulary=VOCABULARY))
    agent = solvi.Agent(Crafting(), knowledge=km, key=place)
    runs = [("world 7, run 1", agent.run(seed=7, steps=150)), ("world 7, run 2", agent.run(seed=7, steps=150)),
            ("world 8 (new)", agent.run(seed=8, steps=150))]

Crafting is any object with reset(seed), actions(state) and step(action) → Outcome — the Environment protocol; solvi.testing.conformance.check_environment checks one of your own.

The same world runs in your browser, with no install: the arcade Space's "Agent and knowledge (1.0)" tab (https://huggingface.co/spaces/solvi-ai/arcade).

The same world, met again

The whole output of the file, verbatim:

Part 1 — the same world met again needs fewer slow decisions
  world 7, run 1: 61 steps, 6 of 6 goals, System 1 0, System 2 61, fallback 0, refused 23
  world 7, run 2: 12 steps, 6 of 6 goals, System 1 11, System 2 1, fallback 0, refused 0
  world 8 (new): 17 steps, 6 of 6 goals, System 1 4, System 2 13, fallback 0, refused 3
  every decision replays: 90 of 90; the knowledge journal verifies: True
  learned: collect_stone needs ["'stone'"] here and a pickaxe ['True']

Part 2 — protection vs justified risk (30 episodes on a world with a breaking bridge to the iron)
  protect : iron in 2 of 30 episodes, fell 1 times, 0 risky crossings, 5.07 goals per episode
  risk    : iron in 24 of 30 episodes, fell 6 times, 27 risky crossings, 5.80 goals per episode
  the gate 'no bridge without the stone pickaxe' held in both: True

The first run explores: 61 steps, every one decided by System 2, 23 actions refused by the world. The second run in the same world takes 12 steps, 11 of them by System 1, nothing refused. In a new world the agent keeps the rules (what an operator needs) and re-learns the map: 17 steps. The last line of part 1 is what the action model learned, readable: collect_stone works with stone here and a pickaxe.

Map facts and rules are kept apart on purpose. Rules are carried across episodes; map facts belong to the map they were seen on, and the first arrival that contradicts a carried map fact drops the carried map — its facts become hints until confirmed again.

Protection by default, risk as an option

Knowledge used as hard checks removes the failures it predicts. It can also cost a goal that needs a risk. Part 2 of the example has a world where the iron lies only across a rope bridge that breaks one time in three (the episode ends), and a written gate: no bridge without the stone pickaxe. Excerpt:

    for name, risk in (("protect", None), ("risk", RiskBudget(max_risk_per_episode=1.0, min_gain_ratio=1.0,
                                                              min_support=1))):
        km = goals(solvi.Knowledge(actions=OutcomeRates(VOCABULARY, only={"bridge": []})))
        km.agenda.gate("bridge_needs_the_tool", lambda s: s["stone_pickaxe"], blocks=["bridge"])
        agent = solvi.Agent(Crafting(bridge=True), knowledge=km, key=place, risk=risk, gain=bridge_gain)

(OutcomeRates is a 20-line action model of the example's own: the outcome rate of each action per situation — any object with observe, predict and fingerprint plugs in.)

With protection (the default) the agent fell once, predicted "refuse" for the bridge from then on, and never crossed again: iron in 2 of 30 episodes, 5.07 goals of 6. With RiskBudget it crossed 27 times against the prediction, when the expected gain outweighed the estimated risk within the episode's budget: it fell 6 times and got the iron in 24 episodes, 5.80 goals. The written gate held in both; gates, hard predictions and the failure memory are never traded.

The honest reading: justified risk lowers the cost of protection. It does not promise to do as well as an agent with no knowledge at all. In a harder dungeon game it won back part of what protection cost, not all of it. Measure it on your own metric before turning it on.

The same idea on a real game map

Example 23 walks the world map of Pokémon Red — 190 places and 447 exits, recorded from a real playthrough (place names and exits only; no ROM bytes, no graphics) — through the game's first fifteen goals, twice. System 1 is two rules over remembered routes; System 2 searches the world map the player builds as it goes; after each goal, what System 2 found becomes System 1's routes. From its output (the per-goal table cut to its totals):

  goal             run 1: moves   S1   S2   run 2: moves   S1   S2   shortest
…
  all goals                 183   36  147             62   60    2         62
…
  → run 2 needs 2 slow decisions instead of 147, and 62 moves instead of 183.

The second run takes the fewest moves possible (62, the "shortest" column), with 2 slow decisions out of 62. And at the end: "245 of 245 stored decisions replay". The recording lives in the repo's spaces/pokemon/, so run it from a clone.

What it does not do

Growth was shown only in environments met again — this crafting toy and the Pokémon map. It was not shown on streams of one kind of decision (classification, matching) or for a support agent with tools (parts 1, 2 and 4 of this series say what the store gives there instead: accountability, not growth).
It does not shorten the first exploration. Run 1 is a search; knowledge pays off on the second visit.
The action model learns only what the environment checks, over your vocabulary. A rule the world does not enforce must be written as a gate.
Rules learned in one world can be over-cautious in another. A condition that held at every success by accident does not transfer; the model then says "unknown", not "wrong".
The knowledge store is measured to 10,000 items, not beyond.

What every item does have: a source (a person, an outcome, a written spec, or a verified System 2 answer — never the agent's own guess), and an exact retraction that takes back everything derived from it.

The series

Part 1: routing tickets with a promise
Part 2: product matching with a rule across answers
Part 3: contract clauses with quotes you can check
Part 4: guarding a support agent's tool calls
Docs: Using solvi: agents and knowledge, mission and honest limits
Full code and outputs: solvi-ai/recipes: 05-environment-agent
Code: github.com/solvi-ai/solvi · pip install solvi

Where does your agent meet the same world again — the same codebase, the same internal tools, the same warehouse? That is where this kind of memory should pay off. And when your agent's knowledge turns out wrong, what would you need to see to trust a retraction of it?

AI assistant keeps giving outdated answers after codebase refactoring, how to update its learned knowledge?

Edward Izgorodin — Wed, 14 Oct 2026 13:56:00 +0000

TL;DR

A refactor changes the code; the assistant's notes about that code stay as they were. Whether it notices depends on two things: does a note point at the code it is about, and is that pointer checked before the note is used?

GitHub Copilot Memory documents both: facts cite code, and the citations are checked against the current branch before use. Claude Code dates its notes and tells the model to verify older ones against current code when it uses them. Cursor advises rules that reference files instead of copying them.

Devin and Cline tie memory to the session it came from or to markdown files that you or the agent keep current. Memory layers (Zep, Mem0, Supermemory, Cognee) trace a fact to a conversation or document, not to code.

After a refactor: keep out of memory what the code already says, point the rest at files and date it, run the checks that exist, replace a stale note instead of appending a correction, and send changed documents through an update path.

A renamed function is easy to find in the repository. It can be much harder to get out of an assistant's memory.

GitHub's engineering post on Copilot's memory says the core challenge is not retrieval but "ensuring that any stored knowledge remains valid as code evolves across branches and time" (GitHub blog, 2026-01-15). Persistence is not the problem. Persistence with no connection to current code is.

A stale note is a stored claim that describes an earlier state of the code and was not revised or invalidated when that state changed.

Code-anchored memory is a stored claim tied to code locations (files, symbols, tests) that can be checked against the current repository before the claim is used.

This is a reading of each vendor's published documentation and shipped files, first read on 2026-10-06 and re-read on 2026-10-10. It is not a hands-on test of the products. Where a page does not describe a behaviour, this article says so rather than claiming the product cannot do it.

Why a refactor leaves the assistant on the old code

A saved note and a repository file have separate lifecycles unless something connects them.

Say an assistant noted that authentication goes through validateSession(). A refactor replaces that function. The repository now holds the new name, but the old note can still be retrieved, because nothing ties it to the changed file and nothing checks it before use. Anthropic describes the same failure for an index of the code: it "returns a function the team renamed two weeks ago, or references a module that was deleted in the last sprint, with no indication that either is out of date" (Anthropic, 2026-05-14).

So two questions matter more than whether a product "has memory":

What does the note point to? Current code, a file, the session it was learned in, a conversation, or a document.
What checks or updates it? A check when it is used, a code change, a scheduled cleanup, a command you run, or nothing that is described.

Copilot Memory, Claude Code, Cursor, Devin and Cline compared

Read from each vendor's documentation and shipped files on 2026-10-10.

Assistant	What a note points to	What signals or checks a stale note	Remember or re-read
GitHub Copilot Memory	Facts with citations to the code that supports them	Citations checked against the current branch before use; unused entries deleted after 28 days	Remembers, then re-reads the cited code
Claude Code	One auto memory directory per git repository, on one machine	Last-modified time on notes; from two full days old, an age note and an instruction to verify against current code	Skips what it can derive from the codebase
Cursor	Version-controlled `.mdc` rules that can reference files	You edit a rule, or Agent does when you tag `@cursor` on a GitHub issue or PR	Rules point at files; search index stays on your machine
Devin	The session where the entry was learned, with the date it was added	Session Insights flags Knowledge items (now moving to Skills) written for a previous version of the codebase	Does not save what is cheap to rediscover
Cline	Markdown Memory Bank files in your project	Update occasions named in the recommended instructions, one of them your `update memory bank`	Reads the Memory Bank after each reset, per the recommended instructions

GitHub Copilot Memory checks cited code against the current branch

In the Copilot Memory docs, Copilot "checks those citations against the current branch to confirm the information is still accurate. Only validated facts are used." The engineering post says how: "the agent is prompted to verify its accuracy and relevance by checking the cited code locations." So the check is an instruction to the agent when it reads a fact, not a rewrite on every commit. When the code disagrees, the agent "is encouraged to store a corrected version of the memory".

The docs add three details that matter after a refactor. A fact captured from a pull request closed without merging has no effect unless the current codebase still supports it. Unused facts are deleted after 28 days, and "The 28-day timer may reset whenever Copilot successfully validates and uses an entry." Repository owners can review and delete repository-level facts by hand. Memory serves the Copilot cloud agent, code review, the CLI and agentic autofix, and, since 2026-08-11, agent chat in JetBrains (GitHub changelog). It "is currently in public preview and is subject to change."

Claude Code auto memory dates notes and warns when they are used

The memory page: "Auto memory is machine-local." All worktrees and subdirectories of one git repository share one directory on that machine, and the files "stay until you or Claude edits or deletes them." Notes have carried timestamps since 2.1.75 (2026-03-13) and an ISO modified write time in their frontmatter since 2.1.214 (2026-07-18) (changelog). The /memory command gives "access to the auto memory folder and the auto memory toggle" (debug your config), so you can read and edit the notes yourself.

The program shipped as Claude Code 2.1.291 adds an age note to any memory two full days old or older, followed by "Verify against current code before asserting as fact." That acts when a note is used, not when the code changes. A link from a note to a file or commit, and a re-check triggered by a code change, are not described on the Claude Code documentation read on 2026-10-10.

Claude Code's first defence is upstream: auto memory skips "anything it can derive from the codebase". Function names, paths and signatures are cheaper to re-read than to keep as prose. For instruction files, /doctor prompt-audit (since 2.1.283, 2026-09-25) reports references to missing files or commands and contradicting files, and changes nothing until you ask. Its default list covers CLAUDE.md, CLAUDE.local.md, AGENTS.md and the rules, skills, commands, subagents and output styles under .claude/ and ~/.claude/. It does not list the auto memory directory, though you can pass a path. The best practices put it in one line: "Treat CLAUDE.md like code".

Cursor rules should reference files, not copy them

Cursor's rules page says it directly: "Reference files instead of copying their contents. This keeps rules short and prevents them from becoming stale as code changes." Rules are .mdc files in .cursor/rules, under version control. You edit them, or Agent does when you tag @cursor on a GitHub issue or PR.

The difference is concrete. A rule that says "follow the interface in src/auth/session.ts" sends the assistant to the current file. A rule that pastes the old interface becomes a second specification, and after the refactor it is the wrong one. Cursor's search builds its index on your machine and "does not store embeddings of your codebase for search"; that is a statement about search, not about rules.

Devin memory links each entry to the session it came from

In Devin's Memory, each entry "links back to the session where it was learned" and is dated when added. That records origin; it points at a past session, not at current code. Session Insights gives as one reason knowledge misled Devin: "Knowledge was written for a previous version of your codebase", listed under Misleading Knowledge so you can update or remove it. Dreaming "reviews your recent sessions alongside your existing memory" and removes "stale notes that no session has used". A check of a note against changed code is not described on the Devin Memory page read on 2026-10-10. The older Knowledge feature "is deprecated"; Devin's release notes date its move into Skills to 2026-09-18.

The Agent Memory Repo specification that this memory is built on recommends a source key, "a link to the agent session where the information was learned", and an added date. A key tying an entry to a code file is not described in the specification, though "Keys are open."

Cline Memory Bank is documentation you keep current

Cline calls its Memory Bank "a documentation methodology". The recommended custom instructions on that page name four occasions to update it, among them "After implementing significant changes" and your own update memory bank. The same instructions say: "After each reset, I rely ENTIRELY on my Memory Bank". These are recommended instructions, not built-in behaviour, and they make a stale Memory Bank expensive. Separately, the extension's source tracks outside edits to files read in the current task "to prevent stale context"; that covers the current task, not the Memory Bank.

Memory layers trace a fact to a conversation or document, not to code

If your agent's memory sits in a separate memory layer, the anchor is the episode, conversation or document the fact came from. Zep ties facts to episodes and says plainly that this "records where information came from. It does not establish that the source content or a derived fact is true." Mem0 keeps in a memory's history "The conversation input that led to this memory change". Supermemory's MCP returns "source document IDs", and its GitHub connector syncs pushed documentation "within approximately 10 minutes", removes deleted files, and excludes code files by default.

What matters after a refactor is what each layer does with a changed document:

Cognee: "add() creates documents; it never updates one." (add). Its update() replaces what was derived from the old content and by default re-ingests "only the chunks the edit touched"; when it rebuilds a document instead, entities that other documents in the same dataset also reference are kept.
Supermemory: "Content changes trigger full reprocessing" (document operations), and "facts supported only by the previous source can disappear after reprocessing" (v5 document updates).
Zep: "Reingesting the same file creates new episodes, even when its contents are unchanged" (ingestion); each extracted fact is then compared with existing ones, and "duplicates merge, and contradictions invalidate older facts rather than leaving conflicting truths side by side." (graph creation).

The lesson is the same in all three: a changed document goes through the update path. Adding it again can leave both versions in memory.

Remember or re-read: what the October 2026 preprints measured

Remembering saves rediscovering. Re-reading avoids trusting yesterday's summary. Anthropic's post explains why Claude Code searches the live code without an index, and names the price: "it works best when Claude has enough starting context to know where to look." Copilot keeps memory and re-reads the cited code.

Three arXiv preprints from October 2026 study this trade-off. None studies refactoring, none is peer-reviewed, and every number below is the authors' own measurement on their own setup.

arXiv 2610.03902 (Wenhui Chu, v1, 2026-10-02) asks, when supporting documents are revoked or replaced, "should it repair memory or re-read current evidence?" On ICU medical records with two 7B models, local repair used 5 to 10 times fewer revision tokens than rebuilding on short records, yet every memory pipeline cost at least twice full re-reading in held-out conditions; in a small development sweep that padded records to about 10,000 tokens with documents the task does not use, memory became cheaper than full re-reading after 2 to 14 uses, "while source-filtered re-reading remained cheapest." The author's own limit: "none of the four primary confirmatory tests reached statistical significance." No code or data are published yet. Its example fits code too: "The raw record is marked revoked; the extracted claim and the summary are not".
MemTrace, arXiv 2610.04838 (Hongming Xu and others; v1 2026-10-04, v2 2026-10-08 with the same results) anchors a coding agent's records to "files, symbols, tests" and checks them "against the current repository state"; with nothing applicable, "the agent reads the current file or symbol, or reruns the required test." The authors: "Persistence alone, however, does not establish current validity". In 84 archived coding-task runs, 57.0% of 9,226 eligible file-bound records met a later change to at least one bound file, which is exposure, not error: "Exposure measures file changes; Trace Validation assesses whether historical evidence remains applicable." The performance runs used one model; sessions are left to future work; no refactoring. The code is published.
StateWise, arXiv 2610.05241 (Yongyuan Peng and others, v1, 2026-10-04) planted one wrong operational record in an agent's persistent memory in each of 150 cases. Even with an explicit reminder to refresh, "refresh remains selective and recovery is limited". Removing the wrong record "reduces harm but leaves the agent without information needed to complete the task." The records were about environment and repository policy, not code after a refactor, and developer responses were simulated.

Together they support a design question, not a product ranking: which notes are worth repairing, and which should be read again from the source.

Replace the stale note, do not append a correction

A correction added beside the old note leaves two notes that disagree, and the old one can still win retrieval. An illustration of the pattern to avoid:

- Session tokens are validated by `validateSession()` in src/auth/session.ts.
- UPDATE 2026-10-10: `validateSession()` was removed; use `verifyToken()`.

Several systems tell the agent to fix the note in place instead. Letta's reflection prompt: "fix the stale entry at the source. Do not append the new version alongside the old." The Agent Memory Repo skill: "Update or remove entries that are out of date instead of adding a contradicting entry." Kevin Liao's essay (version of 2026-10-05): "Facts should be rewritten when truth changes. No appending." Whether a model actually does this varies: Letta's evaluation of 2026-07-28 found that "weaker models make minimal edits that leave stale and contradictory information behind." What a store does with an older record that a newer one contradicts is compared in Agent memory consolidation compared.

A rating moves a note down, it does not say why

Several products take a rating on a memory or an answer, and none of them turns a thumbs-down into the corrected fact. Cognee can fold answer ratings into graph search; self-hosted, by default "the learned feedback signal does not change ranking until you opt in" (feedback system). Its demonstration rates one topic up and another down, because "a symmetric signal would raise both topics equally and leave their relative order untouched", and the same page points disagreeing facts to a separate conflict review, where they are settled "by each source's date, not by a rating." Mem0 takes POSITIVE, NEGATIVE and VERY_NEGATIVE ratings and says "This feedback is used to improve the accuracy of the memories and search results." (feedback); how NEGATIVE lowers a memory is not described on that page. Supermemory down-weights facts its engine inferred "until someone confirms them": once approved, such a fact "Ranks like a stated fact"; once declined, it is "Removed from search entirely" (memory review). In Mnemoverse (the author's company), "Ratings re-rank what comes back next." (agent memory).

A rating can push a stale note down without saying that it is stale or what replaced it. After a refactor, fix the note.

What to do after a refactor

Keep out of memory what the code already says. Function names, paths, signatures and behaviour are for the agent to read from the code; Claude Code skips what it can derive from the codebase, Devin skips what is cheap to rediscover. A prompt such as "Read src/auth/session.ts and confirm the current exports before writing the tests" does more than any remembered summary of that file.
Point what you keep at files, and date it. Cursor's rules advice and Copilot's citations both point at code; Claude Code records write times; the Agent Memory Repo recommends source and added keys. A session link tells you where a note came from, a file reference gives the agent something current to check.
Run the checks that exist. Copilot validates cited facts on its own; repository owners can delete stale ones. In Claude Code, open /memory and read the notes about the code you moved, and run /doctor prompt-audit on your instruction files. In Cursor, update the affected rules. In Devin, review the Misleading Knowledge list in Session Insights (it covers Knowledge items, now moving to Skills). In Cline, run update memory bank.
Replace a stale note instead of adding one beside it (Letta, the Agent Memory Repo). Copilot's agent is encouraged to store a corrected version; GitHub's pages do not say the old fact is removed, and repository owners can delete it by hand.
Send a changed document down the update path, not the add path: Cognee's update(), not add(); in Supermemory, "update the canonical document when facts change" (v5 memory forgetting).

Rules files age with the code too: a rules file is text in a context window. Forgetting over time is a separate question, as are outdated library patterns after a correction.

Common questions

AI assistant keeps giving outdated answers after codebase refactoring, how to update its learned knowledge?

Remove from memory what the code already says, point what you keep at files and date it, run the checks your assistant documents, replace a stale note instead of appending a correction beside it, and send changed documents through an update path. Whether the assistant notices a refactor on its own depends on whether its notes point at code and whether that pointer is checked before use; as of 2026-10-10, GitHub Copilot Memory documents both.

Why does my AI coding assistant still suggest a function I renamed?

Something it reads was written before the rename. If that note points only to an earlier session, conversation or document, nothing in the rename touches it. GitHub Copilot Memory checks a fact's code citations against the current branch before use; Claude Code tells the model to verify notes two full days old or older against current code. Otherwise, rewrite or delete the note, or let the agent read the code again.

Does Claude Code update its memory after a refactor?

Its memory files "stay until you or Claude edits or deletes them." (memory page). The Claude Code documentation read on 2026-10-10 does not describe a re-check of a note when code changes. The program shipped as Claude Code 2.1.291 dates notes and, when a note two full days old or older is used, tells the model to verify it against current code.

How does GitHub Copilot Memory keep facts from going stale?

Facts carry citations to code, and before a fact is used the agent is prompted to check those citations against the current branch; in GitHub's docs, "Only validated facts are used." Unused entries are deleted after 28 days, and the timer may reset on validated use. Repository owners can also delete facts by hand. The feature is in public preview.

Can a thumbs-down rating fix an outdated memory?

No. A rating can move a note down in retrieval where a product documents that, but it does not say why the note is wrong or what replaced it. Cognee's docs say disagreeing facts are settled "by each source's date, not by a rating." Correct or remove the note itself.

Sources and method. Each vendor's pages were first read on 2026-10-06; every quotation on this page was read again on 2026-10-10 (the two Cognee feedback pages on 2026-10-13) and is contiguous text of the source linked beside it (for arXiv papers, the full text behind the abstract page; for Claude Code's shipped instructions, the npm package named below), with the vendor's punctuation. Where a vendor describes an instruction to the model rather than a mechanism, this page says so. Claude Code: memory, best practices, changelog, debug your config, and the shipped program in the npm package @anthropic-ai/claude-code-linux-x64 2.1.291, searched byte by byte; Anthropic, How Claude Code works in large codebases (2026-05-14). GitHub: Copilot Memory, Building an agentic memory system for GitHub Copilot (2026-01-15), JetBrains changelog (2026-08-11). Cursor: rules, search. Devin: memory, Session Insights, Knowledge, release notes. Agent Memory Repo: SPEC.md, SKILL.md. Cline: Memory Bank, FileContextTracker.ts. Letta: reflection.md, Evaluating memory in production agents (2026-07-28). Mem0: memory history, feedback. Zep: source traceability, preparing data for ingestion, how graph creation works. Cognee: update, add, feedback system, feedback demonstration. Supermemory: document operations, GitHub connector, MCP, memory review, v5 document updates, v5 memory forgetting. Research: arXiv 2610.03902 (v1), arXiv 2610.04838 (v1 and v2), arXiv 2610.05241 (v1). Discussion: Kevin Liao's essay (version of 2026-10-05). Mnemoverse: agent memory. No score or performance number for any product appears on this page.

Edward Izgorodin · Mnemoverse · 2026-10-10

Stop Building Listening Dashboards. Build Listening Tools Your Agent Can Call.

Jiahui Miao — Wed, 14 Oct 2026 13:47:35 +0000

I built an open-source listening system, EchoWatch, and made a protocol — MCP — its first interface, ahead of its own dashboard. The next user of a listening tool is not an analyst staring at charts. It is an agent that needs to ask, in the middle of a task, "what changed, how much, and should I care?" — and get an answer it can act on.

Every listening product I have ever opened makes the same assumption: a human will log in, look at a dashboard, and decide what matters. The charts get prettier every year. The assumption gets older. My operation does not have an analyst on staff. It has an AI system that runs my publishing, my monitoring, and my follow-ups — for an audience of exactly one principal: me. A dashboard that waits for me to visit it is a dashboard that gets visited never. So this week I built the listening layer my system actually needed, and I built it agent-first.

EchoWatch collects mentions from live sources, dedupes and stores them locally in SQLite, and exposes them to my agent as four MCP tools: search_mentions, sentiment_trend, hot_events, and generate_brief. The dashboard exists — all six of its routes serve real data — but it is the second surface, not the product. The product is a question an agent can ask at 3 a.m. without waking anyone.

The numbers are unglamorous, and they are real

Five sources are verified live, not stubbed: Hacker News, GDELT (75 mentions in its verification run), RSS, Reddit through its RSS feed (22 mentions), and Bluesky (25), plus a sixth source, a TechCrunch feed, added with zero code through a configuration file. The local database holds 187 mentions, and re-running every fetch stores zero duplicates. That last property is a design decision, not an accident. The mature open-source monitoring suite I benchmarked against — feature by feature, across a 37-row parity matrix — dedupes with a Bloom filter. A Bloom filter has false positives, which means it silently discards real mentions and calls that efficiency. A listening tool that quietly deletes the thing it exists to hear is broken at the load-bearing wall. EchoWatch dedupes with an exact content hash enforced by a unique index: slower to describe, impossible to lie with. In its verification run, a second Hacker News fetch was intercepted 50 out of 50.

The rest of the chain is equally literal. Volume spikes are detected against a rolling baseline and pushed out over a webhook — tested end to end, three true alerts fired, zero false ones on the control set. Briefs render as Markdown and HTML files with code-drawn SVG charts, because a brief an agent can forward beats a page a human must log into. The whole stack is one Python process and one SQLite file, with zero runtime dependencies, and its first GitHub Actions CI run — tests plus a Docker build — passed this morning.

What I refused to copy

Parity was the floor, not the goal. The suite I measured against ships two message queues, four storage engines, and a Java stack, because it was designed for enterprise clusters. Copying its feature list meant refusing its machinery: middleware is a means, not a feature, and no user has ever asked for a second message queue. Ten rows of my 37-row matrix ended as explicit, recorded decisions not to rebuild the heavy parts — each one argued on cost, not skipped in silence.

And one honest limitation, stated the way I require every part of my system to state it: the LLM sentiment backend is wired behind a unified interface, but the machine that built this has no API key, so that path is not yet verified live. What runs today is a dictionary baseline — roughly 190 weighted terms with a negation window — and when no key is present the system labels its own output degraded instead of inventing a score. A listening tool that fabricates confidence is worse than one that admits a gap. The gap closes the day a key lands in the environment; the labeling stays forever.

Listening is a sense. Senses belong to the actor.

Here is the claim I will defend: a dashboard treats listening as reporting — something that already happened, formatted for review. An agent treats listening as a sense — something happening now, wired directly into the thing that acts. When a spike fires in my system, it does not wait for a weekly review meeting. It reaches the same agent that writes, schedules, and escalates, with the brief already attached.

The vendors will keep polishing dashboards, because dashboards are what procurement buys. But the builders wiring agents into real operations are going to route around them, the same way EchoWatch did: local data, exact dedupe, honest degradation, and a protocol where the login page used to be. Stop building listening dashboards. Build listening tools your agent can call.

— Jiahui Miao, founder of vertciti, building a personal AI operating system for one.

Amex's Intent Intelligence: Intent as a Token — Judgment Still Missing

ScriptMasterLabs — Wed, 14 Oct 2026 13:43:02 +0000

American Express published its Intent Intelligence Framework on October 13, 2026 — and it's the first card-network design that treats a customer's intent as a machine-checkable object instead of a checkout click.

The short version: capture and authenticate the customer's purchase intent before the agent buys, carry it as an Intent ID + Proof of Intent Token, and scope the payment credential to exactly that intent. A purchase that doesn't match what was approved can be stopped at authorization — not disputed after settlement.

How it works (from Amex's release)

Create Intent API records the instructions and constraints — merchant, maximum budget, preferences. The customer authenticates and signs off.
The system issues an Intent ID and a Proof of Intent Token carrying those constraints.
The token obtains a secure digital payment credential scoped to the intent. The agent never touches the actual card number.
The ID and token act as controls downstream — validation, fraud prevention, authorization decisioning, dispute resolution.

It's a core service of the ACE Developer Kit (launched April 2026). Amex says it has not seen an agentic-dispute increase yet — this is proactive. Its Trendex survey: 66% of surveyed U.S. consumers likely to use agents to research prices, 61% for recommendations. Amex also name-checked engaging EMVCo and supporting the x402 Foundation's standards work.

What the token can't do

It checks a purchase against approved constraints. It does not score whether the agent's choice inside a broad intent was wise — and it faithfully enforces an intent that was captured wrong and rubber-stamped. Amex's own autonomy example (agents reordering household staples inside price/size constraints) is exactly where that gap opens.

We tested the gap live

This morning (Oct 14, ~09:24 EDT) we ran two Intent-Intelligence-shaped instructions through our public decision gate at scriptmasterlabs.com/api/harness/decide (bands: ≥0.80 auto-act, 0.50–0.79 advisory, <0.50 escalate; decider local-heuristic-v1, calibrated=false):

Mismatch: authenticated intent for white Nike sneakers, size 9, max $120, Nike.com — agent tries black Adidas at $145 from an unlisted seller → 0.59, advisory/hold
Exact match: usual paper towels, $24, usual merchant, inside the authenticated intent → 0.59, advisory/hold

Identical scores. The honest lesson cuts both ways: a gate fed intent as prose cannot discriminate. Amex carries intent as structured fields bound to a token — and structured constraints are mechanically checkable in a way prose never will be.

The layers compose: the token answers "does this match what was approved?", a per-payment confidence score answers "should this fire at all?", and a sealed decision receipt is the dispute file. Our Verdict Protocol was re-verified today: LIVE_SETTLEMENT_VERIFIED, /decide $0.25 USDC, /seal $0.01 USDC, Base, x402 v2.

Full canonical with the dated receipts table, the enforce/cannot-enforce table, Claim Receipts and a 5-step DIY: https://scriptmasterlabs.com/amex-intent-intelligence-framework

ScriptMasterLabs is a Service-Disabled Veteran-Owned Small Business. Coverage-based, not hands-on: Amex published a framework and developer-kit service; we could not verify a public testable endpoint for it today.