DEV Community: Danil Galeev

The agent finished. Who turns off the VM?

Danil Galeev — Thu, 08 Oct 2026 19:24:22 +0000

Suppose a coding agent opens a pull request at 6 p.m. The tests pass, a preview is running, and the reviewer has already logged off.

The agent's task is complete. The machine still has something to do: serve that preview until someone looks at it. Or perhaps it doesn't. Perhaps the preview can be rebuilt tomorrow and the workspace should shut down now.

Either choice is reasonable. Leaving it undecided is how a short coding task turns into a machine nobody remembers starting.

Web Dev Cody's Coder walkthrough shows agents working in remote environments, including an AWS EC2 workspace, with changes and previews available for review. The video is sponsored by Coder. It prompted a narrower question for me: what should stop when an agent finishes?

There are three separate answers: the command, the agent session, and the workspace. A limit on one doesn't automatically control the others.

A test timeout only covers the test

Here is a small guardrail for a Node.js project. It requires Bash, GNU timeout, installed dependencies, and a working npm test script. Run it from the project directory:

#!/usr/bin/env bash
set -u

if timeout --kill-after=30s 10m npm test >test.log 2>&1; then
  printf 'Tests passed. Output: test.log\n'
else
  status=$?
  printf 'Tests failed or timed out (exit %s). Output: test.log\n' \
    "$status" >&2
  exit "$status"
fi

At ten minutes, timeout sends a termination signal. If the command still hasn't exited thirty seconds later, it sends a kill signal. A timeout normally returns 124; a forced kill can return 137. Otherwise, the command's exit status is preserved. GNU documentation

Now imagine the agent reads the failure, changes something, and runs the tests again. Each attempt has a limit. The task as a whole can still continue indefinitely.

That needs a separate deadline or retry budget, enforced by whatever launches and supervises the agent. Putting a timeout inside a script the agent can rewrite is useful for ordinary failures, but it isn't a reliable boundary for the agent itself.

An active workspace may never become idle

An inactivity timer sounds like an answer to the overnight-machine problem. Its usefulness depends on what counts as activity.

Coder's scheduling documentation says that a working agent can extend the workspace's shutdown deadline. This prevents a legitimate task from being interrupted. It also means that an agent stuck doing work may keep its environment alive.

For the hypothetical 6 p.m. pull request, my policy would be: end the agent session when it hands over the result, save the review evidence, and retain the preview only until an explicit expiry time. A longer review window should be a deliberate extension.

The supervisor should also handle the less tidy cases: the agent crashes, loses its connection, or keeps retrying without producing a result. A deadline that depends on the agent successfully reporting completion won't cover those failures.

This is a policy to implement and test, not a claim that every agent platform provides the same controls.

Keep the evidence before removing the workspace

Shutdown is awkward when the only copy of a change or its test output lives on the machine being stopped.

For a task using a Git worktree, the setup might look like this. Run it from an existing checkout, with a new branch name and an unused destination:

task_id="issue-142"
task_dir="../agent-${task_id}"

git worktree add -b "agent/${task_id}" "$task_dir" HEAD

This starts from committed HEAD; it doesn't copy uncommitted edits. A worktree separates working files while sharing repository data. It doesn't isolate credentials, processes, or cloud resources. Git documentation

Before cleanup, preserve the commit remotely and attach the test result to the pull request. Include the command that ran and any failures. If a reviewer needs the preview, record which commit it serves and when it expires.

After review, inspect the worktree from the original checkout. Only remove it once its work and useful logs are preserved:

git -C ../agent-issue-142 status --short

# Run only after checking the status and preserving the work.
git worktree remove ../agent-issue-142

Without --force, Git refuses removal if uncommitted changes or untracked files remain. That includes the test.log from our earlier example. A clean status doesn't prove the commits were pushed, so check that separately.

Removing the worktree leaves the branch and cloud resources in place. Even stopping an EC2 instance is different from terminating it. The infrastructure needs its own cleanup step and retention rules. EC2 documentation

Try one failed task before running ten agents

Before expanding this setup, deliberately run a task that cannot finish. Let its deadline expire. Then check whether the agent stopped, whether the logs survived, and which resources remain active.

Repeat with a successful task whose reviewer is unavailable until the next day. The preview should remain available for the promised window without requiring the agent to keep working.

Those two cases would tell me more about readiness than another successful demo. I want to be able to close the laptop knowing what will still be running tomorrow—and why.

The AI feature nobody is shipping: a system that asks you questions

Danil Galeev — Sun, 04 Oct 2026 10:42:22 +0000

The AI feature nobody is shipping: a system that asks you questions

Open any AI product and you get the same shape: you type a task, it executes, you get a result. Confident and fast. It's also missing the one thing that would have made it correct, which is a question.

I've spent the last year building agents for reporting, CRM, and support workflows. What keeps breaking isn't the model. It's the assumption that the person in front of it already knows how to describe what they want.

The execution trap

We've optimized hard for execution. Give the model a clear task and it will do it, often better than the person who asked. That's real, and it's also where most teams stop thinking.

The trouble starts the moment the request is underspecified, which is most of the time. A real example from my own life. I asked an assistant to sort the email from my kids' schools. Three kids, three schools, a flood of messages about events, forms, and payments. The system said "task received" and started categorizing. It never asked who actually manages the schedule. It never asked whether the information matters differently for a younger child and a teenager. It never asked who picks the kids up on which day.

I got back exactly the task I typed. Not the task I had.

The person who actually owns that calendar is not me. They read those emails first, they know the schedule, they handle the forms. Any system that wants to help with school logistics has to know that, and it can't know it unless it asks. Instead it treated my typed sentence as the whole context and produced a tidy, useless summary.

Why nobody ships the question

Asking is harder than answering, for reasons that are mostly economic.

An assistant that asks four questions before acting feels slower than one that answers instantly. Product teams optimize for the demo, and the demo rewards a fast, confident answer. A model that replies "before I do this, tell me who owns the schedule" looks like a worse product in a 30-second clip, even when it's the better one.

There's a cost angle too. Every clarifying turn spends tokens and time. At the top of the market that's real money: serious agent work runs into tens of thousands of dollars a month, and a single hour of a strong model working a hard task is priced like a junior contractor. Asking questions multiplies the number of turns, so the incentive is to guess and move on.

There's a safety angle vendors rarely say out loud. A system that interrogates you is a system that collects your context. Once it knows who owns the calendar, where the kids go, and what you care about, it holds a profile of your life. That's the data that makes the product valuable, and the data that makes it a liability.

So the current generation of models does the easy half well. It executes, and it rarely asks.

Why the asking model wins

The lab that ships a model that genuinely interrogates you takes the market, and I don't think it's close.

Execution is turning into a commodity. Every serious model can now take a clear instruction and carry it out. What none of them do well is build the picture of your world that makes the instruction meaningful. The model that asks good questions, and remembers the answers, compounds. Each conversation makes the next one better. You can't copy that by releasing a slightly smarter model.

If even a fraction of the people already using AI assistants were asked a few good questions and had the answers stored, you'd have hundreds of millions of filled-in profiles no competitor can replicate. The value sits in the memory of who the user is, not in the model weights.

Today's models still wait for you to volunteer context. They treat a short prompt as a complete specification. The gap between "it did what I said" and "it understood what I meant" is where the next product cycle lives.

How to build it

If you're building an agent, the question step isn't a nice-to-have. It's a node in your graph, and it deserves the same care as your tool calls.

A rough shape:

intake ──> assess_context
             │
             ├── context sufficient ──> plan ──> execute ──> respond
             │
             └── context missing ──> ask ──> (wait for answer) ──> assess_context

Two rules I'd put on that node:

Ask before acting when the stakes are non-trivial. A password reset is fine to just do. Anything touching money, access, schedules, or other people should trigger a question first.
Persist what you learn. A question you ask once and forget is worse than no question at all. The answer belongs in durable memory, keyed to the user, and it should change how the next request is handled.

The loop is the easy part. Deciding what to ask is the hard part, and it's a product problem, not a model problem. The best questions resolve the most ambiguity with the least friction: who owns this, what does "done" look like, what should I never do without asking. Get those three right and the rest follows.

The wider shift

The asking model is one piece of a larger change in how I build with AI.

Stop chasing the word "agent." It's too abstract, and it pushes you to optimize one small piece of a system instead of seeing the whole thing. "An agent that spawned 8,500 sub-agents" sounds impressive and means nothing. I expect the term to fade, the way "prompt engineering" is already fading. Teaching someone to write prompts today is like teaching them to change the screen resolution on an operating system from a decade ago. The models are being built to read your context themselves.

Rebuild the process around the model instead of bolting the model onto the old process. If you're adding an AI module inside a CRM that exists only because software used to be hard to change, you're automating a container that may not need to exist. The useful question isn't how to add AI to this workflow. It's why this workflow has this shape at all.

Pick one primary system and go deep, two at most. Scattering across five half-learned tools is how you end up fluent in none of them.

Access to the strong models will stay expensive and limited. The best systems already sit with a handful of corporations and states. That's not a reason to wait. Build the memory layer, the part that's yours, while the model layer keeps getting rented from someone else.

What I keep coming back to

If I had to bet on one feature for the next year, it isn't a bigger context window or a faster model. It's an assistant that asks me three good questions before it touches anything, and remembers the answers forever.

The model that asks is the model that understands. Everything else is execution, and execution is getting cheap.

I build AI agents for reporting, CRM, and support workflows, with human review where it counts. Based in Potsdam, Germany.

I built an L1 ticket deflector that knows when to stop: LangGraph + human-in-the-loop

Danil Galeev — Sat, 03 Oct 2026 22:57:52 +0000

I built an L1 ticket deflector that knows when to stop: LangGraph + human-in-the-loop

The same tickets arrive at every IT service desk: password resets, VPN that won't connect, "how do I install 7-Zip?", a printer that's offline again. A widely cited industry range puts routine requests at 50–70% of L1 volume — work a script could handle, except nobody trusts a script with the tickets that actually matter.

Building an agent that answers everything is easy. Building one that knows when to hand a ticket to a human is not. I built the second kind and open-sourced it.

Repo: github.com/zedxter/l1-ticket-deflector (MIT)

The design decision that shapes everything

Most "AI helpdesk" demos optimize for deflection rate. That's the wrong target. A high deflection rate is trivial if you let the model guess.

The constraint I optimized for instead: never let the agent act on a sensitive request. Anything touching access, privileges, security, or money goes to a human with prepared context. The agent resolves the boring 30% and escalates the rest — with the relevant KB article already attached, so the human doesn't start from zero.

I called it human-in-the-loop, but the honest framing is narrower: the agent has a hard boundary, and that boundary is the part worth testing.

The graph

The production version is a LangGraph state machine. Three outcomes, no cleverness:

classify ──> no match / low confidence ──> queue (general L1 queue)
   │
   ├── sensitivity = low        ──> auto_resolve ──> notify_user
   └── sensitivity = high/crit  ──> human_review ──> escalate ──> notify_user

The classifier returns a KB article ID and a confidence score. Two gates decide the path:

Confidence gate — below 0.55, the ticket goes to the general queue. No guessing.
Sensitivity gate — high/critical tickets go to service_desk_l2 or security_ops, never to auto-resolve.

Each KB article carries a sensitivity label. That's the trick: the model doesn't decide what's dangerous. The knowledge base does, and it's a human-authored field. The LLM only picks the article.

def route_after_classify(state):
    if not state.get("article_id") or state.get("confidence", 0) < CONFIDENCE_THRESHOLD:
        return "queue"
    if state.get("sensitivity") in AUTO_RESOLVE_SENSITIVITY:
        return "auto_resolve"
    return "human_review"

That's the entire safety story in six lines. Sensitive categories (access grants, license purchases, phishing reports, ransomware) can never reach auto_resolve, no matter what the model says.

Two versions, on purpose

The repo ships two implementations:

demo/ — a zero-dependency Python script. Keyword matching plus stemming, no LLM. It runs on a Raspberry Pi. Its job is to make the mechanics visible and the numbers checkable.
graph/ — the LangGraph agent. RAG over the KB, structured output via Pydantic, tool-calls into the ITSM, MOCK_MODE=1 so you can run the logic without an API key.

I kept the offline version because a demo you can't run is just a claim. Anyone can clone the repo and reproduce the numbers in ten seconds.

The numbers (and why I'm suspicious of them)

On 45 labeled tickets, the offline classifier scored:

Metric	Value
Auto-resolved	25
Escalated to human	18
No match → queue	2
Decision accuracy	100%
Routing accuracy	100%
Sample deflection	55.6%

100% accuracy is a red flag, not a trophy. It means the test set is too small and too clean. 45 tickets, hand-labeled, written by the same person who wrote the KB — of course it scores well. I'm publishing it because it's honest about what it is: a sanity check, not a benchmark. The real test is a client's messy backlog, and that number doesn't exist yet.

The deflection rate is the one I'd defend. 55.6% on this sample, but the ROI model uses a conservative 30%, because market benchmarks for first-year deflection sit at 20–40% and I'd rather be wrong in the client's favor.

The ROI model

Conservative assumptions for a ~500-person company in DACH:

Metric	Value
L1 tickets / month	~1,200
Deflection	30%
Avg handling time	19 min
IT rate (fully loaded)	€45 / hour
Time saved	~114 h/month (~0.7 FTE)
Savings	~€5,130 / month
Retainer	€4,000 / month
ROI	1.3× / month (15× / year)

The retainer is deliberately below the savings. A model where the vendor captures all the value is a model nobody signs. 1.3×/month is not a spectacular number — it's a defensible one, and defensible is what gets a pilot approved.

What's fake and what's real

I want to be precise about this, because "proof of work" usually means "screenshot of a demo."

Real: the classifier, the graph, the routing logic, the KB structure, the ROI math. Clone it, run it, read every line.

Stubbed: the tool-calls. itsm_create_ticket, directory_reset_password, and mdm_push_install return strings, not real ITSM actions. Wiring them to Jira Service Management or Zammad over MCP is a pilot task, not a demo task.

Not built: the Slack/Teams approval UI, telemetry, multilingual KB. They're on the roadmap and I'm not pretending otherwise.

What I'd tell anyone building this

Three things I'd carry into the next version:

Put the safety boundary in data, not in the prompt. A sensitivity field on each KB article is auditable. "The model is instructed to be careful" is not.
Ship the runnable offline version. It costs an afternoon and buys all the credibility.
Publish the suspicious number. 100% accuracy invites the right question — "on how many tickets?" — and answering it honestly is worth more than a flattering metric.

The repo is MIT-licensed and the whole point is that you can pull it apart. If you run an IT desk and want to compare deflection on your own tickets, that's the interesting experiment.

Repo: github.com/zedxter/l1-ticket-deflector

I build AI agents for reporting, CRM, and support workflows — with human review where it counts. Based in Potsdam, Germany.

Updating Xbox controller firmware from Linux: the USB hotplug trap

Danil Galeev — Tue, 29 Sep 2026 18:46:57 +0000

I wanted to update two Xbox controllers without booting my laptop into Windows. Both eventually went from 5.9.2709.0 to 5.23.6.0, with Xbox Accessories confirming Updated! and then No update available for each one.

The update reset the USB connection, and getting the controller back into the VM took most of the work.

The scripts and setup guide are in zedxter/xbox-firmware-linux.

This uses Microsoft's official updater inside a Windows VM hosted on Linux. It does not implement a native Linux firmware flasher or distribute firmware files.

Tested setup and limits

The successful session used openSUSE Tumbleweed with SELinux enforcing, KVM, rootful Docker, Windows 10 22H2 and two Microsoft controllers with USB ID 045e:0b12.

I turned the helpers from this session into configurable scripts. The portable version has tests for event filtering, generated configuration and cleanup protections, but the complete refactored workflow has not yet been retested on hardware. Windows 11 is the default for a new setup; that guest path also remains unvalidated. Firmware 5.23.6.0 was the version Accessories installed on these two controllers; other models may receive a different version.

Why the controller disappeared after USB reset

A firmware update can reset a controller's USB connection. The device address can change, and a bootloader may use a different product ID.

QEMU supports selecting a physical USB port, so I used a configuration shaped like this:

-device usb-host,hostbus=3,hostport=1,id=xbox-controller

The physical port 3-1 is an example. The scripts detect the port you select and generate the arguments.

After a reset, Linux could see the controller, and a fresh libusb context inside the container could see it too. The running QEMU process still could not reacquire it.

The device node was accessible and had the correct SELinux label. QEMU's cached USB object was misleading: its existence did not prove a working connection.

The evidence pointed toward missing hotplug events. libusb's Linux backend consumes processed udev notifications. A USB bind mount provides device access, but does not automatically provide those host events inside a separate network namespace.

systemd's device monitor can disable its udev subscription when /run/udev/control is absent and /dev is not devtmpfs.

I added a monitor marker and a small relay. It forwards original root-origin udev packets only for Microsoft USB device add/remove events on the selected physical port. It excludes USB interfaces and unrelated ports. The repository's architecture notes explain the checks and access boundaries.

Step 1: prepare the host

You need a Linux x86-64 host with KVM, local rootful Docker Engine, Compose v2, Python 3.10+, systemd/udev, and a USB data cable. Allow room for a 4 GB Windows guest and preferably at least 80 GB of free storage. On SELinux hosts, the helpers also use chcon and restorecon.

Disconnect other wired Xbox controllers: preparation temporarily unloads their shared xpad driver. Review the scripts before running them as root.

git clone https://github.com/zedxter/xbox-firmware-linux.git
cd xbox-firmware-linux
python3 scripts/configure.py --list

# Replace 3-1 with the controller's actual physical port.
python3 scripts/configure.py --port 3-1

Configuration creates a local Compose file, a random Windows password in an ignored .env, and storage for the VM. No passwords or VM disks are included in the repository. Windows licensing remains your responsibility.

Step 2: start Windows and the relay

sudo docker compose -f compose.local.json create
sudo python3 scripts/host_session.py prepare
sudo docker compose -f compose.local.json start

If preparation fails, keep the container stopped and run sudo python3 scripts/host_session.py cleanup before fixing the error and retrying.

In another terminal, from the same directory:

sudo systemd-inhibit --what=sleep:idle --mode=block \
  --who=XboxFirmwareUpdate --why='Xbox controller firmware update' \
  python3 -u scripts/relay.py

Keep that terminal open. Look for READY and then FORWARDED add. Keep the laptop on power with its lid open.

Open http://127.0.0.1:8006 and let Windows install. The setup builds on dockur/windows; the repository pins the image used in the original session. The console is bound to localhost.

The relay filters its events. The VM still has privileged USB access, so run it as a trusted maintenance tool.

Step 3: prepare Accessories, then test reconnect

Install available Windows updates, finish guest restarts, and install Xbox Accessories from the Microsoft Store.

We encountered three different blockers:

Windows OS update required: updating Windows resolved it.
Gaming Runtime Services sign-in failed: installing the official Xbox app and signing in resolved our case.
Device Manager reported a working controller, but Accessories did not show it: uninstalling the device without deleting the driver package, followed by a Windows restart, restored discovery.

I only needed these fixes when the corresponding error appeared.

Before starting any firmware update, unplug and reconnect the controller once to the same physical port. Accessories must find it again without a Docker restart, and the relay should report an add event.

sudo python3 scripts/status.py

If that pre-flash test fails, stop and fix discovery first. A guest restart preserves the relay; restarting the Docker container requires starting a new relay.

Step 4: update one controller at a time

Start the available update in Xbox Accessories. Do not disconnect USB, stop the relay, suspend Linux or restart Windows/Docker while updating.

Wait for the explicit success screen. Then reopen the controller details and verify the full firmware version and update status.

The first controller needed a Windows restart after the success screen before Accessories would read its new version. The second reported its new version immediately. Both were confirmed at 5.23.6.0.

Only after verifying the first controller should you swap in the second, using the same cable and port. Repeat the reconnect test and update.

Step 5: finish the session

After every update is finished, shut Windows down through its Start menu. Wait for the container to exit:

sudo docker inspect -f '{{.State.Status}}' xbox-firmware-linux

If Windows is fully shut down but Docker remains running, stop it with sudo docker compose -f compose.local.json stop. Never use this to interrupt firmware updating: Docker can force-stop after its grace period.

The relay exits with its container process. Restore the host settings:

sudo python3 scripts/host_session.py cleanup

The cleanup command refuses a running VM. It removes matching temporary rules, restores the USB label and restores the previous xpad loaded state. Reconnect the cable for Linux. The Windows disk is retained for later use and should remain private.

Firmware did not solve every Bluetooth problem

After updating, the first controller could pair but still failed to reconnect after being turned off and on. A clean reboot and fresh pairing did not resolve it.

On our MediaTek adapter with BlueZ 5.87, testing this option made reconnection work:

[LE]
CentralAddressResolution=0

This is a separate, adapter-wide workaround, not a universal Xbox setting. It does not delete pairing keys or disable encryption. Only the first controller's Bluetooth reconnect was independently verified. The repository includes instructions and rollback guidance; the scripts do not apply it automatically.

For general pairing cleanup after firmware changes, consult xpadneo's troubleshooting guide.

Share reproducible results

If you try the toolkit, report your controller model, host and guest versions, whether the pre-flash reconnect test passed, and the firmware version Accessories confirmed. Remove serial numbers, Bluetooth addresses and keys from logs.

I would test USB rediscovery before clicking Update again. If the controller does not return to Accessories after a simple cable reconnect, a firmware reset is unlikely to go better.

Agent Memory's Real Failure Is Currency, Not Retrieval

Danil Galeev — Wed, 23 Sep 2026 08:14:32 +0000

Retrieval returns the correct record. The record is stale. And because the model states fresh and dead facts with identical confidence, nothing in the output tells you which one you got.

Your agent's memory isn't broken because it can't find the fact. It's broken because the fact it found stopped being true.

That is not a retrieval problem

It's a state-management problem — the same one distributed systems have always had: validity windows, invalidation, cache coherency. A memory layer with none of those is a cache that never expires.

What the numbers say

Memory is now a first-class layer with its own benchmarks:

LoCoMo 92.5
LongMemEval 94.4
BEAM 1M 64.1

...at roughly 6,900 tokens per query on average. The biggest gains are in temporal reasoning (+29.6) and multi-hop (+23.1). And the open problems named in the same report are cross-session identity, temporal abstraction at scale — and staleness.

Stale facts get surfaced. They just don't get acted on.

On the STALE benchmark — a separate 2026 study on stale-fact handling:

a system surfaced the updated fact about 77% of the time, but marked it as worth acting on only ~3%;
one frontier model caught a stale fact 92% of the time when asked directly — and 30% when the question silently depended on it.

The fact wasn't missing. It was present, and discounted.

The pattern that works

Zep's Graphiti shows the shape of the fix: store facts with validity dates, mark contradicting facts invalid instead of overwriting them, and surface only the current version.

The rule

Memory is not "accumulate more." A fact needs a bounded window of validity — otherwise you're shipping an agent that is confidently wrong on a schedule.

How are you handling staleness in your agent's memory layer?

A weekly reporting workflow that stops when its sources are incomplete

Danil Galeev — Wed, 16 Sep 2026 13:18:03 +0000

A weekly report can contain plausible numbers and still be unsuitable for a decision. A missing support export or an old CRM snapshot changes what the report can honestly say.

I built a small reporting demo for MadeBy.Expert to make those conditions visible. It uses synthetic billing, CRM and support snapshots, deterministic calculations and a template narrative. There are no live integrations or language model calls in this version.

Define the numbers before automating the report

The sample distinguishes monthly recurring revenue from revenue collected during the week. Open pipeline is the unweighted value of open opportunities, not a revenue forecast. Support metrics distinguish unresolved tickets at the snapshot from tickets resolved within the reporting period.

Each metric includes a source reference and a definition. Those details give the reviewer something concrete to check against the input.

For a client workflow, I would agree on these definitions before connecting the source systems.

A missing source is a report state

The browser demo lets you switch between complete sources, missing support data and stale CRM data. The report keeps the metrics it can calculate, omits the unavailable ones and records a blocker. It does not substitute zero for a missing value.

The final state comes from that blocker list:

status: blockers.length ? "blocked" : "draft",

For this example, all three sources are required for approval. A snapshot is stale if it was captured before the reporting period ended or more than 24 hours before the report's asOf time. A future timestamp is rejected as invalid input.

Those are choices for this workflow. A daily operations report might need a tighter freshness limit; another report might be useful with an explicitly optional source. The thresholds need agreement with the person using the result.

Here is the actual test covering all three sources and both failure modes. It uses Node's built-in test runner and strict assertions. fixture() loads the synthetic input afresh for each case:

test("missing/stale sources are unavailable and block approval", () => {
  for (const kind of ["billing", "crm", "support"])
    for (const mode of ["missing", "stale"]) {
      const input = fixture();
      if (mode === "missing") delete input.sources[kind];
      else input.sources[kind].capturedAt = "2026-09-12T08:00:00Z";
      const r = buildReport(input);
      assert.equal(r.status, "blocked");
      assert.ok(r.blockers.some((b) => b.startsWith(kind + ":")));
      assert.equal(
        r.metrics.some((m) => m.sourceId === fixture().sources[kind].id),
        false,
      );
      assert.throws(
        () => approveReport(r, r.digest, "Demo reviewer"),
        /Blocked/,
      );
    }
});

The last assertion matters: reporting the problem in the output is insufficient if the approval function still accepts it.

Approval applies to one version of the report

The implementation computes a SHA-256 digest of the report content, which includes a digest of the input. A reviewer supplies the report digest when approving a local export. The approval function recomputes it and checks that the report is still a draft without blockers:

export function approveReport(
  report,
  digest,
  reviewer,
  now = new Date().toISOString(),
) {
  const { digest: stored, ...content } = report;
  if (hash(content) !== stored || digest !== stored)
    fail("Approval digest does not match the current report");
  if (report.status !== "draft" || report.blockers.length)
    fail("Blocked reports cannot be approved");
  if (
    typeof reviewer !== "string" ||
    reviewer.trim().length < 2 ||
    reviewer.length > 100 ||
    /[\r\n]/.test(reviewer)
  )
    fail("A named reviewer is required");
  return {
    ...report,
    status: "approved",
    approval: { reviewer: reviewer.trim(), at: now, digest },
  };
}

Here, hash is SHA-256 over JSON.stringify(value) and fail throws an error. This is an excerpt from the local prototype, not a standalone approval service.

Rebuilding the report after changing the input produces a different digest, so the previous digest cannot approve the new report. The tests also alter a calculated metric directly and check that approval rejects the modified content.

There are limits to this approach. The digest is a consistency check, not a signature or proof of who reviewed the report. The reviewer name is supplied as text; there is no authentication or authorization layer. The JSON serialization is also specific to this implementation, not a canonical format for independent producers.

A deployed approval service would need to establish the reviewer's identity and permissions, store the review event, and bind the delivery action to the approved version. Those parts are not implemented here.

The browser page shows the data conditions and report. The local runnable example handles approval and export; neither sends a report automatically.

Where AI could fit later

A language model might help draft commentary from validated facts. That is a possible extension, not an implemented feature of this demo. I would keep metric calculation, source checks and the approval boundary explicit regardless of how the narrative is produced.

The current prototype makes no claim about client time savings, live connector reliability or model quality. Those need measurement against an actual workflow.

Try the demo, including the missing-source and stale-source cases.

If your team assembles a similar report manually, describe the workflow: which tools supply the data, how often the report is needed, and who checks it. We can discuss whether a small paid pilot with clear success criteria fits.

Typed Contracts Between AI Agents: The Interface That Actually Breaks

Danil Galeev — Tue, 15 Sep 2026 05:44:04 +0000

Most multi-agent failures I've seen don't happen inside an agent. They happen between two of them.

When the handoff between two agents is treated as loosely structured JSON or implicit shared state, the interface is underspecified — and underspecified interfaces fail quietly.

A production handoff should be treated like any other API contract:

an explicit input schema;
an explicit output schema;
required and optional fields;
semantic constraints, not only valid JSON;
a clear rejection and fallback path.

A concrete example

A research agent should not hand an unstructured object to a scoring agent. It should emit something the scorer can actually validate:

class ResearchResult(BaseModel):
    subject_id: str
    evidence: list[EvidenceItem] = Field(min_length=1)
    source_count: int = Field(ge=1)
    confidence: float = Field(ge=0, le=1)

The scorer runs only after that result passes validation. That gate is the important part.

Probabilistic output, consequential action

An LLM output is probabilistic. A database write, an external message, or a business decision is an action. Those two steps should not be directly coupled.

Between them, put a deterministic gate:

Parse the model output.
Validate its schema and business rules.
Reject, retry, route to review, or stop the branch when validation fails.
Execute the action only on an accepted contract.
Record the handoff version and validation result.

Why it changes failure behavior

Without a contract, a missing field becomes a null value, the null becomes a misleading score, and the score becomes a bad CRM update. The symptom shows up several steps away from the cause, and by then the trail is cold.

With contracts, the same failure is localized: ResearchAgent -> ScoringAgent, field source_count, validation rule failed.

That is more than defensive programming. It is architecture for software systems whose internal steps are non-deterministic.

It gives staff engineers independently testable boundaries, helps principal engineers reduce coupling between teams and agents, and gives architects and CTOs a clearer way to reason about compatibility, ownership, and recovery.

The rule

Every probabilistic step that feeds a consequential action needs an explicit validation gate. Typed contracts are one strong way to define that boundary.

The model can be creative inside the boundary. The boundary should not be.

Where have you found the most valuable contract in an agent workflow — between agents, between an agent and a tool, or before the final action?

Building AI Agents That Actually Work: What Nobody Tells You

Danil Galeev — Fri, 04 Sep 2026 09:11:40 +0000

I've been building AI agents for a while now. Not chatbots. Not RAG demos. Real agents that take actions, make decisions, and run autonomously.

Here's what nobody tells you.

The gap between demo and production

Every AI agent framework shows you a 5-line demo that works perfectly. Then you deploy it, and it fails in ways you didn't imagine.

The reason is simple: a demo is a happy path. Production is a graph of failure states.

1. Observability is not optional

Your agent is only as good as your ability to see what it's doing.

When an agent makes a wrong decision, you need to know exactly why. Was it a bad prompt? A hallucinated tool call? Missing context from a previous step?

Log every thought. Audit every action. If you can't replay an agent's decision process, you can't trust it.

2. Tools are the bottleneck, not the LLM

Most people think the LLM is the hard part. It's not. The hard part is the tools.

Your agent needs to call APIs, read databases, write files, send emails. Each of those is a failure point. Network timeout. Auth expired. Schema changed. Rate limited.

Build your tool layer like you build a distributed system. Retries. Circuit breakers. Timeouts. Graceful degradation.

3. The context window is a trap

Long-running agents accumulate context. The more they do, the more context they carry. Eventually, the context window fills with noise, and the agent starts making bad decisions.

Strategies that actually work:

Summarization — compress old context into summaries, don't carry raw history
Structured memory — separate short-term (current task) from long-term (learned patterns)
Forgetting — actively prune irrelevant context. If it didn't matter in the last 10 steps, it probably won't matter now

4. Determinism is underrated

Everyone wants creative agents. What you actually want is predictable agents.

A creative agent that hallucinates a solution is useless. A predictable agent that follows a known pattern is valuable.

Design for determinism first. Add creativity as a controlled parameter.

5. The orchestration layer matters more than the model

You can swap GPT-4 for Claude or Gemini and your agent still works — if your orchestration is solid.

Good orchestration:

Clear state machine
Explicit error handling
Human-in-the-loop for critical decisions
Audit trail for every action

The uncomfortable truth

Building AI agents that actually work is not an AI problem. It's a software engineering problem.

The LLM is the easiest part. Everything around it — tools, observability, state management, error handling, orchestration — that's where the real work is.

And that's also where the real value is.

Your Agent Isn't Losing Memory. It's Rotting.

Danil Galeev — Fri, 28 Aug 2026 18:33:00 +0000

Agents don't forget because the model is bad. They forget because we built it into them. Six hours and forty messages in, what an agent needs is gone — or still there but wrong, drowned in noise.

That's context rot.

I've spent the last month inside a team of always-on AI agents that run for weeks. Different jobs, one codebase, one shared vault of lessons. They review each other's PRs, publish, plan. And they rot. I watched every flavor of it, then built the countermeasures.

What context rot is

Four failures that compound.

Compression loss. When a session runs out of window, a summary replaces raw history. It sounds fine until the agent needs the detail it summarized away. I watched an agent repeat a plan that contradicted a decision two screens earlier. Nobody noticed, because the contradiction lived at a depth that was gone.

Drift. Each turn pulls the agent toward the recent and the loud. Given ten pieces of context, it weights the last five. Over hundreds of turns it quietly solves a different problem than the one you assigned.

Priority blurring. "We decided X" and "we looked at X" feel the same in the tokens. Decisions stop being binding and become suggestions.

Noise accretion. Stale output and dead ends pile up. The signal-to-noise ratio falls, and a degraded ratio looks like a dumber model.

The part nobody warns you about

The cheap fix is "just enlarge the window." I built that reflex out of myself. Longer windows don't fix rot, they defer it — later in the timeline, larger cost per turn, and past some point the model does visibly worse than with a lean window. Remembering everything is a form of forgetting: the important thing drowns.

And rot is per-profile, not per-agent. In a team sharing a codebase, what agent A rotted away still matters to agent B. So the fix must live outside any single session. Memory in one agent's head dies with that session.

What actually works

None of this is exotic.

Summarize to executable facts, not prose. Emit decisions and constraints as explicit, fixed-schema entries the agent can act on. A good summary answers "what is still binding" before "what happened."
Move memory out of the session into a semantic index. Our biggest win. Keep lessons and decisions in a plain markdown vault, index with embeddings, and search returns meaning, not exact strings. "Offer-notification" and "pet-booking" are unrelated strings and near-identical problems; string search never connects them, the graph does — in milliseconds, on one SQLite file and a small local embedding model. No vector DB, no cluster, no bill. Don't ask one long-lived brain to do all the retrieval.
Turn knowledge into skills, not notes. A lesson read once and never referenced is a lesson that rots. The durable form is a skill: a procedure loaded on demand behind a trigger, pulled into context exactly when it applies and out the rest of the time. That's the difference between a memory of what to do and a memory of when to use it. The trigger is the part to get right.
Layer memory by half-life. Instant preferences live in a compact always-on store; entity facts in a structured store you can query; long-lived conventions in a vault you load on purpose. Each level has its own audit cadence, so the always-on layer never bloats and the deep layer never goes stale silently. Cap and prune the always-on store on a schedule: an always-on memory that grows forever is context rot in slow motion.

Patterns to rot-proof an agent

Whatever you pick, these hold.

Make forgetting visible. If a session drops something, drop it loudly: what entered context, what left it, why. You can't manage decay you can't see.

Bind decisions harder than facts. A decision needs a source, a status, an owner. A degraded decision is an incident, not a nuisance.

Retrieve, don't carry. Pull the relevant slice on demand instead of carrying a fat context everywhere. The leaner the working set, the slower the rot.

One source of truth for shared rules. When agents cooperate, conventions belong in one place anyone can read, not duplicated into private memory where they rot out of sync and you end up fighting three versions of the same rule.

Treat memory as code. Schema changes, evals, tests. The vault is source, the index is a produced artifact: rebuild it on a cron and never hand-edit it. A knowledge graph you can't rebuild from source is a liability.

The uncomfortable take

You never beat context rot. You manage it. Design so the cost of rotting is contained and observable, and so the source of truth survives the session because it lives outside it.

Our agents now run week-long cycles with far less decay than the first version at hour three. The difference was never a better model. It was admitting that an agent's context is a perishable working surface, not a mind, and engineering it that way.

I keep getting asked which framework solves this. The answer stays boring: write good compression, index for meaning, encode lessons as triggered skills, cap the always-on memory, and keep long-lived truth somewhere the agent can query and rebuild instead of carry. The agent will look cooler carrying everything. It will also rot faster.

So: how do you decide what an agent is allowed to forget, and how do you make sure it tells you when it does?

When Should You NOT Use an Agent?

Danil Galeev — Thu, 27 Aug 2026 19:21:36 +0000

Everyone is asking "should we use agents?" The real question is "when should we NOT?"

I keep seeing teams bolt an agent on because it's the hot thing — then discover they reinvented a state machine with worse debugging. Agents don't solve a problem by existing. They are a mechanism for deferring decisions to a runtime. When your inputs, tools, and failure modes are well-understood, that deferral buys you nothing but nondeterminism.

The architecture question is not "LLM or not." It is: where does the judgment boundary sit?

Three places the boundary lands

A single agent is a program. For a well-scoped task with a known toolset, you don't need a loop at all. You need a deterministic pipeline — with the LLM as one component, not the orchestrator.

The moment you need a loop, you're building a runtime. A runtime is a different beast. It has observability, tool permissions, credit and rate limits, and a way to explain what it did after the fact. That's not "more AI." That's distributed systems with a language model as the cognitive layer.

The expensive failure is capability you never signed up for. Agents surface things you didn't design for: open-ended tool calls, emergent side-effects, scale that hits budgets or audit. This is where maturity shows — not in the cleverness of the model, but in the constraints around it: entitlements, approval, observability, evals that cover the failure path, not just the happy path.

The tell

Most of what teams call "agent architecture" is a decision-making boundary placed in the wrong spot, papered over with more layers. If you reach for a framework, an orchestrator, a runtime — stop and ask what you're actually deferring, and whether that deferral is working or just making the system harder to debug.

A few heuristics I use before building:

Can the task be expressed as steps with a known order? Pipeline with the LLM as a step — not an agent.
Does the agent invoke known tools with expected outputs? A single governed agent, thin on top of a deterministic core.
Do you need to route mid-task to unexpected states? Only then a real runtime, and only if you accept owning its observability, limits, and audit trail as first-class work.
Is the failure non-recoverable? Then don't put an agent in the loop at all. A wrong tool call inside a runtime can hurt you faster than a slower deterministic path ever will.

Constrain before you automate. The agent looks cooler; it's also much harder to explain three months from now.

So: when you're scoping a new system, what makes you reach for a deterministic pipeline instead of an agent — or the other way? I'd like the rules teams actually run with, not the ones they present in talks.