DEV Community: MadeBy.Expert

The reconciliation step where we use no model at all

Danil Galeev — Sun, 11 Oct 2026 13:42:31 +0000

Our Lead Agent demo prepares a discovery brief for a fictional prospect. The prospect reconciles 3,000 subscriptions a week between HubSpot and Stripe, found 240 discrepancies last week and wants a read-only discrepancy report for an operations lead.

A good brief prepares the first conversation. It does not solve the reconciliation. To make that boundary concrete, we built the reconciliation itself as a separate checked example. It makes no model calls.

Everything below uses a constructed dataset with known answers. There is no live HubSpot or Stripe connection.

Write the comparison contract first

Most of the difficulty in reconciliation is deciding what "the same" means. The example states its contract explicitly:

rows join on one stable subscription ID, exactly, with no fuzzy matching;
both sides must describe the same monthly period;
amounts are EUR, as non-negative safe integer cents;
statuses come from one agreed vocabulary: active, paused, cancelled, past_due.

These are normalized comparison fields, not raw provider schemas. For a real system, someone has to decide whether an amount includes tax, discounts, proration and refunds, and how each tool's statuses map to the shared vocabulary. That decision belongs to the customer's finance or operations owner, not to a model.

Every pair gets one outcome and its reasons

The engine indexes both sides by subscription ID and gives every ID one of three outcomes:

Outcome	When	Amount delta
`match`	Both rows valid, same amount and status	0
`discrepancy`	Amount or status differs, or one side is missing	Known, or `null` when a side is missing
`review`	Duplicate ID, period mismatch, unsupported currency, invalid amount, unknown status	`null`

The core of the comparison is short. Simplified from the demo:

if (reasons.length) outcome = "review";
else {
  amountDeltaCents = billing.amountCents - crm.amountCents;
  if (!Number.isSafeInteger(totalDeltaCents + amountDeltaCents))
    throw new Error("Amount total exceeds safe integer range");
  if (amountDeltaCents) reasons.push("amount_mismatch");
  if (crm.status !== billing.status) reasons.push("status_mismatch");
  outcome = reasons.length ? "discrepancy" : "match";
}

Two choices matter here. A missing counterpart is flagged as a discrepancy, but its monetary value stays unknown: we do not invent the amount of a row that is not there. And if the running total would leave the safe integer range, the run stops instead of rounding.

Each outcome keeps its source rows and reason codes, so a reviewer can see why a pair was flagged without re-running anything.

Check against answers the engine never sees

A reconciliation that reports 240 discrepancies looks right when you expected 240. That is not a test. We needed an independent answer for every row.

The generator produces 3,000 CRM rows, 3,000 billing rows and a separate expected-result manifest:

Expected case	Count
Matching subscriptions	2,730
Amount differences of +€5.00	180
Status differences	60
Unsupported currency (USD), review	30

The engine does not read the manifest. A verifier compares every pair afterwards: classification, reason codes and per-pair amount delta, plus the supported total.

On this dataset, the run classifies all 3,000 pairs as expected. It finds all 240 planted discrepancies with no false positives and no missed cases, and every reason code matches. The supported amount delta is €900.00, with zero cents of error. The comparison itself runs in milliseconds; the build regenerates and re-verifies it each time.

What the numbers mean

The €900 is an amount difference to investigate. It is not recovered revenue or a customer saving.

It also does not cover everything. The 30 USD pairs remain unresolved in the review queue, excluded from the supported total, because the contract has no currency conversion rule. Converting them at some assumed rate would produce a tidier total and a less honest one.

The public page publishes the summary, four source-linked examples, the 30-item review queue with pending decisions, and an evidence file with every input row, expected case and actual outcome. Anyone can check the classifications without our source code.

Why no model?

Joining on an exact ID and subtracting integers are tasks where a model adds cost, latency and a new failure mode without adding judgment. We use models elsewhere in the same story: a decision model selects discovery context, and a generative model drafts the Lead brief. Neither is asked to certify financial arithmetic.

Where a model might help later is in the work around the comparison: explaining a cluster of review cases, or drafting questions for the person who owns the status mapping. That would be a separate, measured addition, not a replacement for the deterministic check.

What this does and does not establish

Zero errors here establishes that this implementation passes its own controlled check on a constructed dataset with declared rules. It does not establish production accuracy, robust entity matching, correct interpretation of a customer's rules, reviewer acceptance, reduced investigation time or savings.

For a real pilot, the useful promise is a process:

Agree the comparison rules with the people who own them.
Check the implementation against an approved reference set, including representative exceptions.
Keep ambiguous cases visible instead of forcing them into a total.
Measure how much investigation work remains for the team.

Only after those steps can anyone make a correctness or economic claim.

You can inspect the reconciliation example and download its evidence. If your team reconciles records between two systems every week, a sample export and your current matching rules are a good starting point for a pilot conversation.

I built an L1 ticket deflector that knows when to stop: LangGraph + human-in-the-loop

Danil Galeev — Sat, 03 Oct 2026 22:57:52 +0000

I built an L1 ticket deflector that knows when to stop: LangGraph + human-in-the-loop

The same tickets arrive at every IT service desk: password resets, VPN that won't connect, "how do I install 7-Zip?", a printer that's offline again. A widely cited industry range puts routine requests at 50–70% of L1 volume — work a script could handle, except nobody trusts a script with the tickets that actually matter.

Building an agent that answers everything is easy. Building one that knows when to hand a ticket to a human is not. I built the second kind and open-sourced it.

Repo: github.com/zedxter/l1-ticket-deflector (MIT)

The design decision that shapes everything

Most "AI helpdesk" demos optimize for deflection rate. That's the wrong target. A high deflection rate is trivial if you let the model guess.

The constraint I optimized for instead: never let the agent act on a sensitive request. Anything touching access, privileges, security, or money goes to a human with prepared context. The agent resolves the boring 30% and escalates the rest — with the relevant KB article already attached, so the human doesn't start from zero.

I called it human-in-the-loop, but the honest framing is narrower: the agent has a hard boundary, and that boundary is the part worth testing.

The graph

The production version is a LangGraph state machine. Three outcomes, no cleverness:

classify ──> no match / low confidence ──> queue (general L1 queue)
   │
   ├── sensitivity = low        ──> auto_resolve ──> notify_user
   └── sensitivity = high/crit  ──> human_review ──> escalate ──> notify_user

The classifier returns a KB article ID and a confidence score. Two gates decide the path:

Confidence gate — below 0.55, the ticket goes to the general queue. No guessing.
Sensitivity gate — high/critical tickets go to service_desk_l2 or security_ops, never to auto-resolve.

Each KB article carries a sensitivity label. That's the trick: the model doesn't decide what's dangerous. The knowledge base does, and it's a human-authored field. The LLM only picks the article.

def route_after_classify(state):
    if not state.get("article_id") or state.get("confidence", 0) < CONFIDENCE_THRESHOLD:
        return "queue"
    if state.get("sensitivity") in AUTO_RESOLVE_SENSITIVITY:
        return "auto_resolve"
    return "human_review"

That's the entire safety story in six lines. Sensitive categories (access grants, license purchases, phishing reports, ransomware) can never reach auto_resolve, no matter what the model says.

Two versions, on purpose

The repo ships two implementations:

demo/ — a zero-dependency Python script. Keyword matching plus stemming, no LLM. It runs on a Raspberry Pi. Its job is to make the mechanics visible and the numbers checkable.
graph/ — the LangGraph agent. RAG over the KB, structured output via Pydantic, tool-calls into the ITSM, MOCK_MODE=1 so you can run the logic without an API key.

I kept the offline version because a demo you can't run is just a claim. Anyone can clone the repo and reproduce the numbers in ten seconds.

The numbers (and why I'm suspicious of them)

On 45 labeled tickets, the offline classifier scored:

Metric	Value
Auto-resolved	25
Escalated to human	18
No match → queue	2
Decision accuracy	100%
Routing accuracy	100%
Sample deflection	55.6%

100% accuracy is a red flag, not a trophy. It means the test set is too small and too clean. 45 tickets, hand-labeled, written by the same person who wrote the KB — of course it scores well. I'm publishing it because it's honest about what it is: a sanity check, not a benchmark. The real test is a client's messy backlog, and that number doesn't exist yet.

The deflection rate is the one I'd defend. 55.6% on this sample, but the ROI model uses a conservative 30%, because market benchmarks for first-year deflection sit at 20–40% and I'd rather be wrong in the client's favor.

The ROI model

Conservative assumptions for a ~500-person company in DACH:

Metric	Value
L1 tickets / month	~1,200
Deflection	30%
Avg handling time	19 min
IT rate (fully loaded)	€45 / hour
Time saved	~114 h/month (~0.7 FTE)
Savings	~€5,130 / month
Retainer	€4,000 / month
ROI	1.3× / month (15× / year)

The retainer is deliberately below the savings. A model where the vendor captures all the value is a model nobody signs. 1.3×/month is not a spectacular number — it's a defensible one, and defensible is what gets a pilot approved.

What's fake and what's real

I want to be precise about this, because "proof of work" usually means "screenshot of a demo."

Real: the classifier, the graph, the routing logic, the KB structure, the ROI math. Clone it, run it, read every line.

Stubbed: the tool-calls. itsm_create_ticket, directory_reset_password, and mdm_push_install return strings, not real ITSM actions. Wiring them to Jira Service Management or Zammad over MCP is a pilot task, not a demo task.

Not built: the Slack/Teams approval UI, telemetry, multilingual KB. They're on the roadmap and I'm not pretending otherwise.

What I'd tell anyone building this

Three things I'd carry into the next version:

Put the safety boundary in data, not in the prompt. A sensitivity field on each KB article is auditable. "The model is instructed to be careful" is not.
Ship the runnable offline version. It costs an afternoon and buys all the credibility.
Publish the suspicious number. 100% accuracy invites the right question — "on how many tickets?" — and answering it honestly is worth more than a flattering metric.

The repo is MIT-licensed and the whole point is that you can pull it apart. If you run an IT desk and want to compare deflection on your own tickets, that's the interesting experiment.

Repo: github.com/zedxter/l1-ticket-deflector

I build AI agents for reporting, CRM, and support workflows — with human review where it counts. Based in Potsdam, Germany.

Agent Memory's Real Failure Is Currency, Not Retrieval

Danil Galeev — Wed, 23 Sep 2026 08:14:32 +0000

Retrieval returns the correct record. The record is stale. And because the model states fresh and dead facts with identical confidence, nothing in the output tells you which one you got.

Your agent's memory isn't broken because it can't find the fact. It's broken because the fact it found stopped being true.

That is not a retrieval problem

It's a state-management problem — the same one distributed systems have always had: validity windows, invalidation, cache coherency. A memory layer with none of those is a cache that never expires.

What the numbers say

Memory is now a first-class layer with its own benchmarks. Mem0, for example, reports 92.5 on LoCoMo and 94.4 on LongMemEval for its managed platform, at about 6,900 tokens per retrieval call. These are the vendor's own figures as published on 13 October 2026; I have not reproduced them.

High retrieval scores answer one question: can the system find the fact? They don't answer whether the fact it found is still true.

Stale facts get surfaced. They just don't get acted on.

The STALE benchmark (May 2026) tests exactly that gap: whether an agent notices that a later observation has invalidated something it stored earlier. The paper reports:

for LightMem, the strongest memory-framework baseline in the comparison, the new evidence appeared in retrieval results for 77.5% of the state-resolution and premise-resistance cases, yet only 3.3% of the old entries it retrieved were judged as needing an update;
on the simplest conflict type, Gemini 3.1 Pro scored 92.0% when asked directly whether the old belief still held, and 30.0% when the question presupposed the old state;
even that strongest model reached only 55.2% overall.

The fact wasn't missing. It was present, and discounted.

The pattern that works

Zep's Graphiti shows the shape of the fix: graph edges carry temporal metadata, contradicting facts are handled by invalidating the old edge instead of overwriting it, and queries can ask for the current state or a point in the past.

The rule

Memory is not "accumulate more." A fact needs a bounded window of validity — otherwise you're shipping an agent that is confidently wrong on a schedule.

How are you handling staleness in your agent's memory layer?

A weekly reporting workflow that stops when its sources are incomplete

Danil Galeev — Wed, 16 Sep 2026 13:18:03 +0000

A weekly report can contain plausible numbers and still be unsuitable for a decision. A missing support export or an old CRM snapshot changes what the report can honestly say.

I built a small reporting demo for MadeBy.Expert to make those conditions visible. It uses synthetic billing, CRM and support snapshots, deterministic calculations and a template narrative. There are no live integrations or language model calls in this version.

Define the numbers before automating the report

The sample distinguishes monthly recurring revenue from revenue collected during the week. Open pipeline is the unweighted value of open opportunities, not a revenue forecast. Support metrics distinguish unresolved tickets at the snapshot from tickets resolved within the reporting period.

Each metric includes a source reference and a definition. Those details give the reviewer something concrete to check against the input.

For a client workflow, I would agree on these definitions before connecting the source systems.

A missing source is a report state

The browser demo lets you switch between complete sources, missing support data and stale CRM data. The report keeps the metrics it can calculate, omits the unavailable ones and records a blocker. It does not substitute zero for a missing value.

The final state comes from that blocker list:

status: blockers.length ? "blocked" : "draft",

For this example, all three sources are required for approval. A snapshot is stale if it was captured before the reporting period ended or more than 24 hours before the report's asOf time. A future timestamp is rejected as invalid input.

Those are choices for this workflow. A daily operations report might need a tighter freshness limit; another report might be useful with an explicitly optional source. The thresholds need agreement with the person using the result.

Here is the actual test covering all three sources and both failure modes. It uses Node's built-in test runner and strict assertions. fixture() loads the synthetic input afresh for each case:

test("missing/stale sources are unavailable and block approval", () => {
  for (const kind of ["billing", "crm", "support"])
    for (const mode of ["missing", "stale"]) {
      const input = fixture();
      if (mode === "missing") delete input.sources[kind];
      else input.sources[kind].capturedAt = "2026-09-12T08:00:00Z";
      const r = buildReport(input);
      assert.equal(r.status, "blocked");
      assert.ok(r.blockers.some((b) => b.startsWith(kind + ":")));
      assert.equal(
        r.metrics.some((m) => m.sourceId === fixture().sources[kind].id),
        false,
      );
      assert.throws(
        () => approveReport(r, r.digest, "Demo reviewer"),
        /Blocked/,
      );
    }
});

The last assertion matters: reporting the problem in the output is insufficient if the approval function still accepts it.

Approval applies to one version of the report

The implementation computes a SHA-256 digest of the report content, which includes a digest of the input. A reviewer supplies the report digest when approving a local export. The approval function recomputes it and checks that the report is still a draft without blockers:

export function approveReport(
  report,
  digest,
  reviewer,
  now = new Date().toISOString(),
) {
  const { digest: stored, ...content } = report;
  if (hash(content) !== stored || digest !== stored)
    fail("Approval digest does not match the current report");
  if (report.status !== "draft" || report.blockers.length)
    fail("Blocked reports cannot be approved");
  if (
    typeof reviewer !== "string" ||
    reviewer.trim().length < 2 ||
    reviewer.length > 100 ||
    /[\r\n]/.test(reviewer)
  )
    fail("A named reviewer is required");
  return {
    ...report,
    status: "approved",
    approval: { reviewer: reviewer.trim(), at: now, digest },
  };
}

Here, hash is SHA-256 over JSON.stringify(value) and fail throws an error. This is an excerpt from the local prototype, not a standalone approval service.

Rebuilding the report after changing the input produces a different digest, so the previous digest cannot approve the new report. The tests also alter a calculated metric directly and check that approval rejects the modified content.

There are limits to this approach. The digest is a consistency check, not a signature or proof of who reviewed the report. The reviewer name is supplied as text; there is no authentication or authorization layer. The JSON serialization is also specific to this implementation, not a canonical format for independent producers.

A deployed approval service would need to establish the reviewer's identity and permissions, store the review event, and bind the delivery action to the approved version. Those parts are not implemented here.

The browser page shows the data conditions and report. The local runnable example handles approval and export; neither sends a report automatically.

Where AI could fit later

A language model might help draft commentary from validated facts. That is a possible extension, not an implemented feature of this demo. I would keep metric calculation, source checks and the approval boundary explicit regardless of how the narrative is produced.

The current prototype makes no claim about client time savings, live connector reliability or model quality. Those need measurement against an actual workflow.

Try the demo, including the missing-source and stale-source cases.

If your team assembles a similar report manually, describe the workflow: which tools supply the data, how often the report is needed, and who checks it. We can discuss whether a small paid pilot with clear success criteria fits.

Typed Contracts Between AI Agents: The Interface That Actually Breaks

Danil Galeev — Tue, 15 Sep 2026 05:44:04 +0000

Most multi-agent failures I've seen don't happen inside an agent. They happen between two of them.

When the handoff between two agents is treated as loosely structured JSON or implicit shared state, the interface is underspecified — and underspecified interfaces fail quietly.

A production handoff should be treated like any other API contract:

an explicit input schema;
an explicit output schema;
required and optional fields;
semantic constraints, not only valid JSON;
a clear rejection and fallback path.

A concrete example

A research agent should not hand an unstructured object to a scoring agent. It should emit something the scorer can actually validate:

class ResearchResult(BaseModel):
    subject_id: str
    evidence: list[EvidenceItem] = Field(min_length=1)
    source_count: int = Field(ge=1)
    confidence: float = Field(ge=0, le=1)

The scorer runs only after that result passes validation. That gate is the important part.

Probabilistic output, consequential action

An LLM output is probabilistic. A database write, an external message, or a business decision is an action. Those two steps should not be directly coupled.

Between them, put a deterministic gate:

Parse the model output.
Validate its schema and business rules.
Reject, retry, route to review, or stop the branch when validation fails.
Execute the action only on an accepted contract.
Record the handoff version and validation result.

Why it changes failure behavior

Without a contract, a missing field becomes a null value, the null becomes a misleading score, and the score becomes a bad CRM update. The symptom shows up several steps away from the cause, and by then the trail is cold.

With contracts, the same failure is localized: ResearchAgent -> ScoringAgent, field source_count, validation rule failed.

That is more than defensive programming. It is architecture for software systems whose internal steps are non-deterministic.

It gives staff engineers independently testable boundaries, helps principal engineers reduce coupling between teams and agents, and gives architects and CTOs a clearer way to reason about compatibility, ownership, and recovery.

The rule

Every probabilistic step that feeds a consequential action needs an explicit validation gate. Typed contracts are one strong way to define that boundary.

The model can be creative inside the boundary. The boundary should not be.

Where have you found the most valuable contract in an agent workflow — between agents, between an agent and a tool, or before the final action?

Building AI Agents That Actually Work: What Nobody Tells You

Danil Galeev — Fri, 04 Sep 2026 09:11:40 +0000

I've been building AI agents for a while now. Not chatbots. Not RAG demos. Real agents that take actions, make decisions, and run autonomously.

Here's what nobody tells you.

The gap between demo and production

Every AI agent framework shows you a 5-line demo that works perfectly. Then you deploy it, and it fails in ways you didn't imagine.

The reason is simple: a demo is a happy path. Production is a graph of failure states.

1. Observability is not optional

Your agent is only as good as your ability to see what it's doing.

When an agent makes a wrong decision, you need to know exactly why. Was it a bad prompt? A hallucinated tool call? Missing context from a previous step?

Log every thought. Audit every action. If you can't replay an agent's decision process, you can't trust it.

2. Tools are the bottleneck, not the LLM

Most people think the LLM is the hard part. It's not. The hard part is the tools.

Your agent needs to call APIs, read databases, write files, send emails. Each of those is a failure point. Network timeout. Auth expired. Schema changed. Rate limited.

Build your tool layer like you build a distributed system. Retries. Circuit breakers. Timeouts. Graceful degradation.

3. The context window is a trap

Long-running agents accumulate context. The more they do, the more context they carry. Eventually, the context window fills with noise, and the agent starts making bad decisions.

Strategies that actually work:

Summarization — compress old context into summaries, don't carry raw history
Structured memory — separate short-term (current task) from long-term (learned patterns)
Forgetting — actively prune irrelevant context. If it didn't matter in the last 10 steps, it probably won't matter now

4. Determinism is underrated

Everyone wants creative agents. What you actually want is predictable agents.

A creative agent that hallucinates a solution is useless. A predictable agent that follows a known pattern is valuable.

Design for determinism first. Add creativity as a controlled parameter.

5. The orchestration layer matters more than the model

You can swap GPT-4 for Claude or Gemini and your agent still works — if your orchestration is solid.

Good orchestration:

Clear state machine
Explicit error handling
Human-in-the-loop for critical decisions
Audit trail for every action

The uncomfortable truth

Building AI agents that actually work is not an AI problem. It's a software engineering problem.

The LLM is the easiest part. Everything around it — tools, observability, state management, error handling, orchestration — that's where the real work is.

And that's also where the real value is.

Your Agent Isn't Losing Memory. It's Rotting.

Danil Galeev — Fri, 28 Aug 2026 18:33:00 +0000

Agents don't forget because the model is bad. They forget because we built it into them. Six hours and forty messages in, what an agent needs is gone — or still there but wrong, drowned in noise.

That's context rot.

I've spent the last month inside a team of always-on AI agents that run for weeks. Different jobs, one codebase, one shared vault of lessons. They review each other's PRs, publish, plan. And they rot. I watched every flavor of it, then built the countermeasures.

What context rot is

Four failures that compound.

Compression loss. When a session runs out of window, a summary replaces raw history. It sounds fine until the agent needs the detail it summarized away. I watched an agent repeat a plan that contradicted a decision two screens earlier. Nobody noticed, because the contradiction lived at a depth that was gone.

Drift. Each turn pulls the agent toward the recent and the loud. Given ten pieces of context, it weights the last five. Over hundreds of turns it quietly solves a different problem than the one you assigned.

Priority blurring. "We decided X" and "we looked at X" feel the same in the tokens. Decisions stop being binding and become suggestions.

Noise accretion. Stale output and dead ends pile up. The signal-to-noise ratio falls, and a degraded ratio looks like a dumber model.

The part nobody warns you about

The cheap fix is "just enlarge the window." I built that reflex out of myself. Longer windows don't fix rot, they defer it — later in the timeline, larger cost per turn, and past some point the model does visibly worse than with a lean window. Remembering everything is a form of forgetting: the important thing drowns.

And rot is per-profile, not per-agent. In a team sharing a codebase, what agent A rotted away still matters to agent B. So the fix must live outside any single session. Memory in one agent's head dies with that session.

What actually works

None of this is exotic.

Summarize to executable facts, not prose. Emit decisions and constraints as explicit, fixed-schema entries the agent can act on. A good summary answers "what is still binding" before "what happened."
Move memory out of the session into a semantic index. Our biggest win. Keep lessons and decisions in a plain markdown vault, index with embeddings, and search returns meaning, not exact strings. "Offer-notification" and "pet-booking" are unrelated strings and near-identical problems; string search never connects them, the graph does — in milliseconds, on one SQLite file and a small local embedding model. No vector DB, no cluster, no bill. Don't ask one long-lived brain to do all the retrieval.
Turn knowledge into skills, not notes. A lesson read once and never referenced is a lesson that rots. The durable form is a skill: a procedure loaded on demand behind a trigger, pulled into context exactly when it applies and out the rest of the time. That's the difference between a memory of what to do and a memory of when to use it. The trigger is the part to get right.
Layer memory by half-life. Instant preferences live in a compact always-on store; entity facts in a structured store you can query; long-lived conventions in a vault you load on purpose. Each level has its own audit cadence, so the always-on layer never bloats and the deep layer never goes stale silently. Cap and prune the always-on store on a schedule: an always-on memory that grows forever is context rot in slow motion.

Patterns to rot-proof an agent

Whatever you pick, these hold.

Make forgetting visible. If a session drops something, drop it loudly: what entered context, what left it, why. You can't manage decay you can't see.

Bind decisions harder than facts. A decision needs a source, a status, an owner. A degraded decision is an incident, not a nuisance.

Retrieve, don't carry. Pull the relevant slice on demand instead of carrying a fat context everywhere. The leaner the working set, the slower the rot.

One source of truth for shared rules. When agents cooperate, conventions belong in one place anyone can read, not duplicated into private memory where they rot out of sync and you end up fighting three versions of the same rule.

Treat memory as code. Schema changes, evals, tests. The vault is source, the index is a produced artifact: rebuild it on a cron and never hand-edit it. A knowledge graph you can't rebuild from source is a liability.

The uncomfortable take

You never beat context rot. You manage it. Design so the cost of rotting is contained and observable, and so the source of truth survives the session because it lives outside it.

In our setup, week-long cycles now hold up better than the first version did at hour three. That is an observation from running them, not a measurement. The difference was never a better model. It was admitting that an agent's context is a perishable working surface, not a mind, and engineering it that way.

I keep getting asked which framework solves this. The answer stays boring: write good compression, index for meaning, encode lessons as triggered skills, cap the always-on memory, and keep long-lived truth somewhere the agent can query and rebuild instead of carry. The agent will look cooler carrying everything. It will also rot faster.

So: how do you decide what an agent is allowed to forget, and how do you make sure it tells you when it does?

When Should You NOT Use an Agent?

Danil Galeev — Thu, 27 Aug 2026 19:21:36 +0000

Everyone is asking "should we use agents?" The real question is "when should we NOT?"

I keep seeing teams bolt an agent on because it's the hot thing — then discover they reinvented a state machine with worse debugging. Agents don't solve a problem by existing. They are a mechanism for deferring decisions to a runtime. When your inputs, tools, and failure modes are well-understood, that deferral buys you nothing but nondeterminism.

The architecture question is not "LLM or not." It is: where does the judgment boundary sit?

Three places the boundary lands

A single agent is a program. For a well-scoped task with a known toolset, you don't need a loop at all. You need a deterministic pipeline — with the LLM as one component, not the orchestrator.

The moment you need a loop, you're building a runtime. A runtime is a different beast. It has observability, tool permissions, credit and rate limits, and a way to explain what it did after the fact. That's not "more AI." That's distributed systems with a language model as the cognitive layer.

The expensive failure is capability you never signed up for. Agents surface things you didn't design for: open-ended tool calls, emergent side-effects, scale that hits budgets or audit. This is where maturity shows — not in the cleverness of the model, but in the constraints around it: entitlements, approval, observability, evals that cover the failure path, not just the happy path.

The tell

Most of what teams call "agent architecture" is a decision-making boundary placed in the wrong spot, papered over with more layers. If you reach for a framework, an orchestrator, a runtime — stop and ask what you're actually deferring, and whether that deferral is working or just making the system harder to debug.

A few heuristics I use before building:

Can the task be expressed as steps with a known order? Pipeline with the LLM as a step — not an agent.
Does the agent invoke known tools with expected outputs? A single governed agent, thin on top of a deterministic core.
Do you need to route mid-task to unexpected states? Only then a real runtime, and only if you accept owning its observability, limits, and audit trail as first-class work.
Is the failure non-recoverable? Then don't put an agent in the loop at all. A wrong tool call inside a runtime can hurt you faster than a slower deterministic path ever will.

Constrain before you automate. The agent looks cooler; it's also much harder to explain three months from now.

So: when you're scoping a new system, what makes you reach for a deterministic pipeline instead of an agent — or the other way? I'd like the rules teams actually run with, not the ones they present in talks.