DEV Community: yongrean

Confidence is enough to decide. It's not enough to do.

yongrean — Thu, 25 Jun 2026 10:18:46 +0000

A classifier confidence of 0.99 is enough to decide a tier. It is not enough to send an email you can't unsend.

Those are two different bars, and most "autonomous" systems use the first one to clear the second. That's the bug.

This is the third post in a series that started as a cheap-model brag and turned into an architecture argument. Post one: a cheap model beat GPT-4o on email triage. Post two: the model only scores four features, and a deterministic rule picks the tier. A commenter, @hannune, pointed at one of those four features:

Your reversibility signal is something I have not seen named explicitly before but it is exactly the right axis for anything that touches irreversible state.

He's right, and it's the cleanest way into the last piece of the design. So: what reversibility actually routes.

The line: can the user undo it with one click?

Most of what a mail agent does is reversible. Archive, un-archive. Trash, restore. Apply a label, remove it. Mark read, mark unread. Re-tier. Snooze. Every one of those is a single click away from undone, so every one of those rides on exactly what post two described — classifier confidence plus a hash of the input bytes that drove the decision. If the model's confident and the inputs are pinned, ship it.

Three actions are not like the others:

export const FLOOR_ACTIONS = ["send_email", "delete_permanent", "forward_external"] as const;

Send (Gmail's undo-send window is 30 seconds, then it's gone). Permanent delete (skips Trash, no recovery path). Forward to an external party (same network effect as send — it's out). For these, reversibility scores near zero, and near-zero reversibility is the signal that says: confidence is necessary but no longer sufficient. You need something the probabilistic layer can't give you.

Why confidence isn't enough: sign the artifact, not the narration

The failure mode here has a name I borrowed from people doing this in crypto: agent-vs-ABI mismatch. The agent narrates a high-level intent — "I sent a polite follow-up to Alice" — and the thing that actually executed did something the narration glossed over: wrong recipient, an edited body, a different attachment. The agent isn't lying. Natural language is lossy by definition; the description and the bytes are allowed to drift.

The cure isn't to verify the narration harder. It's to stop signing on the narration and sign on the deterministic artifact — the actual bytes that will travel to Gmail.

export function sendEmailPayloadHash(input: { to: string; subject: string; body: string }): string {
  const canonical = {
    v: RECEIPT_SCHEMA_VERSION,
    action: "send_email" as const,
    to: input.to.normalize("NFC").trim().toLowerCase(),
    subject: input.subject.normalize("NFC"),
    body: input.body.normalize("NFC"),
  };
  return crypto.createHash("sha256").update(JSON.stringify(canonical)).digest("hex");
}

When you approve a send, the system mints an ActionReceipt that pins this hash — the bytes you actually approved, normalized so a cosmetic edit (Alice@Example.com vs alice@example.com) doesn't false-alarm, and NFC-normalized so composed/decomposed Unicode hashes identically (this matters the moment a body has Korean in it). At execute time it recomputes the hash from the about-to-send bytes and checks:

export function verifyReceipt(receipt: ActionReceipt, expected: { action: FloorAction; currentPayloadHash: string }): void {
  if (receipt.v !== RECEIPT_SCHEMA_VERSION) throw new ActionReceiptSchemaError(receipt);
  if (receipt.action !== expected.action) throw new ActionReceiptMismatchError(receipt, expected.currentPayloadHash);
  if (receipt.payloadHash !== expected.currentPayloadHash) throw new ActionReceiptMismatchError(receipt, expected.currentPayloadHash);
}

Any drift between approve and execute throws and the action is refused. Reusing a send_email receipt to authorize a delete_permanent throws on the action check. Bumping the schema version deliberately invalidates every pending receipt and forces a re-approve under the new shape. The autonomous path fails closed: no valid receipt, no irreversible action.

So reversibility is the router

That's the whole point of naming reversibility as a first-class feature instead of folding it into "risk." It's not decoration on the tier decision — it's the axis that decides which trust model an action even gets:

High reversibility → the probabilistic layer is enough. Confidence + input hash, done.
Near-zero reversibility → drop to the deterministic floor. Confidence got you to "this is worth doing"; the signed artifact is what gets you to "and the bytes are exactly the ones approved."

Two layers, and the feature score picks which one applies. The probabilistic layer is allowed to stay probabilistic precisely because the floor catches the cases where probability isn't enough.

The honest part

Fair question: is this actually wired, or just a module with a TODO? It's wired. The receipt is minted at /approve from the exact bytes you clicked on, executeToolCall refuses any floor action that arrives without a verified receipt (FloorReceiptRequiredError), and send_email re-checks the payload hash in its own path before anything leaves.

The honest edges, because there always are some: of the three floor actions, only send_email is a callable tool today — delete_permanent and forward_external aren't wired as tool cases yet, but the central guard already fails them closed, so a future case physically can't ship a receipt-less side effect. And the autonomous agent runs in SUGGEST mode by default — read-only tools plus propose-only, no mutating power until you opt into AUTO, and even then the floor stands in front of the irreversible three. The brake went in before the autonomous engine gets switched on, which is the only order that isn't reckless. I'd rather show you the guard and its TODOs than claim more than the code does.

The portable version

Separate "confident enough to decide" from "verified enough to do." For anything your system can't undo with one user click, don't trust the model's description of what it's about to do — hash the deterministic artifact at approval, verify it at execution, and fail closed on any drift. Confidence is a fine reason to decide. It is never, by itself, a reason to do something you can't take back.

The whole floor is ~210 readable lines in the open, AGPLv3: github.com/k08200/klorn — packages/api/src/attention-floor.ts. Three posts, one idea: keep the model in the perception layer, and put everything you actually stand behind in code you can read.

I don't trust the LLM to classify my email. So I don't let it.

yongrean — Thu, 25 Jun 2026 05:57:48 +0000

My classifier calls an LLM on every single email. The LLM is not allowed to classify the email.

That sounds like a contradiction. It's the most important design decision in the thing.

A reader named @nazar_boyko left a comment on my last post — the one where a cheap model beat GPT-4o on email triage — and put it better than I did:

Once the LLM is a feature scorer and not the decider, "consistency over genius" falls right out of it, and a cheap fast model is exactly what you want for reading the same four signals the same way every time.

The price upset was the fun headline. This is the actual thesis. So here it is on its own.

The model scores four numbers. That's all it does.

Every inbound email goes to the LLM with one job: read the message and return four scores between 0 and 1.

confidence — how sure you are the other three scores are right
senderTrust — 1.0 a known, important human; 0.3 an automated transactional notice you signed up for; 0.0 anonymous bulk marketing
reversibility — if this got auto-handled and that was wrong, how easy is the recovery? 1.0 trivial undo; 0.0 irreversible ("lost an investor")
urgency — needs attention within hours (1.0) down to informational, no clock (0.0)

The response schema is literally:

{"confidence":0.0,"senderTrust":0.0,"reversibility":0.0,"urgency":0.0,"reason":"short phrase"}

No tier. The model never sees the words PUSH, QUEUE, SILENT, AUTO in its output contract. It reads an email and describes it along four axes. It does not get a vote on what happens next.

A rule I can read decides the tier

What happens next lives in one file, tier-policy.ts, in a function with no model in it:

// 1. Very low confidence → QUEUE. Hiding uncertain mail is the worst failure.
if (f.confidence < 0.5) return "QUEUE";

// 2. Urgent AND sure → wake the user.
if (f.urgency >= 0.7 && f.confidence >= 0.7) return "PUSH";

// 3. Anonymous, no clock, trivially reversible → SILENT (narrow: marketing only).
if (f.senderTrust < 0.2 && f.urgency < 0.2 && f.reversibility > 0.9) return "SILENT";

// 4. Reversible, very sure, not urgent, trusted → AUTO.
if (f.reversibility >= 0.85 && f.confidence >= 0.85 && f.urgency < 0.5 && f.senderTrust >= 0.5) return "AUTO";

// 5. Default → QUEUE. "I'll look at it on my own schedule" is the dominant bucket.
return "QUEUE";

That's the whole decider. Every threshold is a named constant in one object above it, not a magic number sprinkled through a prompt. Order matters — earlier branches win. I can read this in thirty seconds, write a unit test for each branch, and change the policy without touching the model or re-running an eval.

Try doing any of that to "I asked GPT-4o to pick a tier and it picked QUEUE." You can't test it. You can't diff it. You can't explain to yourself why message #4,012 got hidden. The decision isn't anywhere — it's smeared across a weight matrix and a paragraph of prompt.

Why "consistency over genius" falls right out of it

Once the model's only job is to score four signals, the question stops being "which model reasons best about email policy?" and becomes "which model reads the same four signals the same way every time?"

Those are different questions with different answers. The first one points you at the biggest, most expensive model. The second one points you at a cheap, fast, low-variance one — because a frontier model's extra reasoning, applied to a 30-word email, mostly buys you more ways to have an opinion, which is variance, which is the enemy when you've already moved the judgment into a rule. That's why the cheap model won the last post. It wasn't a cost compromise. Splitting scorer from decider is what made the cheap model the correct choice, not just the affordable one.

And because the contract is "four features → tier" and nothing else, the model isn't load-bearing for correctness — it's load-bearing for perception. Proof: when the LLM is down or rate-limited, a keyword fallback produces the same four features with zero model calls, and the exact same rule runs on top. The plumbing doesn't change. The only thing a better model buys you is sharper feature scores on the genuinely ambiguous mail — which is why the one place I'll spend a frontier model is a dial that escalates only the low-confidence tail, and nowhere else.

What this buys you that "let the model decide" can't

Auditability. The policy is a file. Code review covers it. A regression test pins every branch.

Stable learning. When I correct a misclassification, the correction doesn't go fight the model for control of the answer. It becomes an example that nudges the feature scores toward the right values, and the rule — the spine — stays fixed. The thing that learns and the thing that decides are separated on purpose.

A blast radius you chose. AUTO's thresholds sit deliberately high (reversibility ≥ 0.85, confidence ≥ 0.85, trusted sender) so the system structurally cannot auto-handle a destructive or low-trust action. That floor is a number I can point at, not a behavior I'm hoping the model keeps exhibiting.

The honest part

This doesn't make the model's judgment good — it makes the decision layer honest. Garbage feature scores still produce garbage tiers; the rule only guarantees that identical scores always map to the identical tier, and that I can see why. The thresholds were hand-tuned against 50 emails, and calibrating them from accumulated real corrections is still ahead of me, not behind. The keyword fallback, by design, can't emit PUSH — so a total LLM outage degrades urgent mail to "visible in the queue," never "silently hidden," but it does degrade. I'd rather write that down than pretend the split is free.

The takeaway is portable

This isn't really about email. Any time you're handing an LLM a decision with consequences, you can ask the same question: does the model need to decide, or does it need to read? Separate "what the model perceives" from "what the system does about it." Put the second half in code you can read, test, and stand behind. You get auditability, you get to use a cheaper model, and you stop being surprised by your own product.

The judge, the rule, and the thresholds are all in the open — AGPLv3: github.com/k08200/klorn. The decider is packages/api/src/tier-policy.ts, about sixty readable lines. Go see how few of them there are.

I let GPT-4o and a cheaper model fight over my inbox. GPT-4o lost.

yongrean — Wed, 24 Jun 2026 16:59:28 +0000

Here's the scoreboard. Same 50 emails, same prompt, same 4-tier task:

Model	Accuracy	Note
`google/gemini-2.5-flash`	88%	100% recall on urgent mail — never missed one
`google/gemini-2.5-pro`	82%	the "smarter" sibling
`openai/gpt-4o`	82%	the reflex pick
anything cheaper than flash	< 80%	failed my floor, didn't ship

The cheap model didn't tie the expensive ones. It beat them by six points and never missed an email that should have woken me up. The two models I'd have reached for on instinct — the obviously-smarter ones, the ones that cost several times more per token — both came second.

I almost didn't run this comparison at all. That's the part worth your time.

The task

I'm building an email firewall. Every inbound message gets exactly one of four tiers:

SILENT — recorded, never shown (marketing, receipts, FYI)
QUEUE — visible when I choose to look, no notification
PUSH — actually interrupt me
AUTO — reversible, hands-off (classified only, for now)

That's the entire output surface. No suggestion cards, no "AI thinks you should reply" badges. One label per email.

The thing doing the labeling is what I call the judge. It's the part I'd assumed needed a good model — reading an email and deciding whether it's allowed to ring your phone feels like a judgment call, and judgment calls are what you buy a frontier model for.

So I had gpt-4o wired in and I was ready to leave it there. Then I did the boring thing and measured it.

What "measured" means here

The eval set is committed to the repo: packages/api/eval/judge-eval-set.json. Fifty emails, synthetic and PII-free, hand-labeled to encode one specific person's policy — QUEUE is the default, SILENT is narrow (clear marketing only), PUSH is urgent and confident, AUTO is reversible and not urgent. The tier mix is 21 QUEUE / 13 PUSH / 12 SILENT / 4 AUTO.

One command runs a model against it:

pnpm eval:judge   # tsx scripts/poc-accuracy.ts --in=eval/judge-eval-set.json

I ran it across the models I was actually choosing between. Flash won. Not "won on cost, tied on quality" — won on quality, and it happens to be the cheap one. I pinned production to it and wrote the result into the commit message so future-me can't quietly pretend the expensive model was a sacrifice I made for the budget:

flash is not a compromise — it scores 88% / 100% PUSH recall, beating gemini-2.5-pro and gpt-4o (both 82%).

Why cheap wins here

A frontier model sells you reasoning depth. Long chains, hard problems, hold-ten-things-in-your-head problems. Email triage is none of that. It's a short, repetitive, read-four-signals-and-be-consistent problem. You're not paying for capability that moves this needle — you're paying for capability you never touch, and a bigger model's extra "thinking" mostly buys you more chances to overthink a 30-word email.

There's an architecture reason too, and it's the load-bearing one. The LLM never picks the tier. It scores four features per email — confidence, sender trust, reversibility, urgency — and a ~20-line deterministic rule maps those four numbers to PUSH/QUEUE/SILENT/AUTO. The model is a feature-scorer, not a decider. So I don't need a model that reasons brilliantly about email policy. I need one that reads four signals the same way every time. Consistency, not genius. That's exactly the job a cheap fast model is good at — and exactly the job where a bigger model's cleverness becomes variance you don't want.

It also means the policy is auditable without the model in the loop. I can read the rule. I can test it. If the model's down, a keyword fallback produces the same four features so urgent mail still gets through. None of that works if you let the LLM free-hand the answer.

Before you @ me

This is 50 emails. It's small on purpose — it's one person's mental model written down, not a benchmark, and I'm not going to dress it up as one. The set is synthetic, so it tests whether the model applies my policy consistently, not whether it can read the real world. A different inbox with a different owner would draw the lines somewhere else and might rank the models differently.

I'm also only claiming what I measured. Flash hit 100% recall on PUSH — it never sent an urgent email to a quiet tier. I'm not going to invent per-tier numbers for the models that lost; I have their headline accuracy and that's what I'm putting my name on.

What I'm not walking back: on the one task I actually care about, on the set that's sitting in the public repo for you to open, the expensive models lost. That result was stable enough to bet production on.

The lesson is annoyingly cheap

Most of us never run this comparison. "Use the best model" is the default, the best model is the expensive one, and the leaderboard agrees, so why would you waste an afternoon proving the obvious?

Because the leaderboard has never seen your task. MMLU doesn't know what you mean by "urgent." The only eval that ranks models on your problem is the one you write — and when you write it, the ranking stops matching the price tag surprisingly often. The frontier model isn't smarter at your job. It's just smarter at the jobs in the press release.

Write the small eval. Run the cheap model against it before you reach for the expensive one. Worst case you confirm the obvious. Best case you cut your bill and your accuracy goes up, which is a sentence I didn't expect to type either.

The judge, the eval set, and the deterministic rule are all in the open — AGPLv3, OpenAI-compatible, point it at Ollama or vLLM and keep your mail on your own box: github.com/k08200/klorn. The eval set is packages/api/eval/judge-eval-set.json. Open it, label it your way, and go find out which model actually wins on your inbox.

Treat upstream catalogs as mutable: how a free-tier model SKU retirement broke my AI agent

yongrean — Thu, 11 Jun 2026 15:24:22 +0000

Tuesday afternoon, every autonomous cycle in my agent started returning the same error:

[AGENT] Cycle failed: 404 No endpoints found for model: google/gemma-2-9b-it:free

The model hadn't changed in my config. The provider hadn't gone down. The endpoint just... wasn't there anymore. OpenRouter had retired the :free SKU mid-week — no notification, no deprecation window, just gone. Every background classification, every briefing generation, every proactive scan started failing in the same way.

I had a fallback. That was the embarrassing part.

The fallback that didn't fall back

My createCompletion() wrapper had been catching the documented provider failure modes for months:

402 insufficient_credits → walk to next provider
403 daily_quota_exceeded → walk to next provider
429 rate_limited → backoff + retry

What it didn't catch: "the model you asked for doesn't exist anymore." A 404 No endpoints found propagated as a generic error and killed the cycle. The fallback chain never even got consulted because nothing in the existing branches matched.

The mental model was wrong. I'd been treating the model catalog as fixed configuration — something you set once and forget. In reality it's upstream state that can mutate at any moment, just like any other dependency. The retirement was a feature of the provider's catalog management, not a bug.

The fix: walk the free-model chain on retirement signals

The actual patch was short. Two PRs:


ts
// Before: only walked on credit/quota/rate failures
if (isCreditError(err) || isKeyLimitError(err)) {
  return walkFallbackChain(...);
}

// After: also walk when the model itself is gone
if (isModelUnavailableError(err)) {
  markModelUnavailable(model);
  return walkFallbackChain(...);
}
isModelUnavailableError matches on:

HTTP 404 with No endpoints found in body
HTTP 400 with model_not_found code
Anything else the provider emits when the SKU is gone
markModelUnavailable puts the model on a 24h cooldown so the next cycle doesn't try it again immediately. When the catalog refreshes (providers add new SKUs all the time too), the cooldown expires and we retry.

The fallback chain itself is per-provider:


const OPENROUTER_FALLBACK_CHAIN = [
  'meta-llama/llama-3.3-70b-instruct:free',
  'google/gemma-2-9b-it:free',
  'mistralai/mistral-7b-instruct:free',
  'qwen/qwen-2.5-7b-instruct:free',
];
When one entry 404s, we walk to the next. When all of them fail, we fail over to the secondary provider (Gemini direct), which has its own chain. Only when every chain across every provider has been exhausted does the agent give up and surface AllProvidersExhaustedError to the user.

What I should have done from day 1
Three rules I'm internalizing:

1. The upstream catalog is mutable. Hardcoding a single model ID is the same antipattern as hardcoding a single CDN URL. Always have a list. Always make the list cheap to rotate.

2. Distinguish "this model is unavailable" from "the provider is unavailable." They're different failures with different recovery paths. Treating them the same way means you either over-rotate (give up the provider when only one model is gone) or under-rotate (give up entirely when the provider is fine).

3. Cooldowns, not blacklists. When a model disappears, don't kill it forever. Put it on a window. Providers add models back, or you might be hitting a transient 404. A 24h cooldown is much friendlier than a permanent deny-list that requires a code change to undo.

Why this matters beyond one provider
If you're running an agent in production, your model isn't your only upstream dependency:

Vendor's catalog can change
Pricing can change (:free → :paid is a real failure mode)
Rate-limit policies can change
Authentication schemes can change (Google's AQ.-prefix keys rejected by their own OpenAI-compat endpoint is a fun one — I had to write a native adapter for it)
The pattern is the same: treat every assumption about the upstream as a potential dynamic value, and make the recovery path the default, not the exception.

Agents that survive in prod have failover chains, cooldown windows, and degraded modes built in from the start. Not because the upstream is unreliable — because the upstream is alive, and alive things change.

I've been writing about Klorn, an open-source attention firewall for Gmail, where this kind of failure mode hits constantly because the agent runs continuously. Repo: github.com/k08200/klorn · Doctrine: deterministic-floor.md.

If you've shipped agents to prod, what other upstream-mutation failure modes have caught you off-guard?

MCP CI gates need retry receipts for flaky downstreams

yongrean — Mon, 08 Jun 2026 04:43:52 +0000

MCP CI gates need to distinguish two very different failures:

the server is actually broken
the downstream dependency is temporarily flaky

If both become hard failures, CI gets noisy.
If both are ignored, the gate stops meaning anything.

So I shipped @k08200/mcp-probe@1.12.0 with explicit sidecar retry policy for tool-call dry-runs.

The problem

A readiness gate that calls real MCP tools can hit transient downstream failures:

503 Service Unavailable
502 Bad Gateway
504 Gateway Timeout
rate limits
short network timeouts

But auth and permission failures are different. A 401 or 403 usually means the agent will fail in production too.

Those should stay visible unless the contract explicitly says otherwise.

Retry is opt-in per tool

mcp-probe now lets a sidecar contract define retry behavior per tool:

{
  "tools": {
    "logs_query": {
      "input": {
        "query": "service:web status:error",
        "timeframe": "1h"
      },
      "retry": {
        "attempts": 3,
        "delayMs": 1000,
        "retryOn": [429, 500, 502, 503, 504, "timeout", "rate limit"]
      },
      "expect": {
        "status": "pass"
      }
    }
  }
}

The important part: retry is not global magic.

It only happens when the sidecar explicitly opts in.

Receipts still show the flake

If a call fails once and passes on retry, the final result can pass, but the receipt still records every attempt.

That means CI can tolerate a transient downstream blip without pretending the run was clean.

Example shape:

{
  "tool": "flaky_read",
  "status": "pass",
  "source": "sidecar",
  "attempts": [
    {
      "attempt": 1,
      "status": "fail",
      "error": "503 Service Unavailable: transient downstream"
    },
    {
      "attempt": 2,
      "status": "pass"
    }
  ]
}

That is the distinction I want MCP CI gates to preserve:

hard failures should block
transient failures can be retried
pass-after-retry should still leave a receipt

Install

npm install -D @k08200/mcp-probe

Or run directly:

npx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary --receipt-file mcp-probe.receipt.json

GitHub release: https://github.com/k08200/mcp-probe/releases/tag/v1.12.0

npm: https://www.npmjs.com/package/@k08200/mcp-probe

Every "autonomous AI agent" is a customer-support ticket waiting to happen

yongrean — Sun, 07 Jun 2026 16:23:09 +0000

I'm tired of writing apology emails for my own AI.

Last month an agent I was dogfooding cancelled a calendar event I actually cared about. Two weeks before that, a different one auto-replied to an investor with what read like a hostage note from a Slack bot. Both companies have raised more money than I'll see in five years.

The pattern across every "agentic AI" demo on my timeline is the same:

Agent does a thing
Agent emails the user that it did the thing
The thing was wrong
The company ships a fix the following Tuesday

I stopped trusting them. Then I built one that can't do this.

The wedge: agents that wait

Klorn is an approval layer between AI agents and your Gmail / Calendar. The agent does the thinking — reads the email, checks your calendar, drafts the reply, creates the event proposal. Then it stops. Nothing fires until you click approve.

Sounds boring. The constraint is what makes it real.

The constraint that kills "act first, apologize later"

Every meaningful action in Klorn is signed with a payload hash before it fires. send_email literally cannot execute without an ActionReceipt that matches the hash of what was shown to you.

There's an invariant test in the repo that fails the build if anyone — me, a future contributor, an AI agent (the irony) — tries to bypass it. Remove the approval check, the test fails, the build fails, the deploy fails.

You cannot ship a Klorn version that sends emails silently. It's architecturally impossible.

This is the part nobody is building. Every "autonomous agent" demo on my timeline is one feature flag away from the next apology email.

What I shipped this week

The agent loop now runs end-to-end:

Meeting request hits inbox → tier-classified (PUSH / QUEUE / SILENT / AUTO)
Klorn reads the email, checks the calendar for conflicts
Drafts the reply and the calendar event proposal
Both wait as PendingActions in your decision queue
One click → fires

Plus a production bug that would have killed a less paranoid agent: OpenRouter retired a :free model SKU mid-week. Every autonomous cycle died with 404 No endpoints found. The existing failover only covered 402 / 403 / 429 — not "the model is gone." Shipped a multi-model fallback chain on the same provider so losing one upstream SKU never kills the agent.

That fix is the kind of thing you only ship when you trust the boundary the agent runs inside.

Stop hype-cycling, start gating

If you're shipping an "autonomous AI agent" in 2026, three questions:

Can a user prove what was approved is what was sent?
Can a future contributor bypass your approval check?
What is your invariant test?

If the answers are "no", "yes", and "we don't have one" — you're building the next apology email. Stop.

I'd rather build the firewall.

60-second walkthrough above (YouTube · Shorts cut).
Try it free: klorn.ai. PRO auto-applied during private beta.

If you've actually been thinking about where agents should and shouldn't act on their own, I'd love your honest take — even one-line replies. Disagreement especially welcome.

tools/list is not a readiness check for MCP servers

yongrean — Mon, 01 Jun 2026 06:48:53 +0000

The first version of mcp-probe checked the obvious things:

can the MCP server initialize?
does tools/list work?
are tool schemas present?

That was useful, but not enough.

The more I tested real MCP workflows, the clearer the problem became:

tools/list is self-report. CI needs a receipt.

An MCP server can advertise a clean tool catalog and still fail every real call because OAuth handoff, scopes, downstream credentials, row limits, tenant boundaries, or response shapes are broken.

So the latest release of mcp-probe focuses less on "does the process start?" and more on "is CI enforcing the contract an agent actually depends on?"

The new bootstrap flow

npx @k08200/mcp-probe@latest init \
  --target @your-org/your-mcp-server \
  --discover \
  --lock-tools \
  --github-actions

This creates:

mcp-probe.config.json
.mcp-probe.json
.github/workflows/mcp-probe.yml

The important part is what happens during --discover.

mcp-probe connects to the server, reads the live tools/list catalog, and generates a starting contract from the observed tool schemas.

Schema-aware sidecar samples

Older generated samples were too naive. If a schema said:

{
  "type": "object",
  "required": ["location", "count"],
  "properties": {
    "location": { "type": "string", "enum": ["Chicago", "New York"] },
    "count": { "type": "integer", "minimum": 1 }
  }
}

the old fallback might produce empty strings or zero values. That often hit input validation and never tested the real call path.

v1.11.0 now uses schema hints:

default
enum
numeric minimum
string minLength
nested objects
array minItems

So the generated sample becomes:

{
  "location": "Chicago",
  "count": 1
}

It is still only a starting point. You should review generated samples before running them with production credentials, especially for mutating, admin, export, or environment-inspection tools.

Catalog locking

The other new piece is --lock-tools.

With --discover, mcp-probe now writes the observed tool names into expectedTools, so CI fails if a required tool disappears.

With --lock-tools, it also writes allowedTools, so CI fails if unexpected tools appear.

That matters for low-trust agent surfaces. If a server suddenly exposes delete_user, export_all, or rotate_api_key, I do not want that to silently become available to an agent just because tools/list still returns valid JSON.

Example config:

{
  "timeoutMs": 10000,
  "servers": [
    {
      "name": "my-mcp-server",
      "target": "@your-org/your-mcp-server",
      "probeTools": true,
      "toolsFile": ".mcp-probe.json",
      "expectedTools": ["search", "read_record"],
      "allowedTools": ["search", "read_record"]
    }
  ]
}

Receipts

For CI, the workflow can also persist a redacted receipt artifact:

npx @k08200/mcp-probe@latest \
  --config mcp-probe.config.json \
  --github-summary \
  --fail-on-warn \
  --receipt-file mcp-probe.receipt.json

That receipt is the thing I want CI to trust: not the server claiming it has tools, and not an agent claiming what happened later, but an independent probe that actually ran against the boundary.

Try it

npx @k08200/mcp-probe@latest @modelcontextprotocol/server-memory

GitHub: k08200/mcp-probe

Release: v1.11.0

I am especially looking for real Datadog, Supabase, and Gmail MCP recipes. The public fixtures are useful, but the real value is catching auth handoff, permission, tenant-scope, and response-contract failures in CI.

Stop Building AI Assistants. Build AI Firewalls.

yongrean — Thu, 28 May 2026 15:40:23 +0000

Every week another "AI agent for X" launches. Email triage. Calendar coordination. Sales follow-up. PR reviewer. Slack monitor. Meeting summarizer.

I've installed enough of them to see the pattern. Here's the dirty secret nobody mentions in the launch posts:

These tools don't reduce your work. They multiply your notifications.

Each AI tool is configured to be helpful by default. "Helpful" means: "I noticed this thing — here's a notification." Stack a dozen of those, and instead of one inbox to ignore you have twelve. The signal-to-noise ratio gets worse every time you add an AI to your workflow.

The mainstream answer is "just configure each one." Sure. Spend four hours tuning notification settings every time you add a tool, and another four hours when one of them ships a "smarter notifications" update. That's not productivity. That's notification janitorial work disguised as setup.

This is a structural problem. Not a configuration problem.

60-second walkthrough

// Detect dark theme var iframe = document.getElementById('tweet-2060688051920314608-647'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2060688051920314608&theme=dark" }

The wrong question

Every AI tool asks the same thing: "Is this important?"

Wrong question. There is no objective "important." Importance depends on you, right now. A Stripe webhook is important when you're debugging a checkout flow. The same webhook is pure noise during a deep work block. A Slack message from your cofounder is critical at 11am Tuesday and irrelevant at 11pm Friday.

The right question is:

Is this urgent enough to interrupt me, right now, given what I'm doing?

That's not a question any individual AI agent can answer. It's a layer above all your AI agents. None of them have the context. None of them know what the others are doing. None of them know how you're spending the next hour.

So they all default to "I'll just send you a notification, you decide." Which is exactly the experience you have right now: drowning.

What an AI firewall actually looks like

I'm building that layer. It's called Klorn. Here's how it works in practice — and what's already shipping vs what's scope-deferred.

Every incoming email goes through a 4-tier classification:

Tier	Behavior	PoC state
PUSH	Wakes you up. Phone notification.	Classified + alert ✅
QUEUE	Review on your own schedule.	Classified + queued ✅
SILENT	Recorded. Never interrupts.	Classified + logged ✅
AUTO	Reversible, hands-off. Low-risk actions execute; external-facing actions stay approval-gated.	Partial execution: LOW-risk internal (classify, mark read, briefing) auto-executes. MEDIUM (send email, create event) and HIGH (delete) always go through an approve button.

That's the entire surface. No "Call" tier. No fancy automations. Narrow on purpose.

The tier is decided by a 4-feature scorer:

Confidence — how clearly the signal type maps to a tier
Sender trust — your historical reply rate and meeting acceptance for this contact
Reversibility — can the wrong tier be undone without consequence?
Urgency — actual urgency signals, not "URGENT!!!" in the subject line

80% agreement with my hand-labels on 50 real emails. That's the Day 7 PoC gate, met.

Override is GROUP BY, not LLM

When the firewall gets a tier wrong, one click moves the email to the right tier. Your correction doesn't just fix this one email — it becomes ground truth for the next prompt.

The override loop is the wedge. The classifier is replaceable; the alignment signal isn't. Every disagreement is signal, not noise.

Boring + measurable beats fuzzy + ambitious.

Why building this is unpopular in 2026

Building AI firewalls is unsexy. Investors want "AI agents that DO things." Saying "I built a system that does fewer things, more quietly" sounds backwards on a pitch deck.

But every founder I've shown this to has the same reaction: relief. Because they're drowning. Because every productivity tool they bought made their attention worse, not better. The AI agent boom didn't reduce their work. It raised the floor of background notifications.

The default for AI tools should be: shut up unless it actually matters.

Most don't. So I'm building the layer that enforces it from outside, since none of the individual tools will do it on their own.

Where I am

PoC sprint, Week 5, solo. 14-day window ending June 9, 2026.

Day 7 Technical Gate — ≥80% classifier agreement on 50 hand-labeled emails. Met.
Day 14 UX Gate — ≥3/5 ICP demos register "oh, this is different." Pending.

I dogfood it every day. My own inbox runs through the firewall.

Stack: Next.js 15, TypeScript, Prisma, Postgres (Supabase), Claude / OpenAI for the tier reasoning, Gmail for ingest.

The actual unpopular opinion

If your AI tool sends push notifications by default, it's broken. Doesn't matter how good its reasoning is. You can't reason your way out of a notification flood.

The next valuable layer of agentic products won't be more agents. It'll be the firewall that decides which agents are allowed to interrupt you, when.

Try it: klorn.ai
Code: github.com/k08200/klorn

If you're building agentic products and you disagree, I want to hear it. If you've solved it differently, I want to hear that more.

MCP CI gates need receipts: tools/list is not enough

yongrean — Thu, 28 May 2026 11:44:32 +0000

MCP servers are starting to look like normal infrastructure.

That means they need boring infrastructure checks.

The mistake I kept seeing is this:

"The server starts, and tools/list returns a clean schema. Therefore it works."

That is not enough.

An MCP server can pass initialize, advertise every expected tool, and still fail every real call because auth, scopes, tenant boundaries, environment variables, downstream permissions, or read-only roles are broken.

So I pushed mcp-probe@1.8.0 further toward being a real CI readiness gate for MCP servers.

npx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary --fail-on-warn

What changed

1. Warnings can now fail CI

By default, warnings still exit 0. That keeps existing users from getting surprise CI failures.

But production gates often need stricter behavior:

mcp-probe --config mcp-probe.config.json --fail-on-warn

With --fail-on-warn, auth handoff issues, permission warnings, or incomplete readiness receipts can block the workflow.

That matters because many MCP failures are not hard crashes. They are degraded states:

OAuth flow requires a browser redirect the agent cannot complete
a server starts but every tool call returns 401
a database tool works with admin credentials but fails with the intended read-only role
the workflow mentions a probe but does not actually run the production boundary check

2. Doctor now checks the actual workflow receipt

mcp-probe doctor already checked whether a GitHub Actions workflow existed.

But that is not enough either.

The new behavior is stricter: the required flags must appear on the same actual mcp-probe run step.

This should pass:

- run: npx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary --fail-on-warn

This should not count as a complete gate:

- run: npx @k08200/mcp-probe --config mcp-probe.config.json
- run: npx @k08200/mcp-probe ./server.js --github-summary --fail-on-warn

The flags are present somewhere in the workflow, but no single run step proves the intended config is actually being checked with CI summaries and strict warning handling.

That is the difference between "we have a gate" and "the gate is enforcing the thing we trust."

3. Tool call coverage is now tied to expected tools

For config-based checks, you can declare the expected tool catalog:

{
  "servers": [
    {
      "name": "datadog",
      "target": "https://mcp.example.com/mcp",
      "transport": "http",
      "headers": {
        "Authorization": "Bearer ${DATADOG_MCP_TOKEN}"
      },
      "expectedTools": ["logs_query"],
      "forbiddenTools": ["delete_dashboard", "rotate_api_key"],
      "toolsFile": "./datadog.tools.json"
    }
  ]
}

If expectedTools and toolsFile are both set, every expected tool needs a sidecar sample input.

That means CI checks not just "is the tool advertised?" but "did we actually provide a meaningful dry-run sample for the tool an agent depends on?"

4. Sidecar inputs are the real contract

Auto-generated inputs are useful for smoke tests, but they mostly hit schema validation.

Real readiness checks need meaningful inputs:

{
  "tools": {
    "logs_query": {
      "input": {
        "query": "service:web status:error",
        "timeframe": "1h"
      },
      "expect": {
        "status": "pass",
        "not_error_code": [401, 403],
        "requiredFields": ["source", "freshness"],
        "maxRows": 100
      }
    }
  }
}

For database-backed MCP servers, these assertions are the interesting part:

does the read-only role work?
are row limits enforced?
are broad exports/admin actions absent or gated?
are denied writes structured enough for agents to recover?
do results include provenance fields like source and freshness?
does the response avoid leaking secrets, stack traces, or raw internals?

Install

npm install -D @k08200/mcp-probe

Or run directly:

npx @k08200/mcp-probe@latest doctor
npx @k08200/mcp-probe@latest --config mcp-probe.config.json --github-summary --fail-on-warn

GitHub: https://github.com/k08200/mcp-probe
npm: https://www.npmjs.com/package/@k08200/mcp-probe

The goal is simple: CI for MCP should test the contract an agent will actually depend on, not just whether the process starts.

mcp-probe v1.6.0: Stricter GitHub Actions checks for MCP CI gates

yongrean — Tue, 26 May 2026 04:35:59 +0000

I shipped mcp-probe v1.6.0 with a small but useful improvement to mcp-probe doctor.

Previous behavior:

check whether .github/workflows exists
check whether any workflow mentions mcp-probe

That was useful, but too shallow. A workflow can mention mcp-probe and still not run the actual CI gate correctly.

What changed

mcp-probe doctor now warns when the matching GitHub Actions workflow is missing any of these pieces:

actions/checkout@v6
--config <config-file>
--github-summary

Example:

npx @k08200/mcp-probe@latest doctor

If your workflow calls mcp-probe directly but does not use the configured fleet gate, doctor now tells you what is missing before you trust the CI result.

Why this matters

The larger goal of mcp-probe is to make MCP servers testable like normal infrastructure. That means checking more than process startup:

MCP initialize handshake
tools/list discovery
real tools/call dry-runs
sidecar sample inputs
contract assertions for row limits, stable error codes, and leak checks
and now, whether the CI workflow itself is wired correctly

A readiness gate is only useful if the gate is actually installed correctly.

GitHub: https://github.com/k08200/mcp-probe
npm: https://www.npmjs.com/package/@k08200/mcp-probe
Release: https://github.com/k08200/mcp-probe/releases/tag/v1.6.0

mcp-probe v1.5.0: Doctor checks for MCP CI readiness

yongrean — Mon, 25 May 2026 15:40:20 +0000

MCP servers are starting to look like infrastructure. That means the tooling around them needs boring preflight checks, not just optimistic smoke tests.

I just shipped mcp-probe v1.5.0 with a new command:

npx @k08200/mcp-probe@latest doctor

mcp-probe doctor checks whether the current repository is ready to run MCP readiness checks in CI before you even probe an external server.

What it checks

Node.js runtime satisfies mcp-probe requirements
mcp-probe.config.json exists and parses
configured sidecar files exist and have valid tools.*.input objects
GitHub Actions workflows are present and mention mcp-probe

Example:

mcp-probe doctor --config-file examples/self-check.config.json

Output:

mcp-probe doctor
────────────────────────────────────────────────────
  ✓  Node.js version
     Node 24.13.0 satisfies >=20.19.0
  ✓  Config file
     examples/self-check.config.json contains 1 server
  ✓  Sidecar examples/self-check.tools.json
     Found 4 tool entries
  ✓  GitHub Actions workflow
     Found 1 workflow file mentioning mcp-probe
────────────────────────────────────────────────────
  PASS

For automation:

mcp-probe doctor --output json

Why this matters

The earlier releases focused on the MCP server itself:

initialize handshake
tools/list discovery
real tools/call dry-runs
sidecar sample inputs
contract assertions for row limits, metadata, stable error codes, and leak checks

But teams still need to know whether their own probe setup is sane. A broken config file, missing sidecar, or workflow that never invokes the probe should fail early and loudly.

This release is a small step, but an important one: before testing the MCP contract an agent depends on, test that your CI gate is actually wired correctly.

GitHub: https://github.com/k08200/mcp-probe
npm: https://www.npmjs.com/package/@k08200/mcp-probe
Release: https://github.com/k08200/mcp-probe/releases/tag/v1.5.0

Stop building AI inboxes. Build decision layers instead.

yongrean — Mon, 25 May 2026 13:40:43 +0000

I spent six months building an AI-powered email tool. Then I deleted half of it.

Not because the model was bad. Not because the embeddings were off. Because I finally noticed what every "AI inbox" on the market — including the one I was building — was actually doing.

They were surfacing more.

More "smart suggestions". More "priority signals". More "AI-drafted replies waiting for your review". More badges, more banners, more nudges. Every product in the category was racing to add a new surface and call it intelligence.

My six-month-old prototype did all of that. I used it every day. And every morning the inbox was just as loud as the day I started. The model was right about which emails mattered. I still read all the other ones anyway, because they were right there, with a little colored dot suggesting maybe-they-mattered-too.

The model was solving the wrong problem.

The category bug

Look at the leading email tools through this lens:

Superhuman made reading faster. You still read everything.
Shortwave classified smarter. You still read everything.
Motion / Reclaim got more proactive. They added a calendar layer on top of the noise.

None of them subtract. They all add. "AI assistant" became a license to put one more thing in front of you.

The deeper bug: these tools treat email as the primary surface and try to make it better. But email is not what you want. What you want is decisions you have to make. Email is one cheap, unreliable transport that occasionally contains those decisions, buried under hundreds that don't.

Making the transport prettier doesn't fix the signal-to-noise problem. It hides it.

The right abstraction: decision layer

A decision layer doesn't replace your inbox. It sits above mail, calendar, Slack, and any other transport, and it surfaces exactly one thing: items where the system genuinely needs your judgment.

Three properties make a layer a decision layer rather than just "a better inbox":

It subtracts more than it adds. A signal that you've ignored four times in a row should never reach you again. Not muted. Gone.
It treats relationships as data. Two people asking for the same thing are not the same ask. One of them has hit every deadline you've ever had with them; the other ships +3 days late, every time. That should weight the queue.
It refuses to act without your approval. The model can draft, propose, plan. It cannot send, modify, or commit. Approval-before-action has to be a schema-level constraint, not a UI nicety.

None of these are AI features. They are boundary features. The AI is helpful for the classification underneath, but the value lives in what the system refuses to surface.

Here is what each of them actually looks like in production.

Pattern 1 — Closed-loop suppression learning

The single most useful thing the system does is forget.

Every time the user dismisses an attention item, we record a FeedbackEvent with the signal DISMISSED or IGNORED. That table is the cheap part. The interesting part is a job that reads it weekly:

export async function runFeedbackAdaptation(userId: string): Promise<number> {
  const since = new Date(Date.now() - LOOK_BACK_DAYS * 24 * 60 * 60 * 1000);

  const events = await prisma.feedbackEvent.findMany({
    where: {
      userId,
      source: "ATTENTION_ITEM",
      signal: { in: ["DISMISSED", "IGNORED"] },
      createdAt: { gte: since },
    },
    select: { sourceId: true },
  });

  // Join to the attention items themselves so we can bucket by (source, type,
  // priority) instead of just (source, type) — the bucket prevents an
  // over-broad rule from silencing legitimate high-priority signals.
  const items = await prisma.attentionItem.findMany({
    where: { id: { in: events.map(e => e.sourceId) } },
    select: { id: true, source: true, type: true, priority: true },
  });

  const counts = new Map<string, { key: CountKey; count: number }>();
  for (const event of events) {
    const item = itemMap.get(event.sourceId);
    if (!item) continue;
    const bucket = priorityBucket(item.priority);
    const k = suppressionKey(item.source, item.type, bucket);
    const existing = counts.get(k);
    if (existing) existing.count += 1;
    else counts.set(k, { key: { source: item.source, type: item.type, bucket }, count: 1 });
  }

  // Threshold: same tuple dismissed ≥4 times in 30 days → suppress forever.
  const suppressed = [...counts.values()]
    .filter(({ count }) => count >= DISMISS_THRESHOLD)
    .map(({ key, count }) => ({ ...key, dismissCount: count }));

  await remember(userId, "CONTEXT", "attention_suppression_v2", JSON.stringify(suppressed));
  return suppressed.length;
}

The suppression set is then read at the upsert path for every new attention item:

export function isSuppressed(
  set: Set<string>,
  source: string,
  type: string,
  priority?: number,
): boolean {
  if (typeof priority === "number") {
    const bucket = priorityBucket(priority);
    if (set.has(suppressionKey(source, type, bucket))) return true;
  }
  return set.has(suppressionKey(source, type));
}

If the tuple is in the suppression set, the new attention item is forced into SILENT tier — it gets recorded for the audit log, but the user is never paged about it.

A few design choices worth pointing out:

Priority buckets matter. The first version keyed only on (source, type). Dismissing four "due-today commitment" notifications would silence every commitment-due signal, including overdue ones. The current version buckets priority into HIGH / MEDIUM / LOW, so the user can train "I don't care about LOW-priority due commitments" without losing the HIGH ones.
Backwards-compatible key. Memory rows from the previous version are still read; a v1 row without a bucket matches every bucket, so a rollback doesn't lose learned behavior.
10-minute in-process cache. The upsert path is hot — checking the suppression set on every new item against the DB would be wasteful. A 10-minute TTL is short enough that a weekly adaptation run propagates fast and long enough to be free at request time.

Notice what's missing: an LLM. The classifier underneath uses one, but the suppression loop itself is plain counting. The model is not the right tool for "remember what the user doesn't care about". A GROUP BY is.

Pattern 2 — Contact Trust Score

The second feature changed how I think about every productivity tool I've ever used.

When someone makes a commitment to you — "I'll send the deck by Thursday", "let's reconnect next week" — that's a tracked row in a commitment ledger. When the commitment is fulfilled, we record whether it was on-time or late, and update a running tally per contact:

export async function updateTrustScore(
  userId: string,
  contactEmail: string,
  displayName: string | null,
  wasOnTime: boolean,
  daysLate = 0,
): Promise<void> {
  await prisma.contactTrustScore.upsert({
    where: { userId_contactEmail: { userId, contactEmail: email } },
    create: {
      userId,
      contactEmail: email,
      displayName,
      totalCount: 1,
      onTimeCount: wasOnTime ? 1 : 0,
      lateCount: wasOnTime ? 0 : 1,
      totalDelayDays: Math.max(0, daysLate),
      lastUpdatedAt: new Date(),
    },
    update: {
      totalCount: { increment: 1 },
      ...(wasOnTime ? { onTimeCount: { increment: 1 } } : { lateCount: { increment: 1 } }),
      ...(daysLate > 0 ? { totalDelayDays: { increment: daysLate } } : {}),
      lastUpdatedAt: new Date(),
    },
  });
}

That tally rolls up to a badge:

reliable — ≥80% on-time, ≥3 data points
mostly reliable — ≥50% on-time, ≥3 data points
unreliable — <50% on-time, ≥3 data points
unknown — fewer than 3 data points, or stale (no signal in 60+ days)

The stale check is doing real work. A year-old "reliable" badge on someone who has since gone dark shouldn't be load-bearing. Until we get full exponential decay, we demote anyone untouched in two half-lives back to unknown.

The badge gets surfaced as a small chip on the inbox card. But the actually-useful place is inside the agent prompt itself:

export async function buildTrustHintForPrompt(userId: string): Promise<string> {
  const rows = await prisma.contactTrustScore.findMany({
    where: { userId, totalCount: { gte: MIN_DATA_POINTS } },
    orderBy: { lastUpdatedAt: "desc" },
    take: 10,
  });
  if (rows.length === 0) return "";

  const lines = rows.map((row) => {
    const r = computeResult(row);
    const name = r.displayName || r.contactEmail;
    if (r.badge === "reliable")
      return `- ${name}: reliable (${Math.round(r.onTimeRate * 100)}% on-time)`;
    if (r.badge === "mostly_reliable") {
      const delay = r.avgDelayDays > 0 ? `, avg +${Math.round(r.avgDelayDays)}d late` : "";
      return `- ${name}: mostly reliable (${Math.round(r.onTimeRate * 100)}% on-time${delay})`;
    }
    return `- ${name}: unreliable (${Math.round(r.onTimeRate * 100)}% on-time, avg +${Math.round(r.avgDelayDays)}d late) — factor in extra buffer`;
  });

  return `\n## Contact Reliability\nBased on tracked commitments:\n${lines.join("\n")}`;
}

Now when the model decides how urgently to surface "Mina is asking for an update" vs "Sarah is asking for an update", it has actual data on which of them is going to deliver if you give them a polite nudge versus which one needs the deadline restated three times. The prompt isn't fed any feelings about either person. It is fed numbers.

The productivity-tool industry has spent ten years building calendars that don't know which meeting attendees actually show up on time. That's strange.

Pattern 3 — Approval-before-action as a schema constraint

The third pattern is the boring one, and it's the one most AI assistants get wrong.

The model is allowed to draft a reply. It is allowed to propose a calendar move. It is allowed to plan a sequence of actions. It is not allowed to send, move, or commit any of it. Not because we don't trust the model — we sometimes do — but because the user needs to know the surface area of what the system is doing on their behalf, and "silently sent" is a category of bug that never recovers user trust once it happens.

This is enforced at the schema level. Every action the agent proposes lives in a PendingAction row with a status enum. The state machine for that enum is the contract: only one transition (approve()) gets the side effect to actually run. The agent can propose() all day; nothing ships without a deliberate user transition.

The lowest-risk class of actions — internal-only things like blocking calendar time for focus, snoozing an item, setting a reminder — can be marked auto and skip approval. Everything that touches an outside party (sending mail, modifying someone else's calendar) is always gated. The boundary is conservative on purpose. The day a single user discovers their AI assistant silently sent an apology to their VC is the day every AI assistant in the category becomes harder to sell.

What this looks like in practice

The sum of these three patterns is not a smarter inbox. It is a small, quiet queue that contains roughly six to twelve items on any given day. Each item is either an explicit ask, a tracked commitment coming due, or a proposed action waiting for confirmation. The model spent the morning reading and reasoning about a few hundred other things, all of which the system decided you don't need to know about.

When you dismiss an item, the system learns. When a contact reliably delivers, their asks rise. When the model wants to act outside a narrow safelist, it asks first. The result, after a few weeks of training the noise floor, is a queue that feels like it was assembled by someone who actually knows what you ignore.

None of this requires a frontier model. The classifier underneath is a small, cheap LLM with strict cost guards. Almost all of the value is in the boundaries — what the system refuses to surface, what it refuses to do without you, and what it remembers about people you work with.

If you're building anything in this category and you find yourself adding a new surface that shows the user more things, stop and ask whether you'd rather build the thing that subtracts. The market is crowded with smarter inboxes. There is no good decision layer yet.

I'm shipping one at klorn.ai. Not asking for signups — sharing the pattern because I think more people should be building toward it. The closed-loop suppression and trust-score code above are excerpts from the real thing.

Built in TypeScript on Fastify, Prisma, and Postgres. Code patterns shown are production excerpts.