DEV Community: Info Inlet

We Deleted Our Vector Database. Postgres Was Faster.

Info Inlet — Sun, 06 Sep 2026 08:52:41 +0000

We paid for a vector database for a year. It was faster than Postgres at the one thing it does. We deleted it anyway.

- managed vector store      3.2M vectors, its own SLA, its own bill
- sync pipeline             1,100 lines keeping it in step with Postgres
+ one column                embedding  vector(1536)
+ one index                 hnsw

I want to be careful about the "faster," because the headline oversells it and the truth is more useful. On a pure nearest-neighbour benchmark — a bare k nearest vectors, no filter, warm cache — the dedicated store won, every time. It is built for that and it is very good at it.

Our problem was that we never once ran that query in production.

What we actually asked it, every time

Nobody in a real product searches the whole corpus. They search their corpus. Every retrieval we ran looked like this in English:

Find the chunks most similar to this question — among documents this user is allowed to see, that are published, in their workspace, not archived.

That is not nearest-neighbour search. It is nearest-neighbour search with a WHERE clause, and the WHERE clause is the whole problem, because a dedicated vector store filters in one of two bad ways.

Pre-filter, and it walks the metadata first, then does brute-force similarity over what survives — throwing away the ANN index, the only reason you bought the thing.

Post-filter, and it takes the top k by vector distance and then drops the ones that fail the filter. Ask for 10, match on similarity, discover 7 of them belong to a different tenant, return 3. To get 10 back you over-fetch — top 100, top 500 — and hope. Recall becomes a function of how badly the filter correlates with the vector space, which is to say: unknowable, and worst exactly when the filter is selective, which is exactly when you needed it.

We shipped the post-filter version. Then we bolted a fetch-back onto it: take the surviving IDs, go to Postgres for the metadata the vector store didn't hold, filter again to be sure. A second network hop, on every query, to ask the database the question we should have asked it in the first place.

The seam that was actually driving all of it

Here is the part that made me stop optimising the pipeline and delete it instead.

The embedding is not data. It is a pure function of data — run the document through the model, get the vector. It is derived, the way a thumbnail is derived from an image. And we were storing the derived thing in a different database from the thing it was derived from.

Which means every write was two writes:

await db.update(documents, { id, body })          // system 1: the truth
await vectorStore.upsert(id, await embed(body))   // system 2: the copy

Two systems. One of them can fail.

If the embed-and-upsert fails after the document commits — the model API times out, the upsert 500s, the retry also fails and eventually logs a warning nobody reads — you now have a document whose row says one thing and whose searchable vector says another. No exception in your tracker. No alert. Just a document that quietly cannot be found, or worse, one that is found by its old content because the vector is stale.

The mirror case is uglier. Delete a document from Postgres, fail to delete it from the vector store, and now your search confidently returns a chunk of a document that no longer exists — to a user who may no longer be allowed to see it. A deleted row is a permission you revoked. A surviving vector is that permission, still granted, in a system nobody thinks of as holding permissions.

This has a name, or near enough — it is the dual-write problem, the same seam the transactional-outbox people have been shouting about for a decade. If you run a vector store beside your database, you have shipped it. You may just be calling its symptoms "the index is a bit stale sometimes."

The standard fix is a change-data-capture pipeline: tail the Postgres WAL, transform, re-embed, upsert, with a dead-letter for the embeds that fail and a reconciliation job that periodically re-scans everything because you know the pipeline drifts. That works. It is also a distributed system whose entire job is to paper over the fact that a derived value is stored away from its source. It was most of our 1,100 lines.

We deleted the source of the drift instead of building a machine to chase it.

The column

This is the entire vector store now. It is a column on the table that already existed.

create extension if not exists vector;

alter table documents
  add column embedding vector(1536);   -- text-embedding-3-small

vector(1536) is a first-class type from pgvector. The distance operators come with it:

<->   L2 distance
<=>   cosine distance
<#>   negative inner product

And the index that makes it fast is one statement:

create index on documents
  using hnsw (embedding vector_cosine_ops)
  with (m = 16, ef_construction = 64);

HNSW — the good ANN index, the same graph algorithm the dedicated stores use — has been in pgvector since 0.5.0, released August 2023. This is the fact the "just use a real vector database" crowd is usually a year behind on. It is not IVFFlat, it is not a toy, it is the actual state of the art, running inside the database that already holds your data.

The query that was the whole point

The filtered search that cost us two hops and a prayer is now one statement with one planner:

select id, chunk
  from documents
 where workspace_id = $1
   and status = 'published'
   and not archived
 order by embedding <=> $2      -- $2 is the question's embedding
 limit 10;

The WHERE and the ORDER BY embedding <=> are planned together. Postgres decides, per query, whether to use the HNSW index and check the filter, or use a btree on workspace_id and sort by distance — based on how selective your filter actually is. The thing the dedicated store made me choose between at architecture time, the query planner now chooses at runtime, per query, with statistics.

There is one detail that is not optional and is where people get burned, so it gets its own section.

The detail nobody tells you: iterative scan

An HNSW index returns a fixed number of candidates and then your WHERE clause filters them. Selective filter, and you hit the exact post-filter problem I described above — the index hands up 40 candidates, 37 fail your filter, you asked for 10 and get 3. For a while this was pgvector's real weakness and the honest reason to reach for a dedicated store.

It was fixed. pgvector 0.8.0, October 2024, added iterative index scans: when a filtered search comes up short, the index keeps walking the graph and returns more candidates until your LIMIT is satisfied or the search is exhausted.

set hnsw.iterative_scan = 'strict_order';
set hnsw.ef_search = 100;   -- widen the candidate list; the recall/latency dial

If you evaluated pgvector before late 2024, filtered recall is the thing you found wanting, and it is the thing that changed. Re-run your benchmark. This is the single most important sentence in this post.

What got deleted along with the database

The CDC pipeline. WAL tail, transformer, embed worker, upsert, dead-letter, the lot. This was the 1,100 lines.
The reconciliation job. The nightly full-scan that existed only because we knew the pipeline drifted. You do not reconcile a system against itself.
The fetch-back hop. Metadata lives on the same row as the vector. The join is free; it is the same row.
A second SLA, a second bill, a second dashboard, a second thing to be paged about.
"Is the index stale?" — the question that preceded every "why can't the user find this document" investigation, and which no longer has a mechanism to be true.

Ninety days later

	Before	After
Systems in the retrieval path	2	1
p95 filtered retrieval	220ms	41ms
Sync / reconciliation code	1,100 lines	0
Monthly vector-store bill	$840	$0 (folded into RDS)
Documents findable but deleted	non-zero	0

The latency number is the one people fixate on, so let me undersell it correctly: this is not pgvector's ANN beating the dedicated store's ANN. It is not. On the bare benchmark theirs still wins. The 220ms was two network round-trips, a post-filter over-fetch, and a metadata fetch-back — and we deleted all three. We made the path shorter, not the search faster. If your retrieval is a single unfiltered ANN call inside your own VPC, you will not see this.

The last row is the only one I would have done this for. "Documents findable but deleted" was a class of incident. Now it is a state the system cannot be in, because there is no second copy to be wrong.

The five objections, scored honestly

Every one of these was said to me by someone who knew more about vector search than I did.

1. "Postgres can't do ANN at scale."
It has done HNSW since August 2023 — the same algorithm, in-process. "At scale" is the real question and it is objection 2. As a flat "can't," this is a year or two out of date. Cargo cult.

2. "It won't scale to billions of vectors."
Correct, and this is the honest boundary. Somewhere north of tens of millions of vectors at sustained high QPS, a purpose-built store earns its price: better memory layout, real quantization, distributed sharding you do not want to build on Postgres. Ours was 3.2 million at 30 queries a second. Ask what your number actually is before you architect for someone else's. Real — check your number first.

3. "The HNSW index build will melt your database."
This one is real and under-warned. Building an HNSW index over millions of rows is memory-hungry and slow, and it competes with your live traffic. Build it with maintenance_work_mem cranked up, use parallel workers, and treat a full rebuild as a maintenance event, not a migration you run at 5pm on a Friday. halfvec (pgvector 0.7.0) halves the storage and the build cost with negligible recall loss at 1536 dims. Real. Budget for it.

4. "You're coupling your search to your primary database."
We were already coupled — the vectors were derived from the primary and had to be kept in step with it forever. What we removed was the pretence that a copy in another system had decoupled anything. A stale copy is tighter coupling than a column, because a column cannot lag. Cargo cult, in our case — check whether it is in yours.

5. "Dedicated stores have reranking, hybrid search, multi-tenancy features."
Some do, and if you are using them, this trade is different for you. But hybrid search is tsvector and a vector column in the same WHERE, reranking is a cross-encoder call you make after retrieval either way, and multi-tenancy is a column you already have. We were paying for a feature list to use one item on it. Check the list against what you actually call.

When this is the wrong call

I would not do this if:

You are past tens of millions of vectors at high, sustained QPS. Buy the specialised store. The index-build tuning stops being a config change and becomes a full-time interest, and quantization and sharding are genuinely better over there.
Embeddings are the product. If you are running billion-scale semantic search as your core offering, this is your database, not a column, and you should treat it that way.
Your vectors have no source of truth in Postgres. The entire argument here is that the embedding is derived from a row you already store. If the vectors stand alone — you never store the source text, only the vector — then there is no seam to delete and the calculus is just raw ANN performance, which the dedicated store wins.
You need a distance metric or index type pgvector doesn't have. Rare, but check. Do not discover it after the migration.

What I would tell myself a year earlier

The embedding was never a database. It was a column we had exiled to another system, and then hired a pipeline to visit it and keep its story straight.

Before you stand up a vector database, ask the question we skipped: is the vector derived from a row I already store, and are my real queries filtered? If both are yes, you are not buying nearest-neighbour search. You are buying a second copy of your data, a network hop, and a synchronisation problem — to get a WHERE clause your database already had.

pgvector shipped HNSW in 2023 and filtered iterative scan in 2024, and did not send anyone a migration notice.

Four questions I would genuinely like answered in the comments:

What is your actual vector count and peak QPS — the numbers in your dashboard, not the ones in your architecture doc?
If you run a dedicated store, how do you handle deletes propagating from your primary — and are you sure they do?
Has anyone benchmarked pgvector 0.8 iterative scan against a dedicated store on filtered queries specifically? That is the comparison that matters and the one nobody posts.
Who moved the other way — Postgres → dedicated store — and what number forced it?

We Replaced 3 Microservices With One Postgres Table

Info Inlet — Sat, 05 Sep 2026 08:17:59 +0000

Three services died in that pull request.

- notification-svc     1,850 lines
- scheduler-svc        1,340 lines
- retry-svc              910 lines
+ job table               41 lines of SQL
+ one worker             560 lines

Nothing was wrong with any of them. That is the part that took eighteen months to see, and it is the only reason this post is worth your time — because if they had been badly built, the lesson would just be "we wrote bad services and then wrote better ones," which is not a lesson.

They were well built. They were well built around a seam that did not need to exist.

What the three of them actually did

A user places an order. Some time later, they get a receipt. Between those two facts sat:

notification-svc — owned templates and delivery. Talked to the email provider, the push provider, the SMS provider.
scheduler-svc — owned when. Send now, send in 20 minutes, send at 9am in the user's timezone.
retry-svc — owned what happens when the provider is down. Backoff, jitter, dead-letter, the alert when the dead-letter queue got deep.

Between them: two RabbitMQ queues, a Redis for scheduled sets, three deployments, three sets of dashboards, three on-call runbooks. Roughly 4,100 lines, not counting infrastructure.

Write down what all three do in one sentence and the problem becomes visible:

All three existed to move a row from one place to another, later.

That is not a domain. That is a queue. We had built three domain services around a data structure.

The bug that was actually driving all of it

Here is the thing that made me stop and rewrite instead of refactor.

Every path into that system looked like this:

await db.insert(orders, order)        // system 1
await queue.publish('send_receipt', { orderId })   // system 2

Two systems. One of them can fail.

If the publish fails after the insert commits, you now have an order in your database that nothing in the world knows about. No exception in your logs, because you caught it and retried, and the retry also failed, and eventually you gave up and logged a warning nobody reads. No alert. Just a customer who never got their receipt, and a row that looks completely fine.

Do it the other way round and you get the mirror image: a receipt for an order that does not exist.

This has a name — the transactional outbox problem — and if you have ever shipped a queue, you have shipped this bug. You may not have found it yet.

The standard fix is to add an outbox table, a relay process that reads it and publishes, and dedupe on the consumer because that relay is at-least-once. That works. It is also three more moving parts to paper over one seam, and it was on our roadmap.

We built the other thing instead.

The table

This is the entire queue.

create table job (
  id           bigint generated always as identity primary key,
  kind         text        not null,
  payload      jsonb       not null,
  run_at       timestamptz not null default now(),
  attempts     int         not null default 0,
  max_attempts int         not null default 8,
  status       text        not null default 'ready',
  locked_until timestamptz,
  last_error   text
);

-- the only index that matters. Partial, so it stays small:
-- the ready set is a few thousand rows, the table is millions.
create index job_ready_idx on job (run_at)
  where status = 'ready';

run_at is scheduler-svc. attempts and max_attempts are retry-svc. kind and payload are notification-svc. Nine columns.

How a worker takes a job

with claimed as (
  select id
    from job
   where status = 'ready'
     and run_at <= now()
   order by run_at
     for update skip locked      -- ← this line is the whole post
   limit 20
)
update job j
   set status = 'running',
       locked_until = now() + interval '5 minutes',
       attempts = attempts + 1
  from claimed c
 where j.id = c.id
returning j.*;

FOR UPDATE SKIP LOCKED tells Postgres: lock these rows, and if another transaction already has one, don't wait — skip it and take the next.

Twenty workers can run that statement at the same time and no two of them will ever get the same row. No coordination, no leader, no lease service, no Redis. Postgres has shipped this since 9.5, in 2016. Most teams buy a broker to get a behaviour their database has had for a decade.

Two details that are not optional:

Claim, then commit, then work. The transaction above ends the moment the rows are claimed. The worker does the actual sending outside it. Holding a transaction open for the duration of a job is the mistake that gives "Postgres as a queue" its bad reputation — it pins a connection and blocks vacuum for as long as the job runs.

Something has to reap. locked_until is a lease. A worker that dies mid-job leaves its rows in running forever, so a small periodic statement moves expired leases back:

update job set status = 'ready', locked_until = null
 where status = 'running' and locked_until < now();

That is your entire failure-recovery story. It replaced retry-svc.

And then the enqueue

This is the part I actually care about, and it is one line:

begin;
  insert into orders (...) values (...);
  insert into job (kind, payload)
       values ('send_receipt', jsonb_build_object('order_id', ...));
commit;

The order and the job that sends its receipt now commit together, or neither does.

Not "usually." Not "we retry the publish three times." There is no publish. There is no second system. The class of bug where the row exists and the notification does not is no longer unlikely — it is unrepresentable.

We did not solve the outbox problem. We deleted the seam it lives in.

What got deleted along with the services

Things that had been real work, real tickets, real pages, and stopped existing:

Idempotency keys on the consumer. At-least-once delivery meant every handler had to be safe to run twice. Now a job is claimed by exactly one worker in one transaction.
The dead-letter queue and its dashboard. attempts >= max_attempts is a WHERE clause. Inspecting failures is select * from job where status = 'failed'. Retrying them is an update.
Queue-vs-database drift. There is no longer a state of the world where the queue believes one thing and the database believes another, because there is one place.
Two RabbitMQ nodes, a Redis, and three deploy pipelines.
"Which service is this in?" — the question that ate the most engineer-hours of anything on this list, and does not appear in any postmortem.

Ninety days later

	Before	After
Services	3	1
Lines of code	4,100	600
p99 enqueue → execute	4.2s	380ms
Pages / month	9	1
Places a message can be lost	1	0

The p99 number surprised people, so: it is not that Postgres is faster than RabbitMQ. It is that we deleted two network hops and a poll interval. The old p99 was mostly scheduling latency and queue-hop overhead, not broker throughput. Nothing here makes Postgres a faster message bus. It makes the path shorter.

The last row is the only one I would have done this for.

The five objections, scored honestly

Every one of these was said to me, in a review, by someone competent.

1. "A database is not a queue."
It is a durable, transactional, ordered, indexed store with row-level locking and a purpose-built primitive for concurrent consumers. If that is not a queue, the word has stopped meaning anything. Cargo cult.

2. "It won't scale."
It will not scale forever, which is a different sentence. On one unremarkable Postgres box this pattern handles a few thousand jobs a second before you have to think hard. Ask what your actual number is. Ours was 40 jobs a second at peak, and I would bet real money yours is closer to 40 than to 5,000. Real, but check your number first.

3. "You'll blow up the table with dead tuples."
This one is correct and it is the one nobody warns you about loudly enough. A queue table is the highest-churn table you will ever own — every job is an insert, two updates and a delete. Autovacuum's defaults are tuned for tables that do not behave like that. Set them per-table:

alter table job set (
  autovacuum_vacuum_scale_factor = 0.02,
  autovacuum_vacuum_cost_delay = 0,
  autovacuum_analyze_scale_factor = 0.05
);

And delete completed rows on a schedule rather than keeping them forever — move them to a job_archive table if you need the history. Real. Budget for it on day one.

4. "You're coupling your services to one database."
We had one database before. The services shared it. What we removed was the pretence that they did not. Cargo cult, in our case — check whether it is in yours.

5. "This doesn't work for fan-out."
Correct, and it is the real limit. One row goes to exactly one worker; that is the entire design. If you need one event delivered independently to six consumers who each track their own position, you want a log, and you want Kafka, and no amount of SQL is going to be nicer. Real. Different problem.

When this is the wrong call

I would not do this if:

You are above a few thousand jobs a second. Buy the broker. The vacuum tuning stops being a config change and starts being a full-time interest.
You need fan-out or replay. See above. Different data structure, different tool.
Your teams work in four languages. A broker is a lingua franca. SQL and a shared schema are a coupling you have to socialise, and that cost is organisational, not technical, which makes it harder, not easier.
The queue must outlive the database. This is the real trade and it deserves saying plainly: your queue is now exactly as available as your primary. For most teams that is an upgrade, because the queue was never actually more available than the database it fed. But you should choose it on purpose rather than discover it at 3am.

What I would tell myself eighteen months earlier

Check one number before you build three services.

Not "how many messages a second do we expect at scale" — the honest one: how many a second do we do today, and what is the multiple we would need before this hurts? Ours was 40, and the answer was about 100×. We spent eighteen months and three services buying headroom for a load that has still not arrived.

The queue is a solved problem that a lot of us keep re-buying, one service at a time. Postgres solved it in 2016 and did not put out a press release.

Four questions I would genuinely like answered in the comments:

What is your actual jobs-per-second at peak — not your projection, the number in your dashboard right now?
Has anyone here found the autovacuum tuning insufficient at high churn, and what did you move to?
If you moved the other way — Postgres → broker — what number finally forced it?
Who has shipped the transactional outbox and would do it again, versus collapsing the seam instead?

I Replaced 200 Lines of Code With One AI Agent — Here's What Broke

Info Inlet — Wed, 02 Sep 2026 05:18:33 +0000

Here are 200 lines of the most boring code I have ever written, compressed to twelve:

// intent-router.ts
const ROUTES = [
  { when: /\b(invoice|receipt|bill)\b/i,          handler: 'billing' },
  { when: /\b(refund|charge ?back|dispute)\b/i,   handler: 'refunds' },
  { when: /\b(password|2fa|locked out)\b/i,       handler: 'auth'    },
  // … fourteen more, each with a test, each added the day a user hit it
]

export function route(text: string): Handler {
  for (const r of ROUTES) if (r.when.test(text)) return r.handler
  return 'fallback'
}

It took eight months to accumulate. Every regex in it is a scar. It was 91% accurate on our 1,200-row eval set and nobody enjoyed touching it.

Here is what replaced it:

export async function route(text: string): Promise<Handler> {
  const { handler } = await agent.classify(text, HANDLERS)
  return handler
}

Six lines. Same eval set, same afternoon: 96%. Five points better than eight months of regexes, and I deleted 194 lines to get it.

I want to be completely clear about this, because the rest of the article is going to sound like a warning and it isn't one: the agent was better at the job. It never got worse. That number held. The measurement was real.

It still cost me three weeks.

What broke was never the accuracy. What broke was all the stuff the 200 lines were doing that nobody had written down, because nobody had ever needed to.

What actually happened

Three weeks in, support forwarded a ticket. A customer had asked about a duplicate charge and been routed to billing instead of refunds. Wrong queue, four-day delay, angry customer.

I went to look for the error.

There was no error.

The request had succeeded. Status 200. The agent had returned billing, which is a real handler, spelled correctly, in the enum, on schema. Every check in the pipeline had passed because every check in the pipeline was checking whether the answer was well-formed, and it was. The answer was well-formed and wrong, and those two things had never needed to be distinguished before — because when a regex is wrong it falls through to fallback, and fallback is loud.

So I went to reproduce it. Same text, straight into the router.

It returned refunds. Correct.

I ran it again. refunds. Again. refunds. Nine times out of ten it was right. The tenth time it wasn't, and I could not make the tenth time happen on purpose.

That was the actual moment. Not the wrong answer — the unreproducible wrong answer. I had spent a career on a foundation I'd never once articulated: if I run it again with the same input, I get the same thing. Two hundred lines of ugly regex gave me that for free. Six lines of beautiful agent call took it away, and took about five other things with it that I only found by tripping over them one at a time.

Here they are, roughly in the order they bit.

1. Failure stopped announcing itself

When route() was regexes, a miss looked like this:

route("my card got double charged") → 'fallback'

fallback is a real state. It's logged, it's counted, there's a dashboard tile for it, and when it spikes somebody adds a regex. The system's ignorance was a value. It was in the type. You couldn't not handle it.

When route() is an agent, a miss looks like this:

route("my card got double charged") → 'billing'

Confident. Valid. Wrong. There is no fallback because the agent will always pick something — that's what picking means. I had accidentally removed the only channel through which the system could say I don't know, and I removed it in the same commit that made the system more accurate, which is why nobody caught it in review.

The fix is not clever, but you have to decide to want it:

const { handler, confidence } = await agent.classify(text, HANDLERS)
if (confidence < 0.8) return 'fallback'   // put the ignorance back
return handler

The regexes couldn't be uncertain, so they expressed uncertainty structurally, by failing. The agent can be uncertain and expresses it by not mentioning it. If you don't ask, you don't get it.

2. The cache started serving a wrong answer forever

This is the one that turned a bad day into three weeks.

There was a cache in front of the router. Of course there was — it had been there for a year, it was keyed on a hash of the input text, and it had been correct every single day of that year, because a pure function of its input can be cached by its input. That is what pure means.

const key = sha256(text)
const hit = await cache.get(key)
if (hit) return hit
const handler = await route(text)
await cache.set(key, handler, { ttl: '30d' })

Read it now, with a non-deterministic route() behind it, and it stops being a cache. It's a coin flip with a 30-day memory. The first time a phrasing came through, whatever the agent happened to say that time became the permanent answer for that phrasing. If that was the 2% roll, every user who ever phrased it that way for the next month went to the wrong queue. Consistently. Which made it look like a routing rule, not a flake, which is exactly why it took three weeks to find — I was looking for a bug in the rules and there were no rules.

Nothing in that cache changed. Nobody touched it. It didn't break. The thing it was built on top of stopped being true, and it had no way to notice.

Go find your caches, your memos, your useMemo, your idempotency keys, your dedupe-by-hash. Every one of them is a contract that says same input, same output. When you put an agent under one, you are not adding a feature. You are invalidating a contract that something else in your codebase is already relying on, silently, from a file you have not opened.

3. Cost became a function of traffic

The 200 lines cost the same at ten requests a day and ten million: nothing. CPU that rounds to zero. That's not a small property, it's the property that lets you not think about it — you can call route() in a loop, call it twice because it's easier than plumbing the result through, call it on every keystroke.

And we did. There was a place — a live-preview panel — that called route() on input change, debounced at 150ms, because it was free.

It is not free now. Nothing about that call site changed, no one edited that file, and its cost went from zero to a bill.

Worse: the cost isn't just per call, it's per token. A user pasting a long email into the box costs materially more than a user typing "refund". Your unit economics now have a variable in them that your users control and that nobody on the team is tracking, because when the code was regexes there was nothing to track.

4. p99 stopped existing

Old route(): about 0.1ms, and the interesting thing was that the number had a ceiling. Seventeen regexes against a bounded string. There is no input that makes it take a second.

New route(): p50 around 600ms, p99 around 4s, and — the part that matters — no ceiling at all. Not a slow ceiling. None. The upstream can hang. It can rate-limit you. It can 503 in a region. Your p99 is now a property of somebody else's infrastructure and you will find out about it during their incident, not yours.

So the timeout question, which has no good answer:

Timeout at 2s and you turn a 3% slow-tail into a 3% failure rate.
Timeout at 30s and you're holding connections open for half a minute for a routing decision.
No timeout and one bad upstream day exhausts your connection pool and takes down endpoints that have nothing to do with routing.

The regexes never made me pick. Synchronous, bounded, done. I hadn't appreciated that route() being synchronous was load-bearing until making it async rippled through four call sites and one of them was in a hot loop.

5. The tests stopped testing anything

Old test:

expect(route('reset my password')).toBe('auth')

Deterministic, instant, free, and it fails when someone breaks routing. That is the entire job of a test.

You cannot write that test against an agent. Well — you can, and it'll pass most of the time, and it'll go red on a Tuesday for no reason, and within two sprints somebody marks it flaky and skips it. That's not a hypothetical, that is just what happens to a test that fails 2% of the time for reasons no one can act on.

So the tests mutate. They become:

expect(HANDLERS).toContain(await route('reset my password'))

Which asserts that the agent returned a handler. It does not assert that it returned the right handler. It passes if routing is completely broken as long as it's broken into a valid enum value. The suite is still green. The suite is now decorative.

What actually replaces it isn't a unit test at all — it's an eval set that runs on a schedule and reports a rate, plus an alert when the rate moves. That's a real answer and it works. But notice what it costs: it runs on a schedule, not in CI; it reports a distribution, not pass/fail; and it cannot block a merge, because you can't block a merge on a number that jitters. You have swapped a gate for a dashboard, and dashboards are things people have to remember to look at.

6. The code changed without a commit

Six weeks after the swap, accuracy on the nightly eval dropped about a point and a half and stayed there. Nothing in our repo had changed — I checked, twice, and then checked the lockfile.

The model behind the endpoint had been updated.

Sit with that. The behaviour of my production code changed, and there is no commit, no diff, no review, and no line in git log that I can point at. Every instinct I have for "what changed?" is built on the assumption that the answer is in version control. For those six lines, it isn't. It's in somebody else's release notes, if they wrote any.

That's the deepest one, and it's the one that has nothing to do with prompting better. Pin a version where the vendor lets you — and then own the fact that you now have a dependency that expires.

The line I'd write on the wall

Deleting code deletes its guarantees. Guarantees don't show up in the diff.

The pull request showed −194/+6 and a five-point accuracy win. It was, on the evidence available in the pull request, obviously correct, and I would approve it again today.

What the diff could not show was that those 194 lines had been quietly providing determinism, a bounded latency, a zero marginal cost, a fallback state, testability, and a change history — none of which anybody had asked for, all of which other parts of the system had been silently built on top of.

The regexes weren't the feature. They were the reason you could tell when the feature was wrong.

What I actually shipped in the end

Not a rollback. The agent stayed, because it is better. What changed is that it stopped being alone:

export async function route(text: string): Promise<Handler> {
  // 1. Deterministic rules first — cheap, instant, and unambiguous when they hit.
  const certain = RULES.find(r => r.when.test(text))
  if (certain) return certain.handler

  // 2. Agent for everything else, but it's allowed to say "no".
  const { handler, confidence } = await agent.classify(text, HANDLERS)
  if (confidence < CONFIDENCE_FLOOR) return 'fallback'

  return handler
}

Roughly 30 lines of rules survived, not 200 — only the ones that are genuinely unambiguous, where a hit means something. They take about 60% of live traffic at zero cost and zero latency, and, more usefully, they are the only thing in the system capable of disagreeing with the agent. When the nightly eval shows the rules and the agent diverging on a case they both have an opinion on, that's a signal, and it arrives before a customer does.

The cache is keyed on the input and the model version, with a 24-hour TTL instead of 30 days. It's a cost optimisation now, not a correctness assumption.

And fallback is back in the type, where it was for eight months before I got clever.

The four questions I ask now

Before deleting deterministic code in favour of an agent, in this order:

What reads this as if it were pure? Grep for the caches, memos, dedupes and idempotency keys downstream. Each one is a promise you are about to break from a file you aren't editing.
How does this say "I don't know"? If the answer is "it can't", you have removed a state, not just a code path. Put it back before you merge, not after a customer finds it.
What's the ceiling? On latency and on cost. If there isn't one, you now need a timeout and a budget, and both of those are product decisions with no correct answer.
What goes red when this regresses? If nothing does — if the only monitor is a number a human reads on a dashboard — then you don't have a test, you have a hope.

None of that is an argument against agents. I'd make the same swap again; five points is five points and I am never going back to maintaining seventeen regexes by hand.

It's an argument against the thing I actually did wrong, which was to read a −194/+6 diff and believe I was looking at the whole change.

If you've had an agent replace something deterministic and watched a downstream assumption fall over — especially a cache — I'd genuinely like to hear which one. My money's on caches, but I've been wrong about this before.

How to Stop an LLM From Returning Broken JSON — and Why Valid JSON Is the Easy Half

Info Inlet — Tue, 01 Sep 2026 09:48:44 +0000

Here is a model response from a pipeline that reads invoices. It parses. It matches the schema. Every field is the right type.

{
  "verdicts": [
    {
      "doc_id": "doc_a1",
      "risk": "block",
      "reasons": ["already_paid"],
      "explanation": "INV-001 from Acme Supplies was already settled in \"July payables\".",
      "evidence": [
        { "doc_id": "doc_x9", "quote": "Invoice INV-001 · Acme Supplies Ltd · Paid 1,000.00 USD" }
      ],
      "confidence": 0.94
    }
  ]
}

JSON.parse is happy. A validator checking types is happy. And the whole thing is fiction: doc_x9 was never sent to the model, and nothing in the batch says the word paid. The model wrote a citation the way it writes everything else — by producing the most plausible next token — and a citation is a string, so a string is what came out.

That response is going to be shown to a human next to a button that posts money.

Getting valid JSON out of a model is the easy half of this problem, and it's the half everyone writes about. The hard half is JSON that is valid, well-typed, on-schema, and wrong. Below are nine steps, in the order I'd apply them, that take a pipeline from hoping to enforcing. Every one of them is a few lines of ordinary code, and none of them are model-specific — they survive your next provider swap.

First: the four ways model JSON goes wrong

They need different fixes, and lumping them together is why "just add a retry" doesn't work.

#	Failure	Looks like	Fixed by
1	Unparseable	fences, a preamble sentence, a trailing comma	JSON mode, then bracket salvage
2	Parses, off-schema	`confidence: "high"`, a `risk` you don't have a branch for	field-by-field coercion
3	On-schema, invented	an id you never sent, a quote nobody wrote	pointers instead of prose
4	On-schema, true, useless	"confidence 0.97 is below the 1.00 bar"	writing rules the schema can't hold

Steps 1–3 below fix category 1. Step 4 fixes category 2. Steps 5–7 fix category 3 — the expensive one. Steps 8–9 are what happens when all of it fails anyway, which it will.

Step 1: Stop asking for JSON in prose

If your provider has a JSON or structured-output mode, use it, and set temperature to 0 for anything that is a judgement rather than a piece of writing.

async def complete_json(system: str, user: str) -> dict[str, Any]:
    """One call per batch — never per document.

    Ask for JSON (response_format / json mode if the provider has it) and temperature 0: two runs
    over the same books should not disagree about which bill is a repeat. On a parse failure return
    {} — `validate` turns that into "not checked" per item, which is the honest outcome and the
    safe one.
    """

Temperature 0 is not about quality here. It's about two runs over the same input not disagreeing, which is the difference between a check and a coin flip. If a user re-runs the same batch and gets a different verdict, you no longer have a feature — you have a slot machine with a business logo on it.

Rule 1: sampling temperature is a product decision, not a tuning knob. Anything a user can re-run and compare belongs at 0.

Step 2: Shrink the schema until most of it is a closed set

Every free-form string in your schema is a place the model can be creative. Most of them don't need to be strings.

RISK = Literal["block", "review", "clear"]

REASONS = {
    "already_paid", "duplicate_invoice", "vendor_alias", "overpaid", "unallocated_payment",
    "instalment_ok", "credit_note_offsets", "restated_invoice", "currency_mismatch",
}

# Fields a citation may point at. Anything outside this set is dropped — it is how "cite the
# document" stays a checkable instruction instead of a request for a nice sentence.
CITABLE = ("invoice_number", "vendor", "total", "issue_date", "currency", "notes", "ledger")

Three things this buys you that a prose description of the same schema does not:

An unknown value is detectable. "risk": "suspicious" is a bug you can see. "risk": "this looks suspicious to me" is a bug you can only see once a user reports the UI is blank.
The set is a contract you can version. Ours says, in the spec: extend by asking us, not by inventing. A model that invents a tenth reason gets it dropped, not rendered.
You stop translating slugs into sentences on the fly. Keep the closed set for grouping and filtering; keep one free-form explanation for the human. One creative field, not nine.

Leave exactly as much free text as a person actually reads, and no more.

Step 3: Salvage the parse — once, cheaply, and never with a regex

Even in JSON mode, content arrives wrapped in fences or introduced by a sentence often enough to be worth handling. The whole fix is two indexOf calls:

/** Models wrap JSON in fences or a sentence often enough that reading the first bracketed array is
 *  worth more than trusting the response to be clean. */
function parseRanking(raw: string): { id: string; fit: number; reason: string }[] {
  const start = raw.indexOf('[')
  const end = raw.lastIndexOf(']')
  if (start < 0 || end <= start) return []
  try {
    const arr = JSON.parse(raw.slice(start, end + 1)) as unknown
    if (!Array.isArray(arr)) return []
    // …validation continues in step 4
  } catch {
    return []
  }
}

That's the entire acceptable surface area of "JSON repair". First bracket, last matching bracket, one JSON.parse, and a catch that returns your empty case.

What I'd argue against: the libraries and hand-rolled fixers that close unbalanced braces, strip trailing commas, and re-quote keys. They work, which is the problem — they turn a loud failure into a quiet guess. A response truncated mid-object is a response where you don't know what was cut. Repairing the braces gives you a well-formed object that is missing half its verdicts, and nothing downstream can tell.

Rule 2: salvage formatting, never content. If the bytes that describe the answer are incomplete, you do not have an answer, and step 8 is what happens next.

Step 4: Parsing is not validating — coerce every field, individually

This is the step that most pipelines skip, because after JSON.parse returns an object it feels done. Types from a model are suggestions:

return arr
  .map((x) => x as Record<string, unknown>)
  .filter((x) => typeof x?.id === 'string')
  .map((x) => ({
    id: x.id as string,
    fit: Math.max(0, Math.min(100, Number(x.fit) || 0)),
    reason: typeof x.reason === 'string' ? x.reason : '',
  }))

Read what each line assumes will go wrong, because each one has:

filter(typeof x?.id === 'string') — an entry with no id is not an entry. Drop it; don't default it to "" and carry a ghost through the rest of the pipeline.
Number(x.fit) || 0 — handles "87", null, undefined and NaN in one expression.
Math.max(0, Math.min(100, …)) — the model has been asked for 0–100 and will occasionally return 120, or 0.87 because it decided the scale was a fraction. Clamping means your progress bar never renders off the end of its container.
typeof x.reason === 'string' ? x.reason : '' — a missing sentence is an empty sentence, not undefined printed into your UI.

The Python side does the same thing to the same values:

risk = item.get("risk")
if risk not in ("block", "review", "clear"):
    risk = "review"                                    # unknown → the safe branch, not a crash
reasons = [r for r in (item.get("reasons") or []) if r in REASONS]   # unknown members dropped
try:
    confidence = min(1.0, max(0.0, float(item.get("confidence") or 0.0)))
except (TypeError, ValueError):
    confidence = 0.0

Note what isn't here: an exception thrown at the caller. A model returning something odd in one field of one item is an ordinary Tuesday, and it should cost you that field, not the request.

Rule 3: every field gets a defined behaviour for "the model said something else." Written down, in code, next to the field. "That shouldn't happen" is not a behaviour.

Step 5: Let the model point. You render.

Here is the single structural idea, and if you take one thing from this article take this one.

The model does not write the citation. It returns a pointer — {doc_id, field} — and your code renders the text from its own copy of that record.

def _render_evidence(raw: list[dict[str, Any]], index: dict[str, dict[str, Any]]) -> list[Evidence]:
    """Turn (doc_id, field) pointers into text, from our own records.

    This is the step that makes a citation checkable: the model chooses WHICH fact to stand on, and
    the value comes from the data, so it cannot be embellished. A pointer at a document or field
    that was never sent simply does not render.
    """
    out: list[Evidence] = []
    for item in raw or []:
        doc_id = str(item.get("doc_id") or "")
        field = str(item.get("field") or "")
        record = index.get(doc_id)
        if not record or field not in CITABLE:
            continue
        value = str(record.get(field) or "").strip()
        if not value:
            continue
        ref = record.get("invoice_number") or record.get("vendor") or doc_id
        out.append(Evidence(doc_id=doc_id, quote=f"{ref} · {value}"))
    return out

Go back to the response at the top of this article. Under this scheme it cannot exist: doc_x9 isn't in index, so the pointer doesn't resolve, so there is no evidence — and by step 7 the block becomes a review. The invented citation didn't get caught by a checker. It became structurally impossible to express.

The same idea in three lines of TypeScript, on the ranking case:

const byId = new Map(jobs.map((j) => [j.id, j]))
return parsed
  .map((r) => (byId.get(r.id) ? { ...byId.get(r.id)!, fit: r.fit, reason: r.reason } : null))
  .filter((j): j is JobMatch => j !== null)

The model ranks ids we sent. An id it invented finds nothing in the map and disappears. We never render a title, a company or a URL that the model produced — those come from our own row, keyed by an id the model was only allowed to choose.

Rule 4: a model may choose among your facts. It may never author one. Ids, quotes, prices, URLs, names — pointer in, render out.

This is also the honest answer to "how do we stop hallucinated citations?" You don't detect them. You remove the ability to write one.

Step 6: Enforce cardinality, and make a missing answer say so

You asked for one verdict per invoice. You will get: two for the same document, none for another, and one for a document that isn't in this batch.

wanted = [d.doc_id for d in req.invoices[:MAX_INVOICES]]
seen: dict[str, Verdict] = {}

for item in raw.get("verdicts") or []:
    doc_id = str(item.get("doc_id") or "")
    if doc_id not in wanted or doc_id in seen:
        continue  # not ours, or said twice
    # …coerce and store…

# Exactly one verdict per invoice. A missing one is not silence — the client reads absence as
# "not checked", so say that rather than letting the row look clear.
for doc_id in wanted:
    if doc_id not in seen:
        seen[doc_id] = Verdict(doc_id=doc_id, risk="review", reasons=[],
                               explanation="This invoice was not checked.", confidence=0.0)
return [seen[doc_id] for doc_id in wanted]

Three separate guarantees in fifteen lines: only ours (unknown ids dropped), at most one (first wins), at least one (gap filled), and the return is ordered by your input rather than by the model's output order.

The filled gap matters more than it looks. An item with no verdict renders as a row with no warning on it, which a person reads as checked and fine. Absence of an answer must be rendered as an answer, or your UI is quietly lying in the exact place it is meant to be reassuring.

Step 7: Downgrade, don't reject

When a rule is broken, the tempting move is to throw the item away. Usually the better one is to demote it to the outcome that costs least if you're wrong:

evidence = _render_evidence(item.get("evidence") or [], index)
# Cite or downgrade. The prompt says it; this is what makes it so.
if risk == "block" and not evidence:
    risk = "review"
explanation = str(item.get("explanation") or "").strip()
if not explanation:
    risk = "review"

For this pipeline the asymmetry is: "held for a human to look at" costs a few seconds; "cleared automatically" costs a duplicate payment. So the spec says the thing that makes every rule above resolvable:

Default to review when unsure. clear is a claim, not a fallback.

Find the equivalent sentence for your own pipeline before you write the validator, because it decides every else branch in it. A moderation pipeline's safe default is not a search pipeline's safe default. What you must not do is let the safe default be whatever the model happened to say when it broke a rule.

Rule 5: name your cheap direction of failure, in one sentence, and make it the value of every fallback.

Step 8: Choose your failure posture before you need it

The model call will fail. Time out, 500, return prose, return {}. There are three defensible answers and you must pick one per surface:

try:
    raw = await complete_json(SYSTEM_PROMPT + "\n" + FEW_SHOT, user_prompt)
except Exception:  # noqa: BLE001 — a model failure must not 500 someone's books
    raw = {}
    note = "; ".join(filter(None, [note, "the check could not be completed"]))

verdicts = validate(raw if isinstance(raw, dict) else {}, req)

{} flows into the same validator, which fills every row with "This invoice was not checked." The user gets their books, plus an honest note, and nothing auto-posts. Compare the ranking case, where the failure posture is different because the stakes are:

} catch {
  return []   // the caller still has the unranked list, and a search that shows nothing is
}             // better than an error card

Two rules that fall out of doing this a few times:

The enrichment path degrades; the safety path escalates. Ranking, summaries, suggested titles: fail to the unenriched version silently. Anything guarding money, deletion or publication: fail to held for a human, loudly.
Retry structural failures, not semantic ones. An unparseable response or a 503 is worth one retry. A response that parsed and then failed your rules will fail them again — at temperature 0, identically. Change the input or fall back; don't spend a second call proving the first one wasn't a fluke.

Step 9: If the model generated logic, run it before you return it

Last one, for the case where the JSON isn't data but a small program — a formula, a filter, a query, a tool definition. Schema validation says nothing about whether it works.

Our tool builder turns "age calculator from a birth date" into a spec with an expression in it. The checklist it has to pass before it's allowed out:

Valid JSON matching the envelope shape.
Every field id matches ^[A-Za-z_][A-Za-z0-9_]*$, is unique, snake_case.
The expression references only existing field ids and the functions in the allowed list.
The expression parses and evaluates on a sample input — every number field 2, every date field 2000-01-01. If it throws, fix it or fall back.
icon and category are members of their allowed sets.

Rule 4 is the one people leave out and the one that catches the most: a formula referencing a field that isn't there, a division that can only produce Infinity, a date function given a number. You have a whole interpreter sitting right there. Use it as a validator. It costs microseconds and it is the only check in the list that tests the thing the user actually asked for.

The fallback matters too: fix it or fall back to a safe generic tool. Not "return the spec and let the UI throw."

The 60-second audit

Run these four greps against your own codebase. Each one took me under a minute:

JSON.parse( on anything that came from a model, with no .filter or type guard after it. That's steps 4 and 6 missing. Look at what the very next line assumes.
Any string field in your schema that holds an id, a quote, a URL or a price. That's step 5 missing — every one of those is a value the model can invent and your UI will print with a straight face.
catch {} around a model call. Read what the caller does with the empty result. If the answer is "renders as success", you have a silent-lie path.
Enum-ish fields typed string. risk: string, status: string, category: string. Any value outside your branches lands in whatever your else does, and your else was written for a bug you didn't have in mind.

What I'd stop saying

"Just use structured outputs" is now a complete answer to about a fifth of this problem — the fifth where the JSON is malformed. It's a genuine improvement and you should turn it on today.

It does nothing about the other four fifths. Structured output mode will happily hand you a perfectly-formed citation of a document that does not exist, a verdict for an id you never sent, no verdict at all for the one row that mattered, and a confidence of 0.97 where your threshold is 1.00, rendered to a user as a sentence they cannot act on.

The schema you describe to the model is a request. The schema you enforce in code is the only one your product actually has. Nine steps, a few dozen lines, and most of them are filter.

Maximum Update Depth Exceeded: 4 useEffect Dependency Bugs That All Passed Review

Info Inlet — Mon, 31 Aug 2026 07:26:13 +0000

Here's a dependency array that crashed React the first time a user hovered a button:

useLayoutEffect(() => {
  if (!tip || !boxRef.current) return
  const { width, height } = boxRef.current.getBoundingClientRect()
  const x = /* … clamp to the window … */
  const y = /* … above the control, or below it … */
  if (!pos || pos.x !== x || pos.y !== y) setPos({ x, y })
}, [tip, pos])

Nothing in it is wrong by the usual rules. It reads tip, it reads pos, and both are listed. react-hooks/exhaustive-deps is perfectly happy. There's even a guard — that if exists specifically to stop the loop.

It loops anyway. Every time. Maximum update depth exceeded, the first time any tooltip appeared.

I went looking through our own history for the others, and there were three more — same category, four completely different disguises. A spreadsheet that yanked your scroll back to where you started. An onboarding step that showed an error for a request that was never sent. A guided tour that pointed at nothing.

All four shipped. All four passed review. Three of them exist because someone did what the lint rule told them to.

Here's what's actually going on, and the four questions I now ask any dependency array.

The gap the linter can't see

react-hooks/exhaustive-deps answers one question:

What does this effect read?

Your dependency array is asked a different one:

When should this effect run again?

Most of the time those two questions have the same answer, which is why the rule works and why we all stopped thinking about it. Every bug below is a case where they come apart — and because the linter only checks the first question, it will cheerfully sign off on the wrong answer to the second. Worse, it will demand it.

Bug 1: the dependency you measure

Back to the tooltip. Why doesn't the guard hold?

Because x and y are computed from getBoundingClientRect() on an element that has a CSS transform on it. Measuring a translated box hands back subpixel-different widths run to run — 247.99998474121094, then 248.00001525878906. So pos.x !== x is true. Forever.

The sequence is:

effect runs, measures, sets pos
pos changed, so the effect re-runs
it measures again, gets a value a ten-thousandth of a pixel different
the guard says "changed!", sets pos
→ 2

React tears this down at its update-depth limit, which is why it presents as a crash rather than as a slow page.

The fix is one character shorter than the bug:

// Keyed on `tip` ALONE. Re-running on `pos` and comparing the result to decide whether to settle is
// the shape that deadlocks: measuring a translated box hands back subpixel-different widths run to
// run, so the "has it changed?" test never came out false and React tore the render loop down at its
// update-depth limit. One measurement per hover is all this needs — the control does not move under
// the pointer, and anything that WOULD move it (scroll, resize, click) dismisses the tooltip.
useLayoutEffect(() => {
  if (!tip || !boxRef.current) return
  const { width, height } = boxRef.current.getBoundingClientRect()
  // …
  // Whole pixels: keeps the text crisp, and keeps this off the subpixel treadmill above.
  setPos({ x: Math.round(x), y: Math.round(y) })
}, [tip])

Two changes worth separating, because only one of them is the actual fix:

[tip], not [tip, pos]. This is the fix. The effect doesn't need to re-run when its own output changes; it needs to run once per hover. The control cannot move under the pointer, and anything that would move it — scroll, resize, click — dismisses the tooltip.
Math.round. This is the belt-and-braces. It also happens to make the text crisper, because subpixel-positioned text is subpixel-rendered text.

Question 1: is any dependency here something I measured rather than something I was given?

Measured values — getBoundingClientRect, scrollHeight, offsetWidth, anything downstream of layout — are not stable identities. A dependency you measure is a dependency that will never equal itself.

The general shape to be afraid of: an effect that takes its own output as an input, and uses a comparison to decide when to stop. That comparison is now load-bearing, and it's comparing floats you didn't produce.

Bug 2: the dependency whose identity churns

Different file, same family. A spreadsheet keeps the active cell in view — necessary, because the grid is virtualised, so the selected cell may not be rendered at all after keyboard nav or a formula jump.

useEffect(() => {
  const el = gridRef.current
  if (!el) return
  const di = display.indexOf(active.r)
  if (di < 0) return
  const z = zoom / 100
  const top = offsetTop(di) * z
  const bottom = top + rowHeight(active.r) * z
  if (top < el.scrollTop) el.scrollTop = top
  else if (bottom > el.scrollTop + el.clientHeight - HEADER_H * z) /* … */
}, [active.r, display, zoom, offsetTop, rowHeight])

Textbook exhaustive deps. It reads five things; it lists five things. The linter would have added the last four if you hadn't.

The symptom: scroll away from the cell you clicked, and the view snaps straight back to it.

Why: display is a fresh array and offsetTop / rowHeight are fresh callbacks every time the view grid grows — and growing the view grid is precisely what scrolling toward an edge does. So the sequence is: you scroll → the sheet grows → the memo produces a new array identity → the effect re-runs → it scrolls you back to active.r. In both directions, because growing columns invalidates the same memo as growing rows.

The intent was always "when the selection moves". The array said "when any of these five identities change". Those diverged the moment virtualisation was added, and nothing flagged it, because nothing could.

// The layout facts the scroll-into-view effect below needs, held in a ref so they can't RE-TRIGGER
// it. `display` is a fresh array (and offsetTop/rowHeight fresh callbacks) every time the view grid
// grows — which is exactly what scrolling toward an edge does — so having them as deps meant every
// scroll that grew the sheet yanked the view straight back to the cell you had clicked, in both
// directions (growing COLUMNS invalidates the same memo as growing rows). Scrolling away from the
// selection is deliberate; only a change of selection should pull the view back.
const scrollLayout = useRef({ display, zoom, offsetTop, rowHeight })
scrollLayout.current = { display, zoom, offsetTop, rowHeight }

useEffect(() => {
  const el = gridRef.current
  if (!el) return
  const { display, zoom, offsetTop, rowHeight } = scrollLayout.current
  // … same body …
}, [active.r])

This is the pattern the useEffectEvent RFC exists to make official, and you can have it today with four lines and a ref. The rule it encodes:

Question 2: does this effect react to this value, or does it merely read it?

Things it reacts to go in the array. Things it reads go in a ref. The linter cannot tell these apart and will file everything under "reacts to".

If you take one thing from this article, take this one. It's the most common version by a distance, and unlike the subpixel case it never crashes — it just makes your app feel possessed.

Bug 3: the dependency that is your own echo

This one produced the strangest bug report of the four: typing a single letter into "What do you want to make?" printed "Could not check that just now. Try again" — instantly, with no request in flight, and none ever sent.

The setup is one every form in every wizard has. The parent owns the text so it survives stepping away from the step. The child is controlled, and pushes changes up:

const [query, setQuery] = useState(need)

useEffect(() => { setQuery(need) }, [need])   // "sync down when the question changes"

The parent hands that value straight back as need. So need changes for two completely different reasons, and this effect cannot tell them apart:

somebody navigated back into this step with a saved sentence → genuinely new, should reset
the user typed a character, which we pushed up, which came back down → our own echo, should do nothing

Every keystroke re-entered the reset effect. Results wiped. ranFor set to half-typed text. And the render read "no result plus a non-empty question" as a failed check — so for anyone who'd signed up without going through the landing search, the failure notice appeared on the first letter.

The fix is to remember what you pushed:

/** The last value we pushed UP through `onQueryChange`.
 *
 *  The caller hands it straight back as `need` — setup keeps the sentence in its own state so it
 *  survives stepping away from the step — and that echo used to re-enter the reset effect below as
 *  if it were a new question from outside. Every keystroke therefore wiped the results the user was
 *  reading and, for anyone who had signed up without picking anything first, replaced them with a
 *  failure notice for a request that had never been sent. Only a change we did not cause is a new
 *  question. */
const echoed = useRef(need)
const started = useRef(false)

useEffect(() => {
  // Only a question changed from OUTSIDE is a new question — see `echoed`.
  if (started.current && need === echoed.current) return
  started.current = true
  echoed.current = need
  setQuery(need)
  // …
}, [need, /* … */])

Question 3: can this dependency change because of something this component did?

If yes, [dep] is not "when the outside world changes" — it's "when the outside world changes, or when I change". You need to be able to tell those apart, and only you can.

There's a second lesson buried in that bug report, and it's the one I'd actually put on a wall. The error message was rendered by inferring state from what was on screen: no result + non-empty query = failed. So the fix included storing it instead — idle | loading | ok | failed — and rendering the failure line only when a call actually came back empty. Derived state is a guess about the past. If a user can tell the difference between "hasn't asked yet" and "asked and it failed", your component has to be able to as well.

Bug 4: the dependency that wasn't there at all

The last one has an empty-ish array and still breaks, which is why it belongs here — the mistake is the same one wearing a different coat.

A guided tour is handed over the instant setup finishes:

close()            // navigates to /app
startLaunchTour(launch)   // …in the same tick

drive (driver.js) filters out every step whose anchor is not visible at that moment. One tick before the shell paints, that's all of them. So the personalised welcome tour silently degraded to the two popovers that happen to point at nothing.

Nothing here is a dependency-array bug in the literal sense. It's the same misconception one level up: treating "the component ran" as "the thing it points at exists." An effect body running is a statement about React's tree, not about the DOM your library is about to query.

/** Wait for the workspace to actually be on screen.
 *
 *  Setup is a route now, so the tour is started from a page that is in the middle of being replaced
 *  by the shell — `close()` navigates, then hands the tour over in the same tick. `drive` filters out
 *  every step whose anchor is not visible AT THAT MOMENT, so without this the welcome tour arrived
 *  before the rail existed and quietly degraded to the two popovers that point at nothing.
 *
 *  The rail or the composer is enough: both belong to the shell, and either one being laid out means
 *  the workspace has painted. Bounded, because on a narrow layout the rail genuinely never appears
 *  and a tour that waits forever is worse than one that runs over what IS there. */
function waitForShell(): Promise<void> {
  const anchors = ['[data-tour="surfaces"]', '[data-tour="composer"]']
  return new Promise((resolve) => {
    const deadline = Date.now() + 4000
    const tick = () => {
      if (anchors.some(tourTargetVisible) || Date.now() > deadline) resolve()
      else setTimeout(tick, 120)
    }
    tick()
  })
}

Note the deadline. On a narrow layout the rail genuinely never appears, and a tour that waits forever is worse than one that runs over what is there. Any wait-for-the-DOM helper without a bound is a hang you haven't met yet.

Question 4: does this effect assume something outside React has finished?

Mount is not paint. Navigation is not arrival. If you're handing an element to a library that queries the DOM, wait for the element — with a deadline.

The four questions

Print these next to the lint rule, not instead of it:

Is any dependency here something I measured? Measured values never equal themselves.
Does this effect react to this value, or merely read it? Reads go in a ref.
Can this dependency change because of something this component did? Then you must be able to recognise your own echo.
Does this assume something outside React has finished? Mount is not paint. Wait for the element, with a deadline.

The 60-second audit

Grep your own codebase for these four shapes. In our case each one took under a minute to confirm once I knew what I was looking at:

setX inside an effect that lists x. Every one of these is either a loop or a guard doing load-bearing work. Both are worth a second look.
Any dep that is an array, object or function from useMemo/useCallback. Ask what invalidates that memo. If the answer includes anything the user does continuously — scrolling, typing, resizing — you have bug 2.
A controlled child whose parent feeds the value back as a prop. Trace the round trip. If the child can't distinguish its own echo, you have bug 3.
querySelector in an effect, or any third-party library handed an anchor. Ask what guarantees it's there. "The effect ran" is not an answer.

What I'd stop saying

I don't think "just add it to the dependency array" is good advice any more, and I'd said it plenty. It's the right answer to the question the linter asks and the wrong answer often enough to the question your effect is actually asking.

The rule is a great smoke detector. It is not a design review. It cannot tell you that the value you added is measured, or churns, or is your own echo coming home — and in three of the four bugs above, doing exactly what it suggested is what shipped the bug.

The dependency array is not a list of what you read. It's a list of the reasons this should happen again. Write it as an answer to that question and most of this category disappears.

40 Lines of Go That Cut Our LLM Bill by 71%

Info Inlet — Sun, 30 Aug 2026 06:49:55 +0000

On July 30, OpenAI cut GPT-5.6 Luna to $0.20 per million input tokens and $1.20 per million output — down from $1 and $6. An 80% cut. Azure matched it on August 1.

We did what most teams did with that news: nothing. Our gateway sent every request to the strong model, because it always had, and because "just use the cheap model" sounds like a decision that comes back as a support ticket three weeks later.

Then someone put the bill next to the traffic mix, and the awkward part was obvious. The overwhelming majority of our requests were title this thread, summarise this diff, extract the fields from this form, name this file. We were paying frontier prices to generate three-word document titles.

Here's what we shipped instead. It's about forty lines of Go, it moved 81% of requests off the expensive model, and it cut the bill by 71%.

It also broke four things, and those are the interesting part.

The approach that doesn't work: classify the prompt

The obvious design — the one everyone writes first — is a classifier in front of the router:

Look at the incoming request. Decide if it's easy or hard. Send it to the model that matches.

We built this. It's worse than it looks, for three reasons.

You need a model to run the classifier. Now every request pays an extra call before any work happens. A cheap classifier is another Luna call and another 300ms; an accurate classifier is a strong-model call, which is the cost you were trying to avoid.

Short is not easy. "Fix the timezone bug" is eleven characters and needs everything you've got. "Summarise the following 4,000-word RFC" is long and trivial. Length, token count, keyword lists — every cheap heuristic we tried correlated with the shape of the request and not with the difficulty of it.

You're predicting the answer before you've seen it. That's the actual problem. Difficulty is a property of the work, and the only honest way to learn it is to do the work.

The rule: don't classify the prompt. Run the cheap model and judge the output.

Cheap-first with an escalation gate

The design that works is embarrassingly simple:

Send it to the cheap model.
Look at what came back.
If it fails a gate, run it again on the strong model and return that instead.

The thing that makes this viable is arithmetic that surprised us. At our average request shape — roughly 3,000 input tokens and 700 output — a Luna call costs about $0.0014 and a strong-model call about $0.0145. Ten to one.

So a request that gets escalated costs 0.0014 + 0.0145 = $0.0159 instead of $0.0145. About 10% more.

Work out the break-even escalation rate:

cheap + (p × strong) < strong
p < (strong − cheap) / strong
p < (0.0145 − 0.0014) / 0.0145
p < 0.90

You would have to escalate nine times out of ten before the cheap attempt costs you money. We escalate 15% of the time. Cost is not the constraint here, and if you're arguing about whether cheap-first is worth the double billing, you're arguing about the wrong resource.

Latency is the constraint. We'll come back to that.

The router

type Result struct {
    Text   string
    Model  string
    Tokens Usage
}

// A gate returns the reason this answer can't be trusted, or "" to accept it.
type Gate func(req Request, out string, fin FinishReason) string

func (r *Router) Do(ctx context.Context, req Request) (Result, error) {
    if req.ForceStrong || r.alwaysStrong[req.Kind] {
        return r.call(ctx, r.strong, req)
    }

    cheap, err := r.call(ctx, r.cheap, req)
    if err != nil {
        // Availability, not quality. Fall through, don't fail the request.
        return r.call(ctx, r.strong, req)
    }

    for _, gate := range r.gates {
        reason := gate(req, cheap.Text, cheap.Finish)
        if reason == "" {
            continue
        }
        r.metrics.Escalate(req.Kind, reason)

        strong, err := r.call(ctx, r.strong, req) // NB: req, not req+cheap.Text
        if err != nil {
            return cheap, nil // degraded, not failed
        }
        return strong, nil
    }

    r.metrics.Accept(req.Kind)
    return cheap, nil
}

Two lines in there are load-bearing and neither is obvious.

return r.call(ctx, r.strong, req) when the cheap call errors — a 429 or a timeout is an availability problem, and availability problems should not surface to the user as a failed request when you have a second provider sitting right there.

return cheap, nil when the strong call errors — you already have an answer. It may be a worse answer. Shipping a worse answer beats shipping a spinner.

And the comment on the escalation call is the one people get wrong, which is failure mode #4 below.

The four gates

The gates are the whole product. The router is plumbing.

var defaultGates = []Gate{
    SchemaInvalid,   // had to be JSON matching a schema, and wasn't
    ToolArgsMissing, // called a tool, omitted a required argument
    Truncated,       // finish reason != stop
    EmptyOrHedged,   // under 24 chars, or matches the hedge set
}

They are ordered by how cheap they are to evaluate, and every one of them is structural. None of them asks a model to grade another model.

SchemaInvalid does the most work by a distance. Anything with a defined output shape — field extraction, classification, structured summaries — gets validated against the schema you already have. If it doesn't parse or doesn't conform, escalate. This gate alone catches about 60% of our escalations.

ToolArgsMissing is the same idea for function calls. The cheap model picks the right tool far more reliably than it fills in the right arguments, and a missing required argument is a free, exact signal.

Truncated is one field comparison and people skip it constantly. A finish_reason of length means you have a sentence that stops mid-

EmptyOrHedged is the weakest one, and I want to be specific about how weak, because it's the one everybody wants to build first.

We started with the intuitive version: ask the cheap model to say when it isn't confident, then escalate on that. It fired on 0.4% of responses. Our measured error rate on the same traffic was around 15%. The model's self-reported uncertainty was not a signal, it was decoration.

What actually works in that slot is a small, boring list: empty, under 24 characters, or an exact-ish match against a hedge set you build by reading two hundred real failures ("I don't have enough information", "As an AI", "Could you clarify"). Not confidence. Refusal.

The rule: gate on structure you can check, not on the model's opinion of itself.

The four things that broke

1. p50 improved. p95 got worse.

This was immediate and it's the real cost of the design.

	All-strong	Cheap-first + gate
p50	2.9s	1.4s
p95	7.8s	9.6s
p99	11.2s	16.4s

Luna is fast, so the 85% that get accepted got much faster. The 15% that escalate pay for both calls, serially, and they land in your tail.

If you have a latency SLO, that tail is where the design either survives or doesn't. Two things helped: run the gates on the streamed head rather than the finished response where you can, and put a hard escalateBudget on the clock — if the cheap call already burned 4 seconds, return it and log the miss rather than starting a second call you can't afford.

2. You can't stream an answer you might throw away

The whole architecture assumes you get to look at the output before deciding. Streaming assumes you've already committed.

There is no clever fix, only a choice:

Buffer, then decide. Correct, and you've given up time-to-first-token — which for chat is the number users actually feel.
Stream, and never escalate. Fine for anything conversational.
Split your traffic by shape. This is what we do. Structured, non-streamed work — extraction, titling, classification, summarisation, tool selection — goes through the router. Free-form chat streams straight from whichever model that surface is pinned to.

Most of the money was in the first bucket anyway. Structured background work is high volume and nobody is watching a cursor blink at it.

3. The cheap model fails the correctness test long before it fails the eye test

The output is fluent. It's well-formatted. It uses your headings, it hits your tone, it's the right length. It's just wrong.

That's why "does this look like a good answer" gates — including LLM-as-judge in the hot path — did badly for us. Fluency is exactly the axis where the price gap has closed most. Judgement, multi-step reasoning, and knowing what it doesn't know are where it hasn't closed at all.

Structural gates work because they don't have an opinion. Valid JSON is valid JSON.

4. Don't show the strong model the cheap model's answer

Our first version passed the failed attempt along as context — here's a draft, improve it. It seemed obviously more efficient.

It anchors, badly. The strong model inherits the cheap one's framing, keeps its structure, and corrects wording rather than reasoning. On the escalations we hand-checked, the "improve this draft" path was worse than a clean run about a third of the time — and it was worse in the specific way that matters, because it repeated the mistake that triggered the escalation while polishing the prose around it.

Escalation is not a retry. It's a fresh attempt by someone better. Send the original request.

What it did

Per 1,000 requests, at our mix:

	All-strong	Cheap-first + gate
Requests attempted on cheap	0	950
Escalated	—	143
Requests touching the strong model	1,000	193
Cost	$14.50	$4.16

71% off. 81% of requests are served entirely by a model that costs a tenth as much, and the 5% we force to the strong model never enter the router at all.

The forced list is short and it is a policy decision, not a measurement: anything a user is going to send to a customer, anything that writes to production, and anything in the app builder's codegen path. Those never touch the cheap model regardless of what a gate would have said.

When not to do this

Low volume. Under a few hundred thousand requests a month, 71% of your bill is not worth a new component in the hot path. Go negotiate your seat pricing instead.
Long tool-calling chains. Errors compound across steps and the gates only see one step at a time. A cheap model that's right 85% of the time per call is right 44% of the time across five calls.
Anything with a user-visible retry. If your surface already shows a spinner, doubling the tail is worse than paying the bill.
Regulated or audited output. "Which model produced this" becomes a question you have to answer per request. Log Result.Model from day one if there's any chance you're in this bucket — retrofitting it is miserable.

The version you can ship this week

You don't need the whole thing to get most of the money.

Pick one high-volume, non-streamed, structured job. Titling. Field extraction. Tagging. One.
Add one gate: validate against the schema you already have.
Shadow it for a day. Run both models, serve the strong one, log where the gate would have fired. This is the step people skip and it's the one that tells you whether your escalation rate is 15% or 60%.
Flip it, with Result.Model in your logs and a kill switch on the config.

Then measure the escalation rate per request kind, because that's the number the whole design lives on — and it's the number that tells you which job to move next.

The pitch for cheap models in 2026 isn't that they got good enough to replace the frontier. It's that they got cheap enough that checking whether they were good enough is now free.

5 Reasons Your MCP Server Is Slower Than a CLI (and When to Just Delete It)

Info Inlet — Sat, 29 Aug 2026 05:47:24 +0000

There are two ways to let an agent use your product.

A) Ship an MCP server. Define tools, describe them, run a process, the model calls them.

B) Ship a CLI. yourtool do-the-thing --json. The agent runs it in a shell like a human would.

For about a year the answer was obviously A. In 2026 a lot of teams quietly went back to B — and the ones who did aren't posting "MCP is dead" takes, they're posting latency graphs.

Here's what's actually happening. Five costs, in the order they hit you.

1. You pay for every tool, every turn — before the model does anything

An MCP server's tool definitions are loaded into the context window up front. Not when a tool is used. Always.

40 tools × a real JSON schema with descriptions ≈ tens of thousands of tokens
That's paid on turn 1, turn 2, and turn 60
The model has done zero work at this point

A CLI costs you one line in the system prompt. If the agent needs the details it runs yourtool --help — once, only when it matters, and the output can be 30 lines instead of 40 schemas.

Anthropic measured the extreme version of this: a workflow that burned about 150,000 tokens passing tool definitions and intermediate data through the model dropped to about 2,000 tokens when the same tools were exposed as code the agent calls — a 98.7% cut. That gap is not a micro-optimisation. That's the whole bill.

The rule: tool definitions are rent, not purchase. You pay it on every single turn until you delete the tool.

2. A CLI composes. MCP tools don't.

Watch what a shell lets the model write in one action:

yourtool list --json | jq '.[] | select(.status=="failed") | .id' | head -20

One round trip. One result in context.

Now the MCP version:

list_items → 800 rows come back into the context window
model filters them in its own head
get_item × 20
model assembles the answer

Four round trips minimum, and step 1 already poisoned the window with 780 rows nobody wanted.

The shell has had composition for fifty years — pipes, redirection, xargs, exit codes. MCP has a list of function calls. Every "combine two tools" case becomes the model's job, done in tokens, done badly.

The rule: if your users would chain your tools, a protocol with no chaining primitive is the wrong shape.

3. Every byte travels through the model

This is the one that actually shows up as slow.

An MCP tool result goes into the context window. There is no other destination. So:

A 5 MB JSON response doesn't just cost money — it evicts everything useful and you get the classic mid-session quality slide
The model then re-reads that blob on every subsequent turn
You cannot "process and discard"

In a shell, intermediate data can just… stay on disk.

yourtool export --json > /tmp/out.json      # 5 MB, never enters context
jq '.summary' /tmp/out.json                  # 3 lines do

The agent sees three lines. The five megabytes never existed as far as the context window is concerned.

The rule: MCP has no > file. Every result is a broadcast to the model. Design return values like you're paying per character, because you are.

4. A protocol hop that buys you nothing (on a machine with a shell)

Stack them up:

CLI	MCP
exec a binary	spawn/connect a server
argv	JSON-RPC over stdio or HTTP
exit code	protocol errors + tool errors + transport errors
stdout	structured content blocks
—	handshake, capability negotiation, lifecycle

Every row is latency and a failure mode. And they're failure modes your agent handles worse than a non-zero exit code, because "the server disconnected" is not something the model can retry intelligently.

The 2026 spec revision made this concrete for a lot of people: protocol-level sessions were dropped in favour of a stateless core. Sensible for enterprise scale-out — and it broke every server that had quietly built state on top of the session. Nobody's --help output broke that week.

The rule: a transport you don't need is not free. It's latency plus a category of error your caller can't reason about.

5. The model already knows `git`. It has never heard of `create_document_v2`.

This is the underrated one.

git, curl, psql, ffmpeg, gh, jq — the model has seen millions of examples of these. It knows the flags, the idioms, the error messages, and what to do when one fails.

Your bespoke tool surface has zero training examples. So you compensate with description text — which is cost #1 — and you still get the classic failures:

two tools that overlap 80%, picked by coin flip
a tool chosen because its name was closest, not because it was right
the model inventing a parameter that reads plausible and doesn't exist

A CLI that follows Unix conventions inherits all that prior knowledge for free. --json, --dry-run, non-zero exit on failure, errors on stderr. The model has strong priors about every one of those.

The rule: conventions are pretrained context you don't pay for.

The honest part: when MCP is clearly right

None of the above is an argument that MCP is bad. It's an argument that MCP is being used in the one environment where it's weakest — a coding agent that already has a terminal.

Keep the MCP server when:

There is no shell. ChatGPT, Claude's chat surfaces, a mobile app, an embedded assistant. This is the real answer and it's a big one. A CLI is worth nothing to a user who has no machine to run it on.
You don't want to hand the model a shell. MCP is a permission boundary you control, tool by tool. bash is not.
Your users aren't engineers. They will never brew install anything.
You need what the protocol actually provides — resources, subscriptions, sampling, an auth flow that a CLI would have to reinvent badly.
You're a hosted service. There's no binary to install in the first place.

Delete it — or shrink it hard — when:

Your consumers are coding agents that already have a terminal
Your MCP server is a thin wrapper over your own CLI or public API
You have more than ~20 tools and no progressive disclosure
Your tools' most common use is being chained together

The middle path (what most teams should actually do)

You don't have to choose. The pattern that's winning:

Keep MCP thin. Three or four tools — search, fetch, execute — not forty.
Expose the rest as code, not schemas. Let the agent discover and call your API from a sandbox instead of loading every definition up front. That's where the 150k→2k number comes from.
Ship the CLI too. It's usually a day of work over an API you already have, and terminal agents will prefer it without being told.
Make results small by default. --json plus a summary field, with full data behind a flag or a file path.

The 60-second audit

Ask these about your own server:

How many tokens are my tool definitions? (Count them. Actually count them.)
What's my p50 result size? What's my p99?
Are two of my tools doing the same job?
If I deleted the server today, could a competent agent do this with curl and my docs?

If the answer to the last one is yes and your users have terminals — you're paying protocol rent for nothing.

MCP's win was never that it was faster. It was that it made your product reachable from surfaces that have no shell. Use it for exactly that, and stop paying for it where it isn't buying anything.

Why Your AI Agent Gets Dumber After 30 Turns and How to Stop It

Info Inlet — Wed, 26 Aug 2026 04:21:58 +0000

You start a session with an agent. The first half hour is great. It reads your code, follows your instructions, fixes what you asked.

Then somewhere around the middle of the afternoon it starts doing this:

It breaks a rule you gave it at the very start of the session.
It reintroduces a bug it fixed forty minutes ago.
It "helpfully" rewrites a file you told it not to touch.
It gives you a confident answer that contradicts something it said earlier in the same conversation.
It gives up on a task it would have finished an hour ago.

The usual reaction is to blame the model — it got lazy, they nerfed it, this one is worse than last week's.

It's almost never that. What you're looking at has a name, it's been measured, and once you can see it you can design around it.

It's called context rot.

The one-sentence version

An LLM does not use its context window evenly. The more you put in, the less reliably it uses any of it.

That's the whole idea. Everything below is why it happens and what to do.

The important word is reliably. Nothing is deleted. Nothing falls out. The rule you set at turn 3 is still sitting right there in the context at turn 60, perfectly intact — the model has simply become worse at acting on it.

This is why the bug feels supernatural. You can scroll up and see the instruction with your own eyes. It's there. The model can "see" it too. And it still isn't following it.

It isn't your model, and it isn't your prompt

In July 2025, a team at Chroma — Kelly Hong, Anton Troynikov and Jeff Huber — ran a study called Context Rot. They tested 18 models: Claude Opus 4, Sonnet 4, Sonnet 3.7, Sonnet 3.5 and Haiku 3.5; o3, the GPT-4.1 family, GPT-4o, GPT-4 Turbo and GPT-3.5 Turbo; Gemini 2.5 Pro, 2.5 Flash and 2.0 Flash; and three Qwen3 models.

The headline result is short:

Every single one got worse as input length grew.

Not the cheap ones. Not the old ones. All eighteen, including the frontier models of that moment. So if you have been switching models to fix this, that's the wrong lever — you're changing a variable that isn't the cause.

And the reason this matters more now than it did in 2025, not less: context windows got enormous. A million tokens is normal now. Bigger windows didn't fix context rot. They removed the guardrail that used to stop you from hitting it — when the window was small, you got a hard error. Now you get a silent slide in quality with nothing to tell you it started.

Why "30 turns"? It isn't the turns.

Let's kill the number in the title, because it's a symptom, not a cause.

Nothing happens at turn 30. What happens is that by turn 30, a coding agent has typically accumulated something like this:

What's in the window	Roughly
Your system prompt / rules file	1–3k tokens
Tool definitions (often 20–50 of them)	5–15k tokens
Files it read — including versions it has since edited	20–100k tokens
Tool outputs: test runs, stack traces, `grep` results, build logs	20–80k tokens
The actual conversation between you and it	5–15k tokens

Notice the proportions. The part that is actually a conversation is the smallest part of the conversation. Almost everything in the window is machine output that scrolled past you.

So "30 turns" is really just shorthand for "the point where your context is mostly noise". Ten turns of running a failing test suite will get you there faster than a hundred turns of chatting.

The right unit isn't turns. It's how much of the window is still relevant.

The four things that actually make it worse

The Chroma study didn't only measure length. It varied the shape of what was in the window, and this is the useful part, because shape is something you control.

1. Length alone

Even on a task with no reasoning in it at all — copy this list of words back to me, with one odd word inserted — accuracy fell as the list got longer. They tested from 25 words up to 10,000.

At the long end, models stopped copying and started inventing. Gemini models began emitting words that were never in the input at around 500–750 words. Qwen3-8B started producing nonsense from about 5,000.

Let that sit for a second. This is a task with zero difficulty. The instruction is "repeat this." Length alone was enough to break it.

There's a mechanical reason underneath. Attention is quadratic: every token is weighed against every other token. At 100,000 tokens that's 10 billion pairwise relationships to spread a fixed amount of attention across. Nothing is deleted — it's diluted.

2. When what you need isn't worded like your question

They scored how similar each hidden fact was to the question being asked, and found: the less the answer resembles the question, the faster performance falls off with length.

This one has a direct consequence for how you work. When you ask "why is the login flow slow?", the answer might be a line about connection pooling in a config file that never uses the word "login" or "slow". That's a low-similarity target — exactly the kind that degrades fastest in a big context.

Translation: the harder your question is to keyword-match, the more the length is hurting you.

3. Distractors — the big one for coding agents

They added plausible-but-wrong content near the real answer. One distractor measurably hurt. Four compounded it.

They also found the damage isn't uniform — some distractors were far worse than others, in ways that weren't predictable in advance.

Now think about what a coding agent's context looks like after an hour:

Version 1 of the file, which it read
Version 2, after it edited it
Version 3, after it edited it again
A stack trace from a bug that no longer exists
A test failure it already fixed
A plan it wrote, then abandoned

Every stale version is a distractor. They are plausible. They are relevant-looking. They're in the same language, about the same code, sitting right next to the truth. This is the single biggest source of context rot in agentic coding, and almost nobody thinks of it as pollution — it looks like history.

One more detail from the study, worth knowing when you pick a model for long sessions: hallucination behaviour under distractors differed by family. Claude models had the lowest hallucination rates; GPT models the highest. Claude's failure mode was more often to abstain — to say it wasn't sure — where others confidently made something up.

4. Structure — and this one is genuinely strange

They compared a context where the text flowed logically against one where the same sentences were randomly shuffled.

Models did better on the shuffled version.

Nobody has a clean explanation for it yet. But it's a useful warning shot for a habit a lot of us have: carefully assembling a beautiful, well-ordered, coherent block of context on the assumption that tidiness helps the model. On this benchmark, coherent structure did not help. It hurt.

Don't over-read it — this is one benchmark, not a law. But it should stop you from assuming that neatly organising a huge context is doing anything for you. The thing that reliably helps is having less in there.

The 2026 finding almost nobody is talking about

Most articles about context rot stop at the 2025 study. There's a newer one that reframes the whole problem, and I think it's the more useful mental model.

In June 2026, Shijie Xia, Yikun Wang, Zhen Huang and Pengfei Liu published Diagnosing and Mitigating Context Rot in Long-horizon Search. They studied four flagship models across three benchmarks, and named something they call premature termination:

Under extensive context, models give up — or hand back an uncertain wrong answer — long before the context window is anywhere near full.

And the rate of premature termination rises with context length, even after controlling for how hard the question is.

This is a different failure from "it couldn't find the answer". The model stops looking. It ends the search early, on a task it has plenty of room and plenty of capability to keep working on.

If you use coding agents, you have watched this happen and probably filed it under laziness:

It searches two files, doesn't find it, and asks you where it is.
It stops after fixing one of the three things you listed.
It says "this would require a larger refactor" for something it happily did this morning.
It hands back something hedged and half-done rather than finishing.

That's not attitude. That's a measurable effect of a full context window.

The paper also evaluated seven context-management methods across three categories, and their conclusion about why those methods work is the part worth internalising: the methods help because they reduce the premature-termination rate, which lets the model keep exploring. You're not cleaning the context to help it remember. You're cleaning it to stop it from quitting.

The fixes: four moves, and that's the whole toolbox

Everything people call "context engineering" collapses into four moves. The clearest formulation of this is LangChain's — write, select, compress, isolate. Here's each one in plain terms, with what it looks like when you're using an agent rather than building one.

Write — put it somewhere other than the window

Get durable information out of the conversation and into a file the agent can read back when it needs it.

In practice: a rules file (CLAUDE.md, .cursorrules, AGENTS.md), and a plan file the agent keeps updating as it works.

Why this works is subtle and worth spelling out: a decision written in chat at turn 4 is a distractor by turn 40, because five later messages have partially contradicted it. The same decision in a file is a single current version. Files don't accumulate stale copies of themselves the way a conversation does.

Select — bring in only what this step needs

Don't front-load. Pull in the file when it's time to touch the file.

The counter-intuitive habit to break: dumping your whole schema, all your types and six related files "so it has context." Every one of those is a distractor for the 95% of the task they aren't relevant to.

Even tool definitions count. One result LangChain cites: applying retrieval to tool descriptions — showing the model only the relevant tools instead of all of them — improved tool selection accuracy threefold. If you've wired up thirty MCP tools and your agent keeps picking the wrong one, that's not the model being dim; that's forty tool descriptions competing for attention on every single turn.

Compress — keep the meaning, drop the tokens

Summarise and move on. Claude Code does this automatically with auto-compact once you cross about 95% of the window.

The mistake is treating compaction as an emergency measure that fires on its own at the end. By 95% you have already been degrading for a long while. Compact at the seams — when you finish a subtask, when you switch files, when a test finally goes green. You know where the boundaries are. The auto-trigger doesn't.

And trim the biggest offenders by hand. A 400-line stack trace from a bug you already fixed is pure distractor. So is the full output of a test run from three fixes ago.

Isolate — give separate work separate windows

Sub-agents. Anthropic's multi-agent research system had subagents running in parallel, each with its own context window, and only the final result came back to the main agent — not the dozens of tool calls that produced it.

That last clause is the entire benefit. "Search the codebase for every place we handle refunds" might burn 40,000 tokens of grep output to produce one paragraph of answer. Done in the main window, you keep all 40,000. Done in a sub-agent, you keep the paragraph.

The honest tradeoff, from the same source: that system used up to 15× more tokens than ordinary chat. Isolation buys reliability with money. Sometimes that's a great trade and sometimes it isn't.

What to actually do tomorrow

Ordered by effort, cheapest first:

Start a new session at task boundaries. Free, instant, and it's the single most effective thing on this list. Finished the feature? New session. A fresh window with a good rules file beats a long window with perfect history, essentially always.
Move durable decisions into a file, not the chat. Anything you'd be annoyed to repeat is a rules-file line, not a message.
Compact deliberately, at seams. Don't wait for the automatic trigger.
Stop pasting things "just in case." Every extra file is a distractor with a cost and no benefit until it's needed.
Prune your tool list. If your agent has thirty tools loaded and uses four, the other twenty-six are tax on every turn.
Push searching and exploring into sub-agents, so the exploration doesn't live in your main window.
When quality drops, restart — don't argue. Correcting a rotting context adds tokens to a window whose problem is that it has too many. You are treating the disease with more of the disease.

Six signs you have it right now

The agent breaks a rule it followed correctly earlier in the same session.
It reintroduces something it already fixed.
It starts hedging and abstaining on tasks it was completing an hour ago.
It gives up early — "you may want to check X yourself" for something it was doing itself before.
It edits a stale version of a file, one that matches what it read but not what's on disk.
Your fix for all of the above is to explain more, and explaining more makes it worse.

That last one is the tell. If adding context makes it worse, context is the problem.

The reframe

Most of us learned to treat the context window as memory: put the important stuff in, and the model will remember it.

The research points somewhere else. It behaves less like memory and more like a shared workspace with finite attention. Everything you set on the table competes with everything else on the table — including the six things you put there an hour ago and stopped needing.

So the skill isn't getting more into the window. It's knowing what to take out, and when.

A million-token context window is not an invitation to use a million tokens. It's an invitation to be much more careful about which ten thousand you use.

Which of the six signs do you hit most? I'm most curious whether anyone has found a good rule for **when* to restart a session — I still do it on instinct, and instinct is usually about twenty turns too late.*

Your Bundler's Default Target Ships a Blank Screen to iOS 15

Info Inlet — Tue, 25 Aug 2026 04:26:18 +0000

Four tests were failing in our mobile repo before I started last week's work. I fixed the two that had outgrown their screens in about twenty minutes.

The other two were right, and both described bugs that were live, on both app stores, for anyone who happened to own the wrong phone or add the wrong city.

One of them had never produced an error message in its life. The other threw a perfectly good, perfectly loud exception that no one was ever in a position to see.

They are opposite failures, and our test suite was blind to both for exactly the same reason.

We build Xenition, an AI workspace — and the mobile app is Flutter, 382 Dart files, on iOS and Android. Some of its surfaces are not Flutter at all. The diagram canvas is Excalidraw, the 3D surface is three.js, and the notebook runs Python through Pyodide — each one a real web bundle, vendored into the app's assets and rendered in a WebView, because rewriting Excalidraw in Dart is not a thing a small team gets to do.

That decision is defensible. What follows is the invoice for it.

Why a bundled WebView fails in a way native code cannot

When Dart throws, you get a stack trace. When a Flutter widget overflows, the screen turns into yellow-and-black hazard stripes and shouts at you. The framework is built on the assumption that a failure should be loud.

A JavaScript module loaded into a WebView has none of that, and one specific failure inside it is completely silent:

A module that fails to parse reports nothing.

Not a thrown exception. Not a rejected promise. Not an onerror. The parse happens before any of your code exists, so there is no code there to notice it failed. The WebView loads, the host page renders, your Flutter side sees a healthy page load event, and the module — the entire canvas — never runs. The user gets a spinner that spins forever.

Now add the two things that make this specific to mobile:

Your users' WebView is not your WebView. On iOS it's WebKit, at whatever version the OS shipped with, and it does not update independently of the OS.
Your build target is a number you set once, in a different repo, probably by accident.

Those two facts met in our app and produced a feature that had never worked for a slice of our users.

Bug 1: the canvas never started on iOS 15.0–16.3

The Excalidraw bundle is built in our web repo, by npm run build:diagram-host, and the vite config said:

build: {
  outDir: OUT,
  emptyOutDir: true,
  target: 'es2022',
  sourcemap: false,
},

es2022 is a perfectly reasonable thing to type. It's recent-but-not-bleeding-edge, every desktop browser has supported it for years, and on the web — where that bundle also runs, and where the user's Chrome updates itself every six weeks — it is entirely correct.

The problem is one feature in that standard: class static blocks.

class Thing {
  static {
    // initialisation that needs statements, not just an expression
  }
}

WebKit only parses those from Safari 16.4. Our app's ios/Podfile says:

platform :ios, '15.0'

So every user on iOS 15.0 through 16.3 — a real population, on iPhone 6s and 7 hardware that Apple stopped updating — loaded a module that WebKit refused to parse. SyntaxError, thrown by the parser, before a single line of Excalidraw existed to catch it.

What those users saw: they tapped Diagram, and nothing happened. Forever. No error, no toast, no crash we could see in Crashlytics, because nothing crashed. From the app's perspective the page loaded fine.

Fixed in both places it can be fixed

The permanent fix is one line, and it's in the web repo:

build: {
  outDir: OUT,
  emptyOutDir: true,
  // NOT es2022: that emits class static blocks, which WebKit only parses from
  // Safari 16.4. The mobile app supports iOS 15.0 (ios/Podfile), and a module
  // that fails to parse reports nothing — the diagram canvas would simply
  // never start, silently, for every user on 15.0–16.3.
  // mobile/test/vendored_bundle_syntax_test.dart fails if this regresses.
  target: ['safari15', 'chrome90'],
  // Android WebView never ships these; skipping them keeps the APK smaller.
  sourcemap: false,
},

Note the shape of that comment. It says what not to do, why, who it hurts, and what will fail if you undo it. A bare target: ['safari15', 'chrome90'] reads as arbitrary conservatism and the next person to touch the file bumps it back to something modern, for good reasons, in a repo where nothing tests iOS.

But changing the config only helps the next build. The 23 modules already vendored into the app's assets were still the old ones, so they got lowered in place with esbuild:

DIR="assets/diagram/assets"
TARGET="safari15,chrome90"
VERSION="esbuild@0.24.0"
MARKER='static{'   # class static blocks — Safari 16.4+

Two details in that script that took longer to get right than the fix:

Only rewrite modules that actually carry the offending syntax. Re-minifying an already-minified module produces cosmetically different output every time, so touching every module in the bundle would churn all of it on every run and make every diff unreadable. Matching on the marker keeps it idempotent.
Exported names and import specifiers are preserved, so the module graph is unchanged. Local identifiers get renamed and that's fine — nothing outside the module can see them.

The part that makes the fix survive

Here is the uncomfortable truth about a vendored bundle: the fix does not survive a rebuild. Anybody who runs npm run build:diagram-host overwrites assets/diagram/ and brings the modern syntax straight back — silently, again, because the failure was silent the first time.

So the actual deliverable isn't the lowered bundle. It's this test:

/// Guards the vendored Excalidraw bundle against syntax old WebViews can't parse.
///
/// Covers every vendored JS bundle, not just Excalidraw's: the 3D surface ships
/// a three.js host built the same way, and it would fail the same way.
void main() {
  for (final path in ['assets/diagram/assets', 'assets/model3d']) {
    test('$path parses on the oldest iOS we support', () => _check(path));
  }
}

void _check(String path) {
  final dir = Directory(path);
  final modules = dir
      .listSync()
      .whereType<File>()
      .where((f) => f.path.endsWith('.js'))
      .toList();
  expect(modules, isNotEmpty, reason: 'the bundle should contain JS modules');

  // Class static blocks — `class A { static { … } }`. This is the newest
  // syntax an es2022 build emits and the one that actually breaks iOS 15, so
  // it doubles as the marker for "this bundle was rebuilt at the wrong
  // target". Matching the minified form avoids hits on the `static` keyword
  // used for ordinary class members.
  final offenders = [
    for (final f in modules)
      if (f.readAsStringSync().contains('static{')) f.uri.pathSegments.last,
  ]..sort();

  expect(
    offenders,
    isEmpty,
    reason: 'These modules use class static blocks, which WebKit only parses '
        'from Safari 16.4 — on iOS 15.0-16.3 the canvas will never start, '
        'silently: a module that fails to parse reports nothing.\n'
        'Fix it in one of two ways:\n'
        "  * permanent: in web/excalidraw-host/vite.config.ts set target: ['safari15', 'chrome90']\n"
        '  * here and now: run ./tool/lower_diagram_bundle.sh from mobile/\n'
        'Offending modules: $offenders',
  );
}

It is a grep in a trench coat. It does not launch a simulator, it does not parse JavaScript, it does not know what Excalidraw is. It reads files and looks for four characters.

And it is the single highest-value test in that repo, because it converts a silent, device-specific, unreproducible-on-your-machine failure into a red line in CI that explains itself and names both fixes. The reason: string is longer than the assertion. That's deliberate — the person who hits this will be someone who ran an unrelated npm command in an unrelated repo, and every word they need has to be right there.

A guard test's job is not to be clever. It is to be the thing that shouts when the environment quietly changes underneath you.

The generalisation past our stack:

Your build target is a claim about your users' runtimes, and nobody validates it for you. Every bundler defaults to something modern. Every WebView on a phone is frozen to an OS. Those two facts are on a collision course in every app that ships a bundle, and the collision is silent.

If you ship any JS inside a mobile WebView, go and check that number now. It'll take ninety seconds and there's a real chance you find what we found.

Bug 2: the loud one nobody could hear

The second failure is the opposite kind, and that is why it's here.

The world clock — a small utility surface, 56 cities — threw LocationNotFoundException. On Africa/Accra.

No ambiguity, no silence, a named exception with a stack trace. Everything bug 1 wasn't. And it had been shipping just as long, because a loud error in a code path nothing executes is exactly as invisible as a silent one.

Accra is not obscure. It is a capital city, it has been in the IANA time zone database for decades, and Africa/Accra is exactly the id you'd expect to use. The reason it isn't there is that it is a link, not a zone.

The IANA database distinguishes canonical zones from backward-compatibility links — ids that used to be zones, or that duplicate one, kept alive so old configs don't break. Africa/Accra is one of them; the canonical zone it points at is Africa/Abidjan. Same offset, and neither has ever observed daylight saving.

And package:timezone's data/latest.dart ships the canonical zones and none of the links. Not a bug — links are a compatibility layer, they roughly double the table, and a mobile app has every reason to want the smaller one. But it means an id that is correct, current, and documented throws at runtime.

static const _zones = <String, String>{
  'UTC': 'UTC', 'London': 'Europe/London', 'Lisbon': 'Europe/Lisbon',
  // Africa/Accra is a backward-compatibility LINK in the IANA database and
  // data/latest.dart carries none of those. Abidjan is the canonical zone it
  // points at: same offset, and neither has ever observed DST.
  'Accra': 'Africa/Abidjan',
  // …
};

And then the same trap from the other side

While fixing that I found UTC throws too.

getLocation('UTC') — the most obviously valid timezone identifier that exists, the one you'd use in a test as the safe value — raises LocationNotFoundException.

The reason is almost philosophical: UTC is not a row in the database. It is the origin every row is measured from. So the package hands it over as a constant instead:

/// Now, in that city — with whatever rule is in force today.
///
/// UTC is not a row in the zone database; it is the origin every row is
/// measured from, and `getLocation('UTC')` throws. The package hands it over
/// as a constant instead.
tz.TZDateTime _now(String city) {
  final id = _zones[city]!;
  return tz.TZDateTime.now(id == 'UTC' ? tz.UTC : tz.getLocation(id));
}

Two ids, opposite reasons, identical exception. One is too old to be a zone; the other is too fundamental to be one.

And the blast radius was the same for both: adding either city to your clock threw during build, and the whole tool went down — not the one row, the entire surface. A user who added Accra didn't get a broken Accra card. They got a broken world clock, and the only way back was to work out that a city they'd added was the cause.

A lookup that throws instead of returning null turns one bad row into a dead screen. If a table is user-extensible, every lookup against it needs a policy for the id that isn't there.

Why 821 tests caught neither

This is the part I'd want someone to take away, because it's the one that transfers to codebases that ship no JavaScript and no timezones.

We had a smoke test for the tools surface. It built the world clock. It passed, every run, for months.

It built the world clock with its four default cities.

London, New York, Tokyo, Sydney. All canonical zones. All present in data/latest.dart. The test exercised the widget, the state, the rendering, the offset labels, the layout — everything except the fifty-two rows where the bug was.

The replacement is boring and it is the whole point:

/// tools_smoke_test builds this tool with its four default cities, which is why
/// it caught nothing for either of them. This one puts every city on the clock
/// at once, so the whole table gets looked up.
void main() {
  const cities = <String>[
    'UTC', 'London', 'Lisbon', 'Accra', 'Casablanca', 'Paris', 'Berlin',
    'Madrid', 'Rome', 'Lagos', 'Cairo', 'Athens', 'Johannesburg', 'Istanbul',
    // … all 56 …
  ];

  testWidgets('every city on the clock at once resolves and renders', (tester) async {
    // …
  });
}

Fifty-six cities, one widget, one test. It runs in the same time as the old one because the lookup is a map read.

A smoke test that builds the defaults tests the defaults. Every static table your app ships — timezones, currencies, locales, country codes, MIME types, unit conversions — is a list of rows nobody has ever exercised, sitting behind a lookup that throws. The test that covers it is a for loop, and it is nearly free.

We ship 36 locales and 10,002 translation keys. I know exactly which test I'm writing next.

The checklist

What I'd actually run against any app that renders bundled web content:

[ ] What is your bundler's target? Write it down. Compare it to your minimum OS, not your test device.
[ ] Does anything in your build pipeline overwrite vendored assets? If yes, is there a test that fails when it does?
[ ] Is the failure of your WebView content visible from the native side at all — or does a dead module look exactly like a slow one?
[ ] Do you have a timeout on "the canvas is starting"? A spinner with no deadline is how a silent failure becomes a support ticket instead of a bug report.
[ ] For every static table you ship: is there a test that touches every row, or only the defaults?
[ ] For every lookup against those tables: what happens for an id that isn't there — a null you handle, or an exception that takes the screen down?
[ ] Do your device-lab / TestFlight testers include anyone on your oldest supported OS? If not, your minimum is a number in a Podfile, not a claim you've verified.
[ ] When a test fails: is the claim it makes still true? Decide that before deciding what to edit.

What I'd tell someone shipping a WebView in a mobile app

The vendoring decision was right. We would not have a diagram canvas, a 3D surface or a Python notebook on a phone if we'd insisted on writing all three natively, and users don't grade you on which rendering engine drew the thing.

But bundling web content into a native app means you have inherited a second runtime, and you don't control its version. Everything the web taught you about "just target modern browsers" is downstream of automatic browser updates, and an iPhone that stopped getting iOS updates in 2021 is still in someone's pocket, still on your store listing's supported-devices list, still tapping the button that does nothing.

Our diagram canvas never started for those users. Nobody filed a bug, because there was nothing to describe — you tap Diagram, and the app is just a bit rubbish, in a way that's hard to put in an email. It was caught by a test that reads files looking for four characters, on a laptop, in about forty milliseconds.

Both fixes are live. 821 tests pass, and the two silent ones are silent no more.

If you ship a bundle inside a WebView, I'd genuinely like to know what your target says — and whether you knew before you looked. That number is doing more work than almost anything else in your build config, and I'd never once checked ours.

I work on Xenition — one AI workspace for documents, decks, code, apps and media, free to start, on web, desktop, and both app stores: iOS and Android. The mobile app is Flutter — 382 Dart files, 26 feature modules, 36 languages, and, until last week, one canvas that never started.

ChatGPT Plugin Rejected? Here's What the Review Actually Checks

Info Inlet — Mon, 24 Aug 2026 08:11:17 +0000

We submitted our app to OpenAI's directory on a Thursday. Nine days later, this arrived:

One or more of your test cases did not produce correct results. Please re-run all submitted test cases and align tool behavior/output with the documented expected outcomes. Ensure the same test cases pass consistently on both ChatGPT web and mobile.

That is the entire rejection. No tool named, no transcript, no log line. One paragraph and a link to appeal if we thought they'd made a mistake.

They had not made a mistake. Every symptom in that paragraph was real, and finding out why took a full day of running our own tools against production like a stranger. There were four failures: three server bugs, and one that was in the test cases themselves. Two of the three bugs meant tools that had never worked once, in production, for anyone, since the day they shipped.

We build Xenition, an AI workspace, and the submission was our MCP server — 49 tools that let ChatGPT create and edit real artifacts in a user's workspace: documents, decks, boards, ledgers, notes, spreadsheets. If you're preparing a submission this month, this article is the thing I wish I'd read the week before we hit Submit — every bug named, and the one review technique that would have caught all four.

I'll name our own bugs exactly rather than paraphrase them. None of it is product-specific.

Why this review is harder to pass than an app store review

A normal store reviewer taps through your UI. Your UI is a thing you have looked at ten thousand times.

A directory reviewer does something you have almost certainly never done: they take your written test cases, hand the prompts to a model, and watch what your tools actually return — then do it again on a second client. That's the part that broke us. The review surface isn't your product. It's:

A cold account. No state you seeded by hand while debugging.
A one-turn tool call. No follow-up. Whatever your tool returns, that's the whole conversation.
Two clients that word things differently. ChatGPT web and ChatGPT mobile do not phrase the same user intent identically, so your tool receives two different strings for one test case.
Someone reading the model's sentence, not your JSON.

Every one of our four failures lives in that list. None of them was reachable from our own app.

Bug 1: the tools read an empty copy of everything

Our artifact store has a List that selects content and meta as NULL. That is not a bug — it's the query behind the library grid, which renders cards. A card needs a title, a type and a timestamp; shipping every document's full body to draw a wall of tiles would be absurd.

Then the workspace tools arrived — list_tasks, add_task, query_ledger, add_ledger_entry, search_notes — and their lookup helper went through that same List. Every one of those callers then parses the document it just asked for.

Here is what a user of a perfectly healthy workspace got:

Call	Reality	What the tool said
`list_tasks`	a board with 30 cards	`0 tasks`
`add_task`	same board	"the board is empty"
`add_ledger_entry`	a ledger with real invoices	success — having saved a brand-new ledger over the real one

Read that last row again, because it's the one that scared me. Enter an invoice for Acme, then one for Globex, and you are left with a ledger containing only Globex. Not an error. Not a warning. A success message, and the user's data replaced with a single row. The empty parse looked exactly like an empty document, so "add a row to the existing ledger" and "create a ledger with one row in it" became the same code path.

The dates are the humbling part. The List change that started blanking content landed on 20 July. These tools shipped on top of it on 10 August. We submitted on 13 August. They never worked, not once, in production — and three days of our own use didn't reveal it, because nothing in our own app calls the workspace through MCP.

The fix is four lines and a rename:

// ListFull is List with content and meta loaded. Callers that have to READ what is inside each
// artifact — the board's cards, the ledger's rows — need this one, not List.
func (s *Store) ListFull(ctx context.Context, userID, workspaceID, typeFilter, search, projectID string) ([]Artifact, error) {

and at the call site, with the reason written down so nobody quietly swaps it back:

// ListFull, not List: every caller here reads the artifact's document — the board's cards, the
// ledger's rows, the note's body. List blanks content for the library grid.
return s.artifacts.ListFull(ctx, uid, "", typ, query, "")

The lesson generalises well past MCP:

A read path optimised for one caller is a data-loss bug waiting for its second caller.

List returning blank content was a feature for the grid. It became silent destruction the moment a writer used it to decide whether something already existed. If a query drops fields for performance, the name has to say so — List vs ListFull — because the type signature doesn't. Both return []Artifact. Both compile. Only one of them tells the truth about what's inside.

Bug 2: our clarifying question had nobody to ask

Our orchestrator has a clarify gate. Vague brief in, questions back out — good behaviour in a chat window, where a human answers and generation continues.

An MCP tool call is one turn. There is no human in it. So a thin brief made create_slides come back in about two seconds with:

the engine did not return a deck

Correct, technically. The engine had returned a set of questions and there was nobody to answer them, so the tool reported the only thing it could see.

Now the detail that turns this from an ordinary bug into the exact wording of the rejection: whether the gate fires depends on how the brief reads, and ChatGPT phrases the same user intent differently on web than on mobile. Same test case, same account, same minute — passes on one client, fails on the other. "Ensure the same test cases pass consistently on both ChatGPT web and mobile" was not boilerplate. It was a description of our bug, written by someone who had just watched it happen.

The fix is to tell the engine there is nobody home:

// NoClarify because there is nobody here to answer. On a vague brief the orchestrator's clarify
// gate returns questions instead of an artifact, and an MCP call is one turn.
req := engine.Request{Message: brief, UserID: uid, Skill: skill, NoClarify: true}

A flag is a promise, though, and promises get broken by the next person to touch the
orchestrator — so there is also a guard behind it, and the comment on the guard is the part
worth copying:

// Belt-and-braces: NoClarify above should stop this, but if the gate ever fires again the
// caller gets something it can act on rather than "the engine did not return a deck".

If the gate does fire, the tool now returns the questions themselves. A model that receives
three questions can ask the user them; a model that receives "the engine did not return a
deck" can only apologise.

Any interactive gate in a pipeline becomes a hang or a lie when the caller is a machine. Clarify loops, consent prompts, "are you sure?", rate-limit backoff that waits for a retry someone has to press — find every one of them before a reviewer does.

Bug 3: an unknown id reported as "Still working…"

Small bug, worst optics of the three.

check_agent_run polls a background agent. Ask it about a run id that does not exist and it answered:

Still working…

For any id. Forever. A typo'd id, a made-up id, an id from a different account — all of them "still working". A model that receives that will tell the user to be patient, then tell them again, and the user waits for a job that was never created.

Fixed by making a 404 an error rather than an absence of information, and the error names the recovery:

no agent run found with id "nope" — use the missionId that run_agent returned

The general form: "not found" and "not finished" must never collapse into the same reply. One of them is a completed fact about the world; the other is a request to wait. Any status endpoint that answers pending when the truth is nonexistent will strand somebody.

Problem 4: our test cases depended on each other

This one had no server fix, and it's the one I'd bet most submissions get wrong.

Our eight cases looked reasonable in the portal. Case 1 created a deck. Case 2 searched the workspace for the deck case 1 had created — and the expected-outcome text said, helpfully, run case 1 first.

A reviewer who runs them in a different order, in a fresh chat, or on the second device sees case 2 fail. And they are right to. We had written a suite that only passes if it is executed as a script by someone who read our footnotes, and then handed it to strangers on two clients.

Rewritten, every case stands alone. These are the rules we wrote at the top of the file so the next set can't regress:

Self-contained. Seed the demo workspace so the read cases have something to find without a write case running first.
Specific briefs. A vague brief is what fired the clarify gate. NoClarify fixes it server-side, but a specific brief also keeps the output stable enough to describe.
Expected outcomes describe shape, not exact words. A deck's title comes from a model and differs run to run. "A 5-slide deck is created and rendered" passes every time; asserting a literal title does not.
No dependence on which artifact is "most recent" unless the outcome is phrased to allow any of them.
Every case verified against production, from a cold account, on both clients.

An example of the difference, from the same tool:

Version	Expected outcome
Rejected	Returns the deck created in case 1, titled "Orbit Sales Deck".
Resubmitted	`search_artifacts` returns the workspace's sales-onboarding documents as a results list, each row with an "Open" link back into the app. Read-only: nothing is created or changed.

The second one is falsifiable by a stranger. That's the only property that matters.

The part I got right by accident: annotations

While preparing the submission we annotated all 49 tools — readOnlyHint, destructiveHint, openWorldHint — and had to write a justification for each hint on each tool. 147 short paragraphs. It felt like paperwork.

It wasn't, because of one line in the spec that I would have bet money against:

// DestructiveHint and OpenWorldHint are *bool in the Go SDK, and the spec's default when they
// are ABSENT is true.

Absent means destructive. Absent means open-world. Before we filled these in, every tool we shipped — search_artifacts, a pure read, included — was advertising itself to every client as a destructive, open-world operation. Not a cosmetic problem: clients use these hints to decide what needs a confirmation dialog and what can run unattended.

We settled it with constructors instead of scattered literals, one per behaviour class, each carrying the definition in its comment:

// hintsRead: reads the user's own workspace and nothing else.
func hintsRead() *mcp.ToolAnnotations {
    no := false
    return &mcp.ToolAnnotations{ReadOnlyHint: true, DestructiveHint: &no, OpenWorldHint: &no}
}

// hintsAdd: adds something new to the workspace; nothing that already exists is replaced.
func hintsAdd() *mcp.ToolAnnotations {
    no := false
    return &mcp.ToolAnnotations{DestructiveHint: &no, OpenWorldHint: &no}
}

// hintsAct: hands work to a system whose effects this call does not bound — a background agent, or
// a pending action against a connected third-party app. Destructive because what runs is open-ended.
func hintsAct() *mcp.ToolAnnotations {
    yes := true
    return &mcp.ToolAnnotations{DestructiveHint: &yes, OpenWorldHint: &yes}
}

Constructors, not package-level vars, so no two tools can ever share and mutate one annotations value.

Writing 147 justifications also forced the question what does this tool actually do onto tools nobody had asked it about in a while, and three of ours turned out to be annotated the opposite of what their names suggest:

create_app is read-only. It persists nothing. It builds a pre-filled deep link into the builder.
check_3d_model is not read-only. When the poll succeeds it writes the finished mesh onto the artifact.
approve_action is destructive and open-world; deny_action is neither. Approving releases a queued action into a third-party app. Denying closes a request locally.

If a reviewer ever questions our annotations, it will be those three, and the justification text already explains each one. Annotate by what the call does, never by the verb in its name.

The demo account is part of the submission

Two things here cost me hours and would cost you the same.

A duplicate row can be load-bearing. Our seeded workspace held two documents with the same title, so a search case returned the same name twice. It looked sloppy, so I deleted the newer copy — and the grounded-answer case went flaky. It had been returning the same six-step answer every run; afterwards one run answered properly and the next said the provided passages do not describe the onboarding process. Removing the duplicate had thinned the grounding corpus below whatever threshold that answer needed.

Flaky is the precise failure we'd just been rejected for. The copy went back, the answer was consistent across three runs again, and the doc now says so in writing:

A duplicate row reads better than an unreliable answer.

Clean out what your own debugging left behind. Verifying all 49 tools against production left the demo account full of junk — a "Dev Board", "A Budget", four near-identical sales decks. A reviewer opening that account should find a workspace that looks like a real user's, not the wreckage of a test sweep.

Portal friction, so you can plan around it

Four things that cost time and are nobody's bug:

The tool list is virtualised. Page-reading tools only ever return part of it. Filling in each tool's fields one tool at a time was the only reliable way through.
The SPA has a load race. Navigating straight to a deep-linked section sometimes renders ask an organization admin to assign you a role with the api.apps.read permission, and the Skills tab briefly showed no uploaded skill at all. Both times a full reload through the plugin list showed the real state. It is not a permissions problem — I nearly filed a support ticket about a role I already had.
Scan Tools needs an OAuth authorization first. It opens your own consent page and somebody has to sign in there before the scan will run.
The skill safety scan is slow. The portal warns up to two hours. Don't schedule your submit for the last hour of the day.

And one thing worth knowing before you plan a launch: approval does not publish. When it passes, the portal unlocks a publish option and a human presses it. The go-live moment stays yours.

The pre-submit checklist

Everything above, as the list I'll actually run next time:

[ ] Run every tool against production, from an account with no state you created by hand.
[ ] For each read tool: does it see content, or an optimised-away copy of it? Check against a known row count.
[ ] For each write tool: does "I found nothing" ever become "so I'll create a fresh one"? That's a data-loss path.
[ ] Does any tool sit in front of an interactive gate — clarify, consent, confirm — that has no human to answer it?
[ ] Does any status tool report an unknown id as pending?
[ ] Do the test cases pass in any order, in a fresh chat, on web and mobile?
[ ] Does any expected outcome assert model-generated wording? Assert shape instead.
[ ] Are destructiveHint / openWorldHint set explicitly on every tool? Absent means true.
[ ] Is each annotation justified by what the call does, not by its name?
[ ] Is the demo account seeded, and cleaned of what your own testing left behind?
[ ] Did you read the model's sentence for each case, not just your JSON?

What I'd tell someone submitting next week

The rejection email is one paragraph and it will feel unfair. Take it literally instead of personally: "did not produce correct results" and "consistently on both web and mobile" were, in our case, precise technical descriptions of three bugs and a broken suite. We didn't appeal, because there was nothing to appeal — they had run our tools as a user and seen our product lie to them.

The uncomfortable takeaway isn't about any directory. It's this: an MCP server is a second product, and it has its own users, its own state, and its own bugs. Ours had two tools that had never once worked in production, and our own app was structurally incapable of noticing, because our app doesn't call itself through MCP. Every hour of that review found something real.

At the time of writing this, the fixes are live on production, re-verified there — list_tasks reads 30 and goes to 31 after add_task, the ledger keeps both invoices, an unknown run id is an error — and the resubmission hasn't gone in yet. If the second attempt teaches me anything new, I'll write that one too.

If you've built a harness that calls your own MCP server the way a client does — cold account, one turn, both clients, reading the model's sentence rather than your JSON — I'd like to read about it. That's the piece I'm still missing, and it's the only thing that would have caught all four of these before a stranger did.

I work on Xenition — one AI workspace for documents, decks, code, apps and media, free to start, on web, desktop and both app stores. Its MCP server has 49 tools. Two of them, until recently, had never worked.

Nobody Argues With a Prefilled Field

Info Inlet — Sun, 23 Aug 2026 06:23:34 +0000

An agent that remembers you is almost always judged in chat. It recalls something, it says the thing, and if the thing is wrong you can see that it's wrong — it's a sentence, in your reading flow, and you push back. "No, I moved." Memory corrected, one turn, no damage.

Then we wired the same memory into an input box.

We build Xenition, an AI workspace — chat on one side, and on the other a couple of hundred small things you can actually open and use: documents, decks, forms, calculators, converters. Two surfaces, one signed-in user, one memory. Connecting them was the obvious feature, and it took us a while to notice we'd changed what memory is.

A user tells the assistant, in passing, in some conversation weeks ago, that they're 173 cm and 70 kg. Later they open the BMI calculator and both fields are already filled. It feels like magic the first time. It is genuinely good product.

It is also the exact moment the safety properties you were relying on quietly stop applying — because a prefilled field doesn't get read. It gets submitted.

Every example below is our own code rather than a paraphrase, because those are the ones I can quote exactly. None of it is product-specific — if your agent remembers anything and your UI has a form, you have this problem too.

I want to write about that transition, because the agent-memory conversation right now is almost entirely about what memory says. Three of this week's most-read #agents posts are about provenance, authority ranking, and remembering decisions instead of data. All correct. But nothing changes the risk profile of a memory system as much as changing where its output lands, and I haven't seen anyone write that part down.

A remembered fact has two consumers, and only one of them can hedge

This is the whole idea, so let me put it plainly.

When a remembered fact becomes a sentence, it can carry doubt. "I think you mentioned 70 kg — still right?" is a completely legitimate output. Hedging is free. It's one clause.

When the same fact becomes a value, hedging is impossible. There is no way to type "probably 70" into a number input. The field is either empty or it is an assertion — and it's an assertion sitting inside the user's form, above the user's submit button, which means it is now attributed to the user, not to the model.

	fact as a sentence	fact as a field value
Attributed to	the assistant	the user
Wrongness looks like	a claim you can dispute	a default you skim past
Hedging	natural, free	impossible
Cost to correct	one reply	you have to notice it first
Blast radius	that turn	whatever the form does

Two consequences fall out of this, and they're the load-bearing rules for everything below:

The confidence bar for writing into a field is higher than the bar for mentioning in chat. A hunch is an acceptable sentence. A hunch is not an acceptable default.
The UI has to supply the hedge the value can't carry. If the value can't say "probably," the interface around it has to.

The shape of the thing

For context, the pipeline Xenition ended up with. Nothing exotic:

chat turn (signed in)
  → engine extracts durable personal facts → long-term memory, keyed by verified user id

later, user opens a tool/form
  → client sends the field SCHEMA (name, label, type) + instantly-known profile facts
  → server does extractive recall over that user's memory
  → returns { values: { heightCm: 173, weightKg: 70 } }
  → client merges into empty fields only

We built it for forms first — smart forms that fill themselves from what you've already told the assistant — then went to extend it to the roughly two hundred tools next door — and it was that second pass that forced us to actually write the rules down, because the first version had at least three of the bugs below and we'd been lucky.

Rule 1: the user id must be injected by whoever verified it

Start with the one that's a security bug rather than a UX bug.

Most "assist" endpoints in a gateway are a thin passthrough. Client posts a body, gateway forwards it to the engine, engine answers. That's completely fine while the body contains only what the user typed.

Memory recall is the first endpoint where the body has to say whose memory to read. And if that identifier arrives from the client, you have built a cross-tenant read that anyone can perform by editing one JSON field in devtools.

So this endpoint doesn't get the generic passthrough. It gets its own handler, whose entire job is to throw away any client-supplied id and inject the one the auth layer verified:

// dedicated handler, NOT the generic /v1/assist/* passthrough
uid, ok := auth.UserID(r.Context())   // verified at the edge, never from the body
if !ok {
    uid = ""                          // anonymous: profile facts only, no recall
}
body.UserID = uid                     // overwrite unconditionally

Ours is 63 lines and exists purely for that overwrite. Worth every one of them. The rule generalizes: the first time a request body needs to name a principal, your proxy stops being a proxy.

Anonymous is a real case and should work — it just falls back to facts the client already legitimately has (locale, timezone, currency). No recall, no error, no login wall.

Rule 2: extractive only. Empty beats plausible.

If you hand a generative model a field labelled "Weight (kg)" and ask it to fill the form, it will fill the form. It will not return empty. Models are extremely good at producing a plausible adult human.

And plausible is precisely the failure mode, because at the point of use, an invented value and a recalled value are indistinguishable. They're both just a number in a box. The user has no way to tell which one they're about to submit.

So the recall path is strictly extractive: if the fact isn't in memory, the field stays empty. Not "estimated." Not "typical." Empty.

An empty field costs the user four seconds of typing. An invented field costs them a wrong answer with their own name on it.

This is also the single easiest thing to get wrong when you swap the model or rewrite the prompt, and it will not show up in any test that only asserts the response shape — the shape is perfect. Assert on the behaviour: given a user with no stored weight, the weight key must be absent.

Rule 3: precedence, and a merge that cannot clobber

Once more than one thing can fill a field, you need a stated order. Ours, highest wins:

What the user has currently typed — always, no exceptions
An explicit one-shot prefill — they clicked "Open" on a suggestion carrying values
Memory recall — the background thing we're discussing
Context defaults — currency from locale, language from browser

And the merge is fill-empty-only. Never a spread, never a patch that assumes it arrived first:

const patch: Record<string, unknown> = {}
for (const [k, v] of Object.entries(recalled)) {
  const cur = current[k]
  const isEmpty = cur === undefined || cur === null ||
                  (typeof cur === 'string' && cur.trim() === '')
  if (isEmpty && String(v ?? '').trim() !== '') patch[k] = v
}

The bug this prevents is not hypothetical and it is horrible when it happens: the user types their real weight, the recall lands 300 ms later, and the field changes under their cursor to a number from last year. They will not notice. They will submit it.

Which brings us to the timing.

Rule 4: recall races the user's first keystroke, and you lose that race in public

Recall is async and the form is already on screen. So every apply has to re-check the world at apply time, not at request time:

useEffect(() => {
  if (!openId) return
  const ctrl = new AbortController()
  void recall(tool, ctrl.signal).then((values) => {
    if (ctrl.signal.aborted) return
    const s = store.getState()
    if (s.openId !== openId) return      // they already closed it or switched
    applyFillEmptyOnly(s, openId, values) // re-reads current input, see Rule 3
  })
  return () => ctrl.abort()
}, [openId])

Three separate guards — aborted, still-the-same-surface, still-empty — and all three fire in practice. The middle one matters more than it looks: users open a tool, glance, close it, open a different one. A recall for the previous surface landing in the current one writes a stranger's-looking value into an unrelated form.

There's an honest limitation here. Views that hold field state in local useState at mount won't see a late-arriving recall at all. You either lift those fields to the store, or you fall back to the affordance in Rule 5 — which, it turns out, is the better answer anyway.

Rule 5: give the value the hedge it can't carry — and respect fact half-life

Back to the thesis. The value can't say "probably," so the interface has to.

The cheap version is a chip in the footer rather than a silent write:

Fill from memory: 173 cm, 70 kg   [Apply]

One tap, same convenience, completely different accountability — the user chose it, so it's genuinely theirs now.

The question is when to spend that tap, and the answer nobody seems to write down is that personal facts decay at wildly different rates. Provenance discussions usually stop at "where did this come from." The more actionable axis is "how long is this true for":

Fact	Practical half-life	Prefill silently?
Date of birth	Never expires	Yes
Name, email	Years	Yes
Height (adult)	Years	Yes
Home city	Months to years	Yes — but never overwrite a location they set by hand
Weight	Weeks	Offer, don't fill
Salary, job title	Steps at unpredictable moments	Offer, don't fill
"Current project," "what I'm working on"	Days	Don't store as a fact at all

Date of birth is the ideal memory fact: stated once, true forever, tedious to type. In Xenition it silently fills the age calculator, the date-difference tool, and half a dozen forms, and it has never once been wrong. Weight is the worst: it changes silently, the user never thinks to update the assistant, and it feeds calculations that look authoritative. Same storage, same retrieval, entirely different write policy.

If you keep one thing from this post, keep that: the decay rate of a fact should decide whether it gets written or offered. Not the recall confidence score. The confidence is about whether they said it. It tells you nothing about whether it's still true.

Rule 6: "memory off" has to kill the read, not just the write

Every product with memory has a switch in settings. Two things to check, because the second is regularly missed:

The switch must short-circuit recall, not just storage. A user who turns memory off and then sees a form politely fill itself with facts about them has learned something about your switch, and they're going to post about it.
The client-side check is a courtesy that saves a round trip. The server-side check is the feature. Anything enforced only in the client is enforced nowhere.

And while you're there: recall must never throw and never block. If the memory service is down, the form opens empty and on time. {} on any failure, no spinner, no error toast. Memory is an enhancement, and an enhancement that can break the primary interaction isn't one.

Rule 7: the schema is a contract, and nothing tells you when it breaks

The client declares each field as { name, label, type }, and name has to exactly match the state key the view reads. That's the whole contract.

Which means renaming a state key — an ordinary, safe-looking refactor — silently disables prefill for that field. No error. No type failure, because it's a Record<string, unknown> crossing a network boundary. No test failure, because the endpoint still returns 200 with a perfectly-shaped body. The feature just quietly stops existing, and you find out in a month when someone asks whether prefill was always this bad.

The check I want is a build-time assertion that every declared field name appears in the state keys its view actually reads. I don't have it yet — it's the honest gap in this post. Right now what we have is one end-to-end test per prefillable surface, which is more work and less coverage than the static check would be.

What still doesn't work

We don't store per-fact timestamps yet. So the half-life table above is a policy expressed in an extraction prompt, not something enforced by data. It should be a field on the memory record. It will be.
There's no fact editor. Users can turn memory off entirely; they cannot open a list and fix the one number that's wrong. "Off" is a blunt instrument to hand someone whose only complaint is a stale weight.
Normalization is invisible from the client. "I'm 5'8"" → 173 happens engine-side. When it's wrong, the client sees a plausible number and has no way to know it was derived rather than stated.
No static check on the field-name contract (Rule 7).
We measure recall coverage, not recall correctness. Percentage of fields filled is easy and slightly dangerous — it goes up when the system gets more willing to guess, which is the direction you don't want.

The checklist

If you're about to let agent memory touch a form:

Inject the user id server-side. The moment a body names a principal, stop using the generic passthrough.
Extractive only. No stored fact → empty field. Test the absence, not the shape.
State a precedence order and merge fill-empty-only — never overwrite what the user typed.
Re-check at apply time, not request time: aborted, same surface, still empty.
Sort facts by half-life, not by confidence. Slow-decaying facts fill; fast-decaying facts offer.
Mark anything that came from memory and make clearing it one click.
"Memory off" kills recall too, enforced on the server.
Recall never throws, never blocks, never spins. {} and move on.

None of this makes memory smarter. It makes it accountable, which at the point where a remembered fact turns into a value someone is about to submit under their own name, matters considerably more.

If you've built per-fact expiry — real timestamps and a decay policy, not a prompt asking the model to be sensible about it — I'd like to read about it. That's the piece I'm missing.

I work on Xenition — one AI workspace for documents, decks, code, apps and media, with the chat and the tools sharing a single memory. Free to start, on web, desktop and both app stores. It remembers your date of birth. It asks about your weight.

Your Postgres Can Do That

Info Inlet — Sat, 22 Aug 2026 06:41:48 +0000

Here is the architecture diagram for a product with roughly zero users:

Postgres      ← the actual data
Redis         ← cache, and also the queue, and also rate limits
SQS / Kafka   ← the real queue, for when Redis isn't durable enough
Pinecone      ← embeddings
Elasticsearch ← search
A cron box    ← nightly jobs, currently a $5 VPS nobody has SSH'd into since April

Six systems. Six sets of credentials, six failure modes, six things to upgrade, six bills, six places for state to disagree with the other five. A distributed systems problem, chosen voluntarily, before the first paying customer.

The Go gateway behind Xenition has no Redis in go.mod. Jobs are claimed with SKIP LOCKED, search runs on tsvector, and a large share of the schema is jsonb. Not because Postgres is magic — because every extra system has to earn its place by solving a problem we actually have, and at our scale most of them can't clear the bar.

This isn't "Postgres scales forever." It doesn't. It's that the point where it stops is much further out than the diagram above assumes, and everything before that point is operational cost you're paying for scale you don't have.

Below: six things people add a service for, the SQL that does them instead, and — the section that matters most — when you should genuinely leave.

1. A job queue: `FOR UPDATE SKIP LOCKED`

This is the one that surprises people, so it goes first.

SKIP LOCKED landed in Postgres 9.5. It makes the classic "many workers, one table, nobody takes the same row twice" problem trivially correct — a reader skips rows another transaction has locked instead of blocking on them.

CREATE TABLE jobs (
  id           bigserial PRIMARY KEY,
  kind         text        NOT NULL,
  payload      jsonb       NOT NULL,
  run_after    timestamptz NOT NULL DEFAULT now(),
  attempts     int         NOT NULL DEFAULT 0,
  max_attempts int         NOT NULL DEFAULT 5,
  locked_until timestamptz,
  status       text        NOT NULL DEFAULT 'pending'
);

-- The index that makes the claim fast. Partial: pending rows only, so it
-- stays small even when the table holds millions of finished jobs.
CREATE INDEX jobs_claim_idx ON jobs (run_after)
  WHERE status = 'pending';

The claim, which is the whole trick:

UPDATE jobs SET
  status       = 'running',
  attempts     = attempts + 1,
  locked_until = now() + interval '5 minutes'
WHERE id IN (
  SELECT id FROM jobs
  WHERE status = 'pending'
    AND run_after <= now()
  ORDER BY run_after
  LIMIT 10
  FOR UPDATE SKIP LOCKED     -- ← the entire feature
)
RETURNING id, kind, payload;

Run twenty workers against that. None of them will hand you the same job twice, and none will block waiting on another. No broker, no consumer groups, no offset management.

You get the rest of a real queue for the price of ordinary SQL:

-- Retry with exponential backoff
UPDATE jobs SET status = 'pending',
  run_after = now() + (interval '10 seconds' * power(2, attempts))
WHERE id = $1 AND attempts < max_attempts;

-- Dead letter
UPDATE jobs SET status = 'dead' WHERE id = $1 AND attempts >= max_attempts;

-- Reap workers that died holding a lock
UPDATE jobs SET status = 'pending'
WHERE status = 'running' AND locked_until < now();

That last query is your visibility timeout — the thing SQS charges for. Four lines, on a timer.

The part nobody mentions, and it's the real argument:

BEGIN;
  INSERT INTO invoices (...) VALUES (...);
  INSERT INTO jobs (kind, payload) VALUES ('send_invoice_email', ...);
COMMIT;

The row and the job commit together, or neither does. With an external broker this is the dual-write problem, and the industry's answer is the transactional outbox pattern — a table, a relay process, a whole named design pattern — whose entire purpose is to buy back the atomicity you gave away by putting the queue somewhere else. If the queue lives in the database, you never gave it away. There is nothing to buy back.

Throughput: a modest Postgres handles thousands of claims per second this way. Know your actual number before assuming you're above it.

Two of these run in the Xenition gateway right now: outbound webhook delivery for the marketplace, and scheduled social posts. The comment sitting above that second query says exactly what the feature buys — "SKIP LOCKED so two gateways cannot double-post." That is the entire coordination story between instances. There is no second sentence, no leader election, no lease.

2. Pub/sub: `LISTEN` / `NOTIFY`

Need to tell every app server something changed — invalidate a cache, push an SSE event, refresh a dashboard?

-- Fires when the transaction commits, not before. This is the good part.
SELECT pg_notify('artifact_changed', json_build_object('id', 42)::text);

conn.Exec(ctx, "LISTEN artifact_changed")
for {
    n, err := conn.WaitForNotification(ctx)
    // ...
}

Three caveats, because this one is genuinely limited and you should know before building on it:

Not durable. A disconnected listener misses the message entirely. There is no replay.
8000 byte payload limit.
Delivered after commit — which is exactly what you want, and better than most external brokers manage without effort.

The pattern that makes it safe: notify an id, not a payload. The listener reads the row itself. A missed notification then degrades into staleness your next poll or reconnect fixes, instead of a permanently lost update. If you need durable fan-out with replay, that's a real Kafka use case — but be sure you need replay, not just delivery.

3. Cache: it's a table

Yes, really, for a large class of caching:

CREATE UNLOGGED TABLE cache (      -- UNLOGGED: no WAL, much faster writes,
  key        text PRIMARY KEY,     -- contents lost on crash. It's a cache.
  value      jsonb NOT NULL,
  expires_at timestamptz NOT NULL
);

INSERT INTO cache (key, value, expires_at) VALUES ($1, $2, now() + $3)
ON CONFLICT (key) DO UPDATE
  SET value = EXCLUDED.value, expires_at = EXCLUDED.expires_at;

DELETE FROM cache WHERE expires_at < now();   -- on a timer

An indexed primary-key lookup on a warm table costs a fraction of a millisecond. If your alternative is a network hop to Redis, the gap is smaller than intuition suggests.

Redis genuinely wins when you're doing 50k+ ops/sec of pure key-value work, when you want its data structures (sorted sets for leaderboards, streams, HyperLogLog), or when you're deliberately keeping load off a database that is the bottleneck. Those are real reasons. "It's the caching layer, that's just what you use" is not one — adding Redis to relieve a database under no pressure adds a failure mode and removes nothing.

4. Rate limiting

A counter and an upsert:

CREATE TABLE rate_limits (
  subject      text        NOT NULL,   -- user id, ip, api key
  window_start timestamptz NOT NULL,
  count        int         NOT NULL DEFAULT 0,
  PRIMARY KEY (subject, window_start)
);

INSERT INTO rate_limits (subject, window_start, count)
VALUES ($1, date_trunc('minute', now()), 1)
ON CONFLICT (subject, window_start)
  DO UPDATE SET count = rate_limits.count + 1
RETURNING count;

One round trip, atomic, returns the new count so you decide immediately. Honest caveat: a single very hot subject serialises on one row, and at extreme rates you want a sliding window or token bucket rather than fixed buckets. For per-user API limits on a normal product this is enough — and it has the property Redis-based limiters famously don't: it's consistent with the rest of your data, so "did we charge them for this request" and "did we count this request" cannot disagree.

5. Vector search: `pgvector`

CREATE EXTENSION vector;

ALTER TABLE documents ADD COLUMN embedding vector(1536);

CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);

SELECT id, title, content
FROM documents
WHERE workspace_id = $1              -- ← the thing dedicated vector DBs
  AND deleted_at IS NULL             --    make you fight for
ORDER BY embedding <=> $2
LIMIT 10;

Look at that WHERE clause, because it's the entire argument.

Embeddings are never the whole query in a real product. It's always this user's documents, not deleted, in this workspace, maybe from the last 90 days. In a separate vector store, each of those filters is either a metadata field you must duplicate and keep in sync, or a post-filter that ruins your LIMIT — you asked for 10, filtering leaves 2, now you're re-querying with a bigger limit and guessing.

And the sync problem is permanent. Two systems, one source of truth, no shared transaction. A document deleted here and still present there is a data leak, not a stale cache.

With pgvector the embedding is a column on the row. Deleting the row deletes the embedding, in the same transaction, forever. That's the feature.

Dedicated vector databases earn their keep at scale — hundreds of millions of vectors, heavy filtered ANN, sharding across nodes. Below that, you're operating a second database to avoid a column.

6. Full-text search, and cron

Search:

ALTER TABLE documents ADD COLUMN search tsvector
  GENERATED ALWAYS AS (
    setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
    setweight(to_tsvector('english', coalesce(body,  '')), 'B')
  ) STORED;

CREATE INDEX documents_search_idx ON documents USING gin (search);

SELECT id, ts_rank(search, q) AS rank
FROM documents, websearch_to_tsquery('english', $1) q
WHERE search @@ q ORDER BY rank DESC LIMIT 20;

GENERATED ALWAYS AS ... STORED means the column maintains itself — no trigger to write, no update to forget. And websearch_to_tsquery accepts what users actually type, quoted phrases and -exclusions included.

Elasticsearch still wins for fuzzy matching, sophisticated analyzers, faceting at scale, and typo tolerance. It does not win for "let people find their own documents."

Concretely, on our side: team-channel message search and the support inbox both run on tsvector. More interesting is the app builder — Xenition generates working apps for users, and search ships to the generated app as a Postgres full-text scan over that app's own schema. No external engine to provision, per app, forever. That's the multiplier people miss: with search in the database, per-tenant search is an index and a WHERE clause. With a search cluster, every tenant is an operations decision.

Cron: pg_cron if your host offers it — most managed Postgres does now:

SELECT cron.schedule('reap-jobs', '* * * * *',
  $$UPDATE jobs SET status='pending'
    WHERE status='running' AND locked_until < now()$$);

If not, run the loop in your app and guard it with an advisory lock so only one instance acts:

SELECT pg_try_advisory_lock(hashtext('nightly-rollup'));

That's leader election in one line. No ZooKeeper, no etcd, no lease renewal — and the lock releases automatically when the connection dies, which is precisely the failure mode you'd otherwise handle by hand.

When you should actually leave

The credibility section. Every one of these is a real reason, and if you hit one, go:

Signal	Move to
Queue sustained above ~10k jobs/sec, or you need replay, or multiple independent consumer groups	Kafka / NATS
Cache above ~50k ops/sec, or you want sorted sets / streams	Redis
Hundreds of millions of vectors, or heavy filtered ANN across shards	A dedicated vector DB
Fuzzy matching, typo tolerance, heavy faceting, multi-language analyzers	Elasticsearch / Typesense
Analytical scans over billions of rows fighting your OLTP traffic	A column store — ClickHouse, DuckDB, a warehouse
Your write master is genuinely saturated and read replicas are exhausted	Shard, or split the workload out

Note what all six have in common: each is a measurement, not a vibe. "We might need it later" is not on the list. Neither is "this is the standard architecture."

The ordering matters too. Adding a service is a one-way door in practice — once two systems hold state, everything downstream inherits the consistency problem permanently. Extracting later is a migration. Adding early is a tax you pay every day. Do it when the number says so.

The actual thesis

It isn't that Postgres is secretly six products. It's this:

Every system you add is a state boundary, and every state boundary is a place where your data can be two different things at once.

The queue that fired for a row that rolled back. The vector still returned for a document you deleted. The cache that says the user is on the paid plan and the database that says they cancelled. None of those bugs exist inside a single transaction. All of them are the routine, expected cost of running six systems — outbox patterns, CDC pipelines and reconciliation jobs are all standard machinery for managing a problem you chose to have.

Start with one database. Add the second system when a measurement, not an architecture diagram, tells you to. You will be astonished how long that takes, and how much time you get back in the meantime.

For us that's a Go gateway and one Postgres: SKIP LOCKED for jobs, tsvector for search, SSE straight off the same connection pool for realtime, and jsonb in the several hundred places where the shape genuinely varies. Xenition is a full AI workspace — documents, spreadsheets, decks, boards, team chat, an app builder — and none of that has yet produced a number that says add Redis. The day one of the thresholds above trips, we'll add exactly the thing it points at. Not before, and not the other five along with it.

-- Everything above, in one line of philosophy:
BEGIN;
  -- the row, the job, the embedding, the counter, the search index
COMMIT;

DEV Community: Info Inlet

We Deleted Our Vector Database. Postgres Was Faster.

What we actually asked it, every time

The seam that was actually driving all of it

The column

The query that was the whole point

The detail nobody tells you: iterative scan

What got deleted along with the database

Ninety days later

The five objections, scored honestly

When this is the wrong call

What I would tell myself a year earlier

We Replaced 3 Microservices With One Postgres Table

What the three of them actually did

The bug that was actually driving all of it

The table

How a worker takes a job

And then the enqueue

What got deleted along with the services

Ninety days later

The five objections, scored honestly

When this is the wrong call

What I would tell myself eighteen months earlier

I Replaced 200 Lines of Code With One AI Agent — Here's What Broke

What actually happened

1. Failure stopped announcing itself

2. The cache started serving a wrong answer forever

3. Cost became a function of traffic

4. p99 stopped existing

5. The tests stopped testing anything

6. The code changed without a commit

The line I'd write on the wall

What I actually shipped in the end

The four questions I ask now

How to Stop an LLM From Returning Broken JSON — and Why Valid JSON Is the Easy Half

First: the four ways model JSON goes wrong

Step 1: Stop asking for JSON in prose

Step 2: Shrink the schema until most of it is a closed set

Step 3: Salvage the parse — once, cheaply, and never with a regex

Step 4: Parsing is not validating — coerce every field, individually

Step 5: Let the model point. You render.

Step 6: Enforce cardinality, and make a missing answer say so

Step 7: Downgrade, don't reject

Step 8: Choose your failure posture before you need it

Step 9: If the model generated logic, run it before you return it

The 60-second audit

What I'd stop saying

Maximum Update Depth Exceeded: 4 useEffect Dependency Bugs That All Passed Review

The gap the linter can't see

Bug 1: the dependency you measure

Bug 2: the dependency whose identity churns

Bug 3: the dependency that is your own echo

Bug 4: the dependency that wasn't there at all

The four questions

The 60-second audit

What I'd stop saying

40 Lines of Go That Cut Our LLM Bill by 71%

The approach that doesn't work: classify the prompt

Cheap-first with an escalation gate

The router

The four gates

The four things that broke

1. p50 improved. p95 got worse.

2. You can't stream an answer you might throw away

3. The cheap model fails the correctness test long before it fails the eye test

4. Don't show the strong model the cheap model's answer

What it did

When not to do this

The version you can ship this week

5 Reasons Your MCP Server Is Slower Than a CLI (and When to Just Delete It)

1. You pay for every tool, every turn — before the model does anything

2. A CLI composes. MCP tools don't.

3. Every byte travels through the model

4. A protocol hop that buys you nothing (on a machine with a shell)

5. The model already knows git. It has never heard of create_document_v2.

The honest part: when MCP is clearly right

The middle path (what most teams should actually do)

The 60-second audit

Why Your AI Agent Gets Dumber After 30 Turns and How to Stop It

The one-sentence version

5. The model already knows `git`. It has never heard of `create_document_v2`.

1. A job queue: `FOR UPDATE SKIP LOCKED`

2. Pub/sub: `LISTEN` / `NOTIFY`

5. Vector search: `pgvector`