Stakenshake

Posted on Oct 22

A 30-day-old note was obeyed exactly as hard as a 7-year-old one

#ai #programming #opensource #tools

I put a decision in a repository: "do not use TypeScript, the team does not
know it." The date on it was 2019.

Then I asked two model families to choose a stack for a brand-new 2026 service.

Every single response refused TypeScript.

Not because the model was being careful. Because the repository said so, and
the model had no way to tell whether that reason still applied. One response
wrote it out:

"There is no information indicating that the team's capabilities have
changed, so this constraint is retained."

It knew it should check. It had nothing to check against.

The obvious follow-up: change only the date

Same decision text, same task, same models. The only thing that changed was
created.

Age of the note	Recommended TypeScript
no note	12/12
30 days	0/12
1 year	0/12
7 years	0/12
7 years, expired and hidden	12/12

A one-month-old note was obeyed exactly as hard as a seven-year-old one. The
model does not discount age at all — not by a little, by nothing.

A later run at n=20 across three model families put the effect at 78–87
percentage points (Fisher p < 0.0001). The direction was not a small-sample
artefact.

The fix that didn't work

My first instinct was to tell the model the note was old. I added a line to the
prompt: this entry is old, treat it as an assumption to confirm, not a binding
constraint.

It barely moved. At n=3 one model went from 0/3 to 1/3 and the other went the
other way. That is not a safeguard; it is noise. I'm reporting it because the
failed attempt is the reason the design changed.

An agent cannot be trusted to weigh an age it cannot perceive. Asking it to
be skeptical of "old" notes asks it to weigh a variable it does not have.

What did work

Make the expired entry invisible. Not annotated. Not warned about. Absent.

Same note, same task, hidden after expiry: 12/12 recommended TypeScript again.

That is the whole mechanism. Expiry is structural, not advisory.

Two things worth pushing back on

One in five responses did resist the stale note. When they resisted, they
did it by attacking the stated reason — "the team probably knows TypeScript
now" — not by noticing the date.

I ran the same experiment with two different reasons:

Reason in the note	Reads as	Resistance
"the team did not know TypeScript"	a condition that can change	13/60
"this project's fields change constantly"	a property of the project	0/60

So the reason you write matters more than the date. A reason that names a
condition invites the model to re-check it. A reason that states a property
does not.

And the effect is not universal. With real code in front of it, one model
inferred the answer from the repository and ignored the note entirely; another
still preferred the rejected option until the note stopped it.

That is the strongest argument for what this tool is actually for: code
encodes what you chose. Nothing in a repository encodes what you rejected.

What I built

whynot is a folder of Markdown files in your repository that records what you
ruled out and why — especially rejected entries, which are the only kind of
knowledge that cannot be recovered from the code. Every entry expires. When it
lapses, it stops reaching your agent until a human renews it.

---
id: 0001
title: Sessions do not go in Redis
type: rejected
status: live
created: 2026-09-21
expires_on: 2026-12-20
revisit_when: "Redis supports cross-region strong consistency"
evidence: ["pr:412"]
tags: [infra, session]
---

## Conclusion
Sessions must not be stored in Redis.

## Why
We tried Redis with a single writer region. During a us-east outage, reads took
8 seconds; the SLO requires p99 < 500ms.

Install:

npx whynot-cli init

Then register the MCP server with your agent and paste the instruction block
into AGENTS.md or CLAUDE.md.

The design choices follow from the finding

Markdown in git, because if the tool disappears you should still be able to grep your own decisions.
No vector store, because nobody can audit why an embedding put something into the context.
No model call on the write path, because that is where corrupted memories come from.
Expiry by default, because permanence should be a decision, not an accident.

It is MIT: no account, no telemetry, no paid tier, no closed binary.

Check it before you install it

npx whynot-cli audit is read-only. It scans your existing AGENTS.md,
CLAUDE.md, and .cursor/rules for absolute rules that state no condition —
the ones that will be obeyed forever.

Scanned 2 agent config files:
  AGENTS.md  (last changed 412d ago)
  CLAUDE.md  (last changed 8d ago)

3 rules look like permanent constraints:

  AGENTS.md:3
    "Never use TypeScript in this repo."
    → absolute rule with no stated condition; unchanged for 412 days

If it finds nothing, that is useful information too.

What I have not shown

Every experiment here is a controlled, single-turn task. I have not measured
whether this reduces the number of times an agent re-proposes something you
already rejected, over weeks of real work. That is the number that ultimately
matters, and I don't have it.

What I have is a mechanism that works under test, and the scripts to check it.
If you use this for a month and it doesn't help, I'd like to hear that.

DEV Community