This is a submission for Weekend Challenge: Passion Edition
What I Built
I told an agent Never write directly to the database. A lo...
For further actions, you may consider blocking this person and/or reporting abuse
Oooooh that's cool! Love the visuals. I've recently been doing a game series teaching people about how LLMs work and have been pondering on how to best do 'forgetfulness', LLMs are demented was great for the basics, but instruction decay is one I havent really gotten around to. Was thinking something like a 'down the line' decay game, but I think that would be kinda boring, if you have any suggestions, I'm all ears!
given the core idea, I'd say a "context water level rising" with some "how to keep your player above the water" might work here. After all, the LLM do not forget anything from the context, they are just attenuating less and less, but there are some things you can do. It can also demonstrate the recency law (latest instructions are the strongest), and golden instruction construction (which also extremely useful for high context pressure management).
You can find more rules on reporails.com/rules page.
Hm. Maybe like an hour-glass, that fills as the model does work. User gives an instructions, then see how long the LLM runs (fills the hour glass), until it hallucinates the instruction?
that can work too
actually it might be work in a way like a building game? They can build instruction constructs but if they give weak instructions then what they build will fall apart?
I think the learning of these principles can be gamified in many different ways. Just bare in mind that the context creates the push, weak constructs creates the stakes.
Hm... I'm thinking now, how about 1 of those skyscraper building games, where you drop the blocks? That should be a good way to show how a core-instruction builds a foundation, but as you build it's lost, yet ever present and if you drift from the core instruction, or give a bad instruction, it causes context collapse that makes the building topple. If you're aligned with your core instruction, it persists longer, because it progressively re-enforces it, but if you drift, it degrades the stability. Oooh I think that's it, keep an eye open for it tomorrow!
that's even better! It representing nicely the additive nature of the context too!
Do share once you have something to show, looking forward to it
The “rule you cannot afford to lose does not belong in prose” line is the important one.
A lot of teams treat instruction quality as if the only problem is wording. Better wording helps, but it still leaves the rule inside the model’s attention budget. Under long sessions, tool output, retrieval, and competing objectives, even a well-written instruction can become background noise.
The practical split I like is:
“Never write directly to the database” is a perfect example. That should be enforced by the tool surface, credentials, or hook layer before the model ever gets a chance to forget it.
The demo makes the failure visible in a way a checklist usually does not.
Your split is pretty much where I landed too, except I got there the expensive way: by watching a perfectly reasonable "never write to prod" rule sit politely in a CLAUDE.md while the agent, twelve tool-calls deep and thoroughly distracted, went ahead and did the thing anyway.
That's the part "just write better instructions" misses. I spent months making the wording better. It does help. But a well-worded rule and a sloppy one are still both text competing for the same attention budget, and the budget runs out. The rule doesn't break because it was phrased wrong. It breaks because by turn 40 nobody's really reading it anymore, the model included.
Your "observable, not inferred" line is the one I keep coming back to. A model-graded "did you follow the rules?" check hands the question back to the exact component you didn't trust to begin with. A hook at the write boundary doesn't ask for an opinion. It sees the write, says no, and the exit code is the receipt.
And yeah, credentials go even harder than a hook. A hook is still code, so it can have the bug that lets things through. A database role with no write grant can't be talked into anything. When you don't need the capability, taking it away beats guarding it.
Exactly: prohibitions with real side effects should not rely on attention at all. I would pair the runtime hook with a test trace that proves the denial happens after context growth and tool retries; otherwise a policy can be technically present but not enforced at the mutation boundary.
Agree, either a test trace or something else that is outside the harness hook level. For example I have a loop (and hook) for git commit messages, but I have an extra layer outside of the harness - git precommit hooks.