You can jump right to the repo if you want; the README isn't this verbose.
Spending many, many years building products, there's one foundational skill that I've found universally useful, regardless of whether I've been an engineer or a product manager: effectively conveying context and intent. Whether it's a kickoff, hand-off, collaboration, or honestly almost any meeting I've been a part of, this is always one of the first things to come up.
There are a million different corporate terminologies for how we transmit these things: backgrounding, brain-dumps, context transfer, getting alignment, etc.
Regardless of what professional domain you work in, understanding why someone did something the way they did, why they made certain choices, why they prioritized some decisions over others...understanding "why" people do things the way they do is just fundamentally human, I guess.
I think that's probably why it's just so frustrating when our agentic machine friends:
- completely lose the thread
- prioritize the wrong thing, then overcorrect
- implement something that works, but totally miss the intent of what you were trying to do
Well, also because I end up paying for that with tokens, to add insult to injury.
It's frustrating as well when it happens with humans, sure, but with LLMs, it happens much more frequently because—surprise!—they ain't human.
Before I get deeper into the backstory, Semantic Claims is my attempt to solve that problem by colocating plain-language statements of meaningful, intentional, observable behavior with executable tests and the implementation they describe.
Shifting gears for a bit, I want to talk about some techniques and methods I've found useful across the board when building products, in no particular order:
- testing
- given/when/then
- acceptance criteria
- documentation
There's a lot of overlap between these, of course. They encapsulate and encode knowledge and intent, and provide a contextual structure to our professional existence. Without these, it's hard to build a product that remains stable. It just is, please believe me.
When it comes to our side projects, generally speaking, we become our own product managers and engineers, especially in the age of agentic AI.
I've been heads down on a passion project for many months now, building it in layers (hint: it's a full-stack framework, you know, because there simply aren't enough of them). The idea was to start with composable primitives and build up from there, to the uppermost layer. Yada yada, buzzword, jargon, the point is there are layers that compose together. Nothing special, that's often how building things goes.
I've been using AI to help me with the implementation and design review as I work. Mainly, I find AI most helpful in finding out early whether or not a direction I'm thinking about will actually work and scale the way I want. AI can prototype something a million times faster than me, and I can burn through 5 different system design prototypes a day, seeing how they'd look, scale, etc, discarding ideas and choosing the ones that I like best to implement.
As the project comes to life, and the layers come together, inevitably information gets lost, and complexity increases. It doesn't matter how meticulous I am about testing and documentation, or writing clean code. It just happens; entropy, you can't stop it, or I can't 🤷🏻♂️.
I'm sure you know the feeling—agents at the beginning are just brimming with potential, your brain is flying, you're banging through 8 life-changing designs. Then, the project matures, the surface areas grow, and the shininess and brilliance of your AI companions starts to...ebb. You start needing to refresh their context more often, control their attention more strictly, monitor the integrity of the codebase more closely.
This is a signal to me that the complexity of the system is beginning to exceed my effectiveness (or patience) in conveying context and intent to agents for a given task—and it's always right when I need it most!
I imagine this is why I've heard so many stories about AI building 99% of a product successfully and then shitting the bed in the final 1%. Sprawl and complexity have always been the bane of an engineer's existence. The goal is always to keep systems as simple to reason about as possible, and try to never exceed inherent complexity. But, life has other plans. If it didn't, we wouldn't have tech debt.
What am I talking about? Why am I splaining my life to you? Well, because I've been thinking about it a lot (uh oh...), and as a product manager I can't resist a problem (🚨 alert 🚨), and as an engineer I can't resist a methodology (🙅🏻♂️ sweet lord), and when I grow up I want to be Martin Fowler (I'm a grown-ass man).
So of course, I'm building this project, and my brain inevitably begins to spin out on how to rein these agents in. I think about how I'd manage these issues in real life at a product org, which brings me back to the prior-listed tools that I've automatically transferred with me job through job, nothing real notable, but always handy:
- testing
- given/when/then
- acceptance criteria
- documentation
I'm not a monster, of course my codebase has documentation and testing, as well as comments—do you even know what Claude would do if we didn't have load-bearing tests?? But comments and docs get stale. No complaints about tests though.
Yes, I've been a good boy and use the tools we already have today: spec-driven development, design docs, planning mode, loops, etc. They're all very helpful, but I wanted more. I wanted humans and agents entering a domain to quickly understand observable behavior, the terminology used to describe it, and tests that proved the implementation still satisfied it, connecting it all into a compact contextual unit. I know, no matter what, I'll have to constantly revisit specs and docs, but I want a source of truth to draw from, something...verifiable.
I already know colocating tests is great for this, as an engineer. One of the best ways to onboard yourself to a new codebase is by reviewing tests that are nicely colocated with implementation.
I also already know that if, at my job, I stumble into a design doc, ticket, or foreign part of the codebase that disturbs me, I can piece context together by combining all of the aforementioned to understand why something was built.
The best tickets have clear acceptance criteria, and I'm a huge fan of the expressiveness and structure of the humble Given/When/Then.
So, I started to think—could I colocate these things with my code? TDD is already an effective tool to use with agents. TDD from a clear spec is even better. Is there a way I could colocate these specs with tests and use TDD with that? In a structured manner where these micro-specs could do what a good test does, which is describe observable behavior?
Off I went on my magical side-quest, trying to think about it from a human perspective first, because I believe at some level that what's good for humans is good for agents (completely unsubstantiated of course). Also, call me old-fashioned, but at the end of the day, I'm not designing a system for agents, I'm designing a system for myself and other humans, so first and foremost, it has to work for me.
The main constraints I had were:
- colocation
- drift prevention
- high visibility
- structured
- concise
My goal was providing surgical context of any given part in the system. I didn't know what to call these...parts, because I wanted different granularities—anything that could be described by observable behavior. That could be an entire system, a component, or even a tiny shared helper. So, I landed on calling these pieces subjects. Being able to concisely describe the meaningful observable behavior of a given subject; creating a colocated semantic contract that was strong enough to write tests from, and then an implementation to follow (TDD FTW).
So, I started to do this—writing small, structured Markdown files with claims of the observable behavior that gives a subject its meaning. I put those next to a test that proved the contract, which sat next to the implementation being proved. After writing up a design for a part of the system and breaking it down into its respective subjects, the workflow became:
claim -> prove -> implement
Hey, it worked! Soon, my agents and I were groovin' (?) through claims, proofs, and implementations, writing legible, meaningful tests, and creating implementations to match, with filesystems that looked like this:
search/
├── search-results.scenarios.md
├── search-results.scenarios.test.ts
└── search-results.ts
The Markdown would look as simple as this:
# Search results
## §1 Search precedence
### §1.1 Newer searches supersede older results
**Given** an older search is in progress,
**When** a newer search begins and the older search later completes,
**Then** the older result doesn't replace the latest result.
But quickly I realized that there were really two kinds of behaviors. Some described behavior that must remain true whenever certain conditions apply. Others described outcomes whose meaning depended on events occurring in a particular order.
I called the first kind invariants and the second scenarios. The rule of thumb I used is, if changing the order of the relevant events could change the expected result, I would use a scenario; otherwise, I used an invariant.
This was when things started to get interesting. I could write out these claim documents to specify the meaningful, intentional behavior of something, and have an agent review it, challenging me to reduce ambiguity. I could write a spec, and have an agent help me decompose it into coherent subjects with clear semantics. I could have agents write proofs, see how the APIs looked, and discard or iterate until we found shapes I liked, and finally, either I or an agent could write the final implementation.
Soon, I vibe-coded a little checker that verifies the structural links between claim documents and their JS/TS proof files (filenames, identifiers, titles, test structure). An agent couldn’t quietly rename or drop a claim in a proof without the checker catching it. Though of course, human review still needed to determine whether the test actually proved the claimed behavior.
I also quickly realized that I could really scale back on the context I needed to provide upfront to agents for a given task. They're hungry for localized context, so they would quickly locate these semantic contracts and eat them up like little robot candies 🍭.
But I also realized there were some blind spots. There were times when an agent would review a part of the system and want to write a claim document that couldn't be colocated with a single subject because it was really about semantics that lived between multiple subjects. They kept bothering me about this until I said, fine, we can have cross-cutting claims. These files got prefixed with -- and sat at the boundary of where these interactions occurred—so at the closest common directory where all involved subjects lived.
The unexpected benefit was that the agents began to use these cross-cutting claims as part of their context gathering. The difference between localized and system-level reasoning has always resonated with me. It's not that agents don't care about system-level reasoning, it's that they don't always have a clear direction through it, or don't always see the need if a task appears straightforward. When they do explore systems for system-level reasoning, it can be contextually (and financially) expensive if they do it wrong or for too long. These shared claims stood out as bright flags to an agent ls-ing through directories. So rather than trying to intelligently ingest the important contents of code as they traversed a system, it would check locally first, then jump up and review related shared or system-level claims to understand interaction semantics, so on and so forth.
Once the relevant behavior had become accepted claims with executable proofs, I was able to retire most temporary planning documents. Though I still kept high-quality documents around that recorded rationale—Semantic Claims doesn’t replace that.
From a human perspective, I really like this method, which I'm calling Semantic Claims. The reason I like it is because I, also, forget things about my codebase and my product. I don't feel ashamed about it! I'm only human, and I can only hold so many things in my brain at once. This also lets me examine the semantics of my system in isolation, which is a really interesting exercise. When I'm trying to clean off the rust around a part of the codebase I haven't seen in a bit, the claims and proofs are the first things I jump to review.
I was talking to another engineer recently at a conference; he said it sounded interesting, so I put it up in a repo, where I threw in the skills, the JS/TS checker (I'll probably write one for Python too), and even a handy dandy little explorer dashboard, so I could explore the claims and subjects across my codebase:
I'm sure tomorrow when the newest model comes out with infinite context we'll look back and think "wow, remember when we needed to do this", but for now, I've found it to be a real asset to my workflow—I may even keep it around after the agents don't need it because...I might!
Check it out here, and cheers 🥂
Top comments (0)