A spec is only ground truth if the agent can grep it

Most of my repos have an AGENTS.md. It tells the agent how the project works, what not to touch, and why.

On any given turn, I could not tell you whether the agent read it.

That is the quiet problem with "spec-driven" setups. The spec is usually one big markdown file, and the agent may or may not open it. JetBrains has a project out with a different bet, and it is the most interesting part of a launch most people are arguing about for other reasons.

What ThinkRail is

ThinkRail is an agentic IDE from a JetBrains incubator, Apache-2.0, written in TypeScript. It wraps the pi coding agent (version 1.4.0, pinned in the root package.json) in a desktop or browser app. Each workspace is a git worktree, you can run several pi sessions per workspace, and there is a Monaco editor, diffs and review comments around them.

The repo was created on July 8 and had 4,038 stars when I checked on Tuesday evening. It was on GitHub trending with 1,204 stars in a day, and the HN thread "JetBrains Thinkrail" had 239 points and 174 comments. The latest stable release is v0.2.0 from October 18; nightlies have reached 0.3.0-nightly.7.

Worktrees and tabs are not new. The README calls the other part "living specs", and that is the part I read closely.

A spec with an address

In ThinkRail, a spec is a SPEC.md file that sits beside the code it describes. Every one starts with a small YAML header. Here is the real header of the module that implements the whole feature, packages/spec-graph/SPEC.md:

---
id: module-spec-graph
type: module-design
status: active
title: Spec-Graph pi extension
parent: architecture
depends-on: []
tags: [spec-graph, pi-extension]
---

The id is the address. parent points one level up, depends-on points sideways. The types run from goal-and-requirements at the root down through architecture-design, module-design and submodule-design, plus a temporary task-spec. Maciej Gorywoda's post on the ThinkRail blog lays out the same header and suggests starting with nothing more than that and a paragraph in AGENTS.md.

The agent gets seven tools for this graph: spec_grep, spec_get, spec_graph, spec_create, spec_update, spec_delete and spec_validate. spec_get returns a node's header, its resolved links and its path, but not the body; the agent reads the body with the normal read tool. spec_graph returns a bounded slice: a subtree, the ancestors, or the neighbors of one node.

There is no database behind it. The spec says the filesystem is the source of truth. Each read re-scans the tree, re-parses only the files whose modification time or size changed, and skips folders like .claude and .superpowers, where other assistants leave plans with spec-shaped headers that are not this project's specs.

Lookup beats loading

Here is the claim I want to argue. A spec is only ground truth if the agent can look it up the same way it looks up code. A README is something you hope gets loaded. A graph with ids and a grep tool is something the agent can query.

The difference shows up as soon as a project gets big. ThinkRail's own spec graph is about 290,000 words. Nobody wants that in every prompt. But the agent never needs all of it. It needs the spec next to the file it is editing, that spec's parent, and whatever the parent says it depends on. With ids and spec_graph, that is one or two calls, and the rest stays out of context.

That is also how it gets the rule in front of the agent. The extension hooks pi's before_agent_start event and adds a short named section to the system prompt on every run:

pi.on("before_agent_start", (event) => {
  event.systemPromptOptions.sections[SPEC_SECTION] = SPEC_RULE;
});

The rule opens with "Specs are this project's ground truth." What surprised me is the next line. It does not say "read the specs first." It says to consult the relevant spec when the work touches a documented boundary, contract or decision, and that localized work need not read unrelated specs. The repo's own AGENTS.md says the same under the heading "Read context proportionally." The blog post's sample AGENTS.md paragraph is stricter and says to read specs before code. The shipped tool picked cheap lookup over a forced read.

It uses it on itself

The best evidence for the idea is that the ThinkRail repo is built this way. I counted the files ending in SPEC.md through the GitHub API on Tuesday at 6:28 PM PT: 105, plus the two root docs, goal-and-requirements.md and architecture.md.

I downloaded all 107 and ran a short Python script over their headers, a rough version of what spec_validate checks. These are my script's numbers, laid out by hand:

specs with id + type        107 of 107
duplicate ids               0
dangling parent links       0
depends-on links            124, 0 dangling
types                       75 submodule, 30 module, 1 architecture, 1 goal
status                      93 active, 14 draft
deepest parent chain        5 links

A clean graph is one thing. Is it current? For each of the 89 specs with no other spec below them, I asked GitHub for the spec's last commit and then for every later commit that touched its folder. For 64 of them, nothing in the folder has changed since the spec did. 24 had one to four later commits, and one, apps/web/scripts, had ten. A commit count is a rough proxy (a dependency bump may need no spec change, and a fresh spec can still be wrong), but it says the team edits specs in the same change as the code, most of the time.

Where it bends

The graph only keeps context small if the nodes stay small. The project's own writing-specs skill says a spec should be readable in one sitting. Its median spec is about 10 KB. Its largest, apps/web/src/panels/SPEC.md, is 212 KB and 30,640 words under three headings, and 176 of the 593 commits on main have touched it. Seven specs are over 50 KB, which matters because pi's read tool returns at most 2,000 lines or 50 KB per call (I checked that in pi's published read.js, not in a session). So "the nearest spec is one call away" is true for most of the graph and false for the parts that change most.

The tools also stop at the header. spec_update edits frontmatter only, and the prose is still written with the agent's ordinary write and edit. Nothing checks that the prose still matches the code.

And the HN pushback is fair. One commenter on HN argued that agents don't really read anything: a spec gets pulled into context at random and dropped again at random. Another dismissed the specs themselves as machine-written filler. I can't tell who wrote them. A rule in the system prompt makes reading likely, not certain, and ThinkRail's own rule tells the agent it can skip specs for local work. That is a sensible trade, but it is a trade.

Try it today

The README's install for macOS and Linux is one line, curl -fsSL https://raw.githubusercontent.com/JetBrains/thinkrail/main/install.sh | bash, then thinkrail ~/code/my-repo. There is also a DMG on the releases page. I did not install it. The README says you need git and a model provider, that the macOS CLI is signed but not yet notarized, and that basic usage events and error reports are always sent. Once a repo is open, the Specs tool on the right rail shows the graph as a tree.

You don't need the IDE to try the idea, though. Give each module a SPEC.md with an id, a type, a title and a parent, and add a paragraph to AGENTS.md saying where the specs live and when to read them. Plain grep -r "^id:" gets you a crude index today.

So here is what I want to know. How do you keep the agent reading the spec after the first session: a rule in every prompt, a tool it has to call, or a check that fails when the spec and the code disagree?