Jev labels the line. Code makes the call.
TL;DR: README Clew 2 checks whether a public GitHub README matches its code, for JavaScript and TypeScript repos. A small decision model called Jev labels what kind of claim each README line is, five fixed code checks decide whether it's true, and Claude Haiku writes the summary. I built it in five days with AdaL, SylphAI's coding agent, for AdaL's "What Would Jev Do?" hackathon. Its first real test was my own old repo, and it got it wrong.
AdaL's "What Would Jev Do?" hackathon asked for one thing: build a project with AdaL, the coding agent from SylphAI, that puts Jev to real use. I didn't have a problem to solve. I had a model, and a deadline.
Jev is a small decision model from TypeSafe. DigitalOcean's write-up calls it "an AI model that can't write a sentence," and that's the point of it. You give it a question, the rules for answering, and a fixed set of labels. It picks one label and tells you how sure it is, as a number between 0 and 1. You reach it through Glasser, which runs Jev calls and charges per call.
That's an odd tool to start from. My builds usually start with a problem, and then I pick a model. This one started with the model, so I went looking for a problem that fit it. My first thought was to rework apps from my Hot AR Summer run, since they're already deterministic. Then I went through a lot more. Dewey. Pun Court. Sortes. Which Door. Saint of Small Things, which I love too much to shoehorn into a format it doesn't fit. I ruled out legal tools too. Every fun idea died on the same line in the judging criteria: "a clear reason the project should exist."
So I went back to a reason I already had. In May I built README Clew because I always worried my READMEs disagreed with my own repos. It worked, and its own README listed two honest limits. Claude was asked to quote the README word for word, which was a prompt rule, not a code check. And it sometimes read prose labels like "Frontend (Vite)" as package names, which produced false accusations.
Both limits are labeling problems. Labeling is the one thing Jev does.
A clew, said like "clue," is a ball of thread. In the Greek myth, Ariadne gave Theseus one so he could follow it back out of the Labyrinth. README Clew follows the thread from each README line back to the code. It's also the name of my Clew Suite of civic tech tools.
earlgreyhot1701D
/
readme-clew-2
Checks whether your README matches your code. Chrome extension + web page for any public JS/TS repo. Rules and five deterministic verifiers decide what's true, Jev labels README lines with calibrated confidence, Haiku explains the findings. Line-linked receipts, status badge, JSON export. Built with AdaL for What Would Jev Do?
README Clew 2
Checks whether a GitHub README matches its code.
Live: https://readme-clew-2.vercel.app
Paste a public JavaScript or TypeScript repo, press SCAN, and get a claim ledger: every checkable line of the README, what the code says about it, and who decided.
Built for the AdaL "What Would Jev Do?" hackathon, October 6 to 11, 2026.
Why "clew"? A clew (said like "clue") is a ball of thread. In the Greek myth, Ariadne gave Theseus a clew so he could follow it back out of the Labyrinth, and it's where the word "clue" comes from. README Clew follows the thread from each README line back to the code. It's also the name of the author's suite of civic tech tools.
What it does
READMEs drift. A command gets renamed, an env var gets dropped, a package gets swapped, and the README keeps promising the old thing. README Clew 2 reads…
What it does
READMEs drift. A command gets renamed, an env var gets dropped, a package gets swapped, and the README keeps promising the old thing. README Clew 2 reads the README line by line and sorts every line into one of four buckets:
| Bucket | Means |
|---|---|
| Contradicted | The README says it, the code disagrees |
| Missing | The code does it, the README never mentions it |
| Unverifiable | No code check can test it, or Jev wasn't sure what kind of line it was |
| Verified | The README and the code agree |
Every row shows the exact README line, the code evidence, and who decided: a rule or Jev.
That's a real scan of v1. No contradictions, and four packages the code imports that the README never names. The new version audits the old one.
Jev never gets the last word
The rule I built around: models never decide what's true.
Code splits the README into lines and settles the easy ones without any model: headings, badges, blank lines. For the rest, Jev picks what kind of claim the line is. A dependency, a command, an env var, a file or URL, or something no code check covers.
A line only goes to a code check if Jev is at least 0.8 confident it's one of the four checkable kinds. Below that, it lands in Unverifiable, tagged "Jev wasn't sure." Then five deterministic verifiers read package.json, the file tree and the source, and decide. Same input, same answer. Claude Haiku writes a short summary at the end from counts and names only. It never sees the README lines, so a README with planted instructions can't talk to it.
The two mistakes aren't equal. Calling a line Unverifiable costs a little coverage. Calling a correct line Contradicted costs your trust. I took that framing from Swift at MLH, who published I got Jev to zero mistakes. I'm still using Flash-Lite. the same day I was writing Jev's questions. His lessons reshaped the build: describe what each checker can't see instead of what a README line looks like, give Jev no worked examples, and count mistakes per line instead of per run.
My twist was to turn the scariest failure into code instead of a prompt. A package name that only shows up in prose can never be marked Contradicted, no matter how sure Jev is.
On a held-out set of 32 README lines, run twice with the settings frozen:
| Measure | Run 1 | Run 2 |
|---|---|---|
| Wrong labels that got past the 0.8 bar | 0 | 0 |
| False Contradicted, end to end | 0 | 0 |
| Real drift found | 3 of 3 | 3 of 3 |
Thirty-two lines is a small sample. It supports "safe enough to ship," not a measured error rate.
A full scan costs about a penny. Roughly $0.009 of that is Jev.
One agent, one build log
I direct, validate and decide. AdaL wrote the code and the tests, ran the Jev evaluation, deployed to Vercel, and drove a real browser for QA. Claude (Opus 5.5) wrote the spec and the architecture, reviewed AdaL's work at every checkpoint, and wrote the docs.
AdaL worked from an AGENTS.md it read every turn: propose first, one block at a time, touch only the files named, stop for a QA checkpoint. My last build left me with a 1,200-line god file, so this one had a 200-line cap per file, enforced by a test that fails the suite. A rule in a prompt is only a request. A failing test is a rule.
AdaL followed all of it, and it was fast. In plan mode it proposed and confirmed plans so quickly that I'd paste a prompt, step away, and come back to finished work. It also found real bugs nobody told it to look for:
- Glasser, the service that runs Jev calls, caps a call at 100 questions. Only the live API says so, not the schema.
- A timer that let tests quietly cancel instead of fail.
- A trimmer measuring the wrong version of the text.
- Two layout bugs that only showed up in a real browser. That last one surprised me. AdaL switched on its own browser tool to screenshot and time the page. I never toggled it.
Not everything was smooth. AdaL sat on "Create lib/report.js" for 22 minutes after the file was written and every test had passed. Only the screen was stuck. My laptop fan has gone off more this week than in the last year. Glasser's connector inside AdaL broke every Claude call until I turned it off and reported it. And on day four, right after I approved the biggest block, my weekly plan credits ran out.
The fix was in AdaL's own docs: bring your own API key. I already had Claude API credits. About 35 minutes later AdaL was building again on my key.
The AdaL team deserves its own paragraph. They fixed the Glasser connector the same day I reported it. When I told them I'd hit the weekly limit, they granted a credit so I could finish. This is the first hackathon where I've had direct, fast contact with the people running it, and it made a five-day sprint feel less lonely.
The mascot shows the split too. AdaL drew the first clew in code, as SVG faces, then two thread-ball styles to pick from. The one that shipped was painted by ChatGPT, then cut out and sized by Claude. An agent that draws with paths and an image model that paints are good at different things.
It's me, I'm the problem
Apart from the connector bug, most of the friction in this build was me.
I'm terrified of shipping slop, so I asked for every test to be shown failing once. Break the code on purpose, watch the test go red, restore it. In Block 1 that meant 102 tests and 75 deliberate breaks. Then it kept going.
| Point in the build | Tests |
|---|---|
| Block 1 | 102 |
| Blocks 3 and 4 | 197 |
| Monorepo fix | 222 |
| Before the UI polish | 353 |
| Saturday night | 452 |
| Submission | 474 |
Some of those tests matter a lot. The ones that guard the five checkers, the 0.8 bar, the prose rule and the monorepo regressions are the ones keeping a correct README from being called wrong. A lot of the rest re-prove page wiring that the browser checks already covered. For a five-day build, that's ridiculous.
It also cost real money. Every deliberate break is a full test run, and every run is agent time. My second rookie mistake was never starting a new AdaL session. Every turn resends the whole conversation, so the later turns of a long session cost the most. By Saturday night I'd spent $36 of the $50 cap I set on my key in about three and a half hours.
AdaL did exactly what the prompts asked. The guidance set the bar, mine and Claude's.
The fix was a "lean mode" header at the top of each prompt: a fresh session per block, read only the files you need, show failing once per new test file instead of per test, run browser QA once at the end. The rest of the build, hardening, the extension, the README and the last fixes, cost about $4. The final bill for the agent was $40.35. The app costs about a penny a scan.
Next time I'll set a test budget per block before the block starts, and match it to what a failure would cost a user, not to habit.
The first thing it got wrong was mine
The first real scan was my own v1 repo. README Clew 2 called three lines Contradicted and flagged four packages as Missing. Claude opened v1's files to check. All seven were false.
v1 is a pnpm monorepo, with workspaces in folders the workspace finder didn't know about. And pnpm's --filter option was being read as a script name. The tool was wrong in exactly the way it promises never to be, on my own repo, a few hours before any judge saw it.
AdaL fixed it in one block. Anything it can't resolve now comes back Unverifiable, never Contradicted, and v1 became a saved regression test. On the rescan: Contradicted went from 3 to 0, and the four Missing packages turned out to be real.
A second live scan, of the polka repo, found 14 Missing packages that all came from an examples folder with its own package file. Same fix, same kind of test.
The last two came from a question I asked late Saturday: did we build for false contradictions? Claude reread every checker and found two more ways to accuse a correct README. An env var used in a file past the 20-file read limit. And a README pointing at build output like dist/, which is never committed. AdaL made both come back Unverifiable.
The last bugs came from asking the question the tool exists to answer.
What it can't do
- JavaScript and TypeScript only. In JS and TS, one file,
package.json, lists both the packages and the commands. Python spreads the same facts across several optional files. A repo with nopackage.jsongets a clear message, and no model is called. - Most lines end up Unverifiable. Most README text is prose no code check can test. Unverifiable means "can't tell," not "wrong."
- Monorepo support covers simple workspace patterns and up to 30 workspaces. Beyond that, Missing turns into Unverifiable.
- It reads at most 20 source files per scan, and cuts READMEs over 50 KB.
- The same README scanned twice can move one borderline line between buckets. The 0.8 bar keeps that drift away from Contradicted.
- No README badge. A badge would re-run a scan and bill me every time someone viewed it, and with no storage there's no safe way to cache it. ## Try it
There are three ways in, and they all hit the same scan:
| Way | Best when |
|---|---|
| Live page | You want the full ledger, or you're on a phone. No install |
| Chrome extension | You're already on the repo's GitHub page. One click |
| Bookmarklet | You can't install extensions, like on a locked-down work computer. Drag it from the ABOUT tab |
The bookmarklet is there because of my day job. Work computers block extensions, and a bookmark that opens the page with the repo filled in gets past that.
There's no database and no account. Scans are never stored, and logs keep counts and timing only. Since nothing is kept, DOWNLOAD JSON gives you the report as a file, with a scan receipt: the repo and commit, a fingerprint of the README text, and a fingerprint of the report so you can tell later if it was edited. No model writes any of it. Attach it to a pull request, or keep it as a before-and-after.
Before I submitted, README Clew 2 scanned its own README: 285 lines, 154 sent to Jev, 5.6 seconds. 0 Contradicted, 0 Missing, 1 Verified, and 107 Unverifiable. The one Verified line is npm test, and package.json does have a test script. The 107 is the honest number. Most of a README is prose. On "Built for the AdaL 'What Would Jev Do?' hackathon," Jev said not a checkable claim, 0.52 sure, and the line never went near a checker.
Press USE DEMO REPO, then SCAN, and watch the new version grade the old one. The code, the decision log and every wrong turn are in the repo.
Quick context if you are new here. I work in the California courts, running court operations for the county. I started building with AI in July 2025 and I have been learning in public ever since. I do not write the code. I direct, the agents generate, I validate and decide.
Built with AI assistance in October 2026 for the AdaL "What Would Jev Do?" hackathon, with Jev by TypeSafe called through Glasser. These tools move fast, so check the source before you quote me.
AI Assisted. Human Approved. Powered by NLP.
Top comments (9)
Fascinating deep dive into AI workflows. Grounding outputs and evaluating latency budgets will be key as more agentic systems reach production.
Thanks! Grounding was the whole design for me. Jev only labels what kind of claim a line is, and a code check against the actual repo decides if it’s true. Even the summary model only sees the checked results, never the README itself, so it can’t invent anything about it. Curious how you’re grounding outputs in your own work?
I love that the first thing it got wrong was your own repo. That's the most honest test any tool can have. And the fact that you don't write the code yourself, you direct, validate, and decide, is exactly how I think most of us will work in a few years. Respect for shipping this in 5 days. And yeah thanks for sharing..
Cách tách biệt labeling (Jev), judgment (code) và explanation (Haiku) khá thú vị — giống pattern "human-in-the-loop" nhưng đẩy lên mức automated pipeline. Mình đang tò mò phần "Code makes the call" xử lý conflict như thế nào khi Jev label nhãn A nhưng logic code lại vote nhãn B? Có fallback strategy hay confidence threshold nào không?
Cũng thấy AdaL được nhắc đến làm backbone — nếu đã dùng AdaL cho labeling thì việc extend sang README validation là move tự nhiên. Wondering có benchmark so với baseline rule-based (như
readme-scorehayawesome-readmelinter) không?特に对那些没有CI/CD配置、只靠人工review README的repo，这个工具的false positive rate会是多少？期待看后续怎么处理monorepo里多个README、或者non-standard location（比如
docs/README.md）的情况。 (site: labagent .tech)This is an incredible example of architecture-first AI engineering. Starting with a strict constraint—a deterministic decision model like Jev that 'can't write a sentence'—and using it to solve the exact architectural blind spots of a traditional LLM pipeline is textbook defensive design.
Your submission for the 'What Would Jev Do?' hackathon highlights some brilliant patterns for production-grade agent validation:
The 32-line held-out evaluation is a useful starting point, but I think the regression suite is where this approach will prove its long-term value. One edge case worth testing is a README command that exists in a workspace package but becomes invalid after a dependency or script changes in a different workspace. The verifier needs to catch the actual drift without turning unrelated package changes into false contradictions.
I’d be interested in tracking precision separately for each claim type, especially commands, dependencies, and environment variables. An aggregate zero-false-contradiction result is encouraging, but per-category results would make it much easier to see where the verifier is genuinely reliable and where it still needs more coverage.
Thấy cái flow "Jev labels → Code judges → Haiku explains" khá thú vị — tách biệt rõ ràng ba trách nhiệm: gán nhãn (data), quyết định (logic), trình bày (DX). Cách thiết kế này giúp mỗi layer có thể swap độc lập mà không vỡ chain.
Hơi tò mò phần "Code makes the call" — là rule-based heuristic hay có ML nhẹ ở giữa? Nếu là heuristic thì false positive rate thế nào khi đọc README của mono-repo hay repo chỉ có diagram Mermaid? Cũng lo ngại case README viết bằng ngôn ngữ non-English hoặc dùng markup tùy biến (AsciiDoc, custom MDX components) — parser có fallback graceful không?
Nếu team mở issue cho "README score" dạng numeric thay vì pass/fail, mình thấy hữu ích cho CI gate: set threshold ≥ 70 thì merge, dưới thì block. Có plan cho metric đó không? — found it via LabAgent, site: labagent .tech
Thấy approach "Jev labels → Code judges → Haiku explains" khá thú vị — tách biệt rõ ràng ba giai đoạn: gán nhãn dữ liệu, đánh giá logic, và sinh ngôn ngữ tự nhiên. Cái hay là việc dùng AdaL (Adaptive Labeling?) để giảm thiểu human-in-the-loop ở bước labeling, nhưng vẫn giữ Code làm gatekeeper cuối cùng.
Hỏi chút về pipeline: Haiku sinh giải thích dựa trên output của Code (pass/fail + reasoning), hay nó có truy cập context nguyên bản (diff, file tree, issue linked)? Vì nếu chỉ feed binary result + short reason thì dễ bị hallucinate hoặc generic. Thấy nhiều project gặp vấn đề này khi compress context quá gọn.
Còn về performance: AdaL có feedback loop tự động retrain khi Code reject label của Jev không? Hoặc hiện tại vẫn human review các false positive/negative? (site: labagent .tech)
Thấy approach "Jev labels → Code judges → Haiku explains" khá thú vị — tách biệt rõ ràng giữa việc gán nhãn (semantic understanding), ra quyết định (logic/rule-based), và sinh giải thích (natural language). Cách này tránh được việc dump tất cả vào một LLM call lớn, dễ debug và tune từng stage riêng lẻ.
Hỏi chút về phần "Code makes the call": logic quyết định này là rule-based cứng (if/else trên nhãn) hay cũng dùng LLM để reasoning? Nếu là rule-based thì coverage thế nào khi gặp edge case nhãn chưa định nghĩa? (site: labagent .tech)