DEV Community: Ibn Web3

Whylog: three Claude Code agents that say "This was not recorded" instead of guessing why

Ibn Web3 — Sat, 07 Nov 2026 13:28:14 +0000

Your team's code outlives the people who wrote it. The reasons behind it don't. This is how I built three Claude Code agents on Switch that record those reasons while a decision is being made, and that say "This was not recorded" instead of guessing.

The short version

Whylog is three Claude Code agents in one Switch room: a coder, a scribe and a recall agent.
The scribe writes each decision into a room document (DR-1, DR-2...) and marks anything nobody said as "not stated".
The recall agent answers "why did we do it this way?" only from those records and the linked PRs. If nothing covers the question, it says so.

Everything here ran on one Windows machine with Switch Console 0.40.1 and its bundled Mattermost. One person played the engineer. The people and the reasons in the demo are made up.

The problem

Open any old pull request. "Add 5s timeout." "Retry failed deliveries with backoff." The diff shows what changed. It never shows why 5 seconds, why 3 retries, what was rejected, or what would make the team change its mind.

When the person who knew leaves, that reasoning leaves with them. A new hire can ask a chat model, and it will give a confident, plausible reason that nobody ever said.

The record decides. The model doesn't.

That is the one rule Whylog follows. The agents may ask, write and quote. They may not invent a reason.

What I built

Three agents, created from one Switch template, in one private room called whylog-engineering:

whylog-coder works with an engineer on code. Before it picks an option it says the options and trade-offs out loud in the room.
whylog-scribe takes the scribe role. It asks for what is missing, one question at a time: what was chosen, what was rejected and why, who decided, what would make us revisit it. It writes only what a person said.
whylog-recall takes the recall role. It answers only from the decision documents and the GitHub reference, quotes the record, names who said it, and links the PR.

The test subject

I made a tiny webhook sender, whylog-demo, with two merged PRs: #1 adds a 5 second timeout, #2 adds up to 3 retries with backoff. Neither PR description says why. That gap is the whole point.

How Switch is used

Every part of Whylog is a Switch feature.

Switch feature	What Whylog does with it
Agents	Three Claude Code agents on the local machine
Room	One private room where a person and all three agents talk
Roles	`scribe` and `recall`, both exclusive; the agents claim them in the room
Room documents	One document per decision (`DR-1 ...`), following a "How we record decisions" document
References	A GitHub reference to the demo repo, so records link to real PRs
Room instructions	The briefing every agent in the room reads
`!` commands	`!roles`, `!list-documents`, `!list-references`

The room documents carry the idea. A person can open DR-1, read it and edit it. The record does not live only in chat scrollback.

What happened in the demo

Options first. I asked the coder what options we had on each PR. It listed options and trade-offs and gave a pick for each. It also said it had not read the PR diffs, so it described the general option space, not our code.
The scribe records. I told the scribe to take its role and record the decision. It created DR-1 5s timeout and 3 retries with backoff, linked both PRs, and marked every part nobody had said as "not stated".
Filling the gaps. I gave the rejected options (separate connect and read timeouts, because our HTTP client supports one value; a circuit breaker, as too heavy for now) and the backoff. The scribe updated DR-1.
Offboarding. I said I was leaving. The scribe listed the one decision I own, named three gaps, and interviewed me about them. My answers went into DR-1, marked as mine.
Recall, with a record. I asked the recall agent why we use a 5 second timeout and 3 retries. It quoted DR-1, named who decided, linked both PRs and listed what is still not stated.
Recall, without a record. I asked why we did not use a message queue. The answer was "This was not recorded." It named DR-1 as the nearest record and gave no reason.

The scribe takes its role, creates DR-1 and marks the gaps (step 2). Screenshot from the demo run.

!list-documents, then the offboarding interview listing the three gaps (step 4).

The recall agent answers from DR-1, quotes the record and says what is not recorded (step 5).

No record covers the queue question, so the answer is "This was not recorded." (step 6).

Step 6 matters most. Any system can answer when the answer is on file. What a new hire needs to know is where the file ends.

What it still gets wrong

The sources are narrow. The reasons come from the room only. Switch has reference types for Jira, Confluence and Drive, but I connected only GitHub. Whylog does not read tickets, company chat history or research documents.
It is not a retrieval system. The recall agent reads the room's decision documents and the GitHub reference with its own access. There is no search index.
The coder had no clone of the repo. It described options in general and did not read the diffs. The scribe and recall agents rely on what people said and on the PR links.
The demo is one person. One person played the engineer. This is a single-person test, not a real team. I did not test Slack, Teams, Discord or Telegram.
Roles are claimed by the agents. Switch lists the roles as free until an agent takes one. The scribe and recall agents each did in my run.

What got in the way (my #HowISwitch notes)

Create all shows a timeout while the server keeps building. The "Nothing was created" screen is wrong. Retrying made duplicate rooms. Now I click once, wait about five minutes and check the Agents and Rooms lists.
A stale remote host blocked the template. An unreachable WSL host broke the provider check in the form. Deleting the host did not clear it. I fixed it with a template variant that sets the provider and location directly.
Deleting a room returned HTTP 500. I made new team names instead.
Agent replies take 30 to 90 seconds, sometimes longer, with no progress indicator.
Kickoff messages in the template timed out the Console, so the template has none. You start the flow by talking to the agents.

What I would want next: a progress view while Create all runs, a working room delete, a provider check that ignores unreachable hosts, and a way to see which agent is working on a message.

What I learned

"Not stated" is a feature. The most useful thing the scribe does is refuse to fill a blank.
Roles keep jobs clear. One agent writes, one reads. The recall agent cannot edit the record it answers from.
Documents beat scrollback. A decision a person can open and fix is a decision a team will trust.
Say what you did not test. Most of this post is a list of limits. A recall tool that overclaims would be worse than none.

Run it

You need Switch Console 0.40.1 (the Windows early-access build), Docker Desktop for the bundled Mattermost, and Claude Code installed and signed in on the same machine.

Fork or clone whylog-demo, or use your own public repo.
In Switch Console, open Templates, import template/whylog.template.yaml from the Whylog repo.
Fill in team, repo_url and newhire_user.
Click Create all once, and wait. Do not retry.
Open <team>-engineering, run !roles and !list-references, then follow docs/SCENARIO.md.

Links

Repo: https://github.com/ibnweb3/whylog
Demo video: https://youtu.be/dYZPn9NlDb4
Demo subject: https://github.com/ibnweb3/whylog-demo

Built with Claude Code for the Flip the Switch hackathon (WeMakeDevs x DevTools Academy x Switch), Track 01. If you have ever inherited code with no reasons attached, tell me which decision you wish someone had written down.

Your bibliography might be standing on a retracted paper

Ibn Web3 — Sat, 17 Oct 2026 10:45:14 +0000

I tried to predict retracted papers. I failed, and built something better.

Originally published at bedrock-audit.ibnweb3lab.workers.dev/blog. Built for the Graph Hacks hackathon (WeMakeDevs x FalkorDB), Track 01.

Imagine you've spent a year on a literature review. Every source on your list checks out, because your reference manager flags nothing. But two steps back, one of those sources leans on a paper that was retracted years ago, and no tool you use was ever going to tell you.

That gap is what Bedrock is for. You paste a list of DOIs, and it follows every citation path, up to two steps, from your papers to retracted research. For each finding it shows the exact chain, the retraction date and the reasons on record. An AI agent explains the results and proposes edits. It can never apply them.

This is the story of building it in three days: the parts that went well, the part that went badly, and the part where my own agent miscounted.

Code (MIT): https://github.com/ibnweb3/bedrock
Try it: https://bedrock-audit.ibnweb3lab.workers.dev

The idea: a question about paths, not rows

Tools that flag retractions typically check the papers you cite. They don't ask what those papers cite. That second question is a graph question: start at a paper, follow CITES edges, stop when you reach a retracted node, and report the path.

Here is a real path from the corpus:

your reference ──cites──▶ not flagged ──cites──▶ RETRACTED 2014-10-01
10.1016/j.neubiorev       10.1111/ejn.13603      10.1016/j.jpsychires
.2019.09.040                                     .2014.06.018

The middle paper isn't retracted, which is exactly why nothing warns you.

Two rules shaped everything. First, Bedrock states facts about a fixed corpus, it doesn't guess. A paper with "no exposure found" is never called safe, only not found within two hops. Second, the person decides. The agent proposes, a human approves.

The data: public, and kept that way

The retractions come from the Retraction Watch database, distributed by Crossref. The citations come from OpenAlex, which is CC0. I pulled works from 213 journals, 251,645 articles in all, and joined them to 61,299 retracted works. The hackathon rules ask for public data only, so authors were hashed with a secret salt in memory before anything touched disk, and names, emails and ORCIDs were dropped. The graph I publish has no author data at all.

Retraction Watch ─┐
                  ├─▶ FalkorDB (Cypher) ─▶ audit queries ─▶ read-only agent ─▶ you approve
OpenAlex (CC0) ───┘

The graph produces every finding. The model only writes the summary, and a person approves any change.

The graph: a small model, sharp queries

The model is deliberately plain: Paper nodes carrying doi, retracted_at and retraction_reasons, Venue nodes, and two relationships, CITES and PUBLISHED_IN. A retraction is a property, not its own node, which keeps the graph small and makes "as of this date" filters trivial. The two-hop question is one query:

UNWIND $ids AS i
MATCH (p:Paper {id: i})
WITH p
MATCH (p)-[:CITES]->(x:Paper)
WHERE x.retracted_at IS NULL
WITH p, x
MATCH (x)-[:CITES]->(r:Paper)
WHERE r.retracted_at IS NOT NULL AND NOT (p)-[:CITES]->(r)
RETURN p.id, x.id, x.doi, r.id, r.doi, r.retracted_at, r.retraction_reasons
LIMIT $limit

Read it aloud: the paper cites x, x isn't retracted, x cites a retracted r, and the paper doesn't already cite r directly. Everything Bedrock shows, the direct hits, the two-hop paths and the "papers cited alongside it" suggestions for replacements, is a query of this size, run by FalkorDB.

The 59-second query: the planner started in the wrong place

My first audit of nine papers took 59 seconds. The data was small and the question was simple, so something was off. The cause: FalkorDB's planner began at the 61,299 retracted papers and worked back toward my input, instead of starting at the few papers I'd asked about. The fix was the humble WITH p you can see above. It forces the traversal to start from the input papers and expand their handful of references.

Re-timed after the fix: 0.05 to 0.4 seconds for nine papers, and 0.15 to 0.4 seconds for 150.

What I took from it: a graph database makes the shape of the query matter more than the size of the data. When an indexed lookup can anchor a traversal, say so in the query. It's the difference between a demo and a product.

The test that failed: I tried to predict retractions

Bedrock's first idea was ambitious: use graph signals, like who a paper cites, who it shares authors with and which community it sits in, to predict which papers will be retracted. Before looking at any result I wrote down the pass mark: the graph method had to beat the best simple baseline, and the gap had to survive removing the largest journal. Then I ran it on 134,798 articles from journals with a retraction history, 140 of which were retracted within the next two years (cutoff 2020-12-31).

Method	Average precision
Random ranking	0.0012
Journal's past retraction rate	0.0066
Cites a retracted work (one hop)	0.0066
Two citation hops	0.0021
All graph signals combined	0.0041
Journal history plus graph	0.0074

The verdict: no reliable improvement. The best combination beat the journal baseline by less than the uncertainty. The 140 retractions were concentrated in 20 to 27 journals, so the study has low power, and the multi-hop signals did not beat their one-hop versions. The single signal that stood out, "this paper already cites a retracted one", made a paper 5 to 12 times more likely than average to be retracted later, depending on the cutoff. That is a lookup, not a forecast.

I could have shipped a "retraction risk score" anyway. It would have looked impressive and been unearned. Instead I changed the product: Bedrock stopped predicting and started proving. Exact provenance, with the failed test on the front page. That decision, more than any query, is what I'd defend in front of anyone.

The agent: useful, on a short leash

The agent runs on Groq's free tier with openai/gpt-oss-120b. It has three tools, audit_bibliography, explain_finding and find_dois, and every one only reads. There's no edit tool, and a test makes sure it stays that way. The guardrails came from watching it misbehave:

Grounding. Every DOI in the answer must come from your text or a tool result. Models retyped DOIs and corrupted some, so an invented one now raises a visible warning.
Completeness. It once skipped a flagged paper. If the summary omits one, the agent gets a single retry; anything still missing is appended in plain words.
Counts. The same run opened with "one retracted paper" when the audit had found two. Models miscount, so the opening line, "Checked 9 DOIs: 2 retracted, 2 cite a retracted work directly...", is written by code from the audit, never by the model.
Fallback. The findings are computed independently of the model. If the key is missing or Groq is rate-limited, you still get the full audit.

Then the approval step: the server re-runs the audit itself and only applies proposal ids it generated. The browser can't hand it a result to trust.

Shipping it: 2.5 million nodes down to 318,942

The research graph, with authors and two derived edge types for the backtest (1.69M COAUTHOR and 2.68M BIB_COUPLED, both built in Cypher inside FalkorDB), reaches about 2.5 million nodes. Too big to hand to anyone. But an exposure path can only follow CITES between papers I hold, so for the audit most of it is dead weight.

So I built a slim graph with just papers, venues and the citations between held papers: 318,942 nodes and 530,894 edges. Then I tested the claim, not just made it. A script audited 2,501 DOIs on both graphs and compared the complete results: statuses, paths, reasons, coverage and alternatives. Zero differences. (That test also caught a real bug: self-citations had inflated a reference count by exactly one in 37 papers.)

The slim graph ships as a 17 MB DUMP snapshot in the repo. In a container limited to 512 MB and a tenth of a CPU, it restores in about a minute and sits at roughly 300 MB. A judge can run the whole thing with:

git clone https://github.com/ibnweb3/bedrock && cd bedrock
docker compose --profile demo up --build app
# open http://127.0.0.1:8765

I checked that from a fresh clone with no keys: a no-cache build took just over two minutes and the app answered five seconds after starting.

Honest limits

It knows only its corpus: 213 journals, 251,645 articles and the papers they cite, with a Retraction Watch snapshot from Oct 15, 2026.
It looks two hops back. "No exposure found" means nothing was found within that, not that a paper is sound.
Reasons and dates come from Retraction Watch. A flag is a prompt for a person to read, never a finding about an author.
The live demo runs on one small AWS server, so it can be slow if many people try it at once. The one-command setup above always works.

What I learned

Write the pass mark before you run the test. It's the only reason a failure was a result instead of an argument.
In a graph database, anchor your traversals. Query shape beats data size.
Let code state the facts and let the model write the prose. Check its numbers anyway.
Prove your shortcuts. "The slim graph is equivalent" became a claim I could stand behind only after the 2,501-DOI test.

How this was built

I built it solo in three days, with Claude Code writing most of the code and docs from my instructions. I chose the idea, set the rules, approved the experiment's size, decided the pivot from prediction to provenance, picked Groq for the agent, and signed off every publication and deployment. The decision log is in the repo at docs/DECISIONS.md. The code is MIT licensed; the data keeps its own terms (Retraction Watch via Crossref, OpenAlex CC0).

Photo: sedimentary rock layers at Zabriskie Point, Death Valley, by Brigitte Werner, CC0 (Wikimedia Commons).