DEV Community: quintetkit

Bisection Narrowed It Correctly and Still Never Reached the Answer

quintetkit — Tue, 22 Sep 2026 04:16:31 +0000

I built a pipeline that publishes articles to Zenn by git push, using its GitHub
integration. I pushed twelve at once.
Two went live. The other ten did nothing.

No errors. The integration was fine. The push arrived. The deploy reported success.
And the articles did not appear.

It took me a full day to find the cause. The cause itself was not difficult.
What was difficult is that the reason for the failure was never returned to me.

What I tried first (all wrong)

My first assumption was that one file was malformed.

Tried	Result
Confirm the push arrived	It did
BOM / CRLF / control characters	Clean. Identical in shape to the articles that worked
Diff the frontmatter	No difference
Empty commit, redeploy	No change
Change the body, redeploy	No change
Rename the slug (twice)	No change
Wait 4.5 hours, push a single file	No change
Push a brand new article the platform had never stored	No change

That last row killed the hypothesis. A brand new file was not saved either, so
this was not about any individual file.

Half a day gone, spent knocking down guesses one at a time.

Switching to bisection

I stopped guessing and started cutting the input in half.

The repository held twelve articles plus a book (ten chapters and a config). I deleted
the book outright.

Book removed → no change

Then I deleted articles too, down to the two live ones plus one candidate —
three files total.

Three files → no change

That yields a conclusion:

The repository contents are not the cause.

This was a correct conclusion. Bisection worked. It killed hypotheses far faster than
the exhaustive approach that preceded it.

And it still did not reach the answer.

Why it could not

Any experiment you run locally reduces to: change the input, observe the output.
Bisection is one such method — halve the input range, watch the output change.

It assumes the output is observable.

In my case the output was a single bit: the article appeared, or it did not. And the
actual reason for the failure is not returned in the push response, not exposed
through the API, and not written anywhere in the repository. It is displayed only
on Zenn's deploy dashboard.

Which is to say: I had never once looked at the place where the reason lived.

However far you narrow the input, if the output stays one bit wide, one bit is all you
get. When the reason sits somewhere unobservable, local experiments cannot reach it
in principle.

The actual reason

The deploy dashboard said:

The following articles were not deployed because the posting limit was reached:
(four article names)

A posting rate limit. The deploy had succeeded every single time; the articles
were rejected inside it.

Zenn's FAQ states:

The limit logic is not disclosed — explicitly, to prevent gaming
Articles are judged on the number posted in the previous 24 hours
Books have a separate weekly limit
Hitting the limit is temporary; posting resumes after enough time passes

Measured: two got through in 24 hours, and everything after that was dropped.

The very first thing I did — pushing twelve at once — was the cause.

I had the order backwards

The mistake was not using bisection. It was the order.

What I should have done, before starting to narrow anything, is find out where the
failure reason is reported.

In the push response? → no
Through the API? → no
Written back into the repository? → no
On the deploy dashboard? → yes

That check takes five minutes. Instead I spent half a day staring at an output that
was never going to say anything.

"No error appeared" is not the same as working. It may only mean you are not looking
where the errors appear. Those two states are entirely different.

I fell into the same hole again the same day

On dev.to, updating nine published articles through the API. Spaced three seconds
apart, three of them failed with 500 Internal Server Error.

500 is a server-side fault. Read plainly, it means the other end broke. I waited ten
seconds and retried — all three succeeded.

The rate limit was arriving as a 500. Not a 429.

The error you get back does not necessarily describe what happened. "A 500 came
back" means a 500 came back. It does not mean the server broke.

Automation makes you look like a spammer without meaning to

One more thing, and it is not technical.

Publishing twelve articles at once is physically impossible for a person. From the
platform's side, that shape is indistinguishable from a machine flooding the queue.

Zenn does have a policy on AI-written content:

Posting machine-generated text is prohibited
Mass-producing AI-generated posts is grounds for account suspension
But using AI is not itself prohibited
What it asks for is that "a human is the one publishing" — that the author has verified the content

My articles are built from bugs I actually hit and numbers I actually measured. The
substance is on the right side of that policy. The delivery was not.

The rate limit is better understood as a warning than a punishment. Which means
looking for a workaround is the wrong move. Slowing down is the correct fix.

What I changed

Keep exactly one article queued for publication at a time

That is the whole fix. When one goes live, queue the next.

An automated pipeline, left alone, will try to emit everything it is holding. The
limit had to come from what is acceptable to publish, not from what is available to
publish.

Summary

Bisection assumes the output is observable. When the output is one bit wide, you can narrow the range correctly and still never reach the answer
Before you start narrowing, find out where failure reasons are reported. Five minutes
"No error appeared" and "I am not looking where errors appear" are different states
The error code you receive may not describe what happened — a rate limit can arrive as a 500
Automated publishing takes the shape of machine flooding. Rate-limit yourself by what is acceptable to publish, not by what you have available

The bisection I spent half a day on reached a correct conclusion.
One line of a log would have finished it in five minutes.

Related

I publish the configuration for splitting Claude Code into separate personas —
Architect, Coder, Reviewer, Conflict Resolver — under MIT. Copy it, run
./setup.sh, and it works. It does not depend on your tech stack.

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

The Page You Wrote by Hand Is the One That Stops Improving

quintetkit — Sun, 20 Sep 2026 11:23:52 +0000

I wrote a static site generator and added canonical and hreflang to 28 article
pages. Fourteen in Japanese, fourteen in English. Until then, search engines had no
way to know they were translations of each other rather than 28 unrelated pages.

The build passed on the first run. All 28 pages had it.

Two pages did not.

The pages that were missing it were the most linked pages on the site

The landing page, and its English version.

site/index.html          <- product page (Japanese)
site/index.en.html       <- product page (English)
site/articles/*.html     <- 14 articles
site/en/articles/*.html  <- 14 articles

The articles go through a shared page_shell() function. The two landing pages are
hand-written HTML. They predate the generator, and their layout is a one-off, so
there was never a reason to put them on the template.

So when I added the annotations to page_shell(), nothing happened to those two files.

And it was not only the new annotations.

	28 articles	2 landing pages
`rel="canonical"`	yes	no
`hreflang`	yes	no
`og:title` / `og:description`	yes	no
`og:url` / `og:type`	yes	no
`og:image`	yes	yes
`twitter:card`	yes	yes

og:image and twitter:card are there, because I put them in by hand the day I wrote
the page. Everything added to the template since then is missing.

The practical effect: a shared link rendered an image with a guessed title. With no
og:title, a crawler falls back to <title> or whatever heading it finds convincing.

And these two pages are where the READMEs of three repositories and the footers of 24
articles send people. The most linked pages on the site had the least markup on them.

The exception you wrote by hand does not receive later improvements

Stated generally:

Improvements go into the template. Things that do not go through the template do not
get them.

On the day you write it, the hand-written page is usually the better one. You spent
time on it, and you built it for that page specifically. That part is not the mistake.

The mistake is what happens afterward.

Feb   write the landing page by hand   ... it is the best page on the site
May   add OGP to the template          ... only the articles improve
Jul   add structured data              ... only the articles improve
Sep   add hreflang                     ... only the articles improve

The hand-written page stays at the quality it had the day you wrote it. Everything
around it keeps rising, so in relative terms it falls.

The same shape shows up all over a codebase:

the one try/catch that does not go through the shared error handler
the one fetch that does not go through the shared HTTP client
the one console.log that does not go through the logger
the script you run by hand that CI never touches

Every one of those was a reasonable call when it was made. Every one of them stops
receiving what lands afterward.

Checking the output beats normalizing the input

There are two ways out.

A. Remove the exception and put everything on the template.

The right direction, but not always available. My landing page has a one-off layout,
and forcing it through the article template breaks it. Generalizing the template enough
to hold it would make the generation of the other 28 pages more complicated. Adding
complexity to 28 pages for the sake of 2 is not a trade I want.

B. Check the output, whatever produced it.

This is what I did. Look at what came out, not at how it was made.

hreflang has a reciprocity requirement. If A says "the English version is B", then
B has to say "the Japanese version is A". If it does not, the annotation is ignored
entirely. A one-sided declaration does nothing.

So read every HTML file that was produced and check exactly that.

import re, pathlib

BASE = "https://example.github.io"
site = pathlib.Path("site")
ann = {}

for f in site.rglob("*.html"):
    text = f.read_text(encoding="utf-8")
    url = BASE + "/" + str(f.relative_to(site))
    if f.name == "index.html":
        url = url.replace("/index.html", "/")
    # lang -> href
    alts = {m.group(1): m.group(2) for m in re.finditer(
        r'<link rel="alternate" hreflang="([^"]+)" href="([^"]+)"', text)}
    if alts:
        ann[url] = alts

bad = []
for url, alts in ann.items():
    for lang, target in alts.items():
        if lang == "x-default":
            continue
        if not target.startswith(BASE):
            continue                      # external hosts will not annotate back
        if target not in ann:
            bad.append(f"{url} -> {target} has no hreflang of its own")
        elif url not in ann[target].values():
            bad.append(f"{url} -> {target} does not point back")

print(f"pages with hreflang: {len(ann)}  inconsistent: {len(bad)}")
for b in bad:
    print(" ", b)

This check does not know whether a page came from the template. It does not need to.
Hand-written or generated, if the file landed in site/, it is held to the same
standard.

Managing the input means managing every path into it, and you will miss one when a new
path appears. Checking the output is one place, and it holds no matter how many paths
there are.

Some annotations are worth keeping even when they cannot work

Running the check gave this:

pages with hreflang: 32
pointing at external hosts: 20   <- reciprocity cannot hold
inconsistent within the site: 0

The articles are also published on other platforms, and those copies are the canonical
ones. That means hreflang points at an external URL too — and those platforms will
never point back. Reciprocity does not hold, so those 20 annotations are ignored.

I left them pointing outward anyway, because there are only two options:

	reciprocity	agrees with canonical
point at the external URL	no (annotation ignored)	yes
point at my own non-canonical copy	yes	no

A page whose hreflang and canonical disagree gets both signals distrusted. Being
ignored is the cheaper failure.

Sometimes you pick the option you know does not work, because the alternative does
damage. What matters is picking it knowing that. An annotation that is inert because
you chose it and one that is inert because you never checked look identical today, and
differ entirely the day you can fix it.

Takeaways

Improvements land in the template. Anything outside it gets none of them.
A hand-written exception freezes at the quality it had on day one, and falls behind as everything around it rises
The page you put the most care into is the one most likely to be left behind
Check the output rather than normalizing every input — it holds as paths multiply
hreflang requires reciprocity; a one-sided declaration does nothing
An ignored annotation is better than one that contradicts your canonical

Related

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

Write the Same Decision in Two Places and Only One of Them Gets Fixed

quintetkit — Sat, 19 Sep 2026 02:25:41 +0000

There are two storefronts, one domestic and one international, and the
international one is not open yet.

So I made a rule: if the international store is not open, send the reader to the
domestic one. It has an English interface and takes international cards.
Being sent somewhere you can actually buy beats being sent nowhere.

I wrote that rule into the function that generates the footer of each article.

if store.get("gumroad"):
    parts.append(f"The full kit is available here.\n\n{store['gumroad']}")
elif store.get("booth"):
    # until the international store opens, send readers to the domestic one
    parts.append(f"The full kit is on BOOTH ...\n\n{store['booth']}")

It went out to 24 articles. All of them say you can buy it.

The page they link to said "Coming soon"

The function that rewrites the button on the product page lives somewhere else.

url = store.get("booth") if name == "index.html" else store.get("gumroad")
new_btn = off if not url else f'<a class="btn" href="{url}">{label}</a>'

The English button only looks at gumroad. It is not open, so that is None.

<a class="btn" href="#" aria-disabled="true">Purchase Quintet (Coming soon)</a>

Which left the site saying this:

24 articles   "you can buy it here" -> link to the product page
product page  "Coming soon"

Readers arrive at the product page from the articles. The order could not be
worse. Only the people who read a whole article, got interested, and clicked
were told to come back later. The readers most likely to buy were the only
ones turned away.

I did not recognize it as the same decision

In hindsight the two pieces of code decide the same thing:

Which storefront do we send a reader of this language to?

So why did it not register? Because one builds a paragraph of text and the
other rewrites an HTML attribute.

def buy_line(store, lang) -> str:      # returns a Markdown paragraph
def rewrite_lp(store) -> int:          # regex-replaces HTML

Different arguments, different return types, different output formats. When code
does not look alike, the fact that it implements the same policy does not
surface.

Duplication is usually described as the same code written twice. The dangerous
kind is the same decision written twice in two different shapes. The first kind
you can grep for. The second kind you cannot.

What caught it was reading the diff before pushing

Not a test. I read the diff before pushing.

-<a class="btn" href="https://quartet-dev.booth.pm/items/...">Purchase Quintet</a>
+<a class="btn" href="#" aria-disabled="true">Purchase Quintet (Coming soon)</a>

The generated site is committed to a repository, so regenerating it produces a
diff. Checking that nothing came along besides the change you meant is what
works against this class of bug.

What I meant to do was add a fifteenth article. A button had no business being
disabled. The mismatch between intent and diff is the only signal there was.
A workflow that does not read diffs ships this.

One place for the decision, separate places for the presentation

There is a range of possible fixes. Merging the two functions is too far. One
returns Markdown, the other rewrites HTML. Merging them only moves the branching
inside.

Share the decision, nothing else.

def destination(store: dict, lang: str) -> str | None:
    """Which storefront this language's readers go to, or None if none is open.

    This used to be written in two places. A fallback was added to the article
    footer and not to the landing page, so the articles said "you can buy it"
    while the page they linked to said "coming soon". The destination is decided
    here; how it is presented is up to the caller.
    """
    order = ("booth",) if lang == "ja" else ("gumroad", "booth")
    for key in order:
        if store.get(key):
            return key
    return None

It returns the name of the store, not the URL. The callers do not need to know
where to send people so much as which store ended up being chosen:

the article wording differs by store (sending readers abroad needs an explanation)
the landing page only disables its button when there is no store at all

The presentation stays different. Only the decision is shared.

# article
where = destination(store, "en")
if where == "gumroad":
    ...
elif where == "booth":
    ...

# landing page
where = destination(store, "ja" if name == "index.html" else "en")
url = store[where] if where else None

Run every combination

Two stores, each open or not, is four cases. I ran all four and checked that
the articles and the landing pages say the same thing.

booth=no  gumroad=no  -> EN page=#        JA page=#        EN article=none
booth=yes gumroad=no  -> EN page=BOOTH    JA page=BOOTH    EN article=booth
booth=no  gumroad=yes -> EN page=Gumroad  JA page=#        EN article=gumroad
booth=yes gumroad=yes -> EN page=Gumroad  JA page=BOOTH    EN article=gumroad

The second row is the bug. Before the fix, the English page was # there.

Four cases is few enough to run all of them. While the combinations are still
countable, running all of them is faster than arguing about which one is
representative.

I also checked that restoring the original settings produces a zero diff. The
test rewrites configuration, and forgetting to restore it breaks the live
links.

Takeaways

The dangerous duplication is not the same code twice — it is the same decision twice, in two different shapes. grep does not find that
When code does not look alike, the shared policy does not surface
Commit the generated output and read the diff before pushing. The mismatch between what you meant and what changed is the signal
Share only the decision. Sharing the presentation pushes the branching back inside
Have the decision return which option was chosen, not the final value, so callers can present it differently
While the combinations are countable, run all of them — and verify that a test which rewrites configuration puts it back

Related

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

A Default That Is Safe on Create Is Destructive on Update

quintetkit — Fri, 18 Sep 2026 09:47:39 +0000

I wanted to add links to nine already-published articles. Just a bulk update through
my own publishing tool.

The tool tried to unpublish them.

The safe direction of a default flips with the operation

The tool was built like this:

const publish = rest.includes("--publish");

const article = {
  title: fm.title,
  body_markdown: body.trim(),
  published: publish,   // decided by an explicit flag, not by frontmatter
  tags: fm.tags ?? [],
};

published does not come from the article's frontmatter. It comes from --publish on
the command line. Without the flag, you get a draft.

The reason was right there in a comment:

Draft by default. Publish only when --publish is explicit,
because failing to publish is safer than publishing by accident.

For creation, that is correct. Shipping a half-written draft to the world costs more
than forgetting to ship a finished one.

For updates, the same default points the other way.

Forget --publish while updating an existing article and you send published: false.
A live article drops back to draft. The URL stays; it just 404s.

Forgot to publish → notice later, publish then
Un-published by accident → every inbound link and every bit of search equity dies

The same phrase, "err on the safe side", named opposite directions depending on the verb.

The fix was not "infer it more cleverly"

My first instinct was to preserve the existing state on update: if --publish is absent,
keep whatever the article currently is.

I dropped it. You cannot distinguish "I meant to unpublish this" from "I forgot the
flag." Inferring past something you cannot distinguish means that one day someone
genuinely wants to unpublish and is silently ignored.

So it refuses:

// published comes from the explicit flag, so an update without --publish drags a
// live article back to draft. Accidental unpublishing is far more common than
// deliberate, so refuse. The URL survives but 404s, and the backlinks die with it.
if (existing?.published && !publish) {
  throw new Error(
    `"${fm.title}" is already published. Updating without --publish reverts it to a draft.\n` +
    "  Pass --publish to update it.",
  );
}

I ran it without the flag and confirmed it exits 1.

If you want to unpublish, that should be its own operation. Never something that
happens as a side effect of updating.

And then: identity was keyed on a value that changes

The same tool had a second hole — how it found the existing article.

// update if an article with the same title exists, otherwise create
const mine = await call("/articles/me/all?per_page=100");
const existing = mine.find((a: any) => a.title === fm.title);

It matched on the title.

The same day, I fixed a bug in the frontmatter parser. It stripped the outer quotes from
title: "Treating \"It Worked\" as ..." without unescaping what was inside, so
an article had shipped with backslashes in its name.

// before
if (value.startsWith('"') && value.endsWith('"')) value = value.slice(1, -1);

// after
if (value.startsWith('"') && value.endsWith('"')) {
  value = value.slice(1, -1).replace(/\\(["\\])/g, "$1");
}

The moment that landed, fm.title produced a different string. And the published
article still carried the old one.

So find misses. It concludes there is no existing article. It falls through to create.

Had I pushed without noticing, a second copy of the same article would have gone live.

Fixing one bug is what exposed the other

This is not "I hit two bugs." It is one bug whose fix activated the other.

Identity was keyed on a derived value that the tool itself computes. Change how it
is computed and the value changes. Change the value and identity breaks.

The same shape shows up everywhere:

Keying on a normalised name → fixing the normalisation rules orphans your existing rows
Keying on a hash of the body → reformatting makes it a different object
Keying on a display string → correcting a typo in it is an identity change

Never key identity on a value you might fix.

The fix: record the id the other side assigned

id is assigned by the API. Nothing I do changes it. So the tool now records the id of
every article it creates.

{
  "_comment": "filename -> article id. Titles change; identity must not.",
  "ids": {
    "composite-github-action": 4576296,
    "linter-false-positives": 4577641
  }
}

const knownId = store.ids[slug];
let existing = knownId ? mine.find((a) => a.id === knownId) : undefined;

if (knownId && !existing) {
  throw new Error(
    `id=${knownId} (${slug}) is not on the server.\n` +
    "  Either the article was deleted, or this is a different account's key.\n" +
    "  Continuing would create a duplicate, so stopping here.",
  );
}

If there is a recorded id and it is missing, stop. Silently falling through to create
is the single worst behaviour available.

Verifying it

I deliberately changed a title and pushed:

title: "... (Without npm Publishing) (id lookup test)"

-> PUT /articles/4576296

PUT /articles/4576296, not POST /articles. The title no longer matches and it
still resolves to the same article. The article count did not change.

Why it was keyed on the title in the first place

Because I did not want a state file. I wanted the articles to be nothing but Markdown.

I understand the impulse, and the price of it was inferring identity on every run.
That works right up until the material the inference is built on changes.

Designs that hold no state tend to become designs that infer state instead. What you
are inferring in place of storing is worth knowing explicitly.

Summary

A default's safe direction flips with the operation. "Don't publish" is the safe side on create and means "unpublish" on update
Do not infer past an intent you cannot distinguish. Refuse, and make it explicit
Destructive operations get their own verb. Never as a side effect of another one
Do not key identity on a value you computed — fixing the computation breaks identity
Record the id the other side assigned, and stop when a recorded id goes missing
Stateless designs tend to become inference-based designs. Know what you are inferring

Related

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

The Blind Spot Was the Rule I Wrote to Avoid False Positives

quintetkit — Thu, 17 Sep 2026 02:40:25 +0000

I maintain a linter for a configuration format. Its stated principle is on the
first screen of the README:

A finding exists only if the documentation says the thing errors, is skipped,
or is ignored.

One consequence of that principle is a list of things it deliberately does not
check. The clearest entry:

Unknown keys. The documentation says the published schema lags the CLI, so
checking this would flag every new feature.

That is a good rule. I still think it is a good rule. It is also the reason my
own products shipped a broken line in four places for several days.

The linter passed

Before adding a chapter to a paid guide, I ran the linter against every
configuration my own products ship.

product/kit           No problems (5 files checked)
product/oss           No problems (4 files checked)
publish/quartet       No problems (4 files checked)
publish/mdlinkcheck   No problems (4 files checked)

Green everywhere. So I opened the files and read them, which is the only reason
this post exists.

# .claude/agents/coder.md
---
name: coder
description: "..."
tools: *
model: inherit
---

tools: *. I wrote that months ago meaning "this one gets everything".

The documentation says the field takes exact tool names, or the server-level
patterns mcp__<server> and mcp__<server>__*. A bare * is not among them.
It also says that when an entry resolves to nothing, the subagent refuses to
launch, returning an error naming the unresolved entries.

And the documented way to say "everything" is to omit the field.

I cannot prove from here that tools: * fails — I would have to observe a
launch. What I can say is that it is not a documented pattern, and that omitting
the field expresses the same intent with no ambiguity. So I removed the line.

Four copies had it. Two of them were public.

Why the linter could not see it

This is the part worth keeping.

The rule "do not check whether a tool name exists" is correct: the set of tools
grows, a checker can only hold a snapshot, and a snapshot would report every
tool released after it as unknown. One false finding and nobody reads the output
again.

But that rule was written as "do not look at the tools field", and the
thing I needed to catch was not a name at all.

Checking whether Bash is a real tool → needs a current list → goes stale
Checking whether * is a valid entry → needs no list → cannot go stale

* is not a tool name in any version. It never will be. The question is about
the syntax of the field, and I had filed it under identity of the values,
where my own rule told me not to look.

The fix is nine lines and cites the same documentation page:

warn  .claude/agents/coder.md:4
      `tools: *` is not a documented pattern. Omit `tools` to inherit every
      tool available to subagents.
      why: https://code.claude.com/docs/en/sub-agents

To keep the no-false-positives promise honest, the clean fixtures now include
the two things that must not warn: a file with tools omitted, and a file
with disallowedTools: mcp__github, which is a documented pattern. The test
requires zero findings on that directory. 71 tests became 73.

The shape of this mistake

A rule that says "don't check X because it produces false positives" quietly
becomes "don't look at X". Those are not the same rule, and the gap between them
is exactly where a checkable case can sit unchecked.

I have started asking one question about every exclusion I have written:

Is there a sub-case here that needs no external knowledge?

For unknown settings keys, the answer is genuinely no — you cannot tell a typo
from a new feature without the current schema. For tools, the answer was yes,
and I had not asked.

The other thing I would do differently: a green run on your own code is not
evidence about your own code. It is evidence about the intersection of your
code and the rules you happened to write. Reading the files took four minutes
and found what four green runs did not.

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

A Scheduled Job That Works and Never Runs

quintetkit — Wed, 16 Sep 2026 09:08:29 +0000

I have a job that publishes one article a day. Over four days I rebuilt its
scheduling three times, and each rebuild fixed a real failure that the previous
version could not have shown me.

Every version passed the test I gave it. The test was always "does it work
now", never "does it work where it is going to live".

Version 1: a loop in /tmp

nohup bash /tmp/daily-loop.sh &

It ran. It published. Four scheduled items were riding on it.

Then the machine restarted and all four were gone. /tmp is not a place
you put a program, and nohup does not survive a reboot. I knew both of those
facts and still did this, because the thing I was testing was the publishing
logic, and the publishing logic was fine.

Nothing reported the loss. The next morning there was simply no new article,
and no error anywhere, because the process that would have logged the error did
not exist.

Version 2: launchd with KeepAlive

Moved the script into the repository, wrote a launch agent, KeepAlive: true so
it comes back if it dies.

The next morning:

scripts/daily-publish.sh: line 59: node: command not found

A job started by launchd gets almost no PATH. Mine was
/usr/bin:/bin:/usr/sbin:/sbin. Node is in /opt/homebrew/bin. When I ran the
same script from my own shell it worked, because my shell had the PATH.

The part that made this expensive: the same loop also runs a reconciliation step
written in Python, and python3 is in /usr/bin. So that step kept working.
The log filled up with successful runs every fifteen minutes.

A partly working automation is harder to notice than a dead one. The dead
one has no log. This one had a log that looked healthy.

Fix is boring:

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"

I put it in every script the scheduler touches, not only the one that broke.
One entry point was not the whole story: another script could invoke the same
thing from a different parent.

Version 3: the loop was frozen, not running

PATH fixed, publishing worked, and then the log went quiet for three hours.
The process was alive:

$ ps -o pid,etime,command -p 72127
  PID     ELAPSED COMMAND
72127    02:55:52 sleep 900

sleep 900 had been alive for two hours and fifty-five minutes.

That is a laptop. It sleeps. A sleep that is supposed to last fifteen minutes
does not necessarily advance while the system is suspended, so every suspend
adds an unbounded gap to a while true; do work; sleep N; done loop.

The loop was not broken. It was not stuck on a network call. It was doing
exactly what I wrote, and what I wrote assumed wall-clock time keeps flowing.

Version 4: let the scheduler schedule

<key>StartInterval</key>
<integer>900</integer>
<key>RunAtLoad</key>
<true/>

The script now does one pass and exits. No loop, no sleep. StartInterval
is the mechanism built for this: launchd fires the job on wake if the interval
elapsed while the machine was asleep, and it will not start a second instance of
the same label while one is running, so the lock I would otherwise need is gone.

I also deleted the old looping script instead of leaving it beside the new one.
Two entry points for the same job means never being sure which one is running.

One more trap on the way out

I checked whether the interval had registered:

$ launchctl list com.example.daily
{
    "Label" = "com.example.daily";
    "LastExitStatus" = 0;
    ...
};

No StartInterval. I was one command away from concluding the schedule had
silently failed to load and rewriting it again.

$ launchctl print gui/$(id -u)/com.example.daily
    state = not running
    runs = 4
    last exit code = 0
    run interval = 900 seconds

launchctl list does not print every key it holds. launchctl print does.
The absence of a key in one tool's output is not evidence about the job.

What the four versions have in common

Each fix was correct. Each was verified. Each verification was performed in an
environment the job would never actually run in.

I tested the publishing logic in my shell, so the reboot never came up
I tested the launch agent by triggering it once, while my shell's PATH still hung around in my head as "the" PATH
I tested the loop by watching one iteration, which is exactly the case where a sleep that does not advance looks identical to one that does

The rule I ended up with is narrow and I have not managed to argue myself out of
it yet:

When you change where something runs, run it once in the new place before
believing anything you knew about it.

Not a fresh test of the logic. A test in the new environment, of the same logic,
looking for the environment's own failure modes: what the PATH is, what happens
across a reboot, what happens across a suspend, and what the tool you use to
inspect it declines to tell you.

The fourth version has now survived several suspends and one reboot. That is not
proof, but it is the first version whose evidence came from the place it lives.

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

I Removed an allow Rule and the Permission Was Still There

quintetkit — Tue, 15 Sep 2026 02:11:11 +0000

I wanted to stop auto-approving a command, so I deleted its entry from
permissions.allow in my ~/.claude/settings.json.

It was still auto-approved.

Not cached, not stale — the entry existed in the project's
.claude/settings.json as well, and list keys are combined rather than
overridden.

Lists merge. Nothing removes another file's entries

When the same list key is set in more than one settings file, Claude Code
combines the lists.

// ~/.claude/settings.json
"permissions": { "allow": ["Bash(npm run *)"] }

// .claude/settings.json  (checked into the repository)
"permissions": { "allow": ["Bash(git commit *)"] }

// → both apply

The documented intent is that each file can add entries without removing
another file's. Which is reasonable, and has a consequence worth stating
plainly:

No settings file can subtract from another. Not a lower level, not a higher
one. Deleting your own entry removes your own entry and nothing else.

To actually stop something, use deny, or edit the file that adds it.

I had spent a while looking for a caching bug.

The level order, and the row that is actually useful

Highest first:

1  Managed settings          (organisation policy)
2  Command line              (--settings, one session)
3  .claude/settings.local.json
4  .claude/settings.json     (checked in, shared)
5  ~/.claude/settings.json

3 sits above 4, which I did not expect and now rely on. A value your team
set in the shared file can be put back for you alone in settings.local.json —
without editing the file everyone shares.

Nothing beats managed. Not --settings; and --model picks only from the
models the organisation allows.

Environment variables are not a level

They are not in the stack at all, and which one wins is decided per key:

ANTHROPIC_MODEL           beats the model key in any file
ANTHROPIC_DEFAULT_MODEL   applies only when no file sets model

Two variables that both choose a model — one wins, one yields. There is no
general rule to learn here; you check the pair for the key you care about.

An env block inside a settings file is an ordinary key and follows the five
levels.

Seven keys where a lower level beats managed

For seven security-sensitive keys, Claude Code honours the more restrictive
value from a scope that otherwise could not override managed settings — so a
project file can tighten remoteControlAtStartup or crossSessionInbound even
against organisation policy.

In one direction only. A project true for remoteControlAtStartup is
ignored: you can tighten from below, never loosen. And enableArtifact
cannot be turned back on — once anything sets it false, nothing restores
it, managed settings included.

Four model-related keys also break the merging rule and take a whole value from
one file instead. Both lists are on the reference page; what is worth carrying
in your head is that the direction of the exception is not symmetric.

Why I keep writing these down

Every one of these produces the same experience: you changed the file, the
behaviour did not change, and nothing told you why. The causes are different —
scope, merging, precedence, an environment variable — and the symptom is
identical.

Which is also why a checker cannot help with this particular one.
Which files load is decided at run time, so precedence is not knowable from
the files in front of you. /status shows what actually loaded; that is the
tool for this.

/status         # which files were loaded
claude doctor   # the resolved settings, and any skipped rules

The five levels, the merge exceptions and the seven keys, in one table:

https://quintetkit.github.io/en/reference/claude-code-settings-precedence.html

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

The Workflow Ran With Four Personas Instead of Five and Looked Fine

quintetkit — Mon, 14 Sep 2026 01:14:16 +0000

I run Claude Code as five separate subagents with different permissions —
architect, coder, reviewer, conflict resolver, UI designer. One of them stopped
loading.

Nothing said so. The work still got done, by whoever was left. It got
slower, review got softer, declared scopes stopped being respected. Every one of
those looks like "the model is having a bad day".

The symptom has the shape of not working well. The cause was not being
there.

A file in `.claude/agents/` is skipped silently

Five conditions in the frontmatter, any one of which means the file is not
treated as a subagent:

Condition	What happens
No `name`	Treated as documentation — a note kept beside your agents
The opening `---` is not line 1	Read as having no frontmatter, so documentation again
A `name` starting with `-`, or containing `:`	The whole file is skipped
A `name` but no `description`	Skipped
YAML that does not parse	Nothing is read, file skipped

None of these appear in the session. The reason goes to the debug log, which
you see with --debug, which you are not running when things seem fine.

The second row is the one I hit. I had pasted content into a persona file and
an editor put something above the opening ---. From then on the file was
prose sitting next to four agents, and Claude distributed its work to them.

: is unavailable because it is reserved for plugin-scoped names like
my-plugin:reviewer. A leading - is out for the same class of reason.

Copying a persona is where this gets interesting

Adding a persona by duplicating an existing one is the obvious move, and
forgetting to change name is the obvious mistake.

Which of two files with the same name loads is not documented. It is decided
by filesystem read order.

So the same repository can run a different persona on a different machine,
and both machines are "working". I do not have a story about being burned by
this one, and I am fairly sure that is luck rather than care.

`claude plugin validate` gets you part of the way

claude plugin validate .claude/agents
claude plugin validate ~/.claude/agents

It finds frontmatter that fails to parse. Two limits worth knowing:

it looks only at the directory you name — user scope is a separate run
it does not flag a file that parses but has no name

Which is most of the list above. It is worth running and it is not sufficient.

The one that was my own fault

My coder persona had this:

tools: *

meaning "this one gets everything". tools accepts exact tool names or the
server patterns mcp__<server> / mcp__<server>__*. A bare * is not among
them, and an entry resolving to nothing makes Claude Code refuse to launch the
subagent.

The documented way to say "everything" is to omit the field.

Why my own linter could not see that is written up separately.

https://quintetkit.github.io/en/articles/blind-spot-from-refusing-to-check.html

The short version is that the rule I wrote to avoid false positives was phrased
as "don't look at the tools field".

That one is worse than a missing persona, because coder is the one that writes
the code. The other four keep working, so it presents as the coder being
unreliable today.

What I do now

Two checks, and I only run them twice: after setup, and after adding or copying
a persona.

npx @quintetkit/ccheck        # the five conditions, duplicate names, tools: *
claude --debug                # once, to see what was skipped and why

And a habit that is cheaper than either: count them. If the configuration
says five personas, the thing you want to confirm is that five loaded — not that
the work is getting done. The work getting done was never the question.

The five conditions and every frontmatter field, in one table:

https://quintetkit.github.io/en/reference/claude-code-subagent-frontmatter.html

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

I Set a 500ms Timeout and Got 28 Hours

quintetkit — Sat, 12 Sep 2026 01:16:42 +0000

I had an MCP server that occasionally hung, so I gave it a short leash:

{ "mcpServers": { "flaky": { "type": "http", "url": "...", "timeout": 500 } } }

timeout is in milliseconds, so that reads as half a second.

Values below 1000 are ignored. The call falls through to
MCP_TOOL_TIMEOUT, and if that is unset, to its default — about 28 hours.

I had written the tightest limit I could and produced the loosest one available.

Before v2.1.162 a value under 1000 was floored to one second, which is the
behaviour I had in my head. It changed, and nothing in my configuration
changed with it.

Three related things I only learned by reading the whole section:

timeout is a hard wall-clock limit per tool call. Progress notifications from the server do not extend it
A timeout of at least 1000 also acts as a floor on the idle timeout — calls are never aborted for idleness sooner than that
stdio and WebSocket servers have no per-request timer at all

A `url` with no `type` is read as a stdio server

{ "mcpServers": { "example": { "url": "https://mcp.example.com/mcp" } } }

There is no defaulting to http. An entry with no type is a stdio server,
stdio requires command, and so the server is skipped as a configuration error:

MCP server "example" has a "url" but no "type"; add "type": "http" (or "sse" / "ws") to this entry

That message exists from v2.1.202. Before it, the same mistake reported
command: expected string, received undefined — you wrote a url and were told
about a command, which is a long way from the cause.

Under --output-format stream-json it also appears in the system/init event's
mcp_server_errors, so a script can detect that a server never loaded. That is
the only machine-readable path I have found for it.

A cloned repository cannot approve its own servers

This one cost me the most time, because it looks like a permissions bug.

Servers from .mcp.json need approval. You can commit
enableAllProjectMcpServers or enabledMcpjsonServers to the project's
.claude/settings.json — and in an untrusted folder they are ignored. The
server sits at ⏸ Pending approval, never connected, never health-checked.

Which is correct, and obvious once stated: a repository that could approve its
own MCP servers would be a way to run code on anyone who clones it.

Three sources of approval still apply in an untrusted folder: your own
~/.claude/settings.json, managed settings, and settings passed with
--settings.

The pattern across all of these

Every one of them is a configuration that loads. No error, no warning you
would notice, and a running system that is doing something other than what the
file says.

timeout: 500 is the sharpest example: the file says half a second, the system
says a day and change, and both are working as designed.

What I want from a configuration checker, having been on the wrong end of these,
is not more rules. It is that the ones it does have never fire on something
correct — because the reason I stopped reading startup output in the first place
was that nothing there had ever mattered, and by the time something did, I had
already trained myself past it.

npx @quintetkit/ccheck

error .mcp.json:3
      Server "example" has a `url` but no `type`. Add `"type": "http"` (or "sse" / "ws").
warn  .mcp.json:9
      Server "flaky": `timeout: 500` is in milliseconds; values below 1000 are ignored.

The full set, including the required field per transport:

https://quintetkit.github.io/en/reference/claude-code-mcp-json.html

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

The Deny Rule I Wrote Was Never Consulted

quintetkit — Fri, 11 Sep 2026 01:03:22 +0000

I had a rule meant to keep an agent inside one directory:

{ "permissions": { "deny": ["Write(src/generated/**)"] } }

It is valid. It loads. It is never consulted.

File path rules apply to Read(...) and Edit(...). Written for Write,
NotebookEdit, Glob, or the legacy MultiEdit, the rule is accepted and then
never looked at. There is a startup warning; I had not been reading startup
output, because nothing had ever been wrong in it before.

That was the cheap one. Here is the expensive one.

`Bash(git * main)` allows every git subcommand

A * in a Bash rule stands in for whatever text is in its place. So the
earlier it sits, the wider the rule is.

You write	What it actually allows
`Bash(git log *)`	commands starting `git log`
`Bash(git * main)`	every git subcommand — `git push origin main` included
`Bash(* --version)`	every program on the machine

Bash(git * main) reads like "one git command, on main". It permits
git push origin main. It also permits git -c core.pager=… diff main, and
-c makes git run a program you name.

I wrote Bash(git * main) thinking I had allowed a diff.

There is a startup warning for an allow rule with a wildcard before the rest of
the command. Same as above: the rule still applies while the warning scrolls by.

Whitespace decides whether `lsof` is allowed

Bash(ls *)   matches  ls -la,  ls        does not match  lsof
Bash(ls*)    matches  ls -la,  lsof

The space before a trailing * is part of the rule. And Bash(ls *)
matches bare ls only because that trailing * is the rule's only wildcard —
Bash(* --help *) matches npm --help x but not npm --help.

Three more that load and do nothing

An unanchored allow glob is skipped. "*", "B*", "mcp__*" in allow
auto-approve nothing. Allow rules accept a glob only after a literal
mcp__<server>__ prefix, because the rule has to name a server you configured.

A parameter rule on a tool's own content field is ignored.
Bash(command:rm *) is bypassable with a compound command, so it does not
apply. The fields you cannot match this way:

Bash, PowerShell  → command        Grep, Glob   → path
Read, Edit, Write → file_path      WebFetch     → url
NotebookEdit      → notebook_path

Write Bash(rm *), Read(./.env), WebFetch(domain:host) instead.

An mcp__ rule with parentheses is skipped at load. mcp__memory(read)
never applies. It is listed in the invalid-settings dialog and in
claude doctor — neither of which you see in CI.

And one that is only a syntax trap: Bash(ls:*) is the same as Bash(ls *),
but only as a suffix. In Bash(git:* push) the colon is a literal
character and the rule matches nothing at all.

Why this bites harder than it looks

If you are running several agents in parallel, reduced permissions are usually
the safety design. Mine was: the implementing agent may only touch the files
its Issue declared.

I had expressed that as Write(...) path rules. The mechanism I was relying on
was the one that is never consulted. What was actually holding scope was the
Read/Edit rules, the declared scope in the Issue text, and the wording of
the request — none of which I had been treating as the load-bearing part.

The lesson I took is narrower than "read the docs":

A permission rule that is doing nothing looks exactly like one that is
working. Both are silent.

Checking a config

npx @quintetkit/ccheck

warn  .claude/settings.json:14
      `Write(src/generated/**)` is accepted but never consulted. File path rules
      apply only to `Read(...)` and `Edit(...)`.
      why: https://code.claude.com/docs/en/permissions

Building that checker, the tests were the interesting part. It runs the exact
rule strings from the documentation — six that must produce a finding, and
eleven that must not, including Read(./src/**/*.ts) and Bash(ls*).

Eleven-to-six, deliberately. Permission rules have many valid shapes, so calling
a correct one broken is the likely failure, and one false finding is enough for
the whole output to stop being read.

The five forms, with what each does instead of what it looks like:

https://quintetkit.github.io/en/reference/claude-code-permission-rules.html

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

I Put a Setting in My Project Config and It Did Nothing

quintetkit — Thu, 10 Sep 2026 01:48:41 +0000

I put autoMode in .claude/settings.json, committed it, and it had no effect.

The JSON was valid. Startup printed no warning. --debug said nothing. I spent
a while assuming I had the semantics wrong before I found the actual reason:

that key does not apply from that file.

Settings have a scope, and the scope is not in the file

Claude Code's settings reference has a column I had never read carefully. Every
key names the scopes it applies from. Counting it:

Any file                  152 keys
Managed                    39 keys   only from managed settings
User or managed            23 keys   ~/.claude/settings.json, or managed
User, local, or managed     3 keys   settings.local.json but not settings.json
Global config               6 keys   ~/.claude.json only
------------------------------------
                          223 keys, of which 71 are scoped

71 of 223. Put one of those in a project's .claude/settings.json and it is
not an error, produces no warning, and does nothing.

The ones I would expect people to hit:

autoMode                          user or managed
autoMode.classifyAllShell         user or managed
skipAutoPermissionPrompt          user or managed
useAutoModeDuringPlan             user, local, or managed
spellcheck                        user or managed
sandbox.network.strictAllowlist   user or managed
sandbox.filesystem.disabled       user or managed
diffTool                          ~/.claude.json only
autoConnectIde                    ~/.claude.json only

If you run several agents at once you get more approval prompts, so loosening
autoMode is the obvious move — and the obvious place to put it is the project
config you already have open. That is the one place it does not work.

The row that will cost you a debugging session

User, local, or managed     3 keys

settings.local.json is a different scope from settings.json. Three keys
apply from the local one and not from the shared one:

useAutoModeDuringPlan
syncClaudeAiSkills
skipDangerousModePermissionPrompt

So: you put one in settings.local.json, it works. Later you move it into the
shared file so the team gets it, and it silently stops. Nothing changed except
which file it lives in.

This is a manufactured "works on my machine" — with the unusual property that
the person who wrote it is the one for whom it works.

Nested keys are scoped by their full dotted name

The documentation names them the way you would write a path, not the way you
would write JSON:

sandbox.network.strictAllowlist        ← this is what is scoped
sandbox                                ← this is not

Which matters if you go looking. I wrote a checker for this and inspected only
the top level of the settings object, so this:

{ "sandbox": { "network": { "strictAllowlist": true } } }

produced nothing at all. 20 dotted keys were invisible to it, 12 of them
sandbox.*.

How I found out how wrong I was

I had been maintaining this list by hand, copied out of the documentation, with
a comment I wrote myself:

/** そのファイルからは効かないスコープのキー（一部。確実なものだけ） */

"Some. Only the certain ones." I knew it was partial. I did not know how partial.

             actual   I had
Managed          39      14
User or managed  23       0     ← the whole row
Global config     6       6

The user or managed row was missing entirely — 23 keys, including
autoMode, the one that started this.

I had read the table by grouping it: Managed is for organisations deploying
policy, which is not my situation, so I skipped past it — and past the row
sitting next to it, which turned out to mean "applies from your own
~/.claude/settings.json" and was the row I most needed.

Reading it row by row would have worked. Reading it by category is what
failed.

So it is generated now, straight from the scope column, and the count comes out
of the same place the checker reads.

Checking yours

npx @quintetkit/ccheck

warn  .claude/settings.json:63
      `autoMode` applies from user or managed settings only. It has no effect from this file.
      why: https://code.claude.com/docs/en/settings-reference

One thing that took a second pass to get right: ~/.claude/settings.json
is the user scope. Deciding "this key doesn't apply here" from the path alone
would report a correctly placed autoMode on the machine of anyone who runs it
against their home directory — including mine, which has autoMode in it. So
the scope is decided by whether the directory being scanned is your home, not by
what the path looks like.

The full list of 71, and the same list as JSON if you would rather check the
claim than trust it:

https://quintetkit.github.io/en/reference/claude-code-settings-scope.html

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

Every Test Passed and It Could Not Read a Single Real Record

quintetkit — Wed, 09 Sep 2026 01:26:53 +0000

I wrote a tool that reads GitHub Issues and reports the ones whose declared file
scopes overlap.

Eight fixtures, all six rules covered, every test green.

Run against a real repository, it read zero Issues.

What came out

Eleven Issues, one identical complaint for each:

error #21  The scope section is present but declares no path.
error #19  The scope section is present but declares no path.
error #16  The scope section is present but declares no path.
...

The section is there. The contents are not being read.

Here is what a real Issue looks like:

## Scope

```
README.md, .github/workflows/self-check.yml
```

Inside a code block, on one line, comma separated.

My parser treated each line inside a code block as one path, then discarded
anything containing whitespace on the grounds that a path has none.
README.md, .github/... contains a space, so every one of them was thrown away.

What the fixtures said

## Scope

- `src/auth/**`
- `src/routes/login.tsx`

A bullet list. One path per line, in backticks.

Why did I write them that way? Because that is how I assumed people write them.

I also wrote the Issue template those real Issues came from. My tool could not
read the format I designed myself.

Why no test failed

This is the part worth keeping.

The fixtures were written by the same person who wrote the parser, on the same
day, holding the same assumption.

parser      implemented assuming "scopes are written as a bullet list"
fixtures    written assuming "scopes are written as a bullet list"
test        the two agree, so it passes

The assumption is duplicated on both sides, so the test cannot examine it.
What it examines is that the implementation matches the fixtures.

This is not an implementation bug. It is a bug in how the tests were designed,
and no amount of coverage finds it. More coverage adds more paths — through the
same wrong premise.

The fix is not "write more tests"

Ten fixtures instead of eight would still be ten fixtures written by me, holding
the same assumption.

Add one input you did not write.

gh api "repos/owner/name/issues?state=closed&per_page=100" \
  --jq '[.[] | select(.pull_request|not) | {number,title,body}]' \
  > test/fixtures/real-issues.json

Then pin it as a regression test:

// The real Scope sections were a single line inside a code block. The parser
// as written read none of them and called every Issue scope-empty.
const real = JSON.parse(readFileSync(`${F}real-issues.json`, "utf8"))
  .map((i) => ({ id: `#${i.number}`, title: i.title, body: i.body ?? "" }));

ok(!check(real, { files }).some((f) => f.rule === "scope-empty"),
   "reads a Scope written inside a code block");

That test sits outside my assumptions, because I did not author the data. Past
reality did.

The parser side:

if (inFence) {
  // Inside a code block these are usually listed on one line, comma or space
  // separated. Real Issues were written that way, and treating the line as a
  // single path read none of them.
  for (const tok of line.split(/[,\s]+/)) if (tok) out.push(tok);
  continue;
}

The same shape again, the same day

The README said:

Requires Node 22.6 or newer

CI failed.

SyntaxError: Cannot use import statement outside a module

Without "type": "module", Node loads a .ts file as CommonJS. It worked
locally because the local Node is 25.

Same structure:

README    claims it runs on 22.6
CI        verifies on 25
verified      that it runs on 25
not verified  that it runs on 22.6 — which is the claim

The claim and the place it was checked were not the same place.

The fix is to test at the boundary you claim:

strategy:
  matrix:
    node: ["22.18", "24"]

(22.6 turned out to need a flag for type stripping, so the floor moved to 22.18.
I only looked that up because CI failed. Had it passed, the claim would have
shipped false.)

Generalising

Data you produce yourself reproduces your assumptions along with it.

The exposure is highest here:

anything that interprets input from outside (parsers, importers, webhook receivers)
anything handling text other people wrote
any format you defined yourself — the worst case, because you believe people write it the way you specified

One rule covers it:

Anything that interprets input gets one test with input you did not write.

Take one real record, freeze it, make it a regression test. That is one foothold
outside your own assumptions.

And for claims:

Verify at the boundary you claim. If it says "N or newer", test on N.

Takeaways

Fixtures carry the same assumptions as the implementation, so they cannot test those assumptions
Coverage does not help. It adds paths through the wrong premise
Add one input you did not write. Take it from production and freeze it
The riskiest format is the one you defined, because you trust it is followed
A README claim is unverified until it is tested at the boundary it claims
I only checked because CI broke. Had it stayed green, the false claim would have shipped

Related

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review
and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue
parallel execution script and a 11-chapter guide is on the
product page.

The full kit — five personas, the scripts and the complete guide — is available here.

https://quintetkit.gumroad.com/l/quintet

DEV Community: quintetkit

Bisection Narrowed It Correctly and Still Never Reached the Answer

What I tried first (all wrong)

Switching to bisection

Why it could not

The actual reason

I had the order backwards

I fell into the same hole again the same day

Automation makes you look like a spammer without meaning to

What I changed

Summary

The Page You Wrote by Hand Is the One That Stops Improving

The pages that were missing it were the most linked pages on the site

The exception you wrote by hand does not receive later improvements

Checking the output beats normalizing the input

Some annotations are worth keeping even when they cannot work

Takeaways

Write the Same Decision in Two Places and Only One of Them Gets Fixed

The page they link to said "Coming soon"

I did not recognize it as the same decision

What caught it was reading the diff before pushing

One place for the decision, separate places for the presentation

Run every combination

Takeaways

A Default That Is Safe on Create Is Destructive on Update

The safe direction of a default flips with the operation

The fix was not "infer it more cleverly"

And then: identity was keyed on a value that changes

Fixing one bug is what exposed the other

The fix: record the id the other side assigned

Verifying it

Why it was keyed on the title in the first place

Summary

The Blind Spot Was the Rule I Wrote to Avoid False Positives

The linter passed

Why the linter could not see it

The shape of this mistake

A Scheduled Job That Works and Never Runs

Version 1: a loop in /tmp

Version 2: launchd with KeepAlive

Version 3: the loop was frozen, not running

Version 4: let the scheduler schedule

One more trap on the way out

What the four versions have in common

I Removed an allow Rule and the Permission Was Still There

Lists merge. Nothing removes another file's entries

The level order, and the row that is actually useful

Environment variables are not a level

Seven keys where a lower level beats managed

Why I keep writing these down

The Workflow Ran With Four Personas Instead of Five and Looked Fine

A file in .claude/agents/ is skipped silently

Copying a persona is where this gets interesting

claude plugin validate gets you part of the way

The one that was my own fault

What I do now

I Set a 500ms Timeout and Got 28 Hours

A url with no type is read as a stdio server

A cloned repository cannot approve its own servers

The pattern across all of these

The Deny Rule I Wrote Was Never Consulted

Bash(git * main) allows every git subcommand

Whitespace decides whether lsof is allowed

Three more that load and do nothing

Why this bites harder than it looks

Checking a config

I Put a Setting in My Project Config and It Did Nothing

Settings have a scope, and the scope is not in the file

The row that will cost you a debugging session

Nested keys are scoped by their full dotted name

How I found out how wrong I was

Checking yours

Every Test Passed and It Could Not Read a Single Real Record

What came out

What the fixtures said

Why no test failed

The fix is not "write more tests"

The same shape again, the same day

Generalising

Takeaways

A file in `.claude/agents/` is skipped silently

`claude plugin validate` gets you part of the way

A `url` with no `type` is read as a stdio server

`Bash(git * main)` allows every git subcommand

Whitespace decides whether `lsof` is allowed