DEV Community: Kynth Studios

Nearly one in ten published Claude Code skills does not load

Kynth Studios — Fri, 14 Aug 2026 15:36:24 +0000

Cross-posted from The State of Agent Tooling. Dataset: DOI 10.5281/zenodo.21936490, CC BY 4.0.

43,199 of 445,348 published Claude Code artefacts fail a structural check and will not load as published. That is 9.70% of every skill, subagent, plugin and marketplace we could find on GitHub and skills.sh, read from source on 2026-08-14.

Nobody has published this number before, so there is nothing to compare it against yet. That is the reason to publish it.

What was measured

Every artefact was fetched and checked for one thing: whether an agent runtime could register it. Not whether it is good, popular, or maintained.

Kind	Listings	Fail	Rate
Skills	347,382	27,398	7.89%
Subagents	71,430	15,669	21.9%
Plugins	20,212	33	0.16%
Marketplaces	6,324	99	1.57%

Subagents are the broken half

The aggregate hides the finding. Subagents fail at 2.8x the rate of skills, and the reason is structural rather than cultural.

A subagent's name and description are Required: Yes in Anthropic's frontmatter reference. A skill's are not: a skill falls back to its directory name, and to the first paragraph of its body. The same careless file is a working skill and a dead subagent.

4,119 subagents have no name. 1,799 have no description.

Almost all of it is one defect

38,183 of the 43,199 failures — 88.4% — are a YAML block at the top of a file that does not parse.

This is the cheapest class of bug in software, and it is sitting in front of the most-copied artefacts in a fast-growing ecosystem, because nothing in the publishing path checks it. There is no registry gate, no CI convention, and no error surfaced to the author. The file is committed, the repository is starred, and the artefact silently never loads.

Cause	Listings
Frontmatter does not parse	38,183
Subagent has no name	4,119
Subagent has no description	1,799
Manifest is not valid JSON	128
Marketplace lists no plugins	99

Four things this number is not

Not a count of distinct artefacts. It counts listings. The 445,348 come from 14,233 distinct repositories, an average of 31.3 apiece, because this ecosystem is overwhelmingly forked and vendored and the same file is counted once per repository carrying it. The failure rate is the citable figure.

Not an execution result. Nothing was run. Every check is static. A listing that passes can still be wrong, useless or dangerous. Passing means the file parses into something a runtime could register: a floor, not a verdict.

Not a quality judgement. No model scored anything and no human curated the list. Every fatal cause above is a specific, checkable, arguable defect, and the row-level data is published so the argument can be had against the evidence.

Not complete. Discovery is bounded by GitHub's search API, which is rate-limited and does not return everything. What discovery truncated is logged rather than hidden.

A correction, published with the first edition

Preparing this census turned up a defect in the index that produced it.

The index carried a missing-references flag — a skill naming a bundled file that is not in its repository — and it fired on 139,623 listings, 40.2% of every skill. It was wrong roughly seven times in eight, from three bugs in one regular expression:

The extension alternation put js ahead of json, and JavaScript alternation is first-match. So every package.json was captured as a missing package.js, and every res.json(data) inside a fenced code block became a missing file called res.js. 12,178 and 6,946 listings respectively.
Nothing scoped the match to the skill's own bundle. The most-reported "missing files" in the entire index were the framework names Next.js (21,014) and Node.js (19,383), the skill's own SKILL.md (27,002), and CLAUDE.md (11,371) — a file that lives in the user's project and never in the bundle.
A reference written ./run.sh could never resolve against the repository tree, so 1,498 listings were reported missing a file that was sitting right there.

Corrected, 18,657 skills (5.37%) name a bundled file that is genuinely absent.

The flag is a score deduction and was never a load verdict, so the 9.70% headline is unaffected — it comes from the fatal causes above and nothing else.

This is in the report rather than in a changelog because an index that publishes other people's broken metadata has to publish its own.

The data

Published in full under CC BY 4.0, including every failing row, so the headline can be recomputed by anybody who wants to check it.

census.json — every aggregate, plus the exact rule that decides a failure
broken-listings.csv — 43,199 rows, one per failing listing, with its repository, path and cause
listings.csv.gz — the full population, 445,348 rows
The measurement vocabulary — load rate, drift, shipping status and skill decay, each defined as a formula over a named population
The method, written down before the numbers

Cite as: Kynth Studios (2026). The State of Agent Tooling 2026.1. Zenodo. https://doi.org/10.5281/zenodo.21936490

A 301 on a URL you already corrected means the transfer reversed — delete the correction, don't add a hop

Kynth Studios — Thu, 13 Aug 2026 18:33:17 +0000

I run a curated index of agentic dev tooling — 71 entries right now, each one a GitHub repository with an owner, a byline, an outbound link and a chunk of JSON-LD naming the author. Repositories move. GitHub doesn't 404 when they do; it 301s. So every link in the index keeps "working" while the page publishes a name the project stopped using.

The link checker that catches this is 135 lines and treats a redirect as a finding, not a pass. Alive means HTTP 200 and no Location header. That part was easy. The part that bit me was four days long and had nothing to do with redirects being hard to detect.

Corrections are applied on read, at one choke point

The manifest the site renders from is generated — it gets rebuilt from source on every sync, so hand-editing a row survives until the next regeneration and then silently reverts. And in production the rows come from Postgres, which the manifest doesn't govern at all. Two sources, neither of them a place a correction can live.

So corrections live in their own file and are applied in the read model, after both sources, by name:

return items.map((i) => {
  const c = CORRECTIONS[i.slug];
  if (!c) return i;
  return { ...i, repo: c.repo ?? i.repo, owner: c.owner ?? i.owner,
           description: c.description ?? i.description, url: c.url ?? i.url,
           note: c.note };
});

Fields are copied by name so a stray key in the JSON can't overwrite something that was never meant to be correctable. The slug is deliberately not in that list — it's the entry's public address and it's in the sitemap. Changing a slug to fix a link that merely redirects trades a redirect for a 404, which is worse.

The checker has to read through the same choke point

Here's the line that mattered, from the checker:

const items = JSON.parse(await readFile(join(ROOT, "src/data/posts.json"), "utf8"));
const { corrections } = JSON.parse(await readFile(join(ROOT, "src/data/corrections.json"), "utf8"));
for (const item of items) {
  const c = corrections[item.slug];
  if (c) Object.assign(item, { repo: c.repo ?? item.repo, url: c.url ?? item.url });
}

The obvious reason to do this: the raw manifest still carries the pre-transfer path for anything already fixed, so checking it would re-report every correction as a fresh finding forever.

The non-obvious consequence: the checker now probes URLs that only exist because I wrote them down. And those can go stale in a direction I hadn't planned for.

What actually happened

ahujasid/blender-mcp transferred to MCPBlender/blender-mcp. Correction written, note dated, checked 2026-08-08. Four days later the checker reported a 301 — on MCPBlender/blender-mcp, redirecting back to ahujasid/blender-mcp. The transfer had reversed. I re-probed both while writing this: ahujasid returns 200 with no redirect, MCPBlender returns 301 to it, and the REST record's full_name reads ahujasid/blender-mcp.

The instinct is to chase it: point the correction at the new target. That's wrong. The manifest's original value is correct again, so the honest fix is to delete the correction entirely and let the underlying value show through — which also takes the reader-facing note off the page, where it had been telling people the owner had been verified.

For four days the item page's <title>, meta description, byline, outbound link and JSON-LD author all named an owner the project no longer had, under a note saying it had been checked. A wrong fact wearing a verification date is worse than an uncorrected page, because it launders the error as diligence — and a 301 errors nowhere else in the stack, so nothing else was ever going to catch it.

The decision table the checker encodes

Probe result	What it actually means	What happens to the entry
200, no `Location`	Alive at the address published	Nothing
301 on a manifest URL	Repo renamed or transferred	Add a correction with a dated note
301 on a URL that is already a correction	The first transfer reversed	Delete the correction; never add a second hop
404 or 451	Gone or blocked	The entry comes out
429	Nothing was learned	Reported separately as unchecked — not a finding, not a pass

That last row is its own small lesson: HN rate-limits unauthenticated bursts hard, and reporting a 429 as a dead link trains whoever reads the output to ignore it — the one failure mode a checker cannot survive. "Inconclusive" is a different statement from "fine," and the script prints it as one.

The rule now lives in the corrections file's own _readme, next to the data it governs: re-run the checker every pass, and treat a finding on a corrected URL as a correction to remove.

This is how we built Agentwire, an index of agentic developer tooling: https://agentwire.kynth.studio/?utm_source=kynth-devto&utm_medium=social&utm_campaign=kynth

Make a stale number fail `next build`: a claim register whose quoted clauses verify themselves

Kynth Studios — Wed, 12 Aug 2026 22:18:00 +0000

For about a week our landing said civil penalties under the TAKE IT DOWN Act "currently approach $53,000." The FTC's own business guidance says $53,088 per violation. That is not a rounding error in the harmless direction — $53,088 is over $53,000, not approaching it. The commit that fixed it counts 44 instances across 8 files.

The reason it survived is the interesting part. We already had FACTS.json — a register of every claim the site makes that can decay, each with the primary source, the date it was last re-fetched, a verbatim quote from that source, and an appearsOn list of the files that state it. The penalty claim's appearsOn listed two guides. The landing was not in it. So the re-verification pass checked the two files it was told about, found them correct, and moved on.

A register that records where a fact appears is a list somebody has to keep right. The fix was to make pages read the register instead.

`claim()` is the cheap half

export function claim(id: string): Claim {
  const found = CLAIMS.find((c) => c.id === id);
  if (!found) throw new Error(`FACTS.json has no claim with id "${id}"`);
  if (found.status !== "verified") {
    throw new Error(`FACTS.json claim "${id}" is "${found.status}", not verified — it cannot be published`);
  }
  return found;
}

A typo fails the build rather than rendering an empty cell, and an entry parked mid-rotation — ours currently has one, a law-firm article behind a Cloudflare challenge that 403s both curl and a fetch, carried as unverifiable — can never reach a buyer.

`quote()` is the half that actually bites

The problem value(id) doesn't solve: a comparison table often wants half of a registered claim. One of ours reads:

Receipt of a VALID removal request — and the duty is "as soon as possible", with 48 hours as the outer limit

The table's "what it means" cell wants the clause after the dash; the "on what clock" cell comes from a different claim. Retyping that clause into JSX puts a second, unlinked copy of the fact on the page, which is the exact failure the register exists to prevent.

So the clause is passed together with the claim it came out of, and containment is proved at build time:

export function quote(id: string, clause: string): string {
  const norm = (s: string) =>
    s.replace(/[''ʼ]/g, "'").replace(/[""]/g, '"').replace(/\s+/g, " ").trim();
  const registered = claim(id).value;
  if (!norm(registered).includes(norm(clause))) {
    throw new Error(
      `FACTS.json claim "${id}" no longer contains the quoted clause.\n  quoted:     ${clause}\n  registered: ${registered}`,
    );
  }
  return clause;
}

The normalisation is not cosmetic. The register and the JSX disagree about apostrophes and nothing else, so a curly-quote mismatch would fail the build for a reason that has nothing to do with the fact.

I tested the gate rather than assuming it: I edited the registered value from 48 hours as the outer limit to 72 hours, left the table cell alone, and ran pnpm build. It compiled fine, then died in Collecting page data:

Error: FACTS.json claim "tida-clock-starts-on-valid-request" no longer contains the quoted clause. quoted: the duty is "as soon as possible", with 48 hours as the outer limit / registered: … with 72 hours as the outer limit

Failed to collect page data for /, exit code 1. Not a warning, not a runtime 500 on one route — no deployable output at all.

What each tier of coverage actually protects

How a figure reaches the page	Where the link between page and source lives	What happens when the register changes	State in this repo today
Read through `value()` / `quote()`	The import — the page cannot hold a second copy	Build fails, or the cell updates itself	One component, 15 of the register's 35 claims
Hand-typed, listed in `appearsOn`	A path string in JSON, maintained by hand	Nothing fails; the next pass has to grep the listed files	`$53,088` appears 25 times across 15 files; its claim lists 14
Hand-typed, not in `appearsOn`	Nowhere	Nothing at all	How "approach $53,000" survived

The source line under the table is generated too — asOf(...ids) returns the newest verification date across every claim the table used, so the page dates itself instead of carrying a date somebody typed beside it.

Row three is now empty by construction for anything the table renders. Rows one and two are not: 25 hand-typed occurrences of $53,088 remain in prose across 15 files, still guarded only by that appearsOn list.

This is how we built TrustDesk, a hosted NCII takedown desk for covered platforms: https://trustdesk.kynth.studio

`status === 200` let 7,382 non-pages into our iframes — the four checks that replaced it

Kynth Studios — Tue, 11 Aug 2026 10:19:09 +0000

BlockDex indexes shadcn-style component registries, and each item page mounts an iframe of the registry's own demo page with a caption underneath: "this is the component running, not a screenshot of it."

On 2026-08-06 I fetched all 22,313 preview URLs the index had stored. 7,382 of them were not component pages.

Two mistakes stacked

The crawler learned a URL shape per registry — try {base}/docs/{name}, {origin}/components/{name}, and last on the list {origin}/r/{name} — against one sample item name. Whichever shape answered got filled in for every item in that registry and written to preview_url unchecked.

The acceptance test was one line:

const res = await get(url, { accept: 'text/html,*/*' });
if (res.status === 200 && (res.text ?? '').length > 500) { /* learned */ }

{origin}/r/{name} is, on a large class of registries, the item JSON route — the same URL the install command reads. A 4KB JSON body clears length > 500 comfortably. The */* in the accept header forces nothing. And then the second mistake compounded it: an API route sends no X-Frame-Options and no CSP, so our framable() helper —

if (xfo.includes('deny') || xfo.includes('sameorigin')) return false;
const m = csp.match(/frame-ancestors([^;]*)/);
if (!m) return true;

— returned true. The JSON endpoint was marked the most embeddable kind of URL there is. Chrome duly framed it and rendered its own JSON viewer inside the panel, under a caption asserting the component was running.

None of this is detectable at render time. The client watches the iframe's onLoad, and a 404 page, a 500 page and a JSON document all fire onLoad perfectly normally. The only failure a browser surfaces is a frame that never loads at all — which is the one that doesn't happen.

What the sweep actually found

Failure class	Count of 22,313	Why the old test missed it	What rejects it now
Hard 404	4,704	Never tested per item — the one sample name had a page, the other 7,794 items didn't	`res.status !== 200`
JSON body	1,588	4KB body passed `length > 500`; `/` accepted any content type	`content-type` must contain `text/html`
Soft 404	691	200 + HTML + long body — indistinguishable by status. Next.js and Astro answer a missing route with 200 far more often than with 404	`<title>` announces a missing page
Server error	399	Same as hard 404: nothing per item ever ran	`res.status !== 200`

The guard

export async function verifyPreview(url, { timeout = 15000 } = {}) {
  const no = (reason) => ({ ok: false, embeddable: false, reason });

  // `text/html` alone, no `*/*` escape hatch. A registry that answers JSON
  // to this is answering JSON to everybody.
  const res = await get(url, { accept: 'text/html', timeout });
  if (res.error) return no(`transport: ${res.error.slice(0, 60)}`);
  if (res.status !== 200) return no(`http ${res.status}`);

  const type = (res.headers?.get('content-type') ?? '').toLowerCase();
  if (!type.includes('text/html')) return no(`serves ${type.split(';')[0]}`);

  const body = res.text ?? '';
  if (body.length < 500) return no(`body is ${body.length} bytes`);

  const title = body.match(/<title[^>]*>([\s\S]{0,200}?)<\/title>/i)?.[1] ?? '';
  if (isNotFoundTitle(title)) return no(`not-found page (title: ${title.trim()})`);

  return { ok: true, embeddable: framable(res.headers) };
}

The soft-404 check reads the title and nothing else, which was not the first version. Scanning the first 4KB of body condemned two pages that render fine — Apple gallery | 100xUI and Pixel Scroll Transition – unlumen UI — because each carried the string 404 in a nav or an inlined script. Deleting a working preview is worse than letting a broken one through, because the second is visible and the first isn't. So the regex has word boundaries and lives behind unit tests built from real titles: it must catch Component not found - HextaUI and 404: This page could not be found, and must leave Error404 Illustration | someui alone.

Two deliberate asymmetries

docs_url still takes the unverified pattern; preview_url does not. A "Docs on X" chip is a link somebody chooses to follow. The frame is a claim the page makes on their behalf, at 460px, above the fold.

And verification is capped at PREVIEW_CAP=2000 items per registry. Past the cap an item keeps its page, its install command and its access verdict — it just doesn't get a frame nothing checked. After the sweep the index publishes 12,561 verified live previews, 2,858 verified links, and 43,795 items with no preview at all, and the item page's comment block now carries those counted numbers instead of the estimates it used to.

This is how we built BlockDex, a search index for shadcn component registries: https://blockdex.kynth.studio

In zsh, a second `local` on the same name prints it — how five bytes faked a six-day database outage

Kynth Studios — Mon, 10 Aug 2026 14:08:08 +0000

For six consecutive mornings, the freshness sweep behind our free directory list failed at the same place. Its preflight gate probes PostgREST before it's allowed to crawl anything, and every morning the log read:

preflight: PostgREST answered '000', not 200 — waiting 30s (retry 1/3)
preflight: PostgREST answered '000', not 200 — waiting 90s (retry 2/3)
preflight: PostgREST answered '000', not 200 — waiting 240s (retry 3/3)
preflight: PostgREST still answering '000' after 3 retries — giving up

Running the identical probe by hand answered 200 in 0.38 seconds. Same URL, same service key, same machine.

That contradiction is why it survived five days of looking at it. The machine was fine. The credentials were fine. The database was fine. So the theory became timing: a 05:10 job firing before the network came up on wake, and ~6 minutes of backoff not being enough to ride it out. The ladder got widened. It changed nothing, and got reverted.

The only tell was a byte count

The env reader returned 45 bytes for SUPABASE_URL. Parsing the same key out of the same file directly returned 40.

Five bytes. Here's the function they came from:

_lr_env() {
  for f in "$LISTRUN_ROOT/.env" "$LISTRUN_ROOT/.env.local"; do
    [[ -f "$f" ]] || continue
    local v                       # ← the bug
    v=$(grep "^$1=" "$f" | head -1 | cut -d= -f2- | tr -d '"' | tr -d "'")
    [[ -n "$v" ]] && { echo "$v"; return 0; }
  done
  return 1
}

What `local` actually does in zsh

local is typeset. And typeset on a name already declared in the current scope prints the variable instead of quietly redeclaring it:

$ zsh -c 'f(){ local v; local v; }; f'
v=''
$ zsh -c 'f(){ local v=abc; local v; }; f'
v=abc
$ bash -c 'f(){ local v; local v; }; f'
$                                        # bash: nothing

Two properties make this lethal rather than cosmetic. It writes to stdout — which for a function called as $(_lr_env SUPABASE_URL) is the entire return channel. And it prints the current value, so a loop that carried something sensitive would leak it, not just an empty string.

The reason iteration one was silent: SUPABASE_URL lives in .env.local, not .env. The first pass found nothing and continued. The second pass hit local v on a name already declared in that scope, printed v='', then found the real value and echoed it. The caller captured both lines and got:

v=''
https://xxxxxxxx.supabase.co

curl cannot resolve that. It never issued a request at all, and reported 000.

`000` is not an HTTP status

That's the second half of why this took six days. 000 is curl's "no response happened." The log line said PostgREST answered '000' — and PostgREST had answered nothing, because it was never contacted. Six mornings of diagnosis were spent on a component that was never in the request path.

So the branch now says which of the two it's looking at:

if [[ "$code" == "000" ]]; then
  saw="the request never completed (curl '000' — no response, so this is the network or DNS, not PostgREST)"
else
  saw="PostgREST answered '$code', not 200"
fi

The fix itself is local v f declared once above the loop. f joined it because it had never been declared at all and was leaking into the caller's scope.

Two things worth stealing

A retry ladder is the wrong instrument for a bug that fails identically on every attempt. Three failures at 30s, 90s and 240s producing byte-identical output is itself the diagnosis: nothing about the passage of time was touching this. Widening the ladder on the network theory only delayed finding the real cause.

Back off only for failure shapes a wait can fix. The same gate now aborts non-zero on 400/401/403/404/406 instead of retrying, because of a sibling failure: ShipWall's gate probed shipwall_board, which is a Postgres function, not a table. PostgREST answers 404 instantly against a perfectly healthy instance. The gate read that as throttling and exited 0 — so its nightly pass had never run, not once, and the job list reported success every morning.

The blast radius here was wider than the freshness stamp, because the submission worker gates on the same function. Rather than assume, I read the queue: 7 runs, all awaiting_payment, all created 08-01, and the job table empty. Nobody paid during the window, so no customer run actually stood still — but the first payment after 08-03 would have sat with nothing attempted and nothing anywhere saying so.

That preflight gate, and the morning sweep it protects, are how we built ListRun — a submission engine whose free half is a directory list that re-checks itself and publishes what it actually saw.

If your API answers "clear" from an empty result set, your retention policy is product copy

Kynth Studios — Fri, 07 Aug 2026 11:39:55 +0000

I run a free lookup that answers one question: is this nonprofit in good standing? You type an EIN, it checks the IRS auto-revocation list and California's "may not operate or solicit" list, and tells you whether a fundraising platform has to block that org's donation page.

The table behind it, gs_orgs, only holds organizations that appear on one of those lists. Which means the API's happiest answer is produced by finding nothing. The comment at the top of route.ts said so plainly:

The gs_orgs table only holds orgs that appear on a revocation/delinquency list, so "no rows" reads as good standing on the lists we track.

That's a normal, sane design. It's also a landmine, and I stepped on it the week I tried to make the table smaller.

The pruning was correct on its own terms

The ingest pulls both registries in full and had grown to 1,206,313 rows on a Postgres instance shared across the whole estate. Most of it was dead weight: the IRS file carries revocations going back years, and an org revoked in 2013 cannot use the streamlined 15-month reinstatement path this product exists to catch. It's a full re-application, not a buyer.

So the ingest got a retention rule. The relevant part is seven lines:

keep_since = os.environ.get('GS_KEEP_REVOKED_SINCE', '2020-01-01')

def worth_keeping(r):
    if r.get('ca_reg_no') or r.get('ca_registry_status'):
        return True                      # CA-listed: AB 488 buyer at any age
    if r.get('reinstatement_date'):
        return False                     # back in good standing; "clear" is correct
    return (r.get('revocation_date') or '') >= keep_since \
        or (r.get('revocation_posting_date') or '') >= keep_since

331,377 rows survived out of 1,206,313. The table went from 261 MB to 81 MB, and the outreach outputs the product actually acts on were untouched — 716 IRS postings in the trailing 45 days, 8,219 California delinquent or suspended orgs, same as before.

Every number there says this was a good change. And it broke the API without touching a line of API code.

Deleting a row edits a sentence

Look at the two False branches and notice they are not the same kind of false.

The reinstatement_date branch drops a row because the org got its status back. Absence is accurate for that org — the answer "clear" is the answer. Fine.

The date-cutoff branch drops rows that are still revoked. Nothing about their status changed; only my willingness to store them did. But the API has no way to tell those two absences apart. It sees no row and prints the same thing.

So a 2013 revocation — still revoked, still a real problem, still something a fundraising platform is on the hook for — came back from the lookup as clean. Not as an error, not as "unknown". As a confident green panel.

The tempting fix is to unprune. That's wrong: the rows genuinely aren't worth storing, and keeping 874,936 of them to protect one sentence is an absurd trade.

The actual fix is to make the sentence smaller. The clear-state copy used to read:

No IRS auto-revocation and no California delinquency on record for that EIN.

It now reads:

No IRS auto-revocation posted since 2020, and no California delinquency on record for that EIN.

The second version is a claim the data can still support. The first one wasn't true even before the pruning — it just happened to be less wrong.

Keeping them tied together

The failure mode from here is drift: someone widens GS_KEEP_REVOKED_SINCE to 2015, or narrows it to 2022, and the copy stays frozen at "since 2020". Two files, one invariant, no compiler between them.

There's no clever mechanism for this — you can't type-check a sentence against an environment variable. What exists instead is a warning at both ends. The ingest carries a ⛔ block above keep_since saying the cutoff is load-bearing for /api/lookup and that widening it means widening the copy. route.ts carries the mirror image, naming the env var and the component that renders the claim. Whichever file you open first, it points at the other two.

That's a weaker guarantee than a test, and I'd take a test if one were possible. What makes the comments work is that they're specific enough to act on: not "careful here", but the variable name, the file name, and the exact consequence.

The general shape

Any table that stores only exceptions has this property. Blocklists, incident tables, audit-failure tables, sanctions screening, fraud flags — the whole design is "presence means bad, absence means fine," and the moment absence carries meaning, your retention window becomes part of your product's claim. Compaction, TTLs, partition drops, a cheeky DELETE FROM ... WHERE created_at < ... to reclaim disk: all of them silently rewrite what your API is asserting to a user.

Before you prune one of those tables, go find the string that renders when the query comes back empty. That string is a spec. Either the deletion has to respect it, or it has to change with it.

That's how we built the free EIN lookup in GoodStanding — try it on an org you care about, and read the caveat, because it means exactly what it says.

`new Date()` in your Next.js sitemap makes Google discard lastmod — export the page's own date instead

Kynth Studios — Thu, 06 Aug 2026 11:34:16 +0000

I shipped a guide site whose sitemap.ts ended with the most natural line in the world:

{ url: `${SITE}/guides/cra-sbom-requirement`, lastModified: new Date() }

Every URL, every build. Which meant a commit that only touched brand tokens and a motion curve announced to Google that all four legal guides had been rewritten that afternoon. Nothing on those pages had changed in a week.

Why that's worse than no lastmod at all

Google's sitemap documentation says it uses <lastmod> "if it's consistently and verifiably (for example by comparing to the last modification of the page) accurate", and that the value "should reflect the date and time of the last significant update to the page."

A sitemap where nothing is ever older than the last deploy fails both halves. The crawler fetches a page it was told changed, diffs it against what it already has, finds nothing, and learns that this host's dates mean nothing. Then it stops reading them.

That's a cheap loss on an established domain. On a three-week-old host with no inbound links — where three of five guides were still unknown to Google — lastmod is close to the only recrawl signal you own.

One constant, three consumers

The date was already on the page twice: rendered as a visible "Last verified" line, and emitted as dateModified in the page's Article JSON-LD. The sitemap was inventing a third value instead of reading the one that existed.

So the page module exports it, and the sitemap imports it:

// guides/cra-sbom-requirement/page.tsx
export const UPDATED = "2026-07-29";

// sitemap.ts
import { UPDATED as SBOM_UPDATED } from './guides/cra-sbom-requirement/page';

export default function sitemap(): MetadataRoute.Sitemap {
  return [
    { url: `${SITE}/guides/cra-sbom-requirement`,
      lastModified: SBOM_UPDATED, changeFrequency: 'monthly', priority: 0.9 },
    // static pages carry the date of their last real content change, bumped by hand
  ];
}

Next.js accepts an ISO string for lastModified, so no conversion is needed. Static pages that have no UPDATED get a hand-maintained constant with a comment explaining what change earned the bump. The landing page's says 2026-08-04 because a factual correction rewrote the FAQ, its JSON-LD and two components — not because something was deployed that day.

The off-by-one that only surfaced once the dates converged

Collapsing three values into one immediately exposed a bug nobody could have seen while they were independent. The visible date rendered like this:

new Date(updated).toLocaleDateString("en-US", { year: "numeric", month: "long", day: "numeric" })

new Date("2026-08-01") parses date-only strings as UTC midnight. Format that in any negative-offset zone and you get the previous day. Every guide had been showing a "Last verified" date one day behind the dateModified in its own Article schema — human-readable text contradicting structured data on the same page, in a way that only reproduces west of Greenwich. The fix is timeZone: "UTC" in the options bag, with a comment saying it's load-bearing so nobody tidies it away.

The sitemap bug was in sitemap.ts. The date bug was in the article renderer. You find the second one only after the first forces both surfaces to read from the same source.

The harder half: what licenses a bump

Once dates are honest, "should this move?" becomes a real decision. The site makes 34 dated claims about a regulation — deadlines, penalty routes, filing channels — and each lives in a register with the primary source it was fetched from, an asOf, a lastVerified, and a watch note.

The dangerous category is claims that assert something has not happened. One is flagged HIGHEST-DECAY CLAIM IN THIS REGISTER: whether the Commission has adopted an implementing act specifying the SBOM format. It can go false while the repo sits completely still, and if it does, an entire section of a guide is wrong that morning.

So re-verification happens on a schedule the content doesn't. Last check: source fetched, HTTP 200, still nothing adopted. lastVerified in the register moves. The page's UPDATED does not — and therefore neither does lastmod. Verifying is not updating.

The rule that came out of it: a stale-but-true date is worth more than a fresh lie. If you can't name the content change behind a bump, don't bump it.

That register and these guides are how we built PartsProof: https://partsproof.kynth.studio

731 words or 2,042? Why an aggregate's population belongs in the view, not the query string

Kynth Studios — Wed, 05 Aug 2026 15:30:29 +0000

I maintain a corpus of agent instruction files scraped from public repos — AGENTS.md, CLAUDE.md, Cursor rules and friends — and last week I added the page answering "what actually goes in a CLAUDE.md," counted rather than recommended.

The first thing that fell out was two different answers to median length:

basis	files	median words	share with runnable commands
every `CLAUDE.md`	775	2,042	34.2%
repo-root `CLAUDE.md` only	201	731	79.6%

Both are correct. The 574 non-root files — the per-package ones sitting in packages/api/CLAUDE.md — have a median of 2,276 words, which is the opposite of what I'd have guessed. Whatever the cause, they are a different population answering a different question, and someone asking "what should my CLAUDE.md contain" means the one at the root.

So the interesting part isn't the number. It's that the choice of population is a schema decision, and I'd been treating it as a query parameter.

Rule 1: the filter that defines the population goes in the view

The tempting design is /api/sections?root=true. Then the default is wrong, or the caller forgets, and a percentage computed over 775 files gets printed under a heading that says "a CLAUDE.md file." Nobody will ever notice, because the number looks fine.

create or replace view rs_section_stats as
with base as (
  select format, count(*)::int as total
  from rs_configs
  where hidden = false and is_root
  group by format
)
select c.format,
       t.tag                                                 as section_tag,
       count(*)::int                                         as config_count,
       b.total                                               as root_total,
       round(100.0 * count(*) / nullif(b.total, 0), 1)::real  as pct_of_root
from rs_configs c
cross join lateral unnest(c.section_tags) as t(tag)
join base b on b.format = c.format
where c.hidden = false and c.is_root
group by c.format, t.tag, b.total;

is_root appears twice and is optional in neither place. There is no way to call this view and get the other population. The name says pct_of_root, so a column that escapes into a page carries its own basis in the label.

Rule 2: ship the denominator on every numerator row

root_total is repeated identically on all twenty rows, which is redundant on purpose. The alternative is a page that fetches counts here and the total from somewhere else — and "somewhere else" drifts. A different hidden filter, a different date window, a count over the list endpoint that hit a row cap. A percentage with the wrong denominator is the kind of wrong that looks right.

Rule 3: two bases on one page is an invitation to divide across them

I already had a stats table publishing medians over all files. Rather than reuse it, I added a second small view, rs_root_shape, computing length and heading counts on the same root-only basis. It exists purely so a reader — or a future me — cannot pair a median from one population with a share from another in adjacent paragraphs. Duplicating an aggregate is cheaper than the errata.

Rule 4: `GROUP BY` cannot emit a zero

The classifier has a fixed vocabulary of 20 section tags. A group-by returns only tags that some file carries, so for windsurf-rules the view returns 13 rows and for .cursorrules it returns 19. Render that as "the tags" and you've silently claimed the missing 7 aren't categories at all — when "no file in this corpus has a security section" is one of the more interesting things you could say. The endpoint returns sections, plus vocabulary: every tag the classifier can assign. Absence is an answer, and it has to be carried out-of-band because SQL won't produce it.

Why this is a view and not application code

section_tags is a text[]. Getting a share out of it needs an unnest and a group-by — the one aggregate a list endpoint structurally can't serve. /api/configs?tag=build returns the files with a build section, never the fraction. Computing it in the page meant paging a whole format at the handler's 100-row ceiling: eight sequential reads of thirty-eight columns to produce twenty integers. One view, one read.

For the record, the finding that motivated the page: build commands come first in Anthropic's list of what to keep in a CLAUDE.md, and 39.8% of root files have a build section. Hard prohibitions are the cheapest line anyone can write, and 48.3% have those.

This is how we built RuleStack, a measured comparison hub for agent config formats: https://rulestack.kynth.studio

A failed read must throw, not return 0 — how Next.js ISR bakes your fallback into the cache

Kynth Studios — Tue, 04 Aug 2026 11:17:50 +0000

I run a directory site: ~3,500 rows in Postgres, read over PostgREST, rendered by Next.js with revalidate on every route. The headline number in the shell says "Search all 3,516 listings."

For about a day it said "Search all 0 listings" — next to a grid full of listings.

The database was fine by then. Three separate bugs conspired, and all three are the same mistake wearing different clothes: a read that degrades politely on failure, sitting underneath a cache that stores the polite answer.

Bug 1: the cache dropped the header the count lived in

PostgREST returns an exact row count in the Content-Range response header if you ask for it, which is far cheaper than selecting rows and counting them:

const res = await fetch(`${URL}/rest/v1/${table}?${query}&select=id&limit=1`, {
  headers: { apikey: KEY, Prefer: 'count=exact', Range: '0-0' },
  next: { revalidate: 3600 },      // ← the bug
});
const total = Number(res.headers.get('content-range').split('/')[1]);

Next's fetch cache stores the response body and replays it behind a synthetic set of headers. content-range is not one of them. So the first call worked, and every cache hit after it read null and produced 0.

Statically prerendered pages looked perfect — their single fetch happened at build time and was always a miss. Every dynamic route showed zero. If you're caching a fetch for anything that isn't in the body, cache the parsed value instead: mark the fetch cache: 'no-store' and wrap the function in unstable_cache. Cache things that survive being cached.

Bug 2: the zero got memoised for an hour

With the count now memoised, the failure path mattered. The read returned 0 when the response wasn't OK — sensible-looking, and unstable_cache dutifully stored that 0 for the full hour.

So the wrong number outlived the outage. The index came back; the site kept saying zero, and looked completely healthy while doing it. For a directory, "0 listings" isn't a degraded state, it's a false claim about the world.

Fix: throw inside the cached function. unstable_cache stores nothing on a throw, so the next request re-reads and the site heals the instant the database answers.

Bug 3: the page containing the zero was also cached

Here's the one I got wrong twice. The outer catch still returned 0, on what felt like solid reasoning: this zero isn't memoised, so it lasts exactly as long as the outage.

It missed the other cache. Every route declares revalidate. A background revalidation that renders "Search all 0 listings" writes that sentence into the ISR page cache and serves it for an hour on the landing and a day everywhere else. Not caching the zero doesn't help when the page containing the zero is the cached artifact.

The worst instance was /opengraph-image, which drew "0 skills · 0 subagents · 0 plugins" onto the share card and pinned it to every link preview for a day. A share card is the one surface where a wrong number gets screenshotted and outlives your cache entirely.

So it rethrows. Throwing is what makes Next keep the last good copy.

The rule that fell out

Not every read should fail loudly. Two here must degrade: the footer's freshness line, which appears on ~3,500 routes and is one sentence of fine print, and the build-time list behind generateStaticParams, which only decides what gets prerendered. Those go through a separate helper with AbortSignal.timeout(5000) — a plain fetch has no timeout, and a try/catch cannot catch a hang. The footer says "rebuild pending," which is honest, and the build finishes.

Everything else fails. The test is simple: if the page's whole reason to exist is that data, an empty render is a lie, and a lie is what gets cached.

The same family of bug shows up without any cache involved. PostgREST caps a response at 1,000 rows no matter what limit you send. A category count computed in JavaScript over limit=20000 therefore summed the first 1,000 of 3,369 rows and published "1,000 listings" — a suspiciously round number that was precisely the cap. The sitemap did it too: 1,070 URLs emitted against 3,516 real ones, silently dropping the entire long tail. Both now read a Postgres view that does the aggregation server-side.

Truncation, a missing header, a caught exception. Every one of them returns a number rather than an error, and a number is believed.

This is how we built SkillWorks, a scored index of Claude Code skills and subagents: https://skillworks.kynth.studio

Severity belongs to (check, kind), not to the check: fixing 512 false "will not load" verdicts

Kynth Studios — Mon, 03 Aug 2026 11:14:03 +0000

I run an index that scrapes published Claude Code artifacts — skills, subagents, plugins, marketplaces — scores them on four measured components, and publishes a page whose h1 is literally "These will not work."

That page is the whole product. It is also the only place where a bug costs someone else something. A checker's two error directions are not symmetric: a missed defect is a gap, but a false positive is a public accusation against a stranger's repo. Last weekend I found three of them in the same scorer, and they were all the same mistake wearing different clothes.

1. `?? 0` turned a shape mismatch into a verdict

The marketplace check is one line: a marketplace manifest listing no plugins can't install anything.

const n = a.extra?.plugin_count ?? 0;

extra is a shape assembled on the way into the database — after the scorer runs. The artifact the scraper hands the scorer carries plugin_count at the top level. So the read was undefined, ?? 0 made it a confident zero, and every marketplace in the index — all 55 — published a fatal "empty marketplace, will not load." Including anthropics/claude-plugins-official, which lists 276 plugins.

Optional chaining plus a nullish default is how a missing field becomes a measurement. The property access can't fail, so nothing ever throws, and the wrong answer is the plausible one.

2. The same check is not the same severity on every kind

The bigger one. The scorer flagged no-name and no-description as fatal on anything with frontmatter. Reading Anthropic's own references (skills and sub-agents, 2026-08-02):

	`name`	`description`
subagent	Required: Yes	Required: Yes
skill	Required: No — "defaults to the directory name"	Recommended — "if omitted, uses the first paragraph of markdown content"

A skill with neither field still registers. It takes its command from the directory and its routing text from whatever the first paragraph happens to say. That's worse than a written description — it is not a loading failure.

I had generalized the subagent rule across both kinds, so 494 skills, 6 plugins, 2 marketplaces and 10 subagents sat under "these will not work." The fix is that fatality stopped being a property of the flag:

const FATAL_ALWAYS = new Set(['broken-frontmatter', 'broken-manifest', 'empty-marketplace']);

/** Required frontmatter on a subagent; documented as optional on a skill. */
const FATAL_IF_REQUIRED_FIELD = new Set(['no-name', 'no-description']);

/** Does this flag stop the artifact registering, given what kind it is? */
export function isFatal(flag, kind) {
  if (FATAL_ALWAYS.has(flag)) return true;
  if (FATAL_IF_REQUIRED_FIELD.has(flag)) return kind === 'subagent';
  return false;
}

The flags themselves survive — a skill with no description is still genuinely harder to route to, and it still takes a deduction (8 points instead of 30). What changed is that a deduction stopped being a verdict.

archived came off the fatal list in the same pass. An archived GitHub repo is read-only, not unloadable; the files are there and the skill still installs. It was also already the steepest input to the Maintained component, so it was being charged twice.

3. Thresholds you remember instead of read

long-description fired above 1024 characters, with the reason string "the practical ceiling is ~1024." Nobody wrote that number down anywhere. The documented cap is 1,536 characters, over description and when_to_use combined, applied to the listing text to reduce context usage. A remembered number was penalizing descriptions the documentation explicitly allows.

That one's now in a FACTS.json register: every claim the product makes about the outside world, with the primary source URL, the date it was read, and which files depend on it. Internal arithmetic — score weights, grade boundaries — deliberately stays out. Those are the method, not claims about the world.

The check that would have caught all three

Every one of these had the same tell, and it's cheap to look for: if 100% of a category fails your checker, the checker is what's broken. Fifty-five out of fifty-five marketplaces empty. Effectively every skill on the planet missing a required field. That is never a fact about the corpus.

The generalizable version, for anything that grades other people's work: encode the bar as a sentence in the code next to the list (mine is "Claude Code cannot register the artifact — nothing softer qualifies"), keep severity keyed on the pair, not the check, and treat a suspiciously round failure rate as a bug report against yourself.

This is how we built SkillWorks — a scored index of every published Claude Code skill, subagent and plugin.

Commit every raw vendor response: making a benchmark anyone can re-score without an API key

Kynth Studios — Fri, 31 Jul 2026 11:06:52 +0000

I published a benchmark of document-extraction APIs where one of the entrants is my own. That is a credibility problem you cannot write your way out of. The only fix is to make the numbers recomputable by a stranger who trusts nothing I say — including on the runs where my product loses.

The design decision that makes that possible is small: the runner never scores anything it just fetched.

Split the run in half at the raw response

Every vendor call writes one verbatim JSON file and stops there:

// results/raw/<vendor>/<dataset>/<docId>.json
const rec = { vendor, dataset, docId, at: new Date().toISOString(),
              latencyMs: r.latencyMs, raw: r.raw };
writeJson(rawPath(vendor, dataset, id), rec);

// planning a run: anything already on disk is not re-fetched
const ids = allIds.filter((id) => !existsSync(rawPath(adapter.name, ds.name, id)));

Scoring is a separate entry point (--replay) that reads those files and never touches the network. All 1,210 raw responses are committed. Anyone can clone the repo, run pnpm replay && pnpm report, and regenerate the README tables with zero credentials and zero spend.

Three things fall out of that split, and only one of them was the goal.

Reproduction is keyless. That was the point.

Fixing a scoring bug is free. Metrics are where a benchmark is most likely to be wrong, and re-running 500 paid API calls to correct a normalizer is the kind of cost that quietly discourages fixing it. Here a metric fix is a replay.

Crashed runs resume for free. That existsSync filter was written for idempotency and turned out to be the thing that saved the run, because of what happened next.

The stall was not where I was looking

A full run wedged. No error, no exit, just a process sitting on one document forever. My first instinct was rate limiting, and I went looking at retry logic.

The retry logic was fine. It just never ran. My retry wrapper does three attempts with backoff, and the AWS SDK client had maxAttempts: 3 on top of that — but retries only fire when something fails, and a hung socket does not fail. It sits. Every layer of resilience I had was downstream of an error that was never going to arrive.

So the fix was deadlines, not retries. Every fetch in every adapter got signal: AbortSignal.timeout(120_000), including the 3-second job-poll loop, which got its own shorter one. The AWS client got an explicit NodeHttpHandler({ connectionTimeout: 10_000, requestTimeout: 120_000, socketTimeout: 120_000 }). Two commits, one for the fetch calls and one for the SDK client, because the SDK does not respect the same mechanism and I found it as a second stall a half hour later.

If you take one thing from this: a hung request is not a slow request, and no amount of retry configuration converts one into the other. The timeout is what gives maxAttempts something to attempt.

Pre-register the subset before the first call

Selection is the easiest place to cheat, usually without meaning to. So the subset ID lists were committed before the adapters could bill anything, generated by seeded shuffle with the seed in the manifest (20260710). The methodology fixes one rule that costs me points: no document is excluded after seeing results. A vendor failure is a failed document, not a dropped one.

Headline metrics are exact match after type-aware normalization — amounts within 0.005, dates to ISO, strings NFKC-lowercased with punctuation stripped. ANLS (fuzzy Levenshtein similarity) is computed, reported, and never allowed into a headline number, because a similarity threshold is a dial and I would be the one turning it.

What that honesty actually cost

On invoices my API leads at 99.4% field accuracy against 93.1% and 92.2% — at a median 12.8s per document versus Textract's 1.8s. On CORD-v2 line items it gets 41.9% F1 against Textract's 77.1%. On FinTabNet tables it scores 0.791 TEDS against 0.836, with 14 of 100 documents failing outright on model returned unparseable output.

Those last two are in the README, in bold, in the same table style as the wins. Publishing them is the only reason the first number means anything.

The extraction API benchmarked here is Kynth Core, and this is how we built it: https://kynth.studio

The date solver behind a six-document compliance packet: why one backward pass isn't enough

Kynth Studios — Thu, 30 Jul 2026 13:02:44 +0000

Here is the thing we actually ship: a single PDF packet, 30-odd pages, that a small public housing authority hands to its board and its auditor.

Front half is a redline. Their own ACOP (Admissions and Continued Occupancy Policy) and HCV Administrative Plan chapters, with the HOTMA-driven changes marked up and the agency's discretionary elections filled in — asset thresholds, hardship relief, de minimis error policy. Back half is the procedural record that proves the redline was adopted properly: the public notice, the Resident Advisory Board agenda and consultation memo, the hearing script, the board resolution, and a dated adoption memo.

The redline is the part people expect to be hard. It wasn't. The calendar was.

Six documents that quote each other

None of these documents stand alone. The resolution recites the hearing date. The hearing script cites when the notice was published and when comments closed. The adoption memo cites the resolution number and the vote date. The RAB memo cites the consultation meeting and the comment window it sits inside.

So a packet with one wrong date isn't a packet with one typo. It's a packet where the resolution attests to a hearing that, according to the notice, hadn't happened yet. That's exactly the kind of thing an auditor pulls a thread on.

The fix isn't better prompting. It's refusing to let dates be prose at all.

Milestones declare constraints, not dates

Every artifact in the packet declares what it depends on and by how much. Nothing declares when it happens.

const MILESTONES = [
  { id: 'notice',        after: [] },
  { id: 'rab_consult',   after: [{ of: 'notice',        minDays: 10 }] },
  { id: 'comment_close', after: [{ of: 'notice',        minDays: 45 }] },
  { id: 'hearing',       after: [{ of: 'comment_close', minDays: 0 },
                                 { of: 'rab_consult',   minDays: 0 }] },
  { id: 'board_vote',    after: [{ of: 'hearing',       minDays: 0 }],
                         onlyOn: agency.boardCalendar },
  { id: 'effective',     after: [{ of: 'board_vote',    minDays: 0 }] },
]

The minDays values are config, not constants. Some come from federal rules, some from the agency's own bylaws, and they differ per agency — which is the whole reason they're data.

onlyOn is the interesting field. A board doesn't vote whenever the math says it can. It votes on its regular meeting schedule: third Tuesday, or the second Thursday of odd months, or whatever the bylaws say.

Why one backward pass isn't enough

Solving forward from today is easy. Solving backward from a fixed effective date is what the customer actually wants, because there's a wall at 1 January 2027 and the question is "when do we have to start."

Backward looks like simple subtraction until onlyOn shows up. Subtract from the effective date, land on 14 October, discover the board doesn't meet on the 14th, snap back to the 8th. You just moved the vote six days earlier — which means the hearing has to move, which means comments close earlier, which means the notice has to publish earlier. And the RAB consultation, which was comfortably satisfied at the old dates, may now be sitting on the wrong side of its own 10-day floor.

One pass gives you a schedule that looks right and quietly violates a constraint it satisfied before the snap. So the solver iterates:

let dates = seedBackward(MILESTONES, effectiveDate)
for (let i = 0; i < MAX_ITERS; i++) {
  const next = relaxBackward(MILESTONES, dates)  // re-apply every edge, re-snap onlyOn
  if (sameDates(next, dates)) return { ok: true, dates }
  dates = next
}
return { ok: false, reason: 'no fixpoint' }

Each pass only ever moves dates earlier, so it's monotone and it terminates. It's a small fixpoint loop, not a real constraint solver, and that's fine — the graph has six nodes.

Infeasibility is a result, not an error

The loop can push notice into the past. That's not a crash. It's the single most useful thing the packet can tell a small PHA in late 2026: you cannot adopt this on your normal meeting calendar.

So that branch doesn't throw. It renders a different packet — one built around a special meeting, with the notice-of-special-meeting document swapped in and the resolution's recitals rewritten to reference the bylaw provision that authorizes it. Same elections, different procedural path.

Prose last

Once dates exist, generation is narrow. The model's job is to pick the discretionary election given the agency's profile and write the reasoning around it. Every date and every cross-reference is a token substituted from the solved schedule after the fact. The resolution and the hearing script can't disagree, because neither of them ever wrote a date.

That's the reusable bit. If your output is several documents that cite each other, the citations belong in a data structure that gets solved once, not in six generations you then hope agree.

This is how we built QuorumFile — flat-fee HOTMA adoption packets for small PHAs: https://quorumfile.kynth.studio