DEV Community: Ofri Peretz

ESLint Ships in 459 KB, So My Blog Posts Lint Your Code

Ofri Peretz — Wed, 02 Sep 2026 13:07:33 +0000

Every article about a lint rule has the same hole in it.

I paste a snippet, I paste the finding it produces. You think: fine, but does it fire on my code? The article cannot answer — it can only ever show you someone else's code.

So I shipped the linter instead. On the JWT article there is now a Try it live button: paste your own jwt.verify call and the published rule runs on it, in your browser.

The reason is a number I did not expect.

ESLint ships in 459 KB

eslint/universal exports the Linter class with no Node dependencies in its public surface: you hand it source text and a flat config, it hands you messages back.

It is a hard floor, not a polyfill: the ./universal export first appears in ESLint 9.11.0. ESLint 9.10 and every ESLint 8 resolve it to nothing, so this recipe does not degrade on older majors — it fails at build time.

With two real security plugins bundled inside it:

	bytes
raw bundle	1,764,382
`brotli -q 11` locally	370,746
what the CDN actually sends	470,563

Quote the last row, not the second. I first published 362 KB, having confirmed content-encoding: br — which proves the encoding, not the size. The CDN compresses on the fly below -q 11, so the honest number is 459 KB. Same artifact byte for byte; only the compressor differs.

Still the whole argument. At several megabytes you write a blog post about the rule. At 459 KB you ship the rule.

The recipe

The worker holds the linter and never talks to a server:

import { Linter } from "eslint/universal";
import jwt from "eslint-plugin-jwt";

const PLUGINS = { jwt }; // enumerated, never dynamic
const linter = new Linter();

self.onmessage = ({ data: { id, code, pluginId, rules } }) => {
  const findings = linter.verify(code, {
    plugins: { [pluginId]: PLUGINS[pluginId] },
    languageOptions: { ecmaVersion: 2024, sourceType: "module" },
    rules,
  });
  self.postMessage({ id, findings }); // no network, ever
};

The build step is where the real work is — aliases and a banner:

buildSync({
  entryPoints: ["src/workers/lint.worker.ts"],
  outfile: "public/lint-worker.js",
  bundle: true,
  minify: true,
  format: "iife",
  platform: "browser",
  // A worker global has no `process`. ESLint touches more of it than
  // NODE_ENV — cwd, platform, emitWarning — so stub it in a banner.
  banner: {
    js:
      'var process={env:{NODE_ENV:"production"},' +
      'platform:"browser",cwd:function(){return "/"},argv:[]};',
  },
  alias: {
    path: "path-browserify",
    "node:path": "path-browserify",
    fs: shims,
    os: shims,
    util: shims,
    // If anything in your graph pulls oxc-resolver, its native
    // bindings ride in and break the build. Rules never touch it.
    "oxc-resolver": shims,
  },
});

shims is a no-op Proxy, imported but never exercised.

The client seam is a lazy worker. The part worth copying is the failure path:

worker.onerror = () => {
  worker?.terminate();
  worker = null; // ← this line
  for (const entry of pending.values())
    entry.reject(new Error("worker failed"));
};

Nulling the singleton separates a playground that self-heals from one dead until reload. Without it every later lint posts into a corpse and the UI shows a clean result forever: "no findings" and "the analyzer is dead" look identical. So the surface renders "unknown, not clean" on failure, never an empty list.

Three traps

Don't let your framework bundle it. Asking Next.js to build the worker means teaching both webpack and Turbopack about node: schemes: two fragile configs for one artifact. Use esbuild yourself.

A real worker has no process. My spike ran under Node so ReferenceError: process is not defined only appeared in the browser. A spike passing is not the thing working.

Linting the artifact will OOM your editor. ESLint tried to lint the 1.7 MB bundle and died with Abort trap: 6: heap exhaustion, not a native crash. Add public/** to globalIgnores.

Try it

Both run published npm packages, the tarballs npm install gives you:

JWT alg:none — remove "none" from the algorithms array, watch the finding clear.
node-security — three rules on an upload handler.

Nothing you type leaves the page — and the UI says so in as many words, because pasting auth code into a stranger's site deserves an explicit answer rather than an assumption.

Reproduce it

Measured 2026-09-03 against eslint 9.39.4, eslint-plugin-jwt 2.2.14, eslint-plugin-node-security 5.2.3, esbuild 0.28.2, on Node 24.18. The build step itself inherits ESLint's floor, ^18.18.0 || ^20.9.0 || >=21.1.0 — there is no Oxlint variant of this recipe, because eslint/universal is ESLint's own export.

# from the repo root
node apps/blog/scripts/build-lint-worker.mjs
wc -c < apps/blog/public/lint-worker.js                  # 1764382
brotli -q 11 -c apps/blog/public/lint-worker.js | wc -c  # 370746
# what a reader actually downloads — measure this one, not the line above
curl -s -H 'Accept-Encoding: br' https://ofriperetz.dev/lint-worker.js | wc -c   # 470563

Those bytes drift as plugins ship rules; quote them with a date.

Browser-hosted linting is not new; ESLint and typescript-eslint both run excellent playgrounds. What is unusual is placement: not a destination you navigate to, but the rule the paragraph is arguing, running on your code, at the moment you wonder about it.

⭐ Star the repo if you have ever wanted docs that run instead of assert.

I Ran My Plugins Against a Competitor's Own Test Suite

Ofri Peretz — Tue, 01 Sep 2026 06:07:09 +0000

Every "we replace X" claim in the linting space is unfalsifiable, including the ones I have made. The competitor's rules are a list on a README, so comparison collapses into counting rule names.

But eslint-plugin-security ships its test suite inside the npm tarball. That is a corpus: 84 code samples its authors wrote specifically to be caught — and a corpus someone else wrote is the only kind that can embarrass you, which is the whole problem with benchmarking your own tool.

So I stopped counting rules and ran their tests against my plugins.

The method

Their tests use ESLint's RuleTester. Rather than parse samples out of the source, intercept the runner and collect them:

const { RuleTester } = require("eslint");
const captured = [];
RuleTester.prototype.run = (name, rule, tests) => {
  for (const t of tests.invalid ?? []) captured.push(t.code ?? t);
};
for (const f of fs.readdirSync("node_modules/eslint-plugin-security/test/rules"))
  require(`.../test/rules/${f}`);

84 samples. Then lint each one twice — once with their plugin, once with mine — and count which ones produce at least one finding.

The result

	flags
eslint-plugin-security (on its own tests)	71 / 84
my plugins	51 / 84

They win, and the honest headline is that I cover 61% of their corpus. But the distribution matters more than the total, because 29 of my 33 misses are a single rule:

 29  detect-buffer-noassert
  1  detect-disable-mustache-escape
  1  detect-no-csrf-before-method-override
  1  detect-pseudoRandomBytes
  1  detect-unsafe-regex

Excluding that one rule, it is 51 of 55 — 93%.

The obsolete rule

detect-buffer-noassert flags buf.readUInt8(0, true). The noAssert argument told Node.js to skip offset validation, so a read could run past the end of the buffer.

It was removed in Node 10. On Node 24:

const b = Buffer.alloc(4);
b.readUInt8(0, true);   // extra argument ignored
b.readUInt8(99);        // still throws ERR_OUT_OF_RANGE

The parameter has done nothing for about seven years. Their rule still ships in recommended, and their suite tests it across 29 variants — every read* method — which is why one dead rule dominates the gap.

I am not claiming that is wrong of them. Removing a rule breaks configs, and the cost of keeping it is nearly zero. But it is the difference between "61% coverage" and "93% coverage of everything that can still bite you," and no rule-name comparison would ever surface it.

What I catch that they miss

The reverse direction is the part I did not expect. Running their corpus through their own plugin leaves 13 samples unflagged, and my rules catch all 13:

  9  detect-non-literal-fs-filename
  4  detect-child-process

Those are their test cases, written for their rules, that their current implementation does not fire on. Combined, the two plugins flag 84 of 84 — every sample is caught by someone.

Where I lose

Corpus coverage is one criterion and it is the flattering one, so here is the criterion that is not.

On their curated samples my precision looks excellent, because every sample is a real vulnerability. Pointed at ordinary code, it is a different story: I ran my Node rules over 61 files of my own automation scripts last week and got 279 findings, and every one I inspected was a false positive — a timing-attack warning on if (key === -1), a zip-slip warning on a string literal. I have open fixes for those. A corpus benchmark cannot see them, because a corpus contains no boring code.

Coverage and precision are different measurements, and a test suite only measures the first. Anyone quoting one number at you is quoting the one that flatters them — that is measurement bias with a marketing budget. I just did it too: 93% is the number I would put on a slide.

Run it yourself

The whole thing is about 40 lines and no AI, no benchmark harness, no service. Any plugin that ships its tests can be measured this way, in both directions, in a few minutes.

Point it at mine: eslint-plugin-node-security ships its tests too, and I would rather read your numbers than my own.

npm install -D eslint-plugin-node-security

Measured 2026-08-12 against eslint-plugin-security 4.0.1 on ESLint 9.39.2, with my plugins under @typescript-eslint/parser (they are TypeScript-native; the default parser under-reports them by 13 samples). Related: what ground truth caught that unit tests missed, and the maintenance question about this same plugin.

Source at github.com/ofri-peretz/eslint · packages at npmjs.com/~ofriperetz · more at dev.to/ofri-peretz.

If someone ran your test suite against a competitor, what would it show?

My 9-Reviewer AI Gate Failed Articles It Scored 9.1

Ofri Peretz — Wed, 26 Aug 2026 00:01:20 +0000

Nine LLM reviewers score every article I publish. The gate is simple: zero blockers, and a mean score above my floor.

Yesterday it failed an article it had scored 9.1. Then it failed a second one three separate times. Neither reviewer ever disagreed with the writing — all three failures were the harness misreading its own reviewers.

A perfect score, filed as a blocker

The Checklist reviewer returned 10.0 and wrote this under BLOCKERS:

no biography/family/military/location/heritage; no political position;
the numbers are incident/operational facts, so the rule doesn't bite.

That is a clean bill of health. It failed the gate.

The parser dropped lines meaning "nothing here" with /^none/i. The prompt asks reviewers to explain why nothing is wrong, so they write the explanation instead of the word — and "no biography" is not "none".

Why it survives review: the regex is correct for the string it was written against. Nobody writes a test for the sentence a model didn't emit.

The fix wasn't a longer regex. Guessing every phrasing of "nothing is wrong" is unbounded. The prompts already state the contract — fully in scope means BLOCKERS: None and SCORE: 10 — so a 10 that also reports a blocker is self-contradictory:

if (score === 10 && blockers.length) {
  improvements.push(...blockers);  // demote, never drop
  blockers = [];
}

Demoted, not deleted. If a reviewer ever scores 10 on something genuinely broken, the text still reaches a human.

The word that made good reviews look like outages

My runner treats a quota message as a transport failure, so a billing error can never be scored as a bad article. The pattern included a bare rate limit.

It was tested against the reviewer's entire output. My corpus is security articles. Reviewers say "rate limit" constantly:

· Discovery & Hook: USAGE LIMIT — SCORE: 8.0
· Voice & Agenda:   USAGE LIMIT — SCORE: 9.5
· Quality:          USAGE LIMIT — SCORE: 9.0

Three finished reviews, scores visible in the text that got thrown away. That article lost 4 of 9 reviewers and failed two batch runs before I looked. It reads as an account problem, which is exactly why it survived.

The tell separates the two cleanly: a real quota banner is the only thing on stdout. A review that merely discusses limits has a SCORE: line sitting right there.

An excluded reviewer voting anyway

One reviewer is informational — its rubric weights brand fit 40%, which structurally caps anything outside that niche. It was excluded from the gate score.

It was not excluded from blockers. So it vetoed through the back door — and what it files under BLOCKERS is its own arithmetic:

✗ Axis 1 — Content Quality: 8.5
✗ Weighted = 8.5 x 0.6 + 5 x 0.4 = 7.1

Four "blockers" on an article it had just called original and reproducible. A half-applied exclusion is not an exclusion — and a weighted composite is exactly the shape that hides one, because the arithmetic looks like a finding.

The pattern

Every one of these is the harness misreading agreement as disagreement, and all three fail in the same direction: quietly, toward rejection. A false pass is loud — something bad ships and you see it. A false fail looks exactly like a strict gate doing its job, so it can run for weeks while you assume the work is merely not good enough yet. It is the same asymmetry that makes a leaderboard wrong in the flattering direction, and the same reason measurement bias survives longest when it agrees with what you expected.

If you run an LLM as a judge, assert on the disagreements it cannot logically have: a perfect score with a blocker, a reviewer excluded from the score that still blocks, a transport error carrying a parsed result. Those are contradictions, and contradictions are testable without predicting a single word the model will say.

assert.equal(parse("SCORE: 10\nBLOCKERS:\n- none found here").blockers.length, 0);
assert.equal(parse("SCORE: 7\nBLOCKERS:\n- the claim has no date").blockers.length, 1);

Both directions. A one-sided test would have passed on all three bugs — my earlier fix for the opposite failure is what introduced the second one. That is the same lesson ground truth taught me that unit tests could not: a test written from the failure you already know about only ever proves you fixed that one.

More on the tooling behind this at github.com/ofri-peretz/eslint. The three defects above were found on 2026-08-11 across 39 gated articles.

What's the last false negative you found in your own tooling — and how long had it been running?

I Linted My Own Design System. All 8 Breaks Were Pasted In.

Ofri Peretz — Thu, 20 Aug 2026 09:23:07 +0000

Your design system has tokens. Your linter does not know they exist.

eslint-plugin-react has 104 rules and not one of them can tell you that #ffffff1f should have been var(--surface-highlight). That gap is invisible in review, because a hex value in a className looks exactly like every other string.

So I ran four rules over my own code: 401 .tsx files across a design system, a docs site, a blog, and a control-room app. 144 violations in 50 files. The design system defines 192 tokens. It also breaks them.

Raw color literals: 30 findings, and they cluster

The rule is no-raw-color-literal — any hex, rgb(), or rgba() in JSX source.

// packages/ui/src/effects/shimmer-button.tsx:102
"rounded-2xl px-4 py-1.5 text-sm font-medium shadow-[inset_0_-8px_10px_#ffffff1f]"

⚠️ Raw color literal in source — use a design token (R19) | MEDIUM
 Fix: Replace with a CSS custom property (var(--your-token)) or a Tailwind
      theme class wired to it.

Why this survives review: the hex is inside an arbitrary Tailwind value inside a class string. A reviewer scanning for color changes greps for color or bg-. Nobody greps for #.

Here is the part I did not expect. The design system package had 8 of these, and all 8 were in ambient/, effects/, and patterns/ — components I pasted in from component galleries. The core primitives had zero.

The system didn't drift. It leaked at the seam where other people's code came in.

`data-slot`: 67 findings, the biggest by far

⚠️ JSX element has data-testid but no data-slot (R6) | MEDIUM

require-data-slot fires when an element carries a data-testid but no data-slot. Both are hooks; only one is a contract. A data-testid says "a test grabs this." A data-slot says "consumers may style this." Ship the first without the second and every consumer targets your test IDs, and your next refactor breaks their CSS.

67 of my 144 findings were this one rule. It is the least dramatic and the most expensive — a data-testid standing in for a styling contract is a proxy metric with a refactor bill attached.

The pattern

A design system fails at its boundaries, not its center. The primitives I wrote while thinking about tokens obey them. The components I imported while thinking about shipping do not — and they arrive pre-broken, which means no diff ever shows the moment the violation entered.

That is why this has to be a lint rule and not a review convention. Review catches what a diff shows. Pasted code shows up as one clean green addition.

The config

// eslint.config.mjs
import reactFeatures from "eslint-plugin-react-features";

export default [
  {
    files: ["**/*.tsx"],
    plugins: { "react-features": reactFeatures },
    rules: {
      "react-features/no-raw-color-literal": "error",
      "react-features/no-arbitrary-token-class": "error",
      "react-features/require-data-slot": "warn",
      "react-features/no-inline-style": "warn",
    },
  },
];

npm i -D eslint-plugin-react-features

eslint-plugin-react-features on npm · all four rule docs

Start the last two at warn. On a codebase that has never had these rules, require-data-slot alone will be your largest number, and turning it red on day one just teaches people to disable it.

An honest caveat: 47 of this plugin's 61 rules overlap eslint-plugin-react by name. These four do not — nothing in the incumbent enforces a token system, because the incumbent cannot know what your tokens are.

Source at github.com/ofri-peretz/eslint. Every number here came from a run on 2026-08-11 against the four repos named above — four repos I own, which is a sample of one opinion about components, not a ground-truth corpus. I ran my own rule over my own code and reported what it found, which is the only version of benchmarking I trust myself to do. Re-run it on yours and you will get different numbers.

Run one of these on your own components — what's your data-slot number? Mine was 67, and I wrote the rule.

Your Token Is Not Safer in a Cookie. It Is Safer From JavaScript.

Ofri Peretz — Fri, 14 Aug 2026 09:43:04 +0000

Every team I have worked on has had the localStorage argument. Someone says tokens belong
in cookies. Someone else says cookies mean CSRF. Both cite a blog post. The token stays
where it was.

The argument is unwinnable as stated, because both sides are describing the container —
and the container is not what decides the outcome.

The axis everyone argues about

Ask why one mechanism is safer and you get answers about persistence, capacity, expiry,
whether it survives a tab close. Real differences. None of them has anything to do with an
attacker.

The axis that decides it

One question: can JavaScript running on your origin read it?

If yes, anything achieving script execution on your page reads it too. Not encryption —
you have nowhere to put the key. Not obfuscation — the attacker has your bundle. Not "we
only write it after login" — the attacker runs after login too.

Mechanism	Readable by page JS	Under XSS
`localStorage`	yes	gone
`sessionStorage`	yes	gone
IndexedDB	yes	gone
Cache API	yes	gone
Cookie without `HttpOnly`	yes	gone
Cookie with `HttpOnly`	no	survives

Five of those six rows are the same row. The debate spends its energy on a distinction
that exists only in the last line — and it is not the cookie that creates it, it is the
flag.

`HttpOnly` is the boundary

HttpOnly is unusual: a capability the browser withholds from your own code. You cannot
opt back in at runtime. No API reads it, no devtools trick your bundle can perform. The
value goes out on requests to its domain and is otherwise unreachable.

Almost nothing else in the browser works this way. Everything in the first five rows is a
filing cabinet in a public lobby — useful, convenient, not a vault, and never described as
one by anyone who built it.

The corollary people skip

HttpOnly protects the token from being read. It does not stop it being used.

Script on your origin can still fetch with credentials: "include", and the browser
attaches the cookie. The attacker does not hold your session; they operate it, from your
page, while it is open. Smaller blast radius — no exfiltration, no replay next week — but
not nothing.

The honest summary: HttpOnly converts permanent theft into temporary misuse. Real
and worthwhile. Not immunity.

So what counts as sensitive

Stop asking "is this sensitive" — a word that invites negotiation. Ask: if an attacker
had this value, what could they do, and for how long?

Session token — act as the user until expiry.
Refresh token — mint new sessions, far longer. Higher risk, and the one most often parked in localStorage because it needs to persist.
Cached API response with personal data — disclosure, not takeover. The Cache API is the mechanism people forget is storage at all.
Feature flag, theme, draft — nothing. Put it anywhere.

And the CSRF half?

Never on the same axis either. CSRF is the browser attaching a cookie to a request the
user did not intend — a property of ambient authority, not storage — and SameSite
addresses it directly.

That is why the debate feels unresolvable: one camp describes a read primitive, the other
a send primitive. HttpOnly answers the first, SameSite the second — orthogonal flags on
the same cookie.

Where this lands in the standards

Reading storage you should not have access to is
CWE-522 — insufficiently protected
credentials — under
A07:2021.
HttpOnly and Secure are specified in
RFC 6265 §4.1.2.5–6;
SameSite arrived later, in
RFC 6265bis, still
a draft despite universal implementation. Both are on
MDN's Set-Cookie page.
For severity language, use
CVSS and
the OWASP categories.

Why the argument keeps happening

All five equivalent rows are reached the same way: script execution on your origin. So
storage is downstream of the XSS question, and a team that has not settled how untrusted
data reaches a DOM sink is arguing about where to put the token while leaving the door
open. DOM sinks and sources, the
companion piece in this series, works through why those sources outnumber the sinks.

What is in your localStorage right now? Not what you think — open devtools and look. I
have found a refresh token, a full user object with an email, and a cached permissions
array in codebases I would have defended.

I write about measurement and static analysis at
dev.to/ofri-peretz.

innerHTML Has Five Doors. Most Reviews Only Watch One.

Ofri Peretz — Sat, 08 Aug 2026 08:19:02 +0000

I have reviewed the same line of code, written five different ways, and caught it once.

The line is element.innerHTML = something. I catch it when something is obviously a
URL parameter. I miss it when something arrived four frames ago through a message
event, got stored, and is written to the DOM by a function in a different file.

The sink was identical every time. My attention was not.

Sinks are few. Sources are many.

A sink is where data becomes execution: innerHTML, outerHTML,
insertAdjacentHTML, document.write, eval. You can list them on one hand, which is
why they feel solved. Grep for innerHTML. Ban it in review. Done.

A source is where untrusted data enters. Sources are not listable — they are a
property of your architecture, and they multiply with every feature.

That asymmetry is the whole problem. The sink is a syntactic fact. The source is a
trust fact, and trust facts do not survive being passed as a function argument.

The five doors

The same sink, reached five ways. Every one is code I would call normal.

// 1. Direct — the one everybody catches
element.innerHTML = new URLSearchParams(location.search).get("q");

// 2. postMessage — another frame speaks, event.data is a string like any other
window.addEventListener("message", (e) => { element.innerHTML = e.data; });

// 3. FileReader — the user picked the file, which feels like consent
reader.addEventListener("load", () => { preview.innerHTML = reader.result; });

// 4. WebSocket — the connection is authenticated, so the payload feels authenticated
socket.addEventListener("message", (e) => { feed.insertAdjacentHTML("beforeend", e.data); });

// 5. Worker — it is our own worker, running our own code
worker.addEventListener("message", (e) => { results.innerHTML = e.data.html; });

Five doors, one room. On the first, a reviewer says "that's user input." On the other
four, they say some version of "but that's ours."

The word "ours" is doing all the damage

Each of those four defences is a claim about provenance, and each is wrong the same way.

A message event fires for any frame with a handle on your window unless you check
event.origin — the listener does not filter by default. An authenticated WebSocket says
the connection is trusted, not what the other end sent, which may have been stored by a
different user hours earlier. A worker is your code, but the data it posts back is
whatever it was given. And a file the user chose is untrusted precisely because they
chose it.

The failure is not that developers think innerHTML is safe. Everyone knows. The failure
is that danger gets assessed at the sink, where the only visible fact is a string — and
the string looks the same whoever wrote it.

Why generated code lands here

Ask a model to render a preview and it reaches for innerHTML, because that is the
shortest correct-looking answer to "put this markup on the page" and the training corpus
is fifteen years of tutorials using it.

More to the point: a model completing a message handler has the reviewer's problem,
worse. It sees a local scope. It cannot know whether that listener lives in an app that
checks event.origin elsewhere. The sink is local; the trust decision is not.

Which is the interesting part — the information needed to get it right is not present at
the point of writing. A property of the problem, not of the author.

The taxonomy is the useful artifact

Source	Trust claim that fails
URL / query params	none — everyone catches this
`postMessage`	"another frame of ours"
WebSocket message	"the connection is authenticated"
`FileReader` result	"the user chose the file"
Worker message	"it is our own worker"

Read the right column. That is not five vulnerabilities. It is one — provenance assumed
from proximity — wearing five costumes. Sinks tell you where to look; pairs tell you what
to ask.

What it means for detection

Grepping a sink is a heuristic. Following a source to a sink is taint tracking —
the thing that decides your false-positive rate.
Full taint tracking across frames and sockets is not something a linter does; those
boundaries are
the line between linting and SAST.
What a linter can do is match source and sink inside one handler — which covers all four
cases, because the risky pattern is nearly always written in a single function.

Two branches, not one chain

The XSS outcome is CWE-79, under
CWE-74 Injection. The trust failure is
CWE-345 under
CWE-693 — with its child
CWE-346 applying to postMessage
alone, the only case with a page-side origin check. A WebSocket's Origin is validated
server-side at the handshake; FileReader and workers have no origin at all.

Different branches, not one chain. Filing all of it as "XSS" is how the actual defect goes
unrecorded. The taxonomy piece
has the map.

Worth reading in full:
OWASP's DOM-based XSS Prevention Cheat Sheet
and PortSwigger on sources and sinks.
To stop it being a review question at all,
Trusted Types moves
it to the browser — the only place that sees every assignment.

The fix is boring, which is the point

window.addEventListener("message", (event) => {
  if (event.origin !== "https://trusted.example") return;
  element.textContent = event.data;
});

Check the origin. Use textContent for text. Sanitize when you genuinely need markup.
None of that is the hard part.

The hard part is noticing you are in one of the five doorways — because four of them do
not look like doorways. They look like your own code talking to itself.

Which door caught you? I am most interested in postMessage — the origin check is one
line, documented everywhere, and I still find handlers without it in code I wrote myself.

I write about measurement and static analysis at
dev.to/ofri-peretz.

Nobody Writes Bad Crypto. They Write Correct Crypto at Four Layers.

Ofri Peretz — Sat, 01 Aug 2026 09:00:32 +0000

The code review comment I have written most often, and regretted most often, is
"use a stronger algorithm."

It is regrettable because it is almost never the actual defect. It is the defect I can see
from where I am standing. The team swaps SHA-1 for SHA-256, everyone agrees the review was
useful, and the thing that would have mattered stays where it was.

Four layers, each hiding the next

A crypto call has to be right four separate times, and being correct at one layer makes
the next one invisible. AES looks like a decision you already made. It is four.

Layer 1 — the primitive. Right kind of thing? A hash where you needed a
KDF. A cipher where you needed a MAC. Math.random() where you needed a CSPRNG. Everyone
reviews this layer, because it is visible in the function name.

Layer 2 — the mode. Primitive right, mode wrong. AES in ECB encrypts
identical blocks to identical ciphertext — which is why the famous penguin is still
legible after "encryption." Nothing about aes-256-ecb reads as wrong at a glance; it
contains aes and 256, and both are reassuring.

Layer 3 — the parameters. Right primitive, right mode, wrong inputs. A
static IV. An iteration count set in 2015 and never revisited. RSA where someone passed
RSA_PKCS1_PADDING explicitly — Node defaults to OAEP, so this one takes a deliberate
argument to get wrong, which is exactly why it survives review. Each is a single value,
usually a constant, and constants do not attract attention.

Layer 4 — the usage. Everything above correct, and the surrounding code
gives it away. Comparing an HMAC with === and leaking the answer through timing.
Deriving a key correctly, then logging it. This layer is not crypto code at all, which is
why crypto review misses it.

Why layers beat a rule list

A rule list tells you what to grep for. The layer tells you where your team is, and
therefore what you are not going to catch.

My working hypothesis — and it is a hypothesis, not something I have measured — is that
failures cluster by how much crypto attention a team has already had. Never thought about
it: layer 1. Just audited: layer 2, because the audit said "upgrade the primitives" and it
did exactly that. Careful for years: layers 3 and 4.

If that holds, the uncomfortable corollary is that layer 3 and 4 defects live longest
because the code passes every review asking "are we using strong algorithms." The
maturity that fixes the first two layers is the same maturity that stops looking further
down. That is the measurement I most want from this taxonomy.

Detection: the easy layers are the trap

Layers 1 and 2 live in a single call expression. createHash("sha1") and aes-256-ecb
are string literals; matching them is syntactic and any linter does it well.

Layers 3 and 4 are not. Whether an IV is static depends on where it came from — a
data-flow question, the difference between
matching a pattern and following a value.
Whether a comparison is timing-unsafe depends on whether the value is a secret, which no
analyzer knows from the expression alone.

So tool coverage is inversely correlated with how long a defect survives. The layers a
tool covers cleanly are the ones your team already fixes. Read any claim that a tool
"covers crypto" as "covers layers 1 and 2" until shown otherwise — and checking that needs
a corpus with known answers,
not a rule count.

Where this lands in the standards

The layers map to distinct CWEs, which is the useful part —
the taxonomy already encodes the
distinction most reviews collapse:

Layer	CWE
1 — primitive	CWE-327 risky algorithm; CWE-338 weak PRNG
2 — mode	CWE-327 — ECB has no dedicated CWE
3 — parameters	CWE-1204 weak IV generation; CWE-323 reusing a nonce or IV; CWE-916 insufficient computational effort
4 — usage	CWE-208 observable timing discrepancy

Filing all four as "weak crypto" is how the layer 3 and 4 instances disappear from your own
bug data, and then from your priorities.

Prior art worth reading: Egele et al., An Empirical Study of Cryptographic Misuse in
Android Applications (CCS 2013), which found the overwhelming majority of apps using
crypto APIs got at least one layer wrong. A decade old, different platform, and the
layering has not aged at all.
NIST SP 800-175B is the reference for
what each primitive is for, and Bernstein's cache-timing work on AES is the clearest
statement of why layer 4 is not paranoia.

Why generated code lands on layers 2 and 3

A model asked for "AES encryption in Node" produces something that runs. Running is the
constraint it optimizes, and every one of these layers is invisible to that constraint.
ECB runs. A static IV runs. An iteration count of 1000 runs. The code is correct at the
layer a reader checks, because it was trained on a corpus where that is what correct
looked like.

Same reason the human review misses it. The model read the same tutorials you did.

Which layer does your codebase fail at? The answer tends to say more about when the code
was written than who wrote it — layer 2 usually means "audited once, years ago."

I write about measurement and static analysis at
dev.to/ofri-peretz.

I Audited 203 of Our Own ESLint Security Rules. 16% Mislabel Their Own CVSS Score.

Ofri Peretz — Tue, 28 Jul 2026 04:59:46 +0000

Run eslint-plugin-node-security against a Server-Side Request Forgery
pattern and the terminal prints this:

src/fetch-avatar.ts
  4:7  warning  🔒 CWE-918 CVSS:9.1 | HTTP call whose URL argument name
              suggests user input. | LOW

CVSS 9.1. Read the number on the calculator and it's nine tenths of a point
from the maximum possible score a vulnerability can get. Read the word next
to it and it says LOW. Those two facts are printed four characters apart
on the same line, by the same rule, and they disagree with each other.

I write the CWE/CVSS/OWASP metadata that ships in every one of our lint
messages — it's meant to be the thing that tells you, at a glance, how
worried to be. So I went and checked whether it actually does that. I wrote
a 25-line script that reads every rule in the Interlace ESLint ecosystem,
pulls its CVSS score and its shipped severity label, and checks whether the
label matches the official band the score falls in.

33 of 203 rules — 16% — don't. Some of those are defensible once you
know why. Some aren't. All of them are a good excuse to explain what CWE,
CVSS, and OWASP actually measure, because the gap between them is exactly
where lint output stops being trustworthy and starts being decoration.

What the three labels actually measure

A single line of our lint output packs in three unrelated standards. Quick
definitions — there's a full reference for each if you need it:

CWE names what kind of bug it is. CWE-918 is SSRF; CWE-287 is Improper Authentication. A category, not a verdict.
CVSS is a computed 0.0–10.0 score. Its four bands are the ruler this audit uses: 0.1–3.9 Low, 4.0–6.9 Medium, 7.0–8.9 High, 9.0–10.0 Critical. The number is the primary source; the band name is derived from it.
OWASP Top 10 is a category bucket (A01–A10) — an address, not a severity measurement.

Only CVSS is actually a severity measurement. So when a lint rule prints a
severity word, that word has exactly one legitimate source: the CVSS score
next to it. If they disagree, the word is wrong.

Case 1: a 9.1 that prints LOW — and the honest reason it might be fine

no-ssrf flags HTTP calls where the URL argument's name looks
user-supplied (userUrl, req.query.endpoint, targetUri) — a classic
Server-Side Request Forgery
setup, the bug class behind more than one well-known cloud metadata-endpoint
breach:

// flagged — the *name* suggests user input, not a data-flow trace
async function fetchAvatar(userUrl) {
  return fetch(userUrl);
}

The rule's own metadata: cwe: 'CWE-918', cvss: 9.1 — squarely in the
Critical band. The message it prints: severity: 'LOW'.

Here's the part that's genuinely defensible: the rule's own doc comment
calls itself out as "a naming heuristic, not data-flow analysis." It
matches on identifier names, not on whether the value actually reaches an
attacker-influenced source. That's a real limitation — plenty of
userUrl-named parameters are perfectly safe, admin-configured constants.
A low-confidence match that could be a
false positive
does deserve a lower-urgency label than a rule that traced the actual
taint.

The part that isn't defensible: confidence and impact are two different
axes, and severity is only supposed to encode one of them. "How sure are
we this is real" and "how bad is it if it's real" are independent
questions — a low-confidence match on a 9.1-impact bug is still worth
more attention than a high-confidence match on a 3.0. Collapsing both into
one LOW/MEDIUM/HIGH/CRITICAL string means the reader can't tell
which axis they're looking at, and a grep-based triage pass (or an
LLM agent doing the same) will deprioritize a Critical-impact finding on the
strength of a label that was only ever describing detection confidence.

Case 2: a 9.8 with no heuristic excuse at all

require-audience-validation and require-issuer-validation aren't naming
heuristics — they check one deterministic fact: does a jwt.verify() call
pass an audience (or issuer) option? Yes or no, no ambiguity:

// flagged — no audience claim checked
jwt.verify(token, publicKey);

// not flagged
jwt.verify(token, publicKey, { audience: "https://api.example.com" });

Skip audience validation and a JWT minted for any service that trusts the
same signing key gets accepted by this service too — a textbook confused
deputy, CWE-287,
cvss: 9.8. Same story on the issuer side. Both rules print severity: 'MEDIUM'.

There's no false-positive story to hide behind here — the check is exact.
9.8 is a hair under the maximum a CVSS score can reach. MEDIUM sits two
whole bands below where that number lives. If a reader — or a CI gate that
filters on severity string instead of CVSS number — treats MEDIUM findings
as "fix this sprint," a near-maximum-severity auth bypass just got queued
behind a color contrast warning.

Case 3: the one that runs the other way

Not every mismatch under-states. no-unsafe-search-path
— the PostgreSQL search_path hijacking rule I wrote up in full elsewhere —
ships cvss: 7.5 (High band) but prints severity: 'CRITICAL', one band
above its own score. I won't re-run the attack walkthrough here — the
short version, if a label disagreeing with a number bothers you as much as
it bothers me: read that piece for the exploit, come back here for why the
label drifted in the first place.

Why this happens: `severity` is the one field nobody enriches

Every rule funnels its message through one formatter,
formatLLMMessage(), and every formatter call goes through the same
enrichment step first:

// packages/eslint-devkit/src/messaging/formatters.ts
function enrichFromCWE(options) {
  if (!options.cwe) return options;
  const cweData = CWE_MAPPING[options.cwe];
  if (!cweData) return options;

  return {
    ...options,
    owasp: options.owasp ?? cweData.owasp,
    cvss: options.cvss ?? cweData.cvss,
    compliance: options.compliance ?? CWE_COMPLIANCE_MAPPING[options.cwe],
  };
}

There's a canonical lookup table, CWE_MAPPING, and it's correct — its
entry for CWE-918 is { cvss: 9.1, severity: 'CRITICAL' }, internally
consistent, right band. CWE-287 maps to { cvss: 9.8, severity: 'CRITICAL' }. If every rule simply deferred to this table, none of the 33
mismatches would exist.

But look at what enrichFromCWE actually forwards: owasp, cvss,
compliance — and that's the whole list. severity isn't in it. Every
cvss number in our output either comes from this vetted table or from an
explicit override that a human cross-checked against it. Every severity
string is whatever word the rule's author typed into that one call site,
with nothing checking it against the table, the CVSS number, or any other
rule's choice for the same CWE. Two rules can share a CWE, share a CVSS
score, and print two different severity words, and nothing in the pipeline
would ever notice.

That's the actual bug: not "someone mislabeled 33 rules," but "the schema
has a field with no source of truth." Free-text fields drift; that's what
they do when nothing enriches them. The fix is the same one-line pattern
already applied to owasp and cvss — default severity from
CWE_MAPPING[cwe].severity unless a rule explicitly overrides it, the same
options.severity ?? cweData.severity fallback the other three fields
already get. I've filed it; it hasn't shipped as of this writing, and I'd
rather say that than quietly imply it's fixed.

The audit, if you want to run it yourself

Point this at any rule set that ships CVSS numbers and severity strings —
ours included — and it'll tell you exactly where they disagree:

// audit-severity-drift.ts — flags rules where severity doesn't match its own CVSS band
import { readFileSync, globSync } from "node:fs";

const BANDS = [
  [9.0, 10.01, "CRITICAL"],
  [7.0, 9.0, "HIGH"],
  [4.0, 7.0, "MEDIUM"],
  [0.0, 4.0, "LOW"],
] as const;

const bandFor = (cvss: number) =>
  BANDS.find(([lo, hi]) => cvss >= lo && cvss < hi)?.[2] ?? "?";

for (const file of globSync("packages/eslint-plugin-*/src/rules/*/index.ts")) {
  const src = readFileSync(file, "utf-8");
  const cvss = src.match(/cvss:\s*([\d.]+)/)?.[1];
  const severity = src.match(/severity:\s*'([A-Z]+)'/)?.[1];
  if (!cvss || !severity) continue;

  const expected = bandFor(Number(cvss));
  if (expected !== severity) {
    console.log(`${file}: CVSS ${cvss} → ${expected}, shipped ${severity}`);
  }
}

203 rule files scanned, 33 flagged — an 84% clean rate I'd rather publish
honestly than round up. And that rate is itself a snapshot: the rule sources
as they stood when this published (2026-07-06), spread across the ecosystem's
twenty-odd independently-versioned plugins, not one pinned release. The
article's own thesis applies to its own numbers — a printed audit is a cache
too. Re-run the script against today's checkout and trust that count over
mine.

Reading a severity label like a security engineer, not a triage bot

None of this is specific to our plugins — it generalizes to any tool that
prints a severity word next to a CVSS number, including the ones you didn't
write:

The number is the primary source. The word is a cache of it, and caches go stale. If a CRITICAL/HIGH/MEDIUM/LOW string and a CVSS score disagree, trust the score — it's the one with a formula behind it, not a human's word choice.
CWE tells you the shape of the bug, not how loudly to worry. "This is a CWE-918 (SSRF)" is a fact about the code. It says nothing about this instance's severity until a CVSS vector is computed for it specifically.
A low-confidence detector and a low-impact bug produce the same label and mean opposite things. If a tool's docs mention "heuristic" anywhere near a LOW, ask whether LOW is describing the bug or the detector's certainty — they're rarely the same axis, and only one of them tells you whether to actually go read the code.
Don't build a CI gate on the adjective. severity !== 'CRITICAL' as a merge-blocking filter inherits every mislabel in the tool it's reading from. Gate on the CVSS number, or on the CWE list you've decided matters, not on a string a human typed by hand.

The config

npm install --save-dev eslint-plugin-node-security eslint-plugin-jwt

import nodeSecurity from "eslint-plugin-node-security";
import jwt from "eslint-plugin-jwt";

export default [nodeSecurity.configs.recommended, jwt.configs.recommended];

Both plugins print the CWE/CVSS/OWASP line on every finding — that's what
made this audit possible to run against our own output in the first place.
Getting started with eslint-plugin-node-security
and getting started with eslint-plugin-jwt
cover the rest of what each ships.

I'm not writing this to talk our own numbers down — 84% of 203 rules
printing a severity word that matches their own CVSS band is a real bar,
and I'd take it over a tool that doesn't print a CVSS number at all and so
never gets caught disagreeing with itself. I'm writing it because "check
the number, not the adjective" is a rule I only started following rigorously
after writing the script that caught my own tool getting it wrong three
different ways in three different rules.

Your turn: open whatever security linter you already run, pick one
finding, and look up its CVSS vector by hand. If the vector notation is new,
what a CVSS score actually measures
walks the AV/AC/PR/UI fields — and why severity, exploit probability, and
confirmed exploitation are three separate measurements with three separate
owners. Does the severity word next to it still hold up once you've done the
math yourself?

⭐ Star on GitHub if you've ever trusted a severity label more than the score behind it.

I'm Ofri Peretz, a security engineering leader and the author of the
Interlace ESLint ecosystem — domain-specific static analysis for security,
reliability, and performance on the Node.js stack.

ofriperetz.dev · LinkedIn · GitHub

142,076 Weekly Downloads. Zero Releases Since 2021. Is the Niche Defended?

Ofri Peretz — Mon, 20 Jul 2026 04:45:01 +0000

Two reads from the npm registry, same package, same morning (2026-07-19):

downloads, last week:  142,076
last publish:          2021-05-21  (v3.2.0)

serverless-iam-roles-per-function is one of the Serverless Framework ecosystem's most-installed plugins, and it hasn't shipped a release since May 2021. Eleven weeks earlier: 118,761 weekly downloads (measured 2026-05-04). Zero releases in between; 23,315 weekly installs gained.

The package isn't surviving abandonment — its installed base is growing straight through it.

My first read was the common one: "142k a week, the niche is taken, I'm five years late." That's the misconception this article exists to correct. High downloads don't mean a niche is taken. Downloads measure the installed base; maintenance decay measures whether the niche is defended. The gap between those signals is where I decide what to build next, using what I call the incumbent map. Fair warning: I ran the arithmetic in May, shipped against its #1 row on 2026-05-07, and only ran the filter that kills that row while fact-checking this article in July. The method works; I ran half of it. Both halves are below, in the order I should have run them.

How do you build an incumbent map?

Three signals per package, all public, all re-runnable:

Weekly downloads — demand. The installed base a maintained replacement would serve. Downloads are a proxy metric — npm's own account of download counts includes build servers, mirrors, and robots, so the number tracks lockfile momentum, not humans choosing a package. As a relative ranking of where demand pools inside one ecosystem, though, it's the best public signal there is.
Years since last publish — supply-side decay. Read the registry at time[<latest-version>]. Not time.modified — that field moves when npm touches registry metadata: serverless-iam-roles-per-function shows modified: 2022-05-17, a full year after its last release. Trust the version's own timestamp, nothing else.
Open-issue decay — unmet demand. No clean number here, so make the procedure explicit: sort open issues by reactions, list the recurring asks, date the last maintainer reply. For the plugin above: managed-policy attachment and custom role naming — asked for years across multiple issues, no maintainer commitment (the 2018 managed-policy ask drew "open for PRs"; nothing shipped since). And read the replies, not just titles; mine matter later.

The first two signals, as commands:

curl -s https://api.npmjs.org/downloads/point/last-week/<pkg>               # demand
curl -s https://registry.npmjs.org/<pkg> | jq '.time[."dist-tags".latest]'  # decay

Multiply, roughly: demand × decay × unanswered asks, ranked descending. Deliberately not a precise formula — a shortlist generator whose output you still have to judge.

The closest prior art is OpenSSF Scorecard: its Maintained check formalizes the same "is anyone home?" signal for the dependencies you consume — a risk lens. The incumbent map points it the other way: not "should I trust this package?" but "should I compete with it?" I haven't seen that build-side version written down anywhere — a strong claim, and an open invitation to correct me.

What does the serverless incumbent map actually show?

The worked example that produced my roadmap. Two zones, all numbers measured 2026-07-19:

Defended — active maintainers, do not compete on maintenance:

Package	Downloads/wk	Last publish
`serverless-offline`	627,578	2026-06-23
`serverless-esbuild`	388,619	2026-05-21
`serverless-domain-manager`	304,882	2026-07-17
`serverless-step-functions`	208,865	2026-07-13

Abandoned — demand present, nobody home:

Package	Downloads/wk	Last publish	Silence
`serverless-iam-roles-per-function`	142,076	2021-05-21	5.2 yrs
`serverless-plugin-include-dependencies`	52,500	2024-07-22	2.0 yrs
`serverless-plugin-common-excludes`	51,415	2021-07-06	5.0 yrs
`serverless-api-gateway-throttling`	36,528	2023-03-19	3.3 yrs
`serverless-plugin-tracing`	35,709	2017-10-13	8.8 yrs
`serverless-plugin-canary-deployments`	31,562	2022-04-11	4.3 yrs
`serverless-associate-waf`	28,787	2020-04-17	6.3 yrs

The top zone shipped within two months. The bottom zone's median silence: five years — on packages still installed tens of thousands of times a week.

The map moves, in both directions. serverless-associate-waf shows demand finally decaying — down roughly a quarter from my May reading — so abandonment does reach the download number, on a lag of years, not months. And re-pulling before publishing deleted a row: serverless-openapi-documenter, filed in my May notes as a stale documentation-plugin niche, shipped a release on 2026-06-24. Someone re-defended it while I drafted. Row gone. An incumbent map is a snapshot, not a subscription — re-run it the week you commit, not the quarter you planned.

When is a quiet package finished instead of dead?

The honest counter-case, because the map's arithmetic will lie to you twice.

First: stable is not dead. serverless-prune-plugin sits at 414,154 downloads a week (measured 2026-07-19), last publish 2024-10-16. Twenty-one quiet months — and it may simply be done: its job is narrow, its surface barely moves. The separating question: does the surface underneath the package move? IAM — the ground serverless-iam-roles-per-function stands on — moves constantly: new AWS services, new actions, new policy patterns. Five silent years on a moving surface is decay; on a frozen surface, it can be completion. Same silence, opposite meanings — a classic measurement-bias trap.

Second: platform absorption. serverless-plugin-typescript: 151,586 downloads a week (measured 2026-07-19), dormant since 2023-06-05 — by raw arithmetic the juiciest target on the board. It's not on my list: Serverless Framework v4 builds TypeScript natively and won't run build plugins like it unless you explicitly opt out of the native build. That niche isn't undefended; it's dissolving. A replacement would be competing with the platform itself.

An installed base looks like a moat. The publish log says whether anyone still defends it; the platform roadmap, whether the castle is even staying put.

How were the replacement targets actually picked?

The full pipeline, as actually run for the serverless line:

Rank every community plugin by demand × decay (the tables above are the output).
Filter: is the niche still real? — serverless-plugin-typescript fell here first. Then the same filter, run against the current v4 docs instead of my stale notes, reached the top row of my map: v4 ships per-function IAM roles natively — provider-statement inheritance on by default, log/VPC/event-source permissions auto-added, the plugin's own config fields accepted for migration. The official IAM guide credits the plugin for pioneering the feature and says you can now safely remove it. My #1 target wasn't an undefended niche; it was one the platform had absorbed — with honors.
Filter: is the incumbent truly undefended? — re-pull the registry the week you commit; that check removed serverless-openapi-documenter.
Filter: is the unmet demand documented in public — and still unmet? The newest open issue asks "Serverless Framework V4 Support?" — and the thread answers itself: a collaborator's "it works," then a comment pointing at the v4 docs above. The two features its tracker spent years asking for — managed policies, custom role names — shipped in v4's native implementation, not the plugin's. The counter-evidence to my top pick sat in the tracker I was reading for demand. Read the replies.

The worked example deleted two of its own rows — one while I drafted, one while I fact-checked. Here is what the second deletion cost, stated plainly: my May map scored serverless-iam-roles-per-function 23/25 — "the biggest single-plugin opportunity in the portfolio" — and I shipped @interlace/serverless-iam-roles-per-function 1.0.0 off that row on 2026-05-07 (the npm publish date; check it). The filter you just read ran two months later. The package stays published and does what it says; what it is not, anymore, is the line's growth bet — v4 owns that niche now. The ranked ROI analysis lives in the repo, unchanged, dates included — deliberately, as the before-artifact: you can see exactly the map I built from, and exactly what running filter 2 late overturned. Arithmetic generates the shortlist; filters, run against primary sources — before you build — make the decision.

Notice the conflict of interest: the person selling replacements is grading the incumbents — the same conflict as building what I benchmark, with the same mitigation: every number here is a public registry read, stamped with its measurement date, re-runnable in one curl without me in the loop.

The map is step one of entering an ecosystem. Step two keeps the building honest — write down claims and evidence before the marketing exists: the claims registry. Different package types need different metrics once you measure what you built — but the map is how you decide to build at all.

What's the most-downloaded abandoned package in your ecosystem — React components? Terraform providers? Gradle plugins? Two registry reads will tell you. I'm collecting incumbent maps, and I'd genuinely like to see yours.

Quick reference

Signal	What it measures	Where to read it	Failure mode
Weekly downloads	Installed base (demand) — not adoption, not health	`api.npmjs.org/downloads/point/last-week/<pkg>`	Mistaking base for defense; CI/mirror inflation
Years since last publish	Supply-side decay	`registry.npmjs.org/<pkg>` → `time[<latest-version>]` — not `time.modified`	Finished ≠ dead on a frozen surface
Open-issue decay	Unmet demand, stated in public	Issue tracker: sort by reactions, date the last maintainer reply	Reading titles, not replies — the answer may be "the platform shipped it"
Deprecation/archive status + platform roadmap	Whether the niche still exists	Registry `deprecated` field, repo header, framework changelog + docs	High downloads on a dissolving niche
Decision rule	demand × decay × unanswered asks → shortlist → 3 filters	This article	It's a snapshot — re-run the week you commit

References

"numeric precision matters: how npm download counts work" (npm blog, 2014) — npm's own account of what a download includes (build servers, mirrors, robots); the primary source for why downloads can only ever be a demand proxy.
npm registry download-counts documentation — the API the map reads: the endpoints and the data pipeline behind the counts.
Serverless Framework IAM guide — the platform-absorption primary source: v4's native per-function roles, statement inheritance, auto-added permissions, and the credit to the community plugin it replaces.
OpenSSF Scorecard — the closest prior art: its Maintained check operationalizes "is anyone home?" for consumption-side dependency risk; the incumbent map reuses that signal for build-side decisions.

Related deep dives

Proxy Metrics: When the Number Isn't the Thing — why downloads, stars, and every other convenient number measure something adjacent to what you care about
Bias in Measurement — how a real number still produces a wrong inference, including the too-clean-zero forensic
The Claims Registry — step two of the playbook: the evidence discipline that comes before any claim
Different Metrics for Different Package Types — once you build, how the measuring has to change per package type
I Built What I Benchmark. Here's How I Try Not to Cheat. — the conflict-of-interest process this article's mitigation borrows

Star the serverless repo — the map above is its roadmap

Part of the Interlace ESLint ecosystem. Source on GitHub · npm: @interlace · Follow: Dev.to/ofri-peretz · ofriperetz.dev

We Were Wrong About sls remove

Ofri Peretz — Mon, 20 Jul 2026 04:44:23 +0000

The loudest claim in my serverless line was wrong. Early copy for @interlace/serverless-api-gateway-caching implied that the community caching plugin leaves an API Gateway cache cluster running — and billing — after sls remove, and that our cleanup hook fixes it. On 2026-05-04, our own E2E harness ran the community plugin through sls remove on live AWS: 31 seconds, exit 0, no orphans. CloudFormation deleted the stage and took the cache cluster with it. There is no sls remove ghost-billing bug.

The trap itself is real — I had reproduced it on live AWS the day before, 2026-05-03, in about ten minutes for roughly $0.005 of cache-cluster time. It just lives on a different path than the one we were loudest about. This article is the full telling of that re-scoping: the original claim, the two runs that killed and rebuilt it, and why the narrower claim that replaced it does more work than the broad one ever did.

What was the original claim?

The community serverless-api-gateway-caching (v1.11.0) manages the API Gateway cache cluster imperatively — it calls the AWS UpdateStage API during deploy instead of declaring the cluster in the CloudFormation template. CloudFormation never learns the cluster exists. From there, the inference writes itself: if CloudFormation doesn't know about the cluster, teardown won't remove it, so sls remove must leave it behind, billing forever. Our plugin ships a before:remove:remove hook that disables the cluster before teardown — so early copy implied, in effect: the community plugin has a ghost-billing bug on sls remove, and we fix it.

Notice what that claim actually was: an inference from architecture, shipped as if it were a measurement. And it was consistent — repeated the same way across the README, the docs, the comparison table. Consistency is reliability; whether the claim describes the failure that actually happens is validity — and nobody had checked.

What did the measurement actually show?

Two runs on live AWS, one day apart, both against the pinned community release 1.11.0.

2026-05-03 — the trap, reproduced. Deploy with the community plugin (a live stack in 49s), wait for the cache cluster to reach AVAILABLE (4–7 minutes), then do what a real team does when it stops wanting a plugin: delete it from the plugins array and redeploy the still-running service. The redeploy succeeds in ~28 seconds and no plugin code runs — the plugin is gone from the config, so it can't. The cluster afterward: enabled: true, status: AVAILABLE, size: 0.5. Still running, still billing. No CloudFormation event, no log line, no warning. And no exit through the plugin either: sls caching disable and sls caching status both return "command not found" — the community plugin registers no custom commands, so the only escape is dropping to the AWS CLI and patching cacheClusterEnabled to false by hand. Total cost of watching all of this happen: ~10 minutes, ~$0.005 — the recipe is public in docs/ghost-billing-reproduction.md, cheap enough that "it reproduced on my machine" can be your finding rather than my assurance.

2026-05-04 — the loud path, refuted. The community E2E then ran the exact path our copy warned about: sls remove with the plugin still installed. Clean. 31 seconds, exit 0, no orphans — CloudFormation deleted the stage, and the cluster went with it.

The mechanism fits in one sentence: the cache cluster rides on the stage. Delete the stage — which is what sls remove does — and the cluster dies with the stack, no matter who created it. Keep the service deployed and remove only the plugin, and the stage stays up while the one tool that remembered the cluster leaves your toolchain.

I ran that second E2E to collect the evidence for the claim. It came back with the evidence against it — exit 0, no orphans, and a rewrite of my best line of copy. The same discipline that makes you distrust a result that flatters you is the one that makes you run the test that can defeat you, before someone else does.

What is the re-scoped claim?

The old copy didn't get quietly patched; it got re-scoped, on the record. Our claims registry keeps a dedicated table for exactly this case — original copy, what we measured, what's true — and the row for this incident reads, condensed: earlier copy implying "we fix sls remove's ghost-billing bug" was wrong, because there is no sls remove ghost-billing bug. The orphaned-cluster trap lives on the uninstall-while-keeping-service path, where sls caching disable — an explicit offboarding command the community plugin doesn't have — is the actual fix. The before:remove:remove hook stays, verified live (stack deleted in 28s, zero residuals, measured 2026-05-03), but it's defense-in-depth now, not the headline.

That's the whole lifecycle in one incident: claim → reproduction attempt → refutation → re-scope → versioned evidence file. The refuting run is committed as a dated JSON next to the claim row; the trap ships as a recipe anyone can replay.

The stakes are why the claim deserved this much care. The cluster we reproduced was the smallest tier — 0.5 GB at $0.020/hour, $175.20/year. AWS's published cache pricing runs up to 237 GB at $33,288/year, and the charge is per stage, per region, with no flag on orphaned clusters. A number that size deserves a claim pointed at the path where the money actually leaks.

Why is a narrower claim stronger?

Value investors call it margin of safety: you don't pay the price your own estimate says is fair — you pay less, so that being somewhat wrong doesn't ruin you. A claim scoped to exactly what the run files show carries the same margin. The broad claim had none: it rode on an untested inference, and a single ten-minute E2E from anyone could falsify it. Ours did, mercifully before anyone else's.

The narrow claim is also more useful to you. "Run sls caching disable before you remove the plugin from config" is an action. "Beware sls remove" would have taught you to distrust the one command that actually cleans up after itself — misallocating your caution away from the path that costs money and onto the path that doesn't.

Which is the misconception worth naming: "walking back a claim costs credibility." It's backwards. Unmeasured claims cost credibility — on a schedule you don't control. A published re-scope is the cheapest credibility purchase available: it costs one uncomfortable table row and buys back the right to be believed about everything the run files do support. The genre has old prior art — Buffett's shareholder letters have aired his own mistakes for decades, and it's the same model behind how I try not to cheat in my own benchmarks.

I'll admit the first instinct, reading the clean run, was the other genre: quietly delete the sentence and move on. That instinct — fix silently, stay loud — is exactly what a claims registry exists to make impossible, and the reason this article exists at all.

Quick reference

Question	Answer (measured 2026-05-03/04, community plugin v1.11.0)
Does `sls remove` orphan the community plugin's cache cluster?	No — CloudFormation deletes the stage and the cluster with it (31s, exit 0, no orphans)
Where is the actual trap?	Remove the plugin from `plugins` and redeploy the still-running service — no plugin code runs; cluster persists `enabled: true, AVAILABLE`
Is there a plugin-mediated escape (community v1.11.0)?	No — `sls caching disable` / `sls caching status` return "command not found"; manual `aws apigateway update-stage` required
What does an orphaned cluster cost?	$175.20/yr at 0.5 GB up to $33,288/yr at 237 GB — per stage, per region, unflagged by AWS
The safe offboarding path (`@interlace/serverless-api-gateway-caching`)?	`sls caching disable` before uninstalling; `before:remove:remove` covers `sls remove` (verified live: stack deleted in 28s, zero residuals)
Reproduce it yourself	~10 minutes, ~$0.005 — ghost-billing-reproduction.md

The claim that survived is smaller than the one we started with, and it's the only one of the two I'd defend in public — because defending it requires no rhetoric, just the run files. If a claim in our docs matters to you, check whether it has a row in CLAIMS.md; if it doesn't, it isn't allowed to ship. That policy — and the re-scope you just read, kept on the record instead of buried — is the honest answer to "why should I trust a plugin author's comparison table."

⭐ Star the serverless repo — where claims ship with run files, or don't ship

Next in this arc: one step earlier in the story. Before you can re-scope a claim about an incumbent plugin, you have to decide which incumbent is worth challenging at all — the abandoned-incumbent map is how the serverless line made that call.

References

Amazon API Gateway pricing — cache pricing — the per-size price table behind every $/hour and $/year figure in this article.
Cache settings for REST APIs in API Gateway — Amazon API Gateway Developer Guide — the stage-level cache settings (cacheClusterEnabled, cluster size) that make the cluster live and die with its stage.

Bias in Measurement — why a result that flatters you is the one to re-run
Reproducibility vs Replicability — the bar the $0.005 recipe is built to clear
Valid vs Reliable Metrics — how a claim can be perfectly consistent and still measure nothing
The Claims Registry — the table this article's re-scoped row lives in
I Built What I Benchmark. Here's How I Try Not to Cheat. — the same discipline applied to the ESLint benchmarks

Part of the Interlace ESLint ecosystem. Source on GitHub · npm: @interlace · Follow: Dev.to/ofri-peretz · ofriperetz.dev

25% of My Benchmark Verdict Is an Opinion. Here's the Anatomy.

Ofri Peretz — Mon, 20 Jul 2026 04:43:35 +0000

On 2026-05-04, my benchmark harness printed its verdict on two Serverless Framework caching plugins: mine scored 0.88; the community incumbent, 0.3025. Every digit of that gap is arithmetic — seven dimensions, each one measured, normalized, weighted, summed. The raw JSON actually stores my side as 0.8800000000000001, kept as-is: IEEE 754's reminder that a number can be very precise about being an opinion. Because the most consequential input in that file was never measured by any instrument: Lifecycle Correctness is worth 25% of the score because I said so.

A composite score is opinions wearing a number costume — the general argument, and the honesty requirements any composite owes its readers, live in Composite Scores and Weighting, and I won't re-derive them here. What I owe you instead is the costume's pattern, dimension by dimension — including whether the verdict survives when you attack the weights.

What are the seven dimensions — and where does each number come from?

Dimension	Weight	Interlace	Community	Measured from
Lifecycle Correctness	25%	1.0	0.5	local source + live AWS deploy
CLI Surface	15%	1.0	0.0	local source
TypeScript Coverage	15%	1.0	0.0	npm registry*
Maintenance Signal	15%	1.0	0.0	npm registry*
Bundle Weight	10%	0.0	1.0	npm registry*
Hook Coverage	10%	1.0	0.375	local source
Documentation Quality	10%	0.8	0.4	README scan
Composite	Σ=1.0	0.88	0.3025

* Community side only. The Interlace values in these rows came from local source ("source": "local" in the result file) — version 0.0.0 has never been published.

Run stamped 2026-05-04, Node v24.12.0 on darwin/arm64. Two disclosures before anything else. First: my @interlace/serverless-api-gateway-caching was measured from local source at version 0.0.0, while the community's serverless-api-gateway-caching is the published 1.11.0 from the registry. That asymmetry makes one score in the table an artifact, not a measurement: my Maintenance Signal 1.0 rests on daysSincePublish: 0, totalVersions: null, weeklyDownloads: 0 — an unpublished plugin scores perfect maintenance while the community's real 379-day publish gap scores 0. Read that row as unmeasured, not excellent. Second: the lifecycle numbers are not simulations. Both plugins went through a live-AWS end-to-end lifecycle run, and the two E2E result files sit on disk stamped 43 minutes apart from the same night.

The "measured from" column matters as much as the scores. Registry-sourced dimensions are proxy metrics: publish cadence stands in for maintenance, type definitions stand in for TypeScript support. And a README scan is reliable without being automatically valid — it returns the same 0.8 every run; whether 0.8 captures what "good documentation" means to you, the harness cannot answer.

How do kilobytes, hook counts, and pass rates become one scale?

The seven dimensions arrive in incompatible units: bundle size in kilobytes, hook coverage as a count of lifecycle hooks handled, lifecycle correctness as pass/fail steps against a live deploy, documentation as a scan score. A weighted sum over raw units lets the biggest-range unit silently win — the classic mixing-scales trap. So every dimension is normalized to 0–1 before any weight touches it.

The community plugin's Hook Coverage of 0.375 is not a grade someone typed in — it's 3 hooks counted against a ceiling of 8, and that ceiling is my own plugin's full hook-and-command surface, which guarantees my side a 1.0. That denominator is a normalization policy that happens to flatter its author, and it deserves the same suspicion as the bundle curve coming next. On Bundle Weight, my 78 KB unpacked against the community's 47 KB came out 0 against 1 — which tells you the normalization curve is itself a policy decision. A gentler curve would have given my 78 KB partial credit; this one didn't, and I left it that way. Normalization choices change scores without changing a single measurement, so they belong in the published methodology, not in a footnote you have to diff the source to find.

What happens when a dimension can't be measured?

The harness has a standing rule for dimensions it cannot measure — historically the AWS-dependent ones, when there's no live deploy to run against: score them null and never impute a value. An imputed dimension is an invented measurement wearing the same costume as the real ones. But the null path deserves the least flattering disclosure in this article: when nulls are present, compositeScore renormalizes over the measured subset — null dimensions drop out of both numerator and denominator — so a partial run still prints one clean-looking composite. Renormalizing isn't imputation, but it quietly moves weight onto whatever happened to be measurable. The day before this run did exactly that: three of seven dimensions measured, and the harness stored 0.7499999999999999 vs 0.25 — IEEE 754 again, equally precise about a composite built from 40% of the total weight, with nothing in the numbers admitting four dimensions were missing. The n-of-7 label that makes a partial composite honest currently lives one layer up, in the claims registry rather than the harness output — a real gap: "3-of-7, here's what's missing" should be printed by the code, not patched on at the claims layer. In this run the null path went unused — dimensionScores holds numeric values for all seven keys, on both plugins (don't cite the skipped array for this; it was empty in the partial run too) — and I'm telling you anyway, because a policy you only hear about when it's exercised is a policy you can't audit.

Why is Lifecycle Correctness worth 25% — and who decided?

I decided. Here's the reasoning, in tiers.

25% — the damage tier. Lifecycle Correctness is the only dimension where failure means the plugin actively harms your deployment instead of merely annoying you. A caching plugin that mishandles the deploy lifecycle doesn't degrade politely — it leaves your API Gateway stage in a state you didn't ask for.

15% × 3 — the daily-use tier. CLI Surface, TypeScript Coverage, Maintenance Signal: the things you touch or depend on every working day. Individually smaller than correctness, collectively 45% of the score.

10% × 3 — the comfort tier. Bundle Weight, Hook Coverage, Documentation Quality: real costs you can see coming and route around.

Reasonable people could defend a different split — which is the point. The common misconception is that a benchmark score computed from real measurements is objective. The measurements are objective; the weights they run through are editorial, and computed is not neutral. Nor is admitting that unusual: Lighthouse versions its scoring weights and OpenSSF Scorecard publishes its full methodology — the composite-scores canonical walks both. Same page, new Lighthouse version, different score — which means every performance score you've ever pasted into a report was a committee vote you didn't attend. Every index has a committee. The only real question is whether the committee publishes its votes.

Does the winner survive a ±50% attack on the weights?

If a ranking flips when the weights shift, the composite is telling you about the committee, not the plugins. So attack mine as hard as ±50% allows. Halve every weight where my plugin leads: Lifecycle to 12.5%, CLI, TypeScript, and Maintenance to 7.5% each, Hooks and Docs to 5% each. Raise the one dimension where my plugin loses — Bundle Weight — by half, to 15%. Renormalize so the weights sum to 1.0 again.

The composite becomes roughly 0.73 vs 0.42. The order holds, and not by luck: the community plugin leads exactly one dimension out of seven, so no ±50% reweighting exists in which one 10%-class dimension outvotes six others. Push past ±50% and, holding the other six weights in proportion, Bundle Weight needs roughly 43% of the whole composite — against its actual 10% — to flip the winner. The gap is score-broad, not weight-fragile.

That surviving loss deserves its own sentence. My plugin scores 0 on Bundle Weight — 78 KB unpacked against the community's 47 KB — and it's printed in the table above at full weight, in a benchmark I wrote, scoring a plugin I built, under weights I chose. That conflict of interest is real, and the two controls you can audit against it are the published loss and the sensitivity math you can redo on a napkin — the full self-benchmark process is in I Built What I Benchmark. A composite that survives its own worst dimension in public is worth more than one that never shows a loss.

If you'd have weighted it differently, don't argue in the abstract — the whole run is one command: npm run bench:caching from serverless/benchmarks, and this run's output sits at benchmarks/benchmark-results/api-gateway-caching/2026-05-04_v1.0/result.json in the repo. Change the weights, re-run it; if your reweighting flips the order, that's a finding and I want it in an issue. The strongest evidence that this harness follows the data even when it disagrees with me is the next article in this series: the time our loudest serverless claim died under its own E2E run — We Were Wrong About sls remove.

Quick reference

Question	Answer
What is the composite?	Weighted sum over 7 dimensions; weights sum to 1.0
Result (2026-05-04 run)	Interlace 0.88 vs community 0.3025; all 7 dimensions measured
Who sets the weights?	The author — editorial, published, versioned with the results
Weight tiers	Damage 25% · daily-use 15%×3 · comfort 10%×3
Mixed units?	Every dimension normalized to 0–1 before weighting
Unmeasurable dimension?	Null, never imputed; harness renormalizes over measured dims — n-of-7 label lives at the claims layer (this run: 7-of-7)
±50% adversarial reweighting	~0.73 vs ~0.42 — the order holds
The honest loss	Bundle Weight: 78 KB vs 47 KB → scored 0 vs 1, published at full weight
Versions measured	Interlace 0.0.0 (local source, unpublished) vs community 1.11.0 (npm)
Reproduce	`npm run bench:caching` in `serverless/benchmarks` → `benchmark-results/api-gateway-caching/`

External references

OECD/JRC — Handbook on Constructing Composite Indicators: Methodology and User Guide (2008) — the authoritative methodology reference for composite indicators; the normalization and sensitivity sections above are the single-benchmark version of its checklist.
Lighthouse performance scoring documentation — Google's published, versioned scoring weights: the working precedent for treating a weight change as a verdict change.

What's the largest weight you'd assign differently in that table — and which dimension would you demote to pay for it?

Related deep dives

Composite Scores and Weighting — the tier-0 canonical this benchmark instantiates: why composites exist, the honesty requirements, and the Lighthouse / OpenSSF Scorecard case studies
I Built What I Benchmark. Here's How I Try Not to Cheat. — the conflict-of-interest process behind every benchmark on this site
The Claims Registry — the claim→evidence-file contract that this run's result file feeds
Proxy Metrics — why registry-sourced dimensions measure stand-ins, not the thing itself
Valid vs Reliable Metrics — the difference a README scan can't close on its own

Star the serverless repo — you've just read exactly how its scores are built

Part of the Interlace ESLint ecosystem. Source on GitHub · npm: @interlace · Follow: Dev.to/ofri-peretz · ofriperetz.dev

Marketing Pages Rot Silently. Mine Print an Expiry Date on Every Claim.

Ofri Peretz — Mon, 20 Jul 2026 04:43:26 +0000

Around 2026-08-01, a "verification pending" banner goes up on eleven claims in my serverless docs. Nothing will have broken. No competitor will have shipped. The banner goes up because every verified row in that repo's CLAIMS.md is stamped 2026-05-03 or 2026-05-04, the file gives evidence a 90-day warranty, and date arithmetic doesn't care that I wrote the rule.

Line 5 of that file:

If a claim doesn't have a row, it can't ship in the docs. If "Last verified" is older than 90 days, the claim is stale and gets a "verification pending" banner in docs until refreshed.

That line exists because marketing pages rot silently: a claim gets written on the day it's true, nobody dates it, and the sentence sits there aging in public. The fix is one file per product line — the claims registry: every public claim holds a row, every row points at a versioned result file, and every date is an expiry date.

This is the methodology piece — the contract, the lifecycle, the staleness rule — and none of it is JavaScript-specific. If your product page makes claims, it transfers in an afternoon.

What does a claim owe you before it ships?

The contract is deliberately boring. A claim backed by static evidence — a benchmark you can run on a laptop — carries four fields: the claim text exactly as it appears in docs or marketing, the suite that measures it, a link to the latest result file, and the date it was last verified. Verbatim matters: paraphrase is where scope creep hides — if the row is politer than the landing page, the landing page is the claim nobody verified.

One worked row from each product line:

Serverless. The claim "Composite score 88% vs community 30% (7 of 7 dimensions)" points at the api-gateway-caching suite and its result file latest.json, last verified 2026-05-04. Open the file: composite 0.88 for my plugin (measured from local source at v0.0.0, not a published release) against 0.3025 for the community serverless-api-gateway-caching@1.11.0, all seven weighted dimensions scored, including the one I lose outright: bundle weight, 78 KB unpacked against their 47 KB. The row carries the loss because the file does.

ESLint. The sister registry's row reads "Top of leaderboard on the ILB-Arena 40-vuln / 38-safe corpus (1st of 18 plugins tested, 17 security-relevant)" and points at the ilb-arena suite's dated result file, where summary.leaderboard[0].rank == 1. That row survived an audit its neighbor didn't: the registry had been asserting "97.6% precision, 100% recall, 98.8% F1" — figures that appear nowhere in the cited JSON, which reports 100% across the board. The 2026-05-13 audit withdrew the claim rather than substitute the perfect score — 100% on a 40-fixture self-authored corpus is the textbook "regression test, not benchmark" failure mode. The ordinal claim kept its row; the flattering one kept only its receipt, in a "Withdrawn claims" section. The suite column enforces validity — measure what the copy asserts; the result file makes the claim reproducible — regenerate the number without trusting me.

Claims only live infrastructure can verify get a second row flavor: claim text, the E2E step that asserts it, the last release verified. "Cache MISS on first request, HIT within TTL" maps to steps 4–5 of a deploy-and-assert run against real AWS, last verified 2026-05-03.

The contract also kills a third category — the adjective. "Battle-tested," "blazing fast," "production-grade": proxies for measurements nobody made. No suite outputs "battle-tested," so no row, so no ship. The registry doesn't argue with weak claims; it has nowhere to put them.

How does a claim live and die?

Draft → evidenced → stale → re-verified or retired — every transition a visible edit to one file.

Draft. The add-a-claim procedure opens with its most important sentence: don't write the marketing copy first. Build or extend the benchmark until it produces a measurable result for your product and at least one competitor, add the row with today's date, and only then write the copy. The rule is that blunt because my instinct runs the other way — when I draft a README, the adjective arrives before the evidence.

Pending. Unbacked copy isn't deleted — it's routed into a "Pending claims" section next to the suite that would back it. As of today my serverless registry holds 11 verified rows (4 static-evidence, 7 live-evidence) and 2 pending, both "Not started." Pending is the honest parking lot: marketing wishes become a benchmark backlog, priced in engineering hours instead of adjectives.

Re-scoped. The state nobody plans for: the measurement comes back and narrows the claim. Early copy in my serverless line implied we fix a ghost-billing bug in sls remove. Then the community-plugin E2E ran on 2026-05-04: sls remove came back clean — 31 seconds, exit 0, no orphans. The real trap is uninstalling the plugin while keeping the service. The original copy went into a re-scoped row — original text, what we measured, what's true — instead of being quietly rewritten (told in full here). The most useful rows in my registry are the ones that say I was wrong, in writing, next to the run file that proved it.

Retired is a re-scope with nothing left: no narrower version survives the measurement. The row stays; the docs copy goes.

When does a claim go stale — and who decides?

The misconception to kill: "claims go stale when the product changes." They don't. A claim goes stale when its evidence ages past its warranty — the product changing is just one way that happens. Your code can be byte-for-byte identical while the competitor you compared against ships three releases and the runtime you benchmarked goes end-of-life. The sentence didn't move; everything it pointed at did.

That's why staleness is declared, not discovered. Discovered staleness means a reader catches the drift, and the correction arrives after the damage. Declared staleness is date arithmetic: "Last verified" older than 90 days, banner up, mechanically, before anyone is misled. Ninety isn't sacred; it's a warranty one maintainer can honor — pick yours and enforce it the same way.

What keeps this from becoming a refresh treadmill is the version stamp. "eslint-plugin-security has 27.5% recall" rots — a standing claim about the present. "eslint-plugin-security v2.1.1 scored 27.5% recall on our 40-fixture corpus, on ESLint 8.57.0" is a dated measurement no future release can falsify — true forever. Financial statements solved this long ago: a balance sheet is always "as of" a date. Stamped measurements carry zero refresh obligation; only standing claims — "fastest," "most complete," anything in the present tense — sit under the 90-day clock.

The opening countdown is this rule running on its author: the banner lands around 2026-08-01 whether or not I get the re-runs done first. The ESLint registry is already there — one row, a synthetic-corpus speedup last verified 2026-01-02, carries the "re-verify recommended" flag today. That's the system working on me, twice.

How do you adopt this in an afternoon?

Nothing above depends on ESLint, serverless, or even software — the suite can be a load test, an accessibility audit, or a lab assay. The afternoon version:

Create CLAIMS.md at the repo root. The whole starting file:

   # Claims Registry

   > No row, no ship. "Last verified" older than 90 days →
   > "verification pending" banner in docs until refreshed.

   | Claim (verbatim from docs/marketing)   | Suite               | Latest result              | Last verified |
   | -------------------------------------- | ------------------- | -------------------------- | ------------- |
   | "Composite score 88% vs community 30%" | api-gateway-caching | benchmarks/.../latest.json | 2026-05-04    |

Inventory every public claim — README, landing page, package description — verbatim, no paraphrase.
For each, name the evidence (suite + latest result + date) or move it to Pending.
For everything new: evidence first, row second, copy last.
On refresh, commit the new result as a dated file (<suite>/<YYYY-MM-DD>_v<version>/) and bump the date — what you claimed, and when it was last true, becomes repo history.

This isn't an island. The closest neighbor is the OpenSSF Scorecard, which treats a repo's security posture as automated, publicly specified checks pointed at repositories from the outside. The claims registry is the manual, product-side sibling, pointed inward at your own marketing copy (corpus design covers the datasheets-for-datasets lineage). And it pairs with the conflict-of-interest discipline in I Built What I Benchmark: I built the products and the registry that polices them — a suspicious arrangement, which is why every row names the command that would catch me.

Next in this arc: We Were Wrong About sls remove — the re-scope told in full, because the lifecycle is easiest to believe when you watch it delete a claim I liked.

Quick reference

Question	Answer
What gets a row	Every claim in docs/marketing, verbatim — no paraphrase
Static-evidence row	Claim text · suite · latest result file · last-verified date
Live-evidence row	Claim text · E2E step · last release verified
Ship rule	No row → the claim cannot ship in docs
Staleness rule	"Last verified" > 90 days → "verification pending" banner until refreshed
Lifecycle	Draft → evidenced → stale → re-verified or retired; re-scoped when measurement narrows it
New-claim order	Evidence first, row second, marketing copy last
Unbacked copy	Routed to Pending with the suite that would back it — never into docs
Refresh mechanics	Re-run suite → commit dated result file (`<suite>/<YYYY-MM-DD>_v<version>/`) → bump date
The immortal form	Version-stamped: "vX scored Y (measured DATE)" — a dated measurement, true forever

The suite that generates the ESLint result files is public — go check:

⭐ Star the benchmark suite — the result files this registry points at

Before the clock catches yours: what's the oldest undated claim on your product page right now — and would it survive a re-run today?

References

OpenSSF Scorecard — the Open Source Security Foundation's automated checks on open-source repos; cited as the closest prior art for treating claims as named, publicly specified checks rather than prose.
Gebru et al., Datasheets for Datasets (arXiv:1803.09010, later published in CACM) — cited for the documentation-contract idea the registry applies to marketing claims instead of datasets: standardized fields, stated provenance, explicit scope.

Part of the Interlace ESLint ecosystem. Source on GitHub · npm: @interlace · Follow: Dev.to/ofri-peretz · ofriperetz.dev

DEV Community: Ofri Peretz

ESLint Ships in 459 KB, So My Blog Posts Lint Your Code

ESLint ships in 459 KB

The recipe

Three traps

Try it

Reproduce it

I Ran My Plugins Against a Competitor's Own Test Suite

The method

The result

The obsolete rule

What I catch that they miss

Where I lose

Run it yourself

My 9-Reviewer AI Gate Failed Articles It Scored 9.1

A perfect score, filed as a blocker

The word that made good reviews look like outages

An excluded reviewer voting anyway

The pattern

I Linted My Own Design System. All 8 Breaks Were Pasted In.

Raw color literals: 30 findings, and they cluster

data-slot: 67 findings, the biggest by far

The pattern

The config

Your Token Is Not Safer in a Cookie. It Is Safer From JavaScript.

The axis everyone argues about

The axis that decides it

HttpOnly is the boundary

The corollary people skip

So what counts as sensitive

And the CSRF half?

Where this lands in the standards

Why the argument keeps happening

innerHTML Has Five Doors. Most Reviews Only Watch One.

Sinks are few. Sources are many.

The five doors

The word "ours" is doing all the damage

Why generated code lands here

The taxonomy is the useful artifact

What it means for detection

Two branches, not one chain

The fix is boring, which is the point

Nobody Writes Bad Crypto. They Write Correct Crypto at Four Layers.

Four layers, each hiding the next

Why layers beat a rule list

Detection: the easy layers are the trap

Where this lands in the standards

Why generated code lands on layers 2 and 3

I Audited 203 of Our Own ESLint Security Rules. 16% Mislabel Their Own CVSS Score.

What the three labels actually measure

Case 1: a 9.1 that prints LOW — and the honest reason it might be fine

Case 2: a 9.8 with no heuristic excuse at all

Case 3: the one that runs the other way

Why this happens: severity is the one field nobody enriches

The audit, if you want to run it yourself

Reading a severity label like a security engineer, not a triage bot

The config

142,076 Weekly Downloads. Zero Releases Since 2021. Is the Niche Defended?

How do you build an incumbent map?

What does the serverless incumbent map actually show?

When is a quiet package finished instead of dead?

How were the replacement targets actually picked?

Quick reference

References

Related deep dives

We Were Wrong About sls remove

What was the original claim?

What did the measurement actually show?

What is the re-scoped claim?

Why is a narrower claim stronger?

Quick reference

References

Related

25% of My Benchmark Verdict Is an Opinion. Here's the Anatomy.

What are the seven dimensions — and where does each number come from?

How do kilobytes, hook counts, and pass rates become one scale?

What happens when a dimension can't be measured?

Why is Lifecycle Correctness worth 25% — and who decided?

Does the winner survive a ±50% attack on the weights?

Quick reference

`data-slot`: 67 findings, the biggest by far

`HttpOnly` is the boundary

Why this happens: `severity` is the one field nobody enriches