motchalini

Posted on Jul 6

Your AI makes CI green by cheating. I built three GitHub Actions to stop it.

#github #ci #typescript #python

All your checks are green. Before you merge, one question worth asking: is it green because the code was fixed, or because the checker was silenced?

There are two ways to turn a red CI run green. You can fix the problem, or you can make the tool stop reporting it:

Type error? as any or # type: ignore makes it vanish.
Failing test? it.skip or @pytest.mark.skip takes it out of the run. Or it.only quietly skips everything else.
Linter warning? eslint-disable or # noqa and it's quiet.

const user = res.data as any;   // tsc stays green

result = compute()  # type: ignore   — mypy stays green

Silencing is almost always faster than fixing, and AI coding agents (Claude Code, Copilot, Cursor) are very good at fast. To be clear, this isn't a claim that AI is malicious: anything optimizing for "make CI green" — an agent, or a human at 6pm before a release — will sometimes take the shortcut. The problem is that human review is bad at catching it. One as any in a 400-line diff slips through.

So I stopped relying on reviewer willpower and built a mechanical stop: three small GitHub Actions I call the *-ratchet family. Their design stance is the opposite of AI code-review SaaS: no AI, no SaaS, no config.

The ratchet idea

A ratchet is the mechanism that lets a wrench turn one way and never back. All three actions share one loop:

Set a baseline: the number of escape hatches you have today.
If a PR increases the count → the gate fails.
If it decreases → the gate prints IMPROVED, and you lower the baseline to lock it in.
The number can only go down.

The important part: you don't start from zero. Real codebases have Anys and # noqas that exist for defensible reasons. "Keep today's mess, forbid tomorrow's" is an on-ramp a team will actually take — and every cleanup tightens the ratchet by one click.

The three gates

1. type-ratchet — stops type-checker silencing

Counts: TypeScript any (in type position), as any, @ts-ignore, @ts-expect-error; Python Any, # type: ignore.

Why existing tools don't cover it: tsc and mypy --strict don't count their own escape hatches — that's what the hatches are for. Write as any and the type checker goes green without telling anyone it was silenced.

# .github/workflows/type-ratchet.yml
name: Type Ratchet Gate
on:
  pull_request:
    branches: [main]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: motchalini-llc/type-ratchet@v1
        with:
          language: typescript   # python | typescript | auto

Notes: test files (*.test.ts, *.spec.ts) are excluded from the count — mocks legitimately use as any — but you can still pass a typecheck-command (e.g. pnpm exec tsc --noEmit) to type-check everything alongside the count. Baselines are split (baseline-any / baseline-suppress) so you can tighten dynamic types and type-suppressions independently.

One honest caveat: detection is grep-based, so an "any" inside a comment or string can false-positive. That's the price of transparency (you can read exactly what's counted); an opt-in ESLint no-explicit-any backing is on the roadmap for people who want strictness.

2. test-ratchet — stops test dodging

Counts two different sins:

Skipped tests (ratcheted via baseline-skip): Python @pytest.mark.skip/skipif/xfail, pytest.skip(), unittest @skip variants; TypeScript .skip / .todo / .fails on it/test/describe/bench, plus xit / xdescribe.
.only (hard-forbidden at zero, TypeScript): a single it.only makes the runner execute that block and nothing else — the rest of your suite silently stops running while CI stays green. This one isn't baselined; one occurrence is red (disable with forbid-only: false).

Why existing tools don't cover it: eslint-plugin-no-only-tests is the known answer for .only, but it assumes TypeScript and an ESLint setup. I couldn't find a cross-language, zero-config PR gate — and nothing at all that ratchets pytest skips.

# .github/workflows/test-ratchet.yml
name: Test Ratchet Gate
on:
  pull_request:
    branches: [main]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: motchalini-llc/test-ratchet@v1
        with:
          language: typescript   # python | typescript | auto

3. suppress-ratchet — stops linter silencing

Counts: TypeScript eslint-disable (line / next-line / block) and biome-ignore; Python # noqa, # ruff: noqa, # pylint: disable.

Why existing tools don't cover it: linters are designed to respect suppression comments. The more you write, the quieter the linter gets, and nothing in the standard toolchain pushes back on the trend.

# .github/workflows/suppress-ratchet.yml
name: Suppress Ratchet Gate
on:
  pull_request:
    branches: [main]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: motchalini-llc/suppress-ratchet@v1
        with:
          language: typescript      # python | typescript | auto
          baseline-suppress: '12'   # suppressions already in the codebase

Notes: type suppressions (@ts-ignore, # type: ignore) are deliberately not counted here — they belong to type-ratchet, and the same comment shouldn't be double-counted by two gates. Vendor and generated trees (node_modules, dist, .venv, .next, …) are excluded.

Design decisions shared by all three

Zero-dependency bash. Each action is a composite action plus one gate.sh; the machinery is essentially grep and wc. You can read the script and know exactly what's counted — the opposite of a black-box AI reviewer. Your code never leaves your CI.
Zero config. One uses: line. Language is auto-detected from pyproject.toml / tsconfig.json and friends.
One-way only. Increase fails, decrease tightens. Baseline lives in an input or a baseline-file in the repo.
The gates test themselves. Each repo ships clean/dirty fixtures and a self-test workflow asserting clean passes, dirty fails (test- and suppress-ratchet run that matrix for both languages), so the gate can't silently break.
Light version first, heavy version on demand. v1 is a free grep gate. Autofix (mode: fix) and AST-precise detection are on the roadmap, opt-in, with your own LLM API key and a hard cost cap — I'll build them when someone actually asks.
Global count, honestly. The judgment is a repo-wide total; per-file baselines (monorepos) are future work. Coarse, but it's what makes zero-config possible.

Dogfooding numbers

Before publishing I ran all three on my own two repos. Real numbers:

crypto-trading-system (Python, mypy --strict): baseline any=5, type-suppressions 2, skips 0, linter suppressions 4.
meguri (Next.js 15 / React 19, strict TS): any=0 at adoption — so there the same gate runs as keep-it-at-zero insurance rather than a cleanup driver.

The baseline-suppress: 4 was the instructive one. Those four were pre-existing # noqas in test code — F401 unused imports, PLC0415 function-level imports — each left there on purpose. If the gate had demanded zero on day one, I'd have ripped it out by lunch. "Today's 4 are fine; the 5th is not" is what made it adoptable, and that exact number now sits in the README example.

I also opened a demo PR that deliberately added any / as any to watch the gate go red with inline annotations on the offending lines, then closed it. (Deliberate demo — not a caught-the-AI-in-the-act story.)

How they were built

The bash is honestly simple. The hard part was deciding what counts and what doesn't: excluding test files from the any count, keeping type suppressions out of suppress-ratchet, hard-forbidding .only instead of baselining it. Those line-drawing decisions are the product.

Mechanically: solo dev plus Claude Code, two to three days per action, the same playbook three times — build → dogfood on my own repos → publish to the GitHub Marketplace → announce. The skeleton (composite action + gate.sh + clean/dirty self-test + README + roadmap) carries over; only the patterns change. Hence: three siblings.

DEV Community