# The Test That Lied for Weeks

Eduardomr — Sat, 19 Sep 2026 19:59:25 +0000

I lead QA for a mobile product. A few months ago I found an end-to-end flow that asserted an element was not visible, using an identifier our app has never generated in its history. The assertion passed on every run, for weeks. It wasn't testing absence — it was testing a typo, and neither the coverage report nor code review had a way to notice.

That's the blind spot no tool was checking for: a test that is structurally incapable of failing. Coverage says the line executed. Review reads the diff for what it says, not for what it forgot to assert. So I built the check myself — Vigía, a GitHub Action written in TypeScript that reads the tests changed in a pull request and flags the ones that can't fail, before the false confidence becomes permanent.

What "can't fail" actually looks like

Six shapes covered ~90% of what I found in real suites, and they're boring on purpose — boring means they slip past a human reviewer:

it('creates the order', async () => {
  const order = await createOrder(payload)
  // ...and nothing else
})

it('applies the discount', () => {
  expect(applyDiscount(100, 10))          // no matcher chained
})

it('cart exists', () => {
  expect(true).toBe(true)                 // tautology
})

it('rejects an unknown coupon', async () => {
  expect(findCoupon('missing')).rejects.toThrow()   // no await — resolves after the test ends
})

Running Vigía against that exact fixture (it lives in the repo as ejemplo/carrito.test.ts) gives you this, unedited:

$ node dist/cli.js ejemplo

ejemplo/carrito.test.ts:8:3   P1  sin-assercion       "calcula el total" no contiene ninguna aserción.
ejemplo/carrito.test.ts:14:5  P1  expect-sin-matcher  expect() sin matcher encadenado.
ejemplo/carrito.test.ts:18:5  P1  tautologia          expect(true) comparado consigo mismo.
ejemplo/carrito.test.ts:21:3  P3  prueba-omitida      "valida el cupón vencido" está omitida.
ejemplo/carrito.test.ts:25:3  P1  prueba-enfocada     "suma con impuestos" usa .only: el resto de la suite no se ejecuta.
ejemplo/carrito.test.ts:31:5  P1  await-faltante      expect(...).rejects sin await: la prueba termina antes de comprobar nada.

6 hallazgo(s): 5 P1, 0 P2, 1 P3

Six defects, six lines, one small file, all green in Jest a second ago.

The one that actually started this

The sixth shape doesn't fit that list, and it's the reason the whole project exists: an orphan negative assertion. expect(queryByTestId('gasto-monto-9999-duplicado')).toBeNull() looks like a real check. It only becomes worthless once you know the source code never produces that identifier — it only produces gasto-monto-${id} for real values. That check requires reading two things at once: what your tests assert, and what your source can actually generate. So Vigía's seventh rule scans the source tree for every testID literal and template prefix, then cross-references every negative assertion against that set. An identifier that appears only in the test is the tell.

Why regex wasn't enough

My first instinct was a regex pass. It broke on the first self-check: the file that tests the "no assertion" rule contains the string expect( inside a comment describing what not to write, and a text match flagged it as a violation of its own rule. Everything is built on the TypeScript compiler API now — ts.createSourceFile, walk the AST, resolve chained calls like it.skip.each back to their base identifier. An expect in a string or a comment isn't a call expression; the parser knows that even when a regex doesn't.

Where it actually stands

I'm not going to dress this up as "running in production catching bugs daily" — it isn't, yet. What it is: seven detection rules, each with a positive and a negative fixture, 129 passing tests across 4 suites, a CI job where Vigía lints its own source on every push, and a working GitHub Action with a real v1 tag you can drop into a workflow today. The next milestone is exactly what you'd expect — pilot it against a live PR stream and see what real-world test suites do to it that fixtures don't.

If you review test suites for a living, or you're the person a green CI run doesn't fully convince, the repo's on GitHub.

DEV Community: Eduardomr

# The Test That Lied for Weeks

What "can't fail" actually looks like

The one that actually started this

Why regex wasn't enough

Where it actually stands