The test was green. The code had never worked.

Appstruct — Wed, 23 Sep 2026 15:24:00 +0000

Last week I wrote a test, watched it pass, and shipped a feature that was broken the entire time.

The test wasn't wrong. It asserted exactly the right thing, in the right words, about the right values. It just never touched the code it was supposed to be testing — and nothing about a green checkmark tells you that.

Here's the shape of it, because once you've seen it you'll find it in your own repo within the hour.

The setup

We build authorize URLs for a pile of OAuth providers. Most of them want space-separated scopes, because that's what OAuth 2.0 specifies. A couple of them — Meta's Threads, TikTok — document a comma instead.

Getting this wrong is not loud. The whole scope string arrives as one unrecognised scope, so the consent screen grants nothing, and you find out much later from a permissions error that points somewhere else entirely.

So the separator became a property of the provider:

// oauth-providers.ts export const PROVIDERS = { alpha: { scopes: ['read', 'write'] }, // space, the default beta: { scopes: ['read', 'write'], scopeSeparator: ',' }, // comma }
// controller.ts const params = new URLSearchParams({ scope: provider.scopes.join(provider.scopeSeparator ?? ' '), // ... })

And a test:
`const joined = (key: string) => {
const p = PROVIDERS[key]
return p.scopes.join(p.scopeSeparator ?? ' ')
}

it('beta joins with a comma, as the docs require', () => {
expect(joined('beta')).toBe('read,write')
})

it('alpha keeps the OAuth 2.0 default', () => {
expect(joined('alpha')).toBe('read write')
})`

Green. Both of them. Ship it.

The problem

Read the test helper again:

return p.scopes.join(p.scopeSeparator ?? ' ')

That's not a call into the controller. That's a second copy of the controller's logic, living in the test file.

Which means: go back to controller.ts, change it to the old hardcoded .join(' '), and run the suite.

Still green.

The tests assert that PROVIDERS is shaped correctly, and it is. They say nothing whatsoever about the code that builds the URL. I could delete the separator from the controller entirely and ship a broken integration with a full green run behind it.

I know this because I did exactly that, on purpose, to check — and I very nearly didn't.

The fix is boring

One implementation, called by both:

`// oauth-providers.ts
export const scopeParam = (p: OAuthProvider): string =>
p.scopes.join(p.scopeSeparator ?? ' ')
// controller.ts
scope: scopeParam(provider),

// the test
const joined = (key: string) => scopeParam(PROVIDERS[key])`

Nothing about the assertions changed. The expect lines are identical. But now breaking that join turns three tests red instead of zero, because there is only one join left to break.

The rule I use now

For every test you write, name the single line of source whose mutation turns it red.

If you can't name that line, you haven't written a test. You've written a description of the current behaviour that will agree with any future behaviour too.

And then — this is the part people skip — actually go and break that line. Watch it fail. Put it back. It takes twenty seconds and it is the only thing that separates a regression guard from decoration.

I now do this for anything load-bearing. It's caught a lot:

A picker that looked up a pending record by id, with the owner check as a separate if afterwards. Deleting the check kept the suite green. Moving the owner into the where clause made two tests fail when removed.
An adapter that called a token-upgrade endpoint and then handed the old token to everything downstream. Perfectly plausible in review. One mutation, one red test.
Neither of those is a clever bug. Both are the kind that ship.

The second trap: asserting the wrong true thing
There's a cousin of this worth naming, because mutation testing won't save you from it.

We store an OAuth token with an expiry. One provider's token exchange returns a token that lives one hour — and the response carries no expires_in field at all, so nothing in it announces that. Our code stored the token and wrote a 60-day expiry next to it, a number we had made up.

A test like this passes:

expect(info.expiresAt).toBeInstanceOf(Date) expect(daysUntil(info.expiresAt)).toBeCloseTo(60)

It is green, it mutates correctly, and it is worthless — because it asserts that the code does what the code does. The constant under test and the constant in the assertion came from the same place: a guess.

What it produced in practice: connect the account, publish fine for an hour, then fail every scheduled job for two months while the UI shows a healthy connection.

The only test that helps here has an oracle from outside the code — in this case the provider's documented lifetime, and the exchange call you're supposed to make. The test stopped being "is expiresAt set" and became "did we perform the upgrade, and is the stored token the upgraded one, not the short-lived one".

Mutation testing checks that your test is connected to your code. It cannot check that your code is connected to reality. Those are two different jobs and you need both.

Try it on your own repo

Pick the test you'd be most upset to lose. Open the file it covers. Break one line — flip a boolean, drop a !, hardcode a return.

Run the suite.

If it's still green, you just learned something more useful than the test ever told you.

I'm building AppStruct — a workspace where the docs, spreadsheets, slides, whiteboards, CRM and team chat are one product instead of six subscriptions that don't talk to each other. The bugs above are from its integration layer, which is where the interesting ones live. If you want to see what the engineering behind it actually looks like, that's the honest sample.

DEV Community: Appstruct

The test was green. The code had never worked.