DEV Community: Goqa Ai

Writing acceptance criteria a machine can test

Goqa Ai — Fri, 16 Oct 2026 19:26:28 +0000

Most acceptance criteria are written for humans who already share context. That is why they cannot be turned into tests without a conversation.

The failure mode

The user can reset their password.

This is not testable. It does not say what happens to an unknown email, whether the reset link expires, whether an old link works after a second request, or whether the response differs for a registered and unregistered address — which is an account-enumeration decision, not a UX one.

A developer reads it and picks answers. A tester reads it and picks different answers. The bug report is written months later.

The shape that works

Given/When/Then is not ceremony. It forces three things into the open: the starting state, the trigger, and the observable result.

Scenario: reset link expires after one hour
  Given a password reset was requested 61 minutes ago
  When the user opens the reset link
  Then the page explains the link has expired
  And offers to send a new one

Every ambiguity above is now a decision somebody made on purpose.

Four questions that turn a vague criterion into a testable one

What is true before? Not 'the user is on the page' — what state is the data in?
What exactly triggers it? A click, a timeout, a webhook, a cron?
What can an outsider observe? If the only effect is internal, the test has nothing to assert.
What is the nearest failure? Every scenario should have a sibling that fails.

That fourth one catches the most. Teams write the happy path and stop, because the happy path is what the ticket described.

Then generate

Once criteria are in this shape, generating the suite is mechanical — which is the point. GoQA takes criteria in exactly this form and returns BDD cases you can export to Gherkin, Playwright, or Cypress. But the value is upstream: criteria specific enough to generate from are criteria specific enough to build from.

If you take one thing: write the sibling that fails.

The flaky test that was a tenant-isolation bug

Goqa Ai — Fri, 16 Oct 2026 19:26:20 +0000

A database test in our suite failed about one run in eight. Here is the full path from symptom to cause, including the three hypotheses that were wrong, because the wrong ones are the useful part.

The symptom

✖ a pending approval is returned with its platform and action type
  AssertionError: 0 !== 1

And sometimes, instead:

  AssertionError: '—' !== 'gmail'

Same test, two different failures. That should have been the first clue — one bug producing two shapes usually means the failure is upstream of both assertions.

Wrong hypothesis 1: test contention

The obvious read: something else was hammering the database. We had a dashboard running against the same Postgres container.

Evidence killed it. The tests use a different database, connections were 7 of 100, and query round trips were 0.07s. Three consecutive runs passed with the dashboard up.

Wrong hypothesis 2: a CREATE DATABASE race

Each test file drops and recreates its database in before. Fourteen files run in parallel and Postgres serialises CREATE DATABASE. Plausible.

Also wrong — each suite already uses its own database name, so they were never competing for the same one.

Wrong hypothesis 3: test ordering

Maybe a later test decided the approval before the read ran. But the tests are sequential — no concurrency flag — and the seed happens in a before hook.

The actual cause

Our repository wraps every tenant-scoped read like this:

await connection.query("begin");
await connection.query("select set_config('app.tenant_id', $1, true)", [tenantId]);
const result = await work(scope);
await connection.query("commit");

The connection is a single long-lived psql session, not a pool. So when two of these overlap:

A: begin          -- opens A's transaction
B: begin          -- no-op, Postgres warns, B is now inside A's transaction
A: commit         -- ends the transaction B is still using
B: select ...     -- runs with no transaction at all

That last line is the bug. set_config(..., true) is transaction-local — which is exactly what you want, because it stops a pooled connection leaking one tenant's context into another request. But when the transaction ends underneath a caller, the setting goes with it. Every row-level security policy then sees an empty tenant and matches nothing.

Hence 0 !== 1. And when the outer read landed inside a valid context while the joined read did not, '—' !== 'gmail'.

Why this is not a flaky test

A retry would have made it green. It would also have shipped a path where one caller's query can execute under another caller's tenant context — which is the isolation boundary the whole system rests on.

The test was not unreliable. It was intermittently reporting a real defect.

The fix

Serialise callers per connection, so a transaction cannot be opened inside another:

const queues = new WeakMap<SqlConnection, Promise<unknown>>();

const predecessor = queues.get(connection) ?? Promise.resolve();
let release!: () => void;
const turn = new Promise<void>((r) => { release = r; });
queues.set(connection, turn);
await predecessor.catch(() => undefined);
try {
  return await runTransaction(connection, ctx, work);
} finally {
  release();
}

Three details that matter:

The queue slot is claimed before the first await, or two callers entering together read the same predecessor and run as one.
The predecessor's rejection is swallowed. A failed request handed the connection back; inheriting its error would fail an unrelated caller.
release() is in a finally. An unreleased slot stalls every caller behind it — a deadlock is a worse outcome than the bug.

What I would do differently

Reproduce first. I asserted 'test contention' twice before running a loop that would have told me, and both times the story was plausible enough that I nearly stopped there.

The loop that found it is six lines, and it asserts the run completed — because a half-killed test run reports as 'clean' and looks exactly like a pass.

Testing time without waiting for it

Goqa Ai — Fri, 16 Oct 2026 19:26:10 +0000

Anything with an expiry has a first failure that arrives weeks after the deploy. Password reset links, sessions, trial periods, cache entries, retention jobs, token refresh. All of them are testable in milliseconds, and almost none of them are tested at all.

Why these go untested

The honest reason is that the obvious test is unbearable:

test('reset link expires after an hour', async () => {
  const link = await requestReset(user);
  await sleep(60 * 60 * 1000); // ...no
  expect(await useLink(link)).toBe('expired');
});

So it gets skipped, and the expiry logic ships unverified. Which is how you get links that never expire, or expire immediately, or expire correctly but display the wrong message.

Inject the clock

The fix is to stop reading the current time directly:

// Before — untestable
function isExpired(issuedAt) {
  return Date.now() - issuedAt > ONE_HOUR;
}

// After — testable, and no less readable
function isExpired(issuedAt, now) {
  return now - issuedAt > ONE_HOUR;
}

Now the test is instant:

test('reset link expires after an hour', () => {
  const issued = 1_000_000;
  expect(isExpired(issued, issued + 59 * MINUTE)).toBe(false);
  expect(isExpired(issued, issued + 61 * MINUTE)).toBe(true);
});

The boundaries that matter

Once time is a parameter, test the edges rather than the middle:

Exactly at the boundary. Is 3600000ms expired or valid? Pick one deliberately; > and >= are a real decision.
One unit either side. Catches off-by-one in unit conversion, which is the most common bug in this code.
Negative elapsed time. Clock skew and daylight-saving transitions both produce it. What does your code do when now is before issuedAt? Usually something wrong.
Far future. A token issued and checked a year later should be expired, not overflow.

The trap: only injecting it halfway

The common failure is injecting the clock into the check but not the creation. Then your test controls when the link is validated but not when it was issued, and you are back to real time for half the calculation. Both ends need to be parameters.

Where this generalises

Any non-deterministic input is the same problem: randomness, UUIDs, the filesystem, the network. If a function reads the world directly, it can only be tested in the world. Pass the world in.

That is the whole of dependency injection, minus the framework.

If you want the expiry cases enumerated for you, GoQA generates BDD suites from acceptance criteria — including boundary and negative scenarios, which are the ones that get skipped. No sign-up for the demo.

Starting my journey in public: fixing test reliability

Goqa Ai — Mon, 22 Jun 2026 16:09:59 +0000

Hello everyone,

I’ve spent years watching teams struggle with something that should be simple: trusting their tests.

On paper, everything looks fine.

The test suite is large. Coverage is high. CI is green.

But in reality?

Pipelines fail for no clear reason.

Tests break with small UI changes.

And teams hesitate when it’s time to deploy.

Not because they didn’t write enough tests but because maintaining them became the real problem.

This is something my brother and I have spent a lot of time thinking through and working on together.

That gap stuck with us.

So we decided to build something about it.

I’m a co-founder of GoQA AI, an AI-powered quality engineering platform focused on one thing: making tests reliable again.

We’re building a system that:
• Generates tests from user stories
• Runs them across web and APIs
• Adapts when the product changes (without constant rewrites)

No hype. No “AI replaces engineers” narrative.
Just solving a problem most teams quietly deal with: test maintenance.

We’ve started with web and API testing.
"Mobile is next"

Over the next few weeks, I’ll be sharing what we’re learning as we build:
→ How self-healing actually works
→ Practical ways to reduce flaky tests
→ Where AI helps and where it doesn’t
→ What building this in public really looks like

If your team has ever questioned whether failing tests actually mean something…
You’re not alone.
And it’s a problem worth fixing.

Curious to hear from others here, how do you deal with flaky tests today?