DEV Community: Goqa Ai

What good Gherkin looks like, and the four ways it goes wrong

Goqa Ai — Tue, 20 Oct 2026 14:18:22 +0000

Gherkin gets a bad reputation from teams who adopted it, wrote scenarios nobody could read, and concluded the format was the problem. It usually isn't. Here are the four failure modes and what to do instead.

Failure 1: Gherkin as a UI script

# Bad
When I click the button with id "submit-btn"
And I wait 2 seconds
And I click the third row in the table

This breaks when the button gets renamed, and it describes mechanics rather than behaviour. Six months later nobody can tell what it was protecting.

# Better
When the customer submits the order
Then the order appears in their history with status "pending"

The rule: Gherkin describes what the user achieves, the step definitions describe how. If your feature file mentions a CSS selector, the abstraction has leaked.

Failure 2: Given as a setup dumping ground

# Bad
Given a user exists
And the user has an email
And the user is verified
And the user has a subscription
And the subscription is active
And the subscription has 5 seats
And 3 seats are used

Nobody reads this. Collapse it into one meaningful state:

# Better
Given an active subscription with 3 of 5 seats used

The detail lives in the step definition, where it belongs. A Given should name a situation, not enumerate a database.

Failure 3: Multiple Whens

# Bad
When the user adds an item
And the user applies a discount code
And the user checks out
Then the total is correct

Three triggers means three behaviours means the failure tells you nothing about which one broke. And 'the total is correct' is not an assertion — correct according to what?

One When per scenario. If you genuinely need a sequence, the earlier steps are Givens — they are setup, not the thing under test.

Failure 4: Then with no observable

# Bad
Then the system processes the payment
Then the data is saved correctly

Neither is observable. 'Processes' and 'correctly' cannot fail in any specific way, so the step definition ends up asserting something arbitrary, and the scenario passes forever regardless of behaviour.

# Better
Then the order status is "paid"
And a receipt is sent to the customer's email address

The shape that works

Scenario: discount code cannot be applied twice
  Given an order with the code SAVE10 already applied
  When the customer applies SAVE10 again
  Then the order total is unchanged
  And the customer is told the code is already applied

One situation. One trigger. Two observable outcomes. Readable by someone who has never seen the codebase — which is the actual test of whether your Gherkin is any good.

Where generation helps, and where it doesn't

Once criteria are in this shape, enumerating the surrounding cases is mechanical: the expired code, the invalid code, the code that applies to no items in the basket, the code used by a different customer. That is what a generator is genuinely good at — GoQA produces these from criteria and exports them to Gherkin, Playwright, or Cypress.

What it will not do is decide that 'used by a different customer' is the one carrying real risk. That is still your call, and it is the more important half.

Deleting the option instead of defaulting it off

Goqa Ai — Mon, 19 Oct 2026 16:04:22 +0000

We built a headless browser worker that renders our own pages for screenshots and link checks. Here is the design decision I keep coming back to.

The usual approach

interface ContextOptions {
  viewport: { width: number; height: number };
  userAgent: string;
  proxy?: ProxyConfig;        // defaults to none
  credentials?: Credentials;  // defaults to none
}

This is fine, and it is what most libraries do. The proxy is off unless you set it. The credentials are absent unless you supply them.

Why we did not do that

A configuration option that must never be set is a configuration option that will eventually be set.

Not maliciously. Someone six months from now has a legitimate-sounding reason — a rate limit, a geo-restricted page, a login wall on a page we genuinely own — and the option is right there with a sensible-looking type. The code review is one line. Nobody remembers the argument that produced the default.

What we shipped instead

export interface ContextOptions {
  readonly viewport: { readonly width: number; readonly height: number };
  readonly userAgent: string;
  readonly egressAllowlist: readonly string[];
  readonly timeoutMs: number;
}

Four fields. There is no proxy field and no credential field anywhere in the package. A driver implementation cannot offer what the interface never asks for.

There is a test asserting the exact shape, so adding a fifth field is a deliberate act with a failing test attached rather than an autocomplete accident.

The allowlist is not a default either

egressAllowlist is required, and it is derived from the worker's owned-surface policy rather than passed by the caller — so a task cannot widen its own reach. Navigation outside it is refused before a browsing context is created, and every subresource is failed at request time:

case "Fetch.requestPaused": {
  if (this.#hostAllowed(url)) await continueRequest(requestId);
  else { this.#blocked.push(url); await failRequest(requestId, "BlockedByClient"); }
}

That second half matters more than it looks. Without it, rendering a page of ours that embeds a third-party script means our worker makes that third-party request. The allowlist would be true of navigation and false of everything the page pulls in.

The cost

This is genuinely restrictive. The worker cannot log into anything, cannot render a page behind auth, and cannot follow a link off our own domain. Several plausible features are now impossible without an explicit, visible change to a shared interface.

That is the trade. We decided the class of thing we want to be unable to do is worth more than the features we are giving up.

When this applies

Not everywhere. Most options should be options. This pattern earns its cost when:

The capability would be a terms violation, a privacy problem, or a security incident rather than a bug
The pressure to enable it will arrive later, under a deadline, from someone who wasn't in this conversation
The blast radius is other people rather than you

If all three are true, delete the field.

A pragmatic accessibility checklist for teams with no a11y specialist

Goqa Ai — Mon, 19 Oct 2026 16:04:16 +0000

WCAG 2.2 has 87 success criteria. If you are a four-person team shipping weekly, you are not going to work through 87 of anything. Here is the subset that catches most real harm, ordered by how often it goes wrong.

1. Every interactive thing is reachable by keyboard

Put the mouse away. Tab through a complete user journey.

You are checking three things: can you reach every control, can you see which one has focus, and can you get out of everything you get into. A modal you can tab into but not out of is a trap, and it is common.

The usual cause is a div with a click handler. It has no tab stop and no role. Use a button.

2. Every input has a real label

<!-- No. The label is decorative text that happens to sit nearby. -->
<span>Email</span><input type="email">

<!-- Yes. -->
<label for="email">Email</label><input id="email" type="email">

Placeholder text is not a label. It disappears when the user types, which is exactly when they need it.

3. Images say something useful, or nothing at all

Meaningful image: alt describes what it conveys, not what it depicts
Decorative image: alt="" — empty, present. A missing alt makes screen readers read the filename
Image of text: put the text in the page instead

4. Contrast is 4.5:1 for body text

AA needs 4.5:1 for normal text and 3:1 for large. The usual failure is muted grey body text — a design-system default nobody revisited. Devtools will tell you in the colour picker.

5. The heading outline makes sense on its own

Screen-reader users navigate by headings the way you skim by eye. One h1, no skipped levels, and heading tags used for structure rather than for font size.

Read your headings in order with nothing else. If they do not describe the page, your outline is broken.

6. Colour is never the only signal

A red border on an invalid field is invisible to a colour-blind user, and to anyone on a washed-out screen in daylight. Add an icon, or text.

7. Focus is visible

If you find outline: none in your CSS with nothing replacing it, that is a WCAG 2.4.7 failure and a keyboard user is now navigating blind.

What this list does not cover

A lot. Live regions, complex widget semantics, reduced motion, cognitive load, and everything requiring judgement about whether your alt text is actually good.

The honest position: this checklist plus an automated audit gets you past the failures that cause the most harm per unit of effort. It does not make you accessible, and anyone who tells you a tool does is selling something.

Automated tools — including ours — catch roughly a third of issues. Run one, then do the keyboard pass yourself.

Build your test distribution from your incidents, not from a diagram

Goqa Ai — Mon, 19 Oct 2026 16:03:56 +0000

The test pyramid — many unit tests, fewer integration tests, few end-to-end tests — is quoted as though the ratios were derived from something. They were not. It is a heuristic about cost and speed, and it is a reasonable default.

But it is a default, and most teams never check it against their own evidence.

The exercise

Open your last twenty incidents, bugs, or hotfixes. For each one, answer a single question: what kind of test would have caught this before it shipped?

Categorise honestly:

A unit test — logic inside a function was wrong
An integration test — two components disagreed about a contract
An end-to-end test — the behaviour only breaks in a real browser or a real flow
A contract test — an external service changed
None of the above — a config error, an infrastructure failure, a data problem

That last bucket matters, and it is usually larger than people expect. If a third of your incidents are configuration, no test distribution fixes them and you should be spending that effort on deploy validation instead.

What teams usually find

Three patterns come up repeatedly.

The integration-heavy team. Their bugs are almost all seams — an API returning a shape the frontend did not expect, a migration that broke a query, an enum value nobody handled. They have thousands of unit tests, excellent coverage, and their bugs live in the gaps between the units. The pyramid told them to build more of what was already working.

The end-to-end-heavy team. Their failures are in real browser behaviour, real devices, real user flows. The slow expensive tests at the top of the pyramid are the ones earning their keep, and they have spent two years feeling guilty about an inverted pyramid that is correct for their product.

The config team. Half their incidents are not test-shaped at all. Their effort belongs in staged rollouts, health checks, and making bad configuration refuse to boot.

The point is not that the pyramid is wrong

For many teams it is roughly right, and if your evidence produces a pyramid then you have earned it rather than assumed it.

The point is that the shape should be an output of your failure history, not an input from a diagram drawn about somebody else's system in 2009.

Doing it continuously

Add one line to your incident template: what kind of test would have caught this?

After a year you have a distribution derived from your own product rather than a convention. It costs one line and it is the most useful testing metric I know of — considerably more useful than coverage percentage, which tells you what ran rather than what was checked.

Writing acceptance criteria a machine can test

Goqa Ai — Fri, 16 Oct 2026 19:26:28 +0000

Most acceptance criteria are written for humans who already share context. That is why they cannot be turned into tests without a conversation.

The failure mode

The user can reset their password.

This is not testable. It does not say what happens to an unknown email, whether the reset link expires, whether an old link works after a second request, or whether the response differs for a registered and unregistered address — which is an account-enumeration decision, not a UX one.

A developer reads it and picks answers. A tester reads it and picks different answers. The bug report is written months later.

The shape that works

Given/When/Then is not ceremony. It forces three things into the open: the starting state, the trigger, and the observable result.

Scenario: reset link expires after one hour
  Given a password reset was requested 61 minutes ago
  When the user opens the reset link
  Then the page explains the link has expired
  And offers to send a new one

Every ambiguity above is now a decision somebody made on purpose.

Four questions that turn a vague criterion into a testable one

What is true before? Not 'the user is on the page' — what state is the data in?
What exactly triggers it? A click, a timeout, a webhook, a cron?
What can an outsider observe? If the only effect is internal, the test has nothing to assert.
What is the nearest failure? Every scenario should have a sibling that fails.

That fourth one catches the most. Teams write the happy path and stop, because the happy path is what the ticket described.

Then generate

Once criteria are in this shape, generating the suite is mechanical — which is the point. GoQA takes criteria in exactly this form and returns BDD cases you can export to Gherkin, Playwright, or Cypress. But the value is upstream: criteria specific enough to generate from are criteria specific enough to build from.

If you take one thing: write the sibling that fails.

The flaky test that was a tenant-isolation bug

Goqa Ai — Fri, 16 Oct 2026 19:26:20 +0000

A database test in our suite failed about one run in eight. Here is the full path from symptom to cause, including the three hypotheses that were wrong, because the wrong ones are the useful part.

The symptom

✖ a pending approval is returned with its platform and action type
  AssertionError: 0 !== 1

And sometimes, instead:

  AssertionError: '—' !== 'gmail'

Same test, two different failures. That should have been the first clue — one bug producing two shapes usually means the failure is upstream of both assertions.

Wrong hypothesis 1: test contention

The obvious read: something else was hammering the database. We had a dashboard running against the same Postgres container.

Evidence killed it. The tests use a different database, connections were 7 of 100, and query round trips were 0.07s. Three consecutive runs passed with the dashboard up.

Wrong hypothesis 2: a CREATE DATABASE race

Each test file drops and recreates its database in before. Fourteen files run in parallel and Postgres serialises CREATE DATABASE. Plausible.

Also wrong — each suite already uses its own database name, so they were never competing for the same one.

Wrong hypothesis 3: test ordering

Maybe a later test decided the approval before the read ran. But the tests are sequential — no concurrency flag — and the seed happens in a before hook.

The actual cause

Our repository wraps every tenant-scoped read like this:

await connection.query("begin");
await connection.query("select set_config('app.tenant_id', $1, true)", [tenantId]);
const result = await work(scope);
await connection.query("commit");

The connection is a single long-lived psql session, not a pool. So when two of these overlap:

A: begin          -- opens A's transaction
B: begin          -- no-op, Postgres warns, B is now inside A's transaction
A: commit         -- ends the transaction B is still using
B: select ...     -- runs with no transaction at all

That last line is the bug. set_config(..., true) is transaction-local — which is exactly what you want, because it stops a pooled connection leaking one tenant's context into another request. But when the transaction ends underneath a caller, the setting goes with it. Every row-level security policy then sees an empty tenant and matches nothing.

Hence 0 !== 1. And when the outer read landed inside a valid context while the joined read did not, '—' !== 'gmail'.

Why this is not a flaky test

A retry would have made it green. It would also have shipped a path where one caller's query can execute under another caller's tenant context — which is the isolation boundary the whole system rests on.

The test was not unreliable. It was intermittently reporting a real defect.

The fix

Serialise callers per connection, so a transaction cannot be opened inside another:

const queues = new WeakMap<SqlConnection, Promise<unknown>>();

const predecessor = queues.get(connection) ?? Promise.resolve();
let release!: () => void;
const turn = new Promise<void>((r) => { release = r; });
queues.set(connection, turn);
await predecessor.catch(() => undefined);
try {
  return await runTransaction(connection, ctx, work);
} finally {
  release();
}

Three details that matter:

The queue slot is claimed before the first await, or two callers entering together read the same predecessor and run as one.
The predecessor's rejection is swallowed. A failed request handed the connection back; inheriting its error would fail an unrelated caller.
release() is in a finally. An unreleased slot stalls every caller behind it — a deadlock is a worse outcome than the bug.

What I would do differently

Reproduce first. I asserted 'test contention' twice before running a loop that would have told me, and both times the story was plausible enough that I nearly stopped there.

The loop that found it is six lines, and it asserts the run completed — because a half-killed test run reports as 'clean' and looks exactly like a pass.

Testing time without waiting for it

Goqa Ai — Fri, 16 Oct 2026 19:26:10 +0000

Anything with an expiry has a first failure that arrives weeks after the deploy. Password reset links, sessions, trial periods, cache entries, retention jobs, token refresh. All of them are testable in milliseconds, and almost none of them are tested at all.

Why these go untested

The honest reason is that the obvious test is unbearable:

test('reset link expires after an hour', async () => {
  const link = await requestReset(user);
  await sleep(60 * 60 * 1000); // ...no
  expect(await useLink(link)).toBe('expired');
});

So it gets skipped, and the expiry logic ships unverified. Which is how you get links that never expire, or expire immediately, or expire correctly but display the wrong message.

Inject the clock

The fix is to stop reading the current time directly:

// Before — untestable
function isExpired(issuedAt) {
  return Date.now() - issuedAt > ONE_HOUR;
}

// After — testable, and no less readable
function isExpired(issuedAt, now) {
  return now - issuedAt > ONE_HOUR;
}

Now the test is instant:

test('reset link expires after an hour', () => {
  const issued = 1_000_000;
  expect(isExpired(issued, issued + 59 * MINUTE)).toBe(false);
  expect(isExpired(issued, issued + 61 * MINUTE)).toBe(true);
});

The boundaries that matter

Once time is a parameter, test the edges rather than the middle:

Exactly at the boundary. Is 3600000ms expired or valid? Pick one deliberately; > and >= are a real decision.
One unit either side. Catches off-by-one in unit conversion, which is the most common bug in this code.
Negative elapsed time. Clock skew and daylight-saving transitions both produce it. What does your code do when now is before issuedAt? Usually something wrong.
Far future. A token issued and checked a year later should be expired, not overflow.

The trap: only injecting it halfway

The common failure is injecting the clock into the check but not the creation. Then your test controls when the link is validated but not when it was issued, and you are back to real time for half the calculation. Both ends need to be parameters.

Where this generalises

Any non-deterministic input is the same problem: randomness, UUIDs, the filesystem, the network. If a function reads the world directly, it can only be tested in the world. Pass the world in.

That is the whole of dependency injection, minus the framework.

If you want the expiry cases enumerated for you, GoQA generates BDD suites from acceptance criteria — including boundary and negative scenarios, which are the ones that get skipped. No sign-up for the demo.

Starting my journey in public: fixing test reliability

Goqa Ai — Mon, 22 Jun 2026 16:09:59 +0000

Hello everyone,

I’ve spent years watching teams struggle with something that should be simple: trusting their tests.

On paper, everything looks fine.

The test suite is large. Coverage is high. CI is green.

But in reality?

Pipelines fail for no clear reason.

Tests break with small UI changes.

And teams hesitate when it’s time to deploy.

Not because they didn’t write enough tests but because maintaining them became the real problem.

This is something my brother and I have spent a lot of time thinking through and working on together.

That gap stuck with us.

So we decided to build something about it.

I’m a co-founder of GoQA AI, an AI-powered quality engineering platform focused on one thing: making tests reliable again.

We’re building a system that:
• Generates tests from user stories
• Runs them across web and APIs
• Adapts when the product changes (without constant rewrites)

No hype. No “AI replaces engineers” narrative.
Just solving a problem most teams quietly deal with: test maintenance.

We’ve started with web and API testing.
"Mobile is next"

Over the next few weeks, I’ll be sharing what we’re learning as we build:
→ How self-healing actually works
→ Practical ways to reduce flaky tests
→ Where AI helps and where it doesn’t
→ What building this in public really looks like

If your team has ever questioned whether failing tests actually mean something…
You’re not alone.
And it’s a problem worth fixing.

Curious to hear from others here, how do you deal with flaky tests today?