How Much of Your CI Pipeline Is Just Cucumber Scenarios You're Too Afraid to Delete

The CI job just hit 28 minutes. Again.

You pull up the duration report expecting to blame a bloated integration test or a slow environment spin‑up. Instead the longest stage stares back at you: a collection of Cucumber feature files that haven’t caught a real bug in months. Maybe years. They run on every commit, green circle after green circle, while your team mutters about slow pipelines and nobody dares touch them.

Most teams treat those scenarios like documentation. “They describe the system,” someone once said, as if a Gherkin file were a legal contract. Others cling to the sunk cost: a year ago a whole squad spent two sprints writing them, polishing the grammar, aligning step definitions. Deleting them would feel like admitting waste.

Experienced engineers see it differently. They treat a scenario that never fails as a liability you’re paying for on every push. Not neutral. Liable. Compute cycles, developer attention, flake‑debugging time, and the quiet toll it takes on trust in the pipeline.

The principle is blunt: if a test hasn’t failed in the last few sprints, you’re already paying its full cost and receiving nothing in return. That doesn’t mean you delete everything green. But it does mean you audit with the same seriousness you’d use for a memory leak.

What the green wall actually costs

The damage is not abstract. A pipeline bloated with stale scenarios hurts you in five concrete ways.

First, feedback slows. Every extra minute between push and result stretches the loop that tells a developer they’re safe to merge. Multiply across a team and you’re losing hours per week to waiting.

Second, flakiness increases. When you have many scenarios, a single unstable environment variable can produce a handful of failures that are not regressions at all. Engineers learn to retry, then to ignore.

Third, confidence erodes. If half the suite is ceremonial, a genuine failure might be dismissed as “just another flaky test” until it reaches production.

Fourth, maintenance compounds. When you rename a button or re‑orchestrate a flow, someone still has to update the corresponding step definitions. Stale scenarios become tax with every refactor.

Fifth, you lose the ability to reason about coverage. A slim, surgical set of tests tells a clear story about what’s protected. A wall of noise makes it impossible to know what would actually break if you removed something.

How to see the rot

You can’t fix what you don’t measure. The first step is to gather execution data from CI. Most test runners produce a JSON or JUnit report that includes scenario names and outcomes. If your current setup doesn’t, it’s worth a couple of lines of CI script to capture it.

The lowest‑hanging fruit are scenarios that never run at all. Feature files left behind after a feature was rewritten, scenarios that reference pages that no longer exist, step definitions that are unreachable. They’re invisible because the runner skips them silently.

Here’s a TypeScript utility that compares all scenario names in your feature files against what your CI report says actually executed recently. It’s deliberately minimal — you’d want a proper Gherkin parser and unique IDs for production use, but this gives you the shape.

import fs from 'fs';
import path from 'path';

const featureDir = './features';
const reportData = JSON.parse(fs.readFileSync('./ci-report.json', 'utf-8'));
const executedNames = new Set(reportData.scenarios.map((s: any) => s.name));

const stale: string[] = [];
const featureFiles = fs.readdirSync(featureDir).filter(f => f.endsWith('.feature'));
for (const file of featureFiles) {
  const content = fs.readFileSync(path.join(featureDir, file), 'utf-8');
  const scenarioPattern = /Scenario:\s*(.+)$/gm;
  let match;
  while ((match = scenarioPattern.exec(content)) !== null) {
    const name = match[1].trim();
    if (!executedNames.has(name)) {
      stale.push(`${file}: "${name}"`);
    }
  }
}

console.log(`Found ${stale.length} scenario(s) never seen in the last CI run:`);
console.log(stale.join('\n'));

Running that once gives you a list of candidates for immediate deletion. If a scenario isn’t being executed, it’s not guarding anything. Cut it.

The harder category is the scenario that runs but never fails. For that you need a window of history. A small Python script can parse a directory of archived reports and flag any scenario with zero failures in, say, the last 90 days. The logic is straightforward: accumulate pass/fail counts per unique scenario ID and mark those with failure_count == 0. I won’t paste the full script here, but you can write it in twenty lines.

Once you have your stale list, don’t delete everything at once and trigger alarm bells. Remove the bottom 5% and watch a few cycles. Often nothing changes except the pipeline runs a little faster.

The Cucumber tax on simple flows

You know what a login scenario looks like in Gherkin, and you know how much ceremony surrounds it. A feature file with background steps, a half‑dozen step definitions scattered across files, hooks for driver management, perhaps custom parameter types. All to say “fill two fields, click a button, see a dashboard.”

Now look at the same intent expressed as a Playwright test.

import { test, expect } from '@playwright/test';

test('successful login', async ({ page }) => {
  await page.goto('/login');
  await page.fill('[data-testid="username"]', 'user');
  await page.fill('[data-testid="password"]', 'pass');
  await page.click('[data-testid="login-button"]');
  await expect(page.locator('[data-testid="dashboard"]')).toBeVisible();
});

This isn’t a dig at BDD as a practice for collaboration. It’s a recognition that many teams used Cucumber to write UI tests — often years ago when the tooling was different — and those tests are now just verbose wrappers around interactions that the framework itself describes more clearly.

The problem compounds when step definitions grow organically. “Given I