DEV Community: Shanawaz Mohammed

How to Add Quality Gates to a GitHub Actions Pipeline (Step by Step)

Shanawaz Mohammed — Mon, 05 Oct 2026 17:58:19 +0000

Most CI pipelines run tests. Far fewer pipelines actually stop bad code from reaching production. The difference is a quality gate: a check that must pass before the pipeline is allowed to move to the next stage.

In this tutorial, you'll build a GitHub Actions pipeline with four gates:

Lint: code style and obvious errors
Unit tests + coverage threshold: fail if coverage drops below a minimum
API tests: verify the running service behaves correctly
Manual approval: a human signs off before production

By the end, a failed gate will block deployment automatically.

Prerequisites

A GitHub repository with a small Python web service (the same ideas apply to any language)
Basic familiarity with YAML
pytest and pytest-cov in your requirements.txt

Step 1: Create the workflow file

Create .github/workflows/pipeline.yml:

# .github/workflows/pipeline.yml
name: CI/CD Pipeline

on:
  push:
    branches: [main]
  pull_request:

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install ruff
      - run: ruff check .

This first job is your cheapest gate. Linting takes seconds, so it should run first and fail fast.

Step 2: Add unit tests with a coverage threshold

Add a second job. The key line is --cov-fail-under=80, which makes pytest exit with an error if coverage drops below 80%.

  unit-tests:
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: pytest tests/unit --cov=app --cov-fail-under=80

needs: lint creates the gate: this job only starts if linting passed.

Tip: Don't start with an aggressive threshold on a legacy codebase. Measure your current coverage, set the threshold slightly below it, and raise it over time. A gate that always fails gets disabled.

Step 3: Add API tests against the running service

Unit tests check functions in isolation. API tests check that the service actually works when it runs. Start the app in the background, wait for it to be healthy, then run the tests:

  api-tests:
    runs-on: ubuntu-latest
    needs: unit-tests
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - name: Start service
        run: |
          uvicorn app.main:app --port 8000 &
          for i in $(seq 1 30); do
            curl -sf http://localhost:8000/health && break
            sleep 1
          done
      - run: pytest tests/api --base-url http://localhost:8000

The health-check loop matters. Without it, tests often start before the server is ready, which is one of the most common causes of flaky pipelines.

Step 4: Require manual approval before production

GitHub environments let you require a reviewer before a job runs. In your repository, go to Settings → Environments → New environment, name it production, and add yourself under Required reviewers.

Then reference it in a deploy job:

  deploy:
    runs-on: ubuntu-latest
    needs: api-tests
    if: github.ref == 'refs/heads/main'
    environment: production
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/deploy.sh

Now the pipeline pauses after the API tests pass and waits for an approval in the GitHub UI.

Step 5: Protect the main branch

Gates only work if nobody can bypass them. Under Settings → Branches, add a branch protection rule for main and enable Require status checks to pass before merging. Select lint, unit-tests and api-tests.

Pull requests can no longer be merged while any gate is failing.

What you built

Gate	Catches	Typical runtime
Lint	Style issues, unused imports, syntax errors	Seconds
Unit tests + coverage	Logic bugs, untested code	1–3 minutes
API tests	Integration and contract failures	2–5 minutes
Manual approval	Business or timing risk	Human decision

Ordering gates from cheapest to most expensive keeps feedback fast: most problems are caught in the first minute, and expensive checks only run on code that is already in reasonable shape.

Next steps

Run API tests in parallel with a matrix strategy as the suite grows.
Publish test reports as artifacts so failures are easy to debug.
Add a security scanning gate (for example, dependency scanning) before deploy.

Audit-Ready CI/CD: What Regulated Industries Teach Us About Shipping Software

Shanawaz Mohammed — Mon, 05 Oct 2026 17:57:29 +0000

In many startups, a release is a merge and a deploy button. In a regulated industry like banking, every release must also answer a set of questions, often months later, from an auditor:

Who approved this change?
What exactly was deployed?
Was it tested, and where is the evidence?
Could anyone have bypassed the process?

For years, many teams answered these questions with spreadsheets, screenshots and change-request forms filled in by hand. That approach is slow, error-prone and painful for engineers.

My view, after years of working on QA and DevOps in a regulated environment: a well-designed CI/CD pipeline is the best compliance tool you have. And the practices that make a pipeline audit-ready make it better for every team, regulated or not.

Principle 1: The pipeline is the only road to production

If engineers can deploy from their laptops, no amount of documentation will satisfy an auditor, or protect you from mistakes.

In practice, that means:

Production credentials live only in the CI/CD system, never on developer machines.
Branch protection prevents direct pushes to the main branch.
Every production change, including configuration and infrastructure, goes through the same pipeline.

Once the pipeline is the only road, its logs become a complete history of what changed in production.

Principle 2: Separation of duties, enforced by tooling

A classic control in regulated environments is that the person who writes a change should not be the only person who approves it.

Modern tooling makes this easy to enforce rather than just document:

Require at least one reviewer on pull requests, and prevent authors from approving their own.
Use protected deployment environments with required approvers for production.
Restrict who can change the pipeline definitions themselves, because whoever controls the pipeline controls the controls.

Principle 3: Evidence is generated, not collected

The biggest shift is moving from collecting evidence after the fact to generating it automatically during every run.

For each release, the pipeline can produce and store:

Evidence	Source
Linked ticket or change request	Commit message or PR metadata
Reviewer and approver identities	Pull request and environment approvals
Test results and coverage	JUnit/coverage reports stored as artifacts
Security scan results	Dependency and container scan reports
Exact artifact deployed	Image digest or build checksum
Deployment time and target	Pipeline logs

When an auditor asks about a release, you export a record instead of reconstructing history from memory.

Principle 4: Immutable, traceable artifacts

"We deployed version 2.3" is not good enough. Was it built from the same commit that was tested? Was it rebuilt between environments?

Build once, then promote the same artifact through test, staging and production, identified by a checksum or image digest rather than a mutable tag like latest. That gives you a clear chain: this commit → this build → these tests → this deployment.

Principle 5: Quality gates are controls

Automated tests, coverage thresholds and security scans are not just engineering hygiene. In a regulated context, they are the controls. When a gate is defined in code, versioned and enforced on every run, it is far more reliable than a manual checklist.

The flip side: any gate that can be skipped needs its own control. If there is an emergency bypass, it should require extra approval and be logged and reviewed afterwards.

Why this matters outside banking

You don't need a regulator to benefit from these practices. Teams that adopt them get:

Faster incident investigation ("what changed?" has an immediate answer)
Safer releases through consistent, enforced checks
Easier onboarding, because the process lives in code rather than tribal knowledge
A head start if a customer or partner ever asks for SOC 2 or ISO 27001 evidence

Compliance is often seen as the enemy of speed. In my experience, the opposite is true: automating compliance into the pipeline removes manual work, and lets teams ship more often with more confidence.

6 Practical Ways to Fix Flaky Tests in Your CI Pipeline

Shanawaz Mohammed — Mon, 05 Oct 2026 17:56:56 +0000

A flaky test passes sometimes and fails sometimes, with no change to the code. One flaky test is an annoyance. Twenty of them destroy trust in your pipeline: developers start clicking "re-run" by reflex, and real failures slip through because everyone assumes it's "just flakiness again."

After many years working on QA and release pipelines, I've found that most flaky tests come from a small number of root causes. Here are six practical fixes, roughly in the order you should try them.

1. Measure before you fix

You can't fix what you can't see. Before changing any tests, find out which tests are flaky and how often.

A simple approach: re-run your suite several times on the same commit and record which tests produce different results.

# Run the suite 10 times and keep each JUnit report
for i in $(seq 1 10); do
  pytest tests/ --junitxml=reports/run-$i.xml || true
done

Any test that both passed and failed across those runs is flaky. Many CI platforms and test-reporting tools can also track this automatically over time. Rank the flaky tests by failure rate and start with the worst offenders.

2. Replace fixed sleeps with explicit waits

This is the single most common cause I see, especially in UI and API tests:

# Flaky: assumes the page loads within 2 seconds
time.sleep(2)
button = driver.find_element(By.ID, "submit")

On a busy CI runner, two seconds is sometimes not enough. Wait for a condition instead of a duration:

# Stable: waits up to 10 seconds, but continues as soon as the button is clickable
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

button = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.ID, "submit"))
)

The same principle applies to services: poll a health endpoint until it responds rather than sleeping for a fixed time after startup.

3. Make tests independent of each other

If test B only passes when test A runs first, you have hidden shared state. It usually shows up when tests run in parallel or in a different order.

Common culprits:

Tests that share a database row or user account
Global variables or singletons modified by one test
Files written to a shared temporary folder

Fix: each test creates its own data and cleans it up. In pytest, fixtures are the natural tool:

import uuid
import pytest

@pytest.fixture
def new_customer(api_client):
    customer = api_client.create_customer(name=f"test-{uuid.uuid4()}")
    yield customer
    api_client.delete_customer(customer.id)

A quick way to expose order dependencies is to run your suite in random order (for example with the pytest-randomly plugin) and see what breaks.

4. Control time, randomness and external services

Tests that depend on things outside your control will eventually fail:

Time: a test that checks "created today" can fail when run near midnight. Freeze or inject the clock.
Randomness: seed random generators so failures are reproducible.
External APIs: a third-party sandbox that is slow or down will fail your build. Mock it in unit tests, and keep a small number of real integration tests in a separate stage.

5. Quarantine, don't ignore

When a flaky test can't be fixed immediately, move it into a quarantine group that still runs but does not block the pipeline:

@pytest.mark.quarantine
def test_payment_callback_timeout():
    ...

# Blocking stage: everything except quarantined tests
pytest -m "not quarantine"

# Non-blocking stage: quarantined tests, results reported but not enforced
pytest -m quarantine || true

Two rules make quarantine work:

Every quarantined test has an owner and a ticket.
The quarantine list is reviewed regularly. Quarantine is a waiting room, not a graveyard.

6. Treat automatic retries as a last resort

Retry plugins can make the pipeline green again quickly, but they hide the problem. If you do use retries, limit them (one retry at most), log every retry, and track retried tests as flaky so they still get fixed.

The payoff

Fixing flakiness is unglamorous work, but the return is large. A pipeline that developers trust gets faster feedback, fewer "re-run and hope" cycles, and real failures that get investigated instead of ignored.

Start small: measure this week, fix your top five flaky tests next week, and add a quarantine process so new flakiness never quietly piles up again.