TestFly Part 3: CI/CD Quality Gates, Parallel Execution & CI Metadata

Enterprise CI/CD with TestFly

In modern continuous delivery pipelines, flaky tests and degraded pass rates must not block deployments silently. TestFly features dedicated CI subsystems:

1. CiEnvironmentDetector: Auto-detects GitHub Actions, Jenkins, GitLab CI, CircleCI, TeamCity, and Bitbucket pipelines.
2. BuildThresholdEnforcer: Enforces strict quality gates on build completion.
3. JUnitXmlReporter: Generates machine-readable TEST-TestFly.xml for CI dashboard integration.

---

1. Configuring CI Quality Gates in testfly.yml

execution:
  mode: local
  parallel: methods
  threadCount: 4
  maxActiveSessions: 4

ci:
  failOnPassRateBelow: 95.0 # Fails the build if pass rate < 95%
  maxFlakyTests: 2          # Fails if more than 2 tests are flaky

If quality thresholds are breached, TestFly throws a BuildQualityGateException and exits with a non-zero code.

---

2. GitHub Actions Multi-Browser Matrix Workflow

name: TestFly E2E Regression

on:
  push:
    branches: [ master, main ]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        browser: [ chrome, firefox ]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: 'temurin'
          java-version: '17'
          cache: 'maven'

      - name: Run TestFly Tests
        run: mvn clean test -Dbrowser.name=${{ matrix.browser }} -Dbrowser.headless=true

      - name: Upload TestFly HTML Report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: testfly-report-${{ matrix.browser }}
          path: target/testfly-report.html