DEV Community

Shell QA
Shell QA

Posted on

End-to-End Setup Guide: Integrating Playwright + Cucumber with Harness CI

#testing #devops #playwright #ci

Integrating end-to-end (E2E) automation suites into enterprise CI/CD pipelines requires robust reporting, dynamic execution controls, and seamless artifact management.

Here is a guide on setting up a Node.js + Playwright + Cucumber.js test suite using Harness CI, configured with dual-repository dependencies, parallel execution capabilities, and dashboard-ready reporting.

Key Architectural Setup

  • Two-Repo Architecture:

    • Repository A (Application Automation Repo): Contains application-specific feature files, page objects, and pipeline definitions.
    • Repository B (Shared Framework Repo): Hosts core framework utilities, custom assertions, and base drivers consumed as a pinned dependency.

  • Tech Stack: Node.js, Playwright, Cucumber.js, Allure/JUnit reporting.

Step 1: Configure Harness Connectors & Secrets

Set up these foundational resources within your Harness account:

Connectors:

  • GIT_CONNECTOR: Grants access to both application and framework GitHub repositories.

  • K8S_CONNECTOR: Manages the Kubernetes build infrastructure.

Secrets:

  • CONNECT_URL, CONNECT_USERNAME, and CONNECT_PASSWORD (and proxy settings if required).

Step 2: Configure Pipelines

Import your execution configurations using YAML files inside .harness/:

  • Standard Run (.harness/e2e-poc.yaml): Used for fast PR checks.

  • Parallel Regression (.harness/e2e-regression-parallel.yaml): Used for scheduled, high-volume regression runs.

Replace placeholders such as , , and to map to your cluster environment.

Step 3: Define Pipeline Triggers

Set up two primary execution workflows:

  • Pull Request (PR) Trigger:

    • Event: Pull Request to main/POC branch.
    • Runtime Variables: cucumberTags=@smoke

  • Scheduled Nightly Trigger:

    • Event: Scheduled Cron.
    • Runtime Variables: cucumberTags=@regression, cucumberParallel=4

Step 4: Test Report & Artifact Collection

To ensure test metrics display properly on the Harness dashboard, configure both JUnit parsing and raw artifact archiving.

Generated Outputs:

  • reports/junit-report.xml (parsed by Harness for test metrics)

  • reports/cucumber_report.json

  • reports/allure-results & HTML reports

  • test-results/screenshots & test-results/traces (captured on failure)

Harness UI Configuration Steps:

  • Under execution reporting options, enable JUnit Test Report Collection and point it to reports/junit-report.xml.

  • Enable Artifact Collection with the following paths:

    • reports/**
    • test-results/**

  • Optional: Set strictReporting=true to force pipeline failure if report outputs are missing.

Step 5: Best Practices for Headless Environments

  • Browser Fallbacks: Configure cross-platform handling in your hooks file to run branded browsers (like Edge) locally, while falling back to standard headless containers (e.g., Chromium) in CI Linux pods.

  • Branch Protection: Enforce GitHub branch rules requiring status checks from Harness to pass prior to merging PRs.

  • Shared Framework Management: For quick POCs, pin Repository B as a Git dependency in package.json. For production, publish the framework as a versioned private package.

Top comments (0)