DEV Community

Shell QA
Shell QA

Posted on

Checklist: Onboarding End-to-End Automation Frameworks to Harness CI

#devops #testing #automation #ci

Successfully onboarding an automated test suite to Harness CI requires configuring infrastructure placeholders, secrets, pipelines, and branch protection rules.

Here is a 10-step checklist to help you onboard your end-to-end (E2E) automation pipelines seamlessly.

Step 1: Replace Infrastructure Placeholders

Ensure your pipeline YAML definitions (e.g., .harness/e2e-poc.yaml and .harness/e2e-regression-parallel.yaml) contain your specific environment values:

  • ORG_ID: Harness Organization Identifier

  • PROJECT_ID: Harness Project Identifier

  • GIT_CONNECTOR: Harness Git Connector for GitHub Enterprise access

  • APP_REPO_NAME: Target repository in owner/repo format

  • K8S_CONNECTOR: Kubernetes connector for build infrastructure

  • K8S_NAMESPACE: Kubernetes namespace where build pods run

Step 2: Configure Environment Secrets

In Harness, set up the following runtime secrets:

  • CONNECT_URL

  • CONNECT_USERNAME

  • CONNECT_PASSWORD

Step 3: Setup PR Validation Pipeline

  • Import your short-run pipeline YAML into Harness.

  • Save it as your PR Validation Pipeline.

  • Run a manual validation test using runtime overrides:

    • TargetEnv = qa
    • cucumberTags = @smoke

Step 4: Verify Artifact Generation

Confirm that the initial execution correctly generates and uploads all required outputs:

  • JUnit Report: reports/junit-report.xml

  • Test Reports: reports/**

  • Failure Artifacts: test-results/** (screenshots, traces)

Step 5: Setup Nightly Parallel Pipeline

  • Import your parallel pipeline YAML into Harness.

  • Save it as your Nightly Regression Pipeline.

  • Run a manual validation test with target concurrency parameters:

    • TargetEnv = qa
    • cucumberTags = @regression
    • cucumberParallel = 4

Step 6: Configure Automated Triggers & Branch Protection

  • PR Trigger: Configured on pull requests with cucumberTags=@smoke.

  • Nightly Schedule Trigger: Configured on a nightly cron schedule with cucumberTags=@regression and cucumberParallel=4.

  • GitHub Branch Protection: Enable branch protection on target branches requiring the Harness PR pipeline status check to pass before merging.

Once both the PR and Nightly execution runs pass with green status checks, your pipeline onboarding is complete!

Top comments (0)