Supercharging Your CI/CD: Integrating TestSprite AI Testing with GitHub Actions

ExecuteAutomation — Thu, 23 Apr 2026 02:48:35 +0000

In the evolving landscape of software quality assurance, AI-driven testing is no longer a luxury—it’s a necessity for speed and scale. TestSprite, an AI-powered testing platform, offers a way to generate and execute end-to-end tests using agentic workflows. However, the real power of these tools is unlocked when they are integrated directly into your CI/CD pipeline.

In this article, I will demonstrate how to bridge the gap between containerized applications and cloud-based AI testing agents which is TestSprite in our case and using it with GitHub Actions.

The Challenge: Testing Private Containers from the Cloud

When your application runs in a managed environment like Vercel or Netlify, integration is straightforward. But what if your app is running in a local Docker container or a private GitHub runner?

Because the TestSprite cloud needs to "see" your application to test it, you face a networking hurdle. You cannot simply point the AI to localhost. To solve this, the video introduces a three-tier architecture: Build, Tunnel, and Execute.

Phase 1: The Build

The process starts with a standard GitHub Actions runner. Using Docker Compose, the workflow spins up both the front-end (e.g., Vite) and the back-end (e.g., Node/Express) services. This ensures the environment is a perfect replica of your production stack.

Phase 2: The Tunnel (The Secret Sauce)

To make the private container accessible to TestSprite without exposing it permanently to the internet, the tutorial utilizes the Cloudflare Tunneling library (cloudflared).

Temporary URL: The tunnel creates a random, dynamic URL (e.g., *.trycloudflare.com).
Dynamic Routing: This URL is passed to the TestSprite agent, serving as the BASE_URL for all test execution.
Security: The tunnel is temporary and only exists for the duration of the test execution phase.

Phase 3: Integration and Execution

To trigger the tests, you need to configure your GitHub repository with a few essential pieces:

API Key: Securely store your TestSprite API key in GitHub Repository Secrets as TESTSPRITE_API_KEY.
Workflow YAML: Define a .github/workflows/ci.yml file that calls the test-sprite/run-action@v1.

Once pushed, the pipeline automatically triggers on every Pull Request.

GitHub Action YAML file

Here is how the GH Action Pipeline will look like

name: CI

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      issues: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Build and start app containers
        run: docker compose up -d --build

      - name: Wait for frontend to be ready
        run: |
          echo "Waiting for frontend on port 5173..."
          timeout 60 bash -c 'until curl -s http://127.0.0.1:5173 > /dev/null 2>&1; do sleep 2; done'
          echo "Frontend is up!"

      - name: Wait for backend to be ready
        run: |
          echo "Waiting for backend on port 4000..."
          timeout 60 bash -c 'until curl -s http://127.0.0.1:4000/employees > /dev/null 2>&1; do sleep 2; done'
          echo "Backend is up!"

      - name: Install and start Cloudflare Tunnel
        run: |
          # Install cloudflared
          curl -sL https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb -o cloudflared.deb
          sudo dpkg -i cloudflared.deb

          # Start tunnel for frontend and capture the public URL
          cloudflared tunnel --url http://127.0.0.1:5173 --no-autoupdate > /tmp/cloudflared.log 2>&1 &
          sleep 10

          # Extract the tunnel URL from the logs
          TUNNEL_URL=$(grep -o 'https://.*\.trycloudflare\.com' /tmp/cloudflared.log | head -1)
          echo "Tunnel URL: $TUNNEL_URL"
          echo "TUNNEL_URL=$TUNNEL_URL" >> $GITHUB_ENV

      - name: Verify tunnel is working
        run: |
          echo "Testing tunnel URL: $TUNNEL_URL"
          curl -sI "$TUNNEL_URL" | head -5

      - name: Rewrite hardcoded URLs in test scripts
        run: |
          echo "Replacing http://127.0.0.1:5173 with $TUNNEL_URL in test files..."
          find testsprite_tests -name '*.py' -exec sed -i "s|http://127.0.0.1:5173|$TUNNEL_URL|g" {} +
          echo "Replacing http://localhost:5173 with $TUNNEL_URL in test files..."
          find testsprite_tests -name '*.py' -exec sed -i "s|http://localhost:5173|$TUNNEL_URL|g" {} +
          # Show a sample to verify
          grep -r 'trycloudflare' testsprite_tests/*.py | head -3 || echo "No replacements found - check URL pattern"

      - name: Run Testsprite Action
        uses: TestSprite/run-action@v1
        with:
          testsprite-api-key: ${{ secrets.TESTSPRITE_API_KEY }}
          github-token: ${{ secrets.GITHUB_TOKEN }}
          base_url: ${{ env.TUNNEL_URL }}
          blocking: 'true'

Real-World Results: AI-Powered PR Reports

The most impressive part of this integration is the feedback loop. Once the tests are complete:

PR Blocking: If tests fail, the Pull Request is automatically blocked from merging, ensuring code quality.
Comprehensive Dashboards: TestSprite provides a detailed dashboard within the PR, showing exactly which cases passed or failed.
Video Evidence: Perhaps most helpfully, the platform provides video recordings of the AI agent interacting with the browser, making it incredibly easy to debug failures.

The most impressive part of this integration is the feedback loop. Once the tests are complete:

PR Blocking: If tests fail, the Pull Request is automatically blocked from merging, ensuring code quality.

Comprehensive Dashboards: TestSprite provides a detailed dashboard within the PR, showing exactly which cases passed or failed.

Video Evidence: Perhaps most helpfully, the platform provides video recordings of the AI agent interacting with the browser, making it incredibly easy to debug failures.

Conclusion

Integrating AI testing into GitHub Actions transforms QA from a bottleneck into a seamless part of the development cycle. By combining Docker, Cloudflare tunneling, and TestSprite, teams can achieve high-confidence deployments with minimal manual intervention.

For a detailed walkthrough of the code and configuration, check out the full video by Execute Automation:

DEV Community: ExecuteAutomation