Catch Visual Bugs Before They Hit Production: Automated Visual Regression Testing for Astro with Diffy and Cloudflare

Yuri Gerasymov — Thu, 04 Jun 2026 19:51:01 +0000

You push a CSS tweak. It looks fine locally. The PR gets approved. You deploy — and two hours later someone messages you: "Hey, the navigation is broken on mobile."

Sound familiar? This is exactly the gap that visual regression testing fills. And if you're building with Astro and deploying to Cloudflare Pages, you can close that gap automatically on every pull request — without any manual QA steps.

This post walks through setting up Diffy with your Astro + GitHub + Cloudflare workflow so that every PR gets its own visual comparison against production before it's merged.

What Is Diffy?

Diffy is a screenshot-based visual regression testing tool. It takes screenshots of your pages across multiple breakpoints, compares them against a baseline (typically production), and highlights any visual differences — layout shifts, missing elements, broken styles, unexpected changes.

What sets Diffy apart from rolling your own Playwright/Puppetteer setup is its focus on zero false positives. Dynamic content like ads, carousels, and video embeds are the big problem of visual diffing — they change every time and flood you with noise. Diffy handles this by masking and freezing dynamic elements so your diffs stay meaningful.

The Stack

Here's what this integration connects:

Astro — your framework, with pages deployed as a static/SSR site
Cloudflare Pages — which spins up a unique preview URL for every branch/PR automatically
GitHub Actions — which orchestrates the workflow
Diffy — which does the visual comparison and posts results back to the PR

The flow looks like this:

PR opened → Cloudflare deploys preview → GitHub Action fires →
Diffy CLI compares preview vs. production → Result posted to PR check

Step 1: Set Up Your Diffy Project

First, create a project in Diffy. During setup you'll:

Add the URLs you want to test (your key pages — home, blog, landing pages, etc.)
Configure breakpoints (e.g. 375px mobile, 1280px desktop)
Take an initial set of screenshots from your production environment — this becomes the baseline

Once your project is created, grab two things you'll need later:

Your Project ID (from the project settings URL)
An API key (from https://app.diffy.website/#/keys)

Step 2: Connect Diffy to Your GitHub Repo

In Diffy, go to Project Settings → Notifications → GitHub and add your repository URL in the format:

https://github.com/{username}/{repo}

Then install the Diffy GitHub App on your repository. This is what gives Diffy permission to post check results back to your pull requests.

Step 3: Add Secrets to GitHub

In your GitHub repository, go to Settings → Secrets and variables → Actions and add:

A secret called DIFFY_API_KEY with your API key
A variable called DIFFY_PROJECT_ID with your project ID

(Secrets are encrypted and hidden in logs; variables are visible but not sensitive — hence the split.)

Step 4: Add the GitHub Actions Workflow

Create .github/workflows/visual-regression.yml in your repo:

name: Visual Regression

on:
  check_run:
    types: [completed]

jobs:
  compare:
    name: Run Diffy Comparison
    # Only fire on successful Cloudflare Workers and Pages check runs.
    if: >-
      github.event.check_run.check_suite.conclusion == 'success' &&
      github.event.check_run.app.name == 'Cloudflare Workers and Pages'
    runs-on: ubuntu-latest
    timeout-minutes: 15
    permissions:
      contents: read

    steps:
      # Extract the preview URL from the check run summary and resolve the PR number.
      - name: Get deployment URL and PR number
        id: info
        uses: actions/github-script@v9
        with:
          script: |
            const summary = context.payload.check_run.output.summary || '';
            const urlMatch = summary.match(/https:\/\/[a-z0-9-\.]+\.workers\.dev*/);
            if (!urlMatch) {
              core.setFailed('Could not find a *.workers.dev URL in the check run summary');
              return;
            }
            core.setOutput('deployment_url', urlMatch[0]);

            const branchName = context.payload.check_run.check_suite.head_branch || ''
            if (!branchName) {
              core.setFailed('Could not find a head branch name');
              return;
            }
            core.setOutput('branch_name', branchName);

      - name: Download Diffy CLI and trigger comparison job
        run: |
          wget -q -O /usr/local/bin/diffy \
            https://github.com/diffywebsite/diffy-cli/releases/latest/download/diffy.phar
          chmod +x /usr/local/bin/diffy

          diffy auth:login "${DIFFY_API_KEY}"

          diffy project:compare "${DIFFY_PROJECT_ID}" prod custom \
            --env2Url="${{ steps.info.outputs.deployment_url }}" \
            --commit-sha="${{ github.event.check_run.head_sha }}" \
            --name="PR ${{ steps.info.outputs.branch_name }}"
        env:
          DIFFY_API_KEY: ${{ secrets.DIFFY_API_KEY }}
          DIFFY_PROJECT_ID: ${{ vars.DIFFY_PROJECT_ID }}

Let's break down what's happening:

The trigger (check_run: completed) means this workflow fires whenever any check run finishes on the repo. The if: condition filters it down to only Cloudflare deployments that completed successfully — so you're not burning CI minutes on failed builds.

The first step uses actions/github-script to parse the Cloudflare check run's output summary and extract the preview URL (which looks like your-branch.your-project.pages.dev). It also grabs the branch name for labeling the comparison.

The second step downloads the Diffy CLI, authenticates, then runs project:compare with your production environment as the baseline and the Cloudflare preview URL as the comparison target. The --commit-sha flag is what links the Diffy result back to the specific commit so GitHub knows which PR check to update.

What You Get

After a PR is opened and Cloudflare finishes its deployment, the workflow automatically kicks off. Within a few minutes you'll see a new check appear on your PR:

✅ Passed — no visual differences detected, safe to merge
❌ Failed — visual differences found, with a link to the Diffy comparison

Clicking through takes you to a side-by-side diff view in Diffy where you can inspect exactly which pages changed, across each breakpoint you configured. If a change is intentional (you did update the hero section after all), you can approve it in Diffy to bring the check back to passed.

Why This Matters for Astro Projects

Astro's component model makes it easy to share layout components across dozens of pages. That's a superpower — but it also means a CSS change in a shared component can ripple across your entire site in ways that are hard to catch in code review.

If you make your changes to the site with Claude (or any other AI) you would want to have a safety net that will catch any unintentional changes.

Visual regression testing is that net. With this setup, every PR on your Astro site gets automatically checked against production before it ever has a chance to merge — with zero manual steps required from you or your reviewers.

PHPStorm picking up wrong index.php

Yuri Gerasymov — Thu, 03 Jan 2019 03:44:07 +0000

Recently I was setting up Drupal development environment with docksal. I used traditional drupal composer setup (https://github.com/drupal-composer/drupal-project) so drupal is in the /web folder on the server.

I have mapped main root folder in PHP Storm

So I was really surprised when I started my debugging session and discovered some weird file being started.

Looks like PHP Storm tries to find first index.php it can find and then starts session like it is being debugged.

Solution for this is to map folder "web" and actual index.php so PHP Storm can find proper file easier.

As result proper index.php is displayed as being debugged.

Happy debugging!

Automating checking your Drupal's updates with Pantheon multidev

Yuri Gerasymov — Sun, 16 Dec 2018 23:38:26 +0000

Drupal updates can be very different. Some of them -- easy patches that you just roll out and forget. Some of them -- break your site. Tricky part is you never know how updates will behave on your site until you actually tried them out.

This is why it is very tricky to give estimates to clients how long it is going to take. They usually do not appreciate answer 1 to 20 hours depending on some random facts.

In this way rolling out updates got delayed and delayed. And then we get to situations after half a year or a year that we know for sure site will be broken after updates. And now hero time begins.

Would it be nice if site would tell you not only the fact that it needs updates but also if it is going to break or not after rolling them out.

Nowadays, thanks to Pantheon's multidev, it is technically possible to automate checking how your updates will behave on the site.

Main idea is to regularly check updates (using drush command) then if updates are found create a separate environment and roll updates there. Afterward to ensure that they didn't break the site (at least visually) we could run some visual regression testing. So in result we have way more predictable answer about "how much efforts it will take to roll updates out".

Here is a full article tutorial about how to set it up http://docs.diffy.website/tutorials/put-your-sites-updates-on-autopilot-with-pantheon-multidev-and-visual-testing.

For sure fixing smaller updates is much easier than fixing big break after year of delays.

DEV Community: Yuri Gerasymov