Automate Visual Testing for Astro Sites with Cloudflare and Diffy

A minor CSS adjustment passes local checks, gets merged, and suddenly your mobile navigation collapses. This scenario is all too common in front-end development—until now. By integrating visual regression testing into your Astro and Cloudflare Pages workflow, you can automate these checks and catch layout shifts before users do.

This guide walks through setting up Diffy to compare every pull request against production, ensuring visual consistency without manual intervention.

Why Visual Regression Testing Matters for Astro Sites

Static site generators like Astro streamline development, but even small changes can introduce unintended side effects. A tweaked padding value might render correctly on desktop but break responsiveness on mobile. Or a font update could shift elements in ways that aren’t visible in local previews. These issues often slip through manual testing, only to surface after deployment.

Visual regression testing addresses this by:

• Capturing screenshots of key pages across multiple breakpoints
• Comparing them against a production baseline
• Flagging discrepancies like shifted layouts or missing elements

Tools like Diffy excel at this because they handle dynamic content—ads, carousels, or embedded videos that change between tests—by masking or freezing those areas during comparisons. This reduces false positives that plague homegrown solutions built with Puppeteer or Playwright.

The Stack: Astro, Cloudflare Pages, Diffy, and GitHub

This integration bridges four components to create a seamless workflow:

• Astro: Your static or server-side rendered site framework
• Cloudflare Pages: Automatically deploys a unique preview URL for each pull request
• GitHub Actions: Orchestrates the process and triggers visual tests
• Diffy: Executes the visual comparison and reports results back to pull requests

The flow is simple:

1. A developer opens a pull request
2. Cloudflare Pages deploys a preview URL for the branch
3. GitHub Actions detects the successful Cloudflare deployment
4. Diffy compares the preview against production
5. Results appear as a GitHub check on the pull request

Step-by-Step Setup: Diffy + Astro + Cloudflare Pages

Step 1: Configure Your Diffy Project

Begin by creating a new project in Diffy. During setup, you’ll define:

• The URLs to test (homepage, blog posts, landing pages, etc.)
• Breakpoints for responsive testing (e.g., 375px for mobile, 1280px for desktop)

Diffy will then capture an initial set of screenshots from your production site. These become the baseline for all future comparisons. After creating the project, note two critical details:

• Your Project ID (found in project settings)
• An API key (generated in your Diffy account)

Step 2: Link Diffy to Your GitHub Repository

In the Diffy dashboard, navigate to Project Settings → Notifications → GitHub and add your repository URL in the format:

`

Next, install the Diffy GitHub App on your repository. This grants Diffy permission to post check results directly to pull requests.

Step 3: Secure Your API Credentials in GitHub

Head to your GitHub repository’s Settings → Secrets and variables → Actions. Add:

• A secret named DIFFY_API_KEY with your Diffy API key
• A variable named DIFFY_PROJECT_ID with your Project ID

Secrets are encrypted and hidden in logs, while variables remain visible but non-sensitive. This setup ensures security without compromising functionality.

Step 4: Deploy the GitHub Actions Workflow

Create a new workflow file at .github/workflows/visual-regression.yml with the following configuration:

name: Visual Regression

on:
  check_run:
    types: [completed]

jobs:
  compare:
    name: Run Diffy Comparison
    if: >-
      github.event.check_run.check_suite.conclusion == 'success' && 
      github.event.check_run.app.name == 'Cloudflare Workers and Pages'
    runs-on: ubuntu-latest
    timeout-minutes: 15
    permissions:
      contents: read

    steps:
      - name: Extract deployment URL and PR number
        id: info
        uses: actions/github-script@v9
        with:
          script: |
            const summary = context.payload.check_run.output.summary || '';
            const urlMatch = summary.match(/https:\/\/[a-z0-9-\.]+\.pages\.dev/);
            if (!urlMatch) {
              core.setFailed('Could not find a *.pages.dev URL in the check run summary');
              return;
            }
            core.setOutput('deployment_url', urlMatch[0]);

            const branchName = context.payload.check_run.check_suite.head_branch || '';
            if (!branchName) {
              core.setFailed('Could not find a head branch name');
              return;
            }
            core.setOutput('branch_name', branchName);

      - name: Download Diffy CLI and run comparison
        run: |
          wget -q -O /usr/local/bin/diffy \
            
          chmod +x /usr/local/bin/diffy

          diffy auth:login "${DIFFY_API_KEY}"
          diffy project:compare "${DIFFY_PROJECT_ID}" prod custom \
            --env2Url="${{ steps.info.outputs.deployment_url }}" \
            --commit-sha="${{ github.event.check_run.head_sha }}" \
            --name="PR ${{ steps.info.outputs.branch_name }}"
        env:
          DIFFY_API_KEY: ${{ secrets.DIFFY_API_KEY }}
          DIFFY_PROJECT_ID: ${{ vars.DIFFY_PROJECT_ID }}

#### How the Workflow Works

• Trigger: The workflow runs whenever a check_run completes, but only for successful Cloudflare deployments. This avoids wasting CI minutes on failed builds.
• URL Extraction: A script parses the Cloudflare check run’s summary to extract the preview URL (e.g., your-branch.your-project.pages.dev) and the branch name.
• Diffy CLI Execution: The workflow downloads the Diffy CLI, authenticates using the API key, and runs a comparison between the preview URL and your production baseline. The --commit-sha flag ensures results link back to the correct commit.

What to Expect After Setup

Once configured, every pull request triggers this workflow automatically. Within minutes, a GitHub check appears on the pull request with one of two outcomes:

• ✅ Passed: No visual differences detected. The PR is safe to merge.
• ❌ Failed: Visual discrepancies found. A link in the check directs you to Diffy’s comparison view, where you can inspect side-by-side differences, masking applied regions, and annotations highlighting problematic areas.

This system eliminates the guesswork in visual testing, ensuring that every deployment meets your design standards—without manual oversight.

Looking Ahead: Smarter, Faster Front-End Testing

As front-end frameworks evolve, so do the tools for maintaining quality. By integrating automated visual regression testing into your Astro and Cloudflare workflow, you’re not just catching bugs—you’re future-proofing your deployments. Future enhancements could include expanded breakpoint coverage or deeper integrations with other CI/CD pipelines, making visual testing an effortless part of your development cycle.