DEV Community: Jeremy SFEZ

Why Playwright visual testing doesn't scale

Jeremy SFEZ — Fri, 11 Apr 2025 09:18:26 +0000

You touch a shared component and knowing it will affect multiple screens.

So you run your Playwright tests locally to regenerate the visual screenshots. It takes 40 minutes...

Once it's done, you go through the diffs. Some changes are expected, others are noise; fonts shift slightly, anti-aliasing artifacts, platform quirks.

You commit the new images and push your branch.

Then you notice a real visual bug you missed. Back to the tests. Another 40 minutes.

And if someone else touched the same screenshots? Merge conflicts, reruns, frustration.

This isn't just slow. It breaks your flow.

And this isn't a tooling problem. It's a model problem.

Why local visual testing breaks down in a team

You have to run the full test suite locally just to regenerate screenshots before pushing
You're comparing against screenshots you pulled which may already be outdated
Screenshots live in Git, polluting diffs and causing merge conflicts
Tests are flaky due to environment inconsistencies (OS, browser, rendering)
Reviewing visual diffs is hard when you're limited to raw image diffs

Bootstrap a solution? A misguided approach

It's technically possible to run visual testing with Playwright in CI. You can use GitHub Actions and the artifacts system to store and compare screenshots automatically. Here is a full guide that explains how.

It includes maintaining custom scripts, juggling artifact storage, reverse-engineering Git history to find the right baseline, and constantly debugging flaky tests from subtle rendering differences.

The result is a fragile, time-consuming setup with no built-in UI to review visual changes. It works, but it's far from ergonomic or team-friendly.

A different approach: generate in CI, compare to production

A scalable visual testing workflow needs to flip the model.

Instead of running tests locally and storing screenshots in Git, what if:

Screenshots were generated automatically in CI
Comparisons are always made against the actual state of your production
No one had to commit images manually
Reviews in a dedicated UI where you can inspect visual diffs, accept or reject them, and track changes history

This is the model Argos follows.

You push your branch. Screenshots are captured automatically in CI.
They're compared to the single source of truth: main.
You get a visual diff. No manual test runs, no image files in Git, no guesswork.

You review the changes in the UI. If they look correct, you click “Accept” and done!

Visual reviews that fit into your workflow

Argos integrates tightly with GitHub and GitLab. Visual diffs are posted as comments with status checks directly in your pull requests.

You can:

See what changed in every pull request
Accept changes with a single click
View the history of a given page or component's appearance across time

You review inline, with full context, and move on without pushing updated screenshots or rerunning tests.

Less noise, more trust

Argos also tackles another major pain point: stability.

Because screenshots are captured in a controlled CI environment, and processed with Argos' own stabilization algorithm:

You benefit from consistent comparisons across all environments
You see fewer diffs, and the ones you do see are meaningful
Your workflow becomes streamlined, allowing you to focus on shipping features

Stabilizing visual tests with custom Playwright scripts is a never-ending task.
At Argos, we've built a dedicated stabilization algorithm based on years of studying why visual tests fail.
It automatically handles animation disabling, font loading, image normalization, and many subtle sources of false diffs, so you don't have to.

You can trust the output.
You don't need to double-check every time.

Setting it up Argos with Playwright takes 5 minutes

Integrating Argos into a Playwright project is straightforward.
Basically, replace toHaveScreenshot with argosScreenshot in your tests.

// example.spec.ts
import { argosScreenshot } from "@argos-ci/playwright";
import { test } from "@playwright/test";

test("example test", async ({ page }) => {
  await page.goto("https://playwright.dev");
  await argosScreenshot(page, "homepage");
});

And, update your playwright.config.ts file to upload screenshots to Argos at the end of the test run.

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  reporter: [
    // Use "dot" reporter on CI, "list" otherwise (Playwright default).
    process.env.CI ? ["dot"] : ["list"],
    // Add Argos reporter. Upload on CI only.
    ["@argos-ci/playwright/reporter", { uploadToArgos: !!process.env.CI }],
  ],

  // Setup test debugging on CI.
  use: {
    trace: "on-first-retry",
    screenshot: "only-on-failure",
  },
});

Curious? Try Argos on your next pull request: argos-ci.com

Conclusion: This isn't a plugin. It's a different model.

toMatchSnapshot isn't broken.
It's just not designed for projects that scale: for teams, CI, or long-lived codebases with multiple contributors.

At some point, what was once a convenience becomes a bottleneck.
The visual testing tool becomes the thing slowing you down.

When that happens, the right move isn't to stop testing visuals.
It's to adopt a model that fits the way your team actually works.

How to Stabilize Flaky Screenshots for visual testing

Jeremy SFEZ — Tue, 27 Feb 2024 07:00:00 +0000

Visual testing is the most efficient way to test your app as it's fast and robust (doesn't require maintenance). Yet, flaky tests create frustration and erode developers' trust in their testing suite, potentially leading to the abandonment of visual testing altogether. In this article, I'll describe the common culprits responsible for flaky screenshots and how to fix them. I'll also introduce Argos's solutions to stabilize screenshots.

What is a Flaky Test?

In visual testing, a flaky test produces screenshots that differ randomly between test runs without any changes to the code. The common causes are often network latency, lazy loading, animations, and dynamic content like dates and external scripts.

Disclaimer

Flaky Tests mean Flaky UX — Guillermo Rauch

Flaky screenshots are usually a sign of underlying instability in the app's UI. To minimize flaky tests, the first step is to minimize random behaviors and ensure the UI is as stable as possible. For example, if you have a news website, you should rely on a stable dataset for your test environment.

Network Latency

Network latency can cause flaky screenshots if assets (font, images, ...) are not fully loaded before the screenshot is taken.

To stabilize screenshots, you must ensure your resources are fully loaded before capturing screenshots. For example, the argosScreenshot command waits for resources to load before taking a screenshot.

Lazy Loading

Lazy loading can cause flaky screenshots if the screenshot is taken before the elements are fully loaded.

To stabilize screenshots, use a loading indicator and wait for its removal before proceeding. Aligning with ARIA specification, I recommend using the [aria-busy] attribute on your loader component. At Argos, the argosScreenshot command delays the screenshot capture until all components with the [aria-busy] attribute are removed.

Example of a loader component:

<div id="loader" aria-busy />

Dynamic Content

By nature, dynamic elements like dates or time introduce variability in screenshots. The easiest solution is to hide these elements by injecting CSS before taking a screenshot. With Argos, you can use the data-visual-test attribute to hide these elements from the screenshot.

<div id="clock" data-visual-test="transparent">...</div>

For dates, another solution is to use a fixed date for your test environment. You can see here how to do it with Cypress. But this solution is not always possible in a real-world scenario.

Animation

Capturing a screenshot during an animation will lead to flaky screenshots. They must be either completed or paused before taking a screenshot. With Argos, CSS animations are automatically paused, but JavaScript animations require manual intervention or hiding with data-visual-test.

Note: If an animation causes layout shifts, it's recommended to remove it from the DOM with data-visual-test="removed" instead of merely hiding it with data-visual-test="transparent".

External Scripts and Iframes

External scripts and iframes (e.g. Intercom snippet) introduce flakiness due to network latency or rendering inconsistencies. There are several solutions to avoid flakiness, consider the following in this order:

Avoid loading external scripts in your test environment.
Inject CSS to hide their UI impacts.

await argosScreenshot(page, "homepage", {
  argosCSS: `
    .__argos__ iframe {
      display: none;
    }
  `,
});

Wait for the iframe's content to be loaded before taking the screenshot. If you follow this path, I recommend reading Debbie O'Brien's article about how to test an iframes with Playwright.

Data inconsistency

The two main causes of data inconsistency are randomized data seed and missing sorting order. Usually, it's easy to fix by using a fixed dataset for your test environment. But if not, you can use the data-visual-test attribute to hide the area where the data is displayed but still keep the layout consistent in the screenshot.

<article>
  <div class="title" data-visual-test="blackout">Breaking news: XXX</div>
  <div class="content" data-visual-test="blackout">...</div>
</article>

Mobile Status Bars

Mobile status bars introduce flakiness in screenshots due to their dynamic nature: notifications, battery, and time. The best practice is to hide the mobile status bar before taking a screenshot, crop it out, or ignore this area in the comparison. Argos offers a mask option for the latter.

Example with Argos' WebDriverIO integration:

import { argosScreenshot } from "@argos-ci/webdriverio";
import { browser } from "@wdio/globals";

describe("Integration test with visual testing", () => {
  it("covers homepage", async () => {
    await browser.url("http://localhost:3000");
    await argosScreenshot(browser, "homepage", {
      mask: [{ x: 0, y: 0, width: 1170, height: 120 }],
    });
  });
});

Browser-Related Inconsistencies

Browser-related inconsistencies are challenging because they are generated from the browser itself, not your code. Addressing them requires in-depth research and development. Nevertheless, Argos provides built-in solutions for most of them, ensuring screenshot consistency across runs.

Common browser inconsistency causes:

Glitches: border radius, shadow, scrolling bar, and caret visibility
Rendering: text aliasing, image rendering
Navigation: random focus, random hover, and scrolling position

Cropped Screenshots Due to Layout Shift

Sometimes, flaky tests produce cropped screenshots because Playwright (or another browser automation library) measures the page size before a layout shift, then takes the screenshot after the shift. The best way to fix this is to identify and correct the root cause of the layout shift. It can be challenging!

Capturing Screenshots on CI/CD or Locally

Capturing screenshots on different machines introduces variability, as each device's settings can affect the output. Using a CI/CD pipeline for screenshot capture standardizes the environment and enforces stability, ensuring every screenshot is produced under identical conditions.

Conclusion

Flaky tests are a common challenge in visual testing, but once you know the patterns, it will be easier to stabilize your screenshots. The most important thing is to not give up on any flaky test and risk introducing a bug in your app. Keeping your test suite stable might seem tough and thankless at times, but in the end, it's always worth the effort.

I hope these best practices help you stabilize your visual testing suite. If you have any questions or want to share your experience, feel free to reach out or join Argos' Discord community.

Time-Travel Debugging: A Playwright Traces Tutorial

Jeremy SFEZ — Wed, 17 Jan 2024 10:32:29 +0000

Ever wished you could replay a test that failed in your CI? That's where Playwright Traces come in! This guide shows how it seamlessly fits into your CI workflow, making bug-hunting easier and more effective.

Debugging Trace in Action

Let's jump right into a practical scenario. In the video below, I'll show how to use the Playwright Trace Viewer to inspect a failed test, identify the issue, and apply a fix.

In this video, I focus on resolving a complex locator issue. Besides this, Playwright Trace Viewer offers an extensive array of features for pinpointing test failures, including network logs, and console messages or step-by-step snapshots, not covered here.

Integrating with Your CI

1. Record Traces when a Test Fails

To automatically record traces on your CI, simply update your Playwright config file with this option:

use: {
  trace: 'on-first-retry',
}

Now, whenever a test fails, a trace file is generated, ready for download or online viewing with a third-party reporter.

2. Open Trace Files

To open a Trace file, either upload it to trace.playwright.dev or run the command:

npx playwright show-trace <TRACE_FILE_PATH>.zip

3. Third-Party Reporting with Argos

To enhance your workflow, you can integrate your CI with Argos for direct access to Playwright Traces online. This seamless connection eliminates the need to download trace files, offering a more streamlined approach.

Check out the Argos Playwright setup guide for easy integration steps.

TWC is not a revolution but…

Jeremy SFEZ — Wed, 27 Dec 2023 17:37:49 +0000

I'm a true fan of shadcn/ui as it offers out-of-the-box components that are highly customizable. By blending TailwindCSS for styling and theming, along with Radix for accessibility, it provides a smart kick-start for product development.

However, at some point, you will have to create your own React-TypeScript components, which can become complicated, even for coding a simple styled div:

import { cn } from "@/lib/utils"

const Card = React.forwardRef<
  HTMLDivElement,
  React.HTMLAttributes<HTMLDivElement>
>(({ className, ...props }, ref) => (
  <div
    ref={ref}
    className={cn(
      "rounded-xl border bg-card text-card-foreground shadow",
      className
    )}
    {...props}
  />
))

Enter TWC, that dramatically simplifies this with a straightforward syntax:

import { twc } from "react-twc";

const Card = twc.div`rounded-xl border bg-card text-card-foreground shadow`;

Among other features TWC offers some useful helpers like handling conditional classes, cva support and conditional props.

import { twc, TwcComponentProps } from "react-twc";

type ButtonProps = TwcComponentProps<"button"> & { $primary?: boolean };

// Conditional ClassNames
const Button = twc.button<ButtonProps>((props) => [
  "font-semibold border border-blue-500 rounded",
  props.$primary ? "bg-blue-500 text-white" : "bg-white text-gray-800",
]);

import { twc, TwcComponentProps } from "react-twc";

// Adds `target` and `rel` attributes on external links
const Anchor = twc.a.attrs<AnchorProps>((props) =>
  props.$external ? { target: "_blank", rel: "noopener noreferrer" } : {},
)`appearance-none size-4 border-2 border-blue-500 rounded-sm bg-white`;

Integration with Other Libraries

TWC can be used to add TailwindCSS classes on several design systems components including Radix or React Aria.

Takeaway

TWC is not a revolution, but it's a lib crafted by an experienced developer (gregberge), that addresses the frustrations of building React components in modern stacks. For me, it's a ❤️ tool that enhances development experience, and my productivity.

Enforce API Stability with Zod and Argos

Jeremy SFEZ — Thu, 14 Dec 2023 08:00:00 +0000

API changes frequently result in broken applications, largely due to the difficulty in monitoring their ripple effects. This article discusses the practical benefits of using Zod to enforce API stability and detect regressions through visual testing with Argos.

Let's see the advantages of this method before diving into the technical stuff.

Key Benefits for Developers

Fast Feedback: Quick identification of API contract issues.
Reliability: Detects both type and visual regressions effectively.
High Coverage: Covers all pages with a single line of code.
No Maintenance: No need to update your tests when UI or API changes.

Zod: Streamlining Type Safety

Zod is a TypeScript-first library for schema declaration and validation. As it excels in enforcing type safety, we can use it to ensuring API response integrity and maintains the API contract over time.

With this method frontend developers can rely on type safety, while backend developers can quickly identify regressions in API responses.

// Zod schema example for API response testing
import { z } from "zod";

const SneakerSchema = z.object({
  id: z.number(),
  picture: z.string(),
  price: z.number(),
  description: z.string(),
  brand: z.string(),
});

const getSneakers = async () => {
  const response = await fetch("/api/sneakers");
  if (!response.ok) throw new Error("API error");
  const data = await response.json();
  SneakerSchema.parse(data); // Validates against the schema
};

Zod can be used to ensure API integrity, but manually checking each route response after each update is impractical. Argos streamlines this, safeguarding the entire app more efficiently.

Argos: Simplifying Visual Testing

Argos integrates with CI/CD pipelines, facilitating UI change verification within pull requests. Basically, it streamlines visual regression detection and give feedback into GitHub directly.

It takes a few lines in your E2E test suite to cover your whole application:

// Playwright E2E test: screenshot app's pages
import { argosScreenshot } from "@argos-ci/playwright";
import { test } from "@playwright/test";

const baseUrl = "http://localhost:3000";
const pages = [
  { name: "homepage", path: "/" },
  { name: "product-list", path: "/explore" },
  { name: "product-detail", path: "/preview/404758" },
  { name: "contact", path: "/contact-us" },
];

test("screenshot pages", async ({ page }) => {
  for (const { name, path } of pages) {
    await page.goto(`${baseUrl}${path}`);
    await argosScreenshot(page, name);
  }
});

The Combined Impact of Zod and Argos

Utilizing Zod in tandem with Argos empowers developers to swiftly pinpoint and address broken pages caused by API modifications throughout their application.

When Zod flags a type error during runtime, it throws an error in console and render a blank page. That's where Argos steps in to detect this visual regression, and then delivers prompt feedback during code reviews, enhancing the overall development process.

The synergy of Argos and Zod redefine API contract testing, seamlessly bridging data integrity with UI consistency.

Conclusion

This powerful combination not only makes monitoring API changes across applications both efficient and cost-effective but also integrates effortlessly into your workflow. It's a toolset that empowers developers with rapid feedback, streamlining the path to a more robust and swiftly delivered product.

Startup Testing Dilemma: E2E or Not E2E?

Jeremy SFEZ — Mon, 16 Oct 2023 14:13:15 +0000

In a fast-growing startup I'm part of, a question sparked intense debates among senior developers: with all the setup and maintenance time it cost, is E2E testing really worth our time?

We all prioritize fast delivery, and figuring out how to keep that momentum in long term was the crux of our chats

Quick note: The other perspective would be to bypass E2E, let a few bugs sneak by QA, and patched things up when they showed in production.

The E2E Test Paradox in Startups

End-to-end (E2E) tests are often seen as the gold standard in quality assurance. They replicate real user actions and make sure everything in the application runs smoothly. But in a startup environment, they can be a double-edged sword.

Imagine investing hours crafting a detailed E2E test suite for a feature. The next week, based on new insights, that feature undergoes a revamp or is replaced entirely. Now, not only do you need to rewrite the feature, but you also have the unenviable task of overhauling the test suite. This cycle can be both mentally exhausting and time-consuming.

To counter the challenges posed by E2E testing and its associated maintenance costs, we explored an alternative: visual testing.

🧿 Visual Testing: Agile’s Best Friend

In the dynamic landscape of startups, where changes are frequent, visual testing emerges as the optimal solution. Here's why:

Trust in Every Change: With visual testing, changes become transparent. Approve expected updates and reject unintended alterations.
Minimal Effort, Maximum Coverage: Efficiently use one line of code to visually verify a full page's layout and content, eliminating the need for extensive scripts.
Cut the Maintenance Chain: Even if your product evolves constantly, no need to update the test code. Simply approve or reject the updated screenshots.

⚒️ Our Go-To Tool

We've been using Argos for visual testing. It's a dev tool that allows to review the changes a pull request brings. It's easy to onboard and it slides right into our workflow.

// Homepage test file. No maintenance needed! 
import { test } from "@playwright/test";
import { argosScreenshot } from "@argos-ci/playwright";

test('homepage', async ({ page }) => {
  await page.goto("http://localhost:3000");
  await argosScreenshot(page, "homepage");
});

An example of Argos build, where the dev can review the PR's visual changes.

⚖️ Balancing Act: E2E and Visual Testing

While visual tests shine in adaptability, the value of E2E tests remains unmatched for mission-critical functionalities. The key is balance. For core features with less frequent changes, E2E tests guarantee stability. For others that are in flux, visual tests provide adequate coverage without the maintenance headache.

Conclusion

In the fast-paced world of startup development, agility is critical. While robust testing remains essential, the type of testing needs to align with the product's stage and volatility. Visual testing provides the flexibility startups need without compromising on quality, making it a crucial part of any developer's toolkit today.

Have you faced similar testing dilemmas in your startup? How did you navigate them? Share your experiences in the comments.

How to Connect Your Google Sheet to ChatGPT

Jeremy SFEZ — Thu, 16 Feb 2023 08:20:30 +0000

If you're looking to leverage the power of AI to interact with your Google Sheet content, connecting your sheet to ChatGPT is a great way to get started. In this post, we'll walk through the steps required to connect your sheet to ChatGPT and start using its capabilities to answer questions about your data.

Setting Up Google Apps Script

Before we get started with ChatGPT, you'll need to set up Google Apps Script. If you're not already familiar with Google Apps Script, you can get started with the documentation here.

Javascript code snippet

The code is available in this GitHub gist.

// Load OpenAI API key from script properties
const openaiApiKey = "YOUR_API_KEY";

// Load Google Sheet data
const sheet =
  SpreadsheetApp.getActiveSpreadsheet().getActiveSheet() ||
  SpreadsheetApp.getActiveSpreadsheet().getSheets()[0];
const data = sheet.getDataRange().getValues();

// Fetch response from OpenAI's completion API
function generateResponse(prompt) {
  if (!openaiApiKey) throw new Error("OpenAI API key not found.");
  const openaiUrl = "https://api.openai.com/v1/completions";
  const response = UrlFetchApp.fetch(openaiUrl, {
    method: "post",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${openaiApiKey}`,
    },
    payload: JSON.stringify({
      prompt,
      model: "text-davinci-003",
      max_tokens: 1024,
      n: 1,
      stop: null,
      temperature: 0.5,
      presence_penalty: 0,
      frequency_penalty: 0,
      best_of: 1,
    }),
  });

  const responseData = JSON.parse(response.getContentText());
  return responseData.choices[0].text.trim();
}

function gpt(prompt) {
  const response = generateResponse(
    `${data.map((row) => row.join("\t")).join("\n")}\n\n${prompt}`
  );
  Logger.log(response);
  return response;
}

Connecting Your Google Sheet to ChatGPT

To connect your Google Sheet to ChatGPT, you'll need to follow these steps:

Copy the code provided above and paste it into Google Apps Script.
Replace "YOUR_API_KEY" with your ChatGPT API key.
Save and execute the code.
Allow the necessary Google authorizations.

Once you've completed these steps, you'll be able to use the =gpt() formula in your Google Sheet to query ChatGPT about your sheet content. This will allow you to get intelligent responses to your questions based on the data in your sheet.

Conclusion

Connecting your Google Sheet to ChatGPT is a powerful way to take advantage of AI to interact with your data. With the help of Google Apps Script, it's easy to set up and start using in your sheet. By following the steps outlined above, you'll be able to get started quickly and start benefiting from the power of ChatGPT in no time.

Automating Visual Testing with Playwright, Argos and GitHub Actions

Jeremy SFEZ — Wed, 25 Jan 2023 07:37:06 +0000

Photo de John Noonan sur Unsplash

Visual testing allows developers to identify and fix visual bugs in applications before they become a problem. By automating visual testing, developers can ensure that their web applications look and function as intended when they make changes and updates.

In this guide, we will show how to automate visual testing with Playwright, Argos, and GitHub Actions. The goal is to be notified of visual changes on each pull-request directly within GitHub, and fix visual bugs before they make it into production.

Install the Argos GitHub App

To get started with Argos, you'll first need to install the Argos GitHub App. This will allow Argos to access your repositories, add test statuses on pull-request, and notify you when visual changes are detected.

To install Argos app, go to the GitHub Marketplace, subscribe to an Argos plan and complete the installation.

Playwright and Argos setup

Playwright is a powerful testing framework that provides a great developer experience and good out-of-the-box features for end-to-end testing. To install Playwright, execute the following command in your terminal and answer the questions in the prompt:

npm init playwright@latest

To use Playwright with Argos, you will also have to install the following packages:

npm install --save-dev @argos-ci/cli  @argos-ci/playwright

@argos-ci/cli: easiest and fastest way to upload screenshots to the Argos.
@argos-ci/playwright: improve screenshot stability and expose CSS to hide content.

Capture the screenshots

Once Playwright and Argos app are installed, you can create test files to capture screenshots of your application. Use the argosScreenshot function provided by @argos-ci/playwright insure stable screenshots.

For example, the following test file will take a screenshot of your homepage:

// tests/homepage.spec.ts
import { test } from "@playwright/test";
import { argosScreenshot } from "@argos-ci/playwright";
test.describe("Homepage", () => {
  test("take screenshot", async ({ page }) => {
    await page.goto("http://localhost:3000/");
    await argosScreenshot(page, "homepage");
  });
});

To run the test, use the playwright test command:

npx playwright test

If everything is set up correctly, the test should pass and you should find a new screenshot in the /screenshots folder.

Integrate with GitHub Actions

With the screenshots captured, you can now integrate the Visual Testing into your GitHub Actions Continuous Integration. To do this, you have to create a new GitHub Actions workflow file that runs Playwright tests and uploads screenshots to Argos.

Create a new file called .github/workflows/main.yml in the root directory of your repository with the following content:

# file: .github/workflows/main.yml
name: CI
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 18
      - name: Install dependencies
        run: npm ci
      - name: Run tests
        run: npx playwright test
      - name: Upload screenshots to Argos
        run: npx @argos-ci/cli upload screenshots

This workflow defines a job that runs on pull requests to the main branch. The job does the following:

Checkout the code from GitHub
Setup Node.js 18
Install dependencies
Run the tests with Playwright
Upload the screenshots to Argos

Once you have added the workflow file, you can commit and push the changes to your repository. GitHub will automatically start running the workflow on each pull request.

Monitor Visual Changes

You can now start monitoring visual changes in your web application. Argos will detect and report any visual changes between your pull request branch and your main branch.

GitHub check status

By clicking on "Details" link, you can review the results of the comparison in Argos.

Argos example build

Read this documentation if you need more information about visual testing workflow.

Conclusion

In this guide, we have shown you how to automate visual testing with Playwright, Argos, and GitHub Actions. By integrating visual testing into your CI workflow, you can catch and fix visual bugs before they make it into production. With Argos, you can easily configure baseline, monitor visual changes, and collaborate with your team to fix issues. By following the steps outlined in this guide, you can be sure that your web application looks and functions as intended, even as you make changes and updates.

We hope this post has been helpful. If you're interested in learning about how Argos can help you improve the quality of your web applications, take a look at our documentation. And if you're ready to start automating your visual testing, feel free to sign up for a free trial and see how Argos can help you catch bugs.

If you need more information about Visual Testing, it would be a pleasure to help you on our Discord community.

How to improve developer experience (DX) with visual testing

Jeremy SFEZ — Wed, 21 Dec 2022 08:56:48 +0000

As a developer, there's nothing more frustrating than spending hours, if not days, working on a feature or bug fix, only to have it break the visual layout of the app. This not only slows down the development process, but it can also lead to frustration and loss of confidence in the quality of the work being delivered.

That's where visual testing comes in.

Developer experience (DX)

Developer experience refers to the overall satisfaction and efficiency of development tools, including design, usability, documentation, support, and working workflows.

A good developer experience is important for maintaining a high level of productivity and software quality. Efficient tools and workflows allow developers to iterate quickly, and ultimately resulting in higher software quality and positive experiences for developers and users.

Visual testing to ship with confidence

Visual testing is a type of testing that focuses on the visual aspects of an application, such as layout, design, and user interface elements. One way visual testing can be implemented is through the use of screenshot comparison tools. These tools allow developers to take screenshots of the app before and after a change is made.

With visual testing tools like Argos, the screenshots are
automatically compared, and if any differences are detected, the developer is alerted, allowing them to fix the potential issues before it becomes a problem.

In addition, a visual tests suite can be created to ensure that the visual aspects of the app are consistent across different devices and screen sizes and catch visual issues early on. By implementing visual testing, developers can be confident that their work won't cause any visual issues that could break the app.

Conclusion

Overall, visual testing is invaluable for any developer looking to deliver high-quality, bug-free applications. Automatic visual testing improves software crafting workflow by giving developers more confidence in feature delivery and save time and effort in the long run.

Photo by devn on Unsplash

Set up Argos with Cypress and GitHub Actions in minutes

Jeremy SFEZ — Fri, 09 Dec 2022 13:47:05 +0000

Argos visual testing

Argos is a beautiful open-source app for visual testing.
It is easy to install and less expensive than it's competitors.

Visual testing is a scalable way to ensure the visual integrity of your web application and catch any potential visual bugs before they become a problem.

Argos build example

When visual testing is combined with a CI workflow, you get an immediate feedback cycle that highlights visual diffs across browsers and screen resolutions for each pull request.

5 steps installation

Setting up Argos is blazing fast if you're working on a project that already uses Cypress for end-to-end testing and GitHub Actions as CI.

Step 1 - Install the GitHub app

Subscribe to a plan on GitHub Marketplace and follow the installation tunnel to authorize the GitHub app.

Argos is free up to 5K screenshots monthly.

Step 2 - Install Argos' packages

Install Argos CLI and the Argos Cypress plugin in your project.

npm i --save-dev @argos-ci/cli @argos-ci/cypress

Step 3 - Configure Cypress

Add the following code to cypress/support/index.js to enable the Argos Cypress plugin and use the argosScreenshot command.

// file: cypress/support/index.js

import "@argos-ci/cypress/support";

Step 4 - Take a screenshot

In your Cypress test files, use the argosScreenshot command to take a screenshot of the page for visual testing with Argos. For example:

// file: cypress/e2e/homepage.cy.js

describe("Homepage", () => {
  it("screenshots the page", () => {
    cy.visit("http://localhost:3000");
    cy.argosScreenshot("home");
  });
});

The argosScreenshot command increases screenshot stability, but you can use the cypress' native screenshot command instead.

Step 5 - Configure GitHub Actions

Add a command to your GitHub Actions config file to upload the screenshots to Argos.

# file: .github/workflows/ci.yml

name: CI
  build:
    runs-on: ubuntu-latest
      - uses: actions/checkout@v3

      # 👉 Insert the steps to execute Cypress tests

      # 👇 Insert this command to upload screenshots to Argos
      - name: Upload screenshots to argos-ci.com
        run: npx @argos-ci/cli cypress/screenshots

Congratulation

That's it! Argos will automatically receive screenshots of your pages when you push a commit to GitHub. In addition, Argos will compare them to previous versions to ensure no unexpected changes.

Review visible changes

You can now review the visual diffs on Argos and avoid regression. In your pull request, click on details link in front of the Argos check to open the build on Argos.

DEV Community: Jeremy SFEZ

Why Playwright visual testing doesn't scale

Why local visual testing breaks down in a team

Bootstrap a solution? A misguided approach

A different approach: generate in CI, compare to production

Visual reviews that fit into your workflow

Less noise, more trust

Setting it up Argos with Playwright takes 5 minutes

Conclusion: This isn't a plugin. It's a different model.

How to Stabilize Flaky Screenshots for visual testing

What is a Flaky Test?

Network Latency

Lazy Loading

Dynamic Content

Animation

External Scripts and Iframes

Data inconsistency

Mobile Status Bars

Browser-Related Inconsistencies

Cropped Screenshots Due to Layout Shift

Capturing Screenshots on CI/CD or Locally

Conclusion

Time-Travel Debugging: A Playwright Traces Tutorial

Debugging Trace in Action

Integrating with Your CI

1. Record Traces when a Test Fails

2. Open Trace Files

3. Third-Party Reporting with Argos

Further Reading

TWC is not a revolution but…

Integration with Other Libraries

Takeaway

Enforce API Stability with Zod and Argos

Zod: Streamlining Type Safety

Argos: Simplifying Visual Testing

The Combined Impact of Zod and Argos

Conclusion

Startup Testing Dilemma: E2E or Not E2E?

The E2E Test Paradox in Startups

🧿 Visual Testing: Agile’s Best Friend

⚒️ Our Go-To Tool

⚖️ Balancing Act: E2E and Visual Testing

Conclusion

How to Connect Your Google Sheet to ChatGPT

Setting Up Google Apps Script

Javascript code snippet

Connecting Your Google Sheet to ChatGPT

Conclusion

Automating Visual Testing with Playwright, Argos and GitHub Actions

Install the Argos GitHub App

Playwright and Argos setup

Capture the screenshots

Integrate with GitHub Actions

Monitor Visual Changes

Conclusion

How to improve developer experience (DX) with visual testing

Developer experience (DX)

Visual testing to ship with confidence

Conclusion

Set up Argos with Cypress and GitHub Actions in minutes

Argos visual testing

5 steps installation

Step 1 - Install the GitHub app

Step 2 - Install Argos' packages

Step 3 - Configure Cypress

Step 4 - Take a screenshot

Step 5 - Configure GitHub Actions

Congratulation

Review visible changes

Credit and sources