<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Babatunde Hammed</title>
    <description>The latest articles on DEV Community by Babatunde Hammed (@2unde).</description>
    <link>https://dev.to/2unde</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4183149%2Fd4bbcb4e-dcd4-40d0-972e-1026990e6b38.png</url>
      <title>DEV Community: Babatunde Hammed</title>
      <link>https://dev.to/2unde</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/2unde"/>
    <language>en</language>
    <item>
      <title>Fails only in CI, part 2: the causes readers taught me</title>
      <dc:creator>Babatunde Hammed</dc:creator>
      <pubDate>Thu, 12 Nov 2026 12:09:06 +0000</pubDate>
      <link>https://dev.to/2unde/fails-only-in-ci-part-2-the-causes-readers-taught-me-3hf8</link>
      <guid>https://dev.to/2unde/fails-only-in-ci-part-2-the-causes-readers-taught-me-3hf8</guid>
      <description>&lt;p&gt;Last week I wrote about &lt;a href="https://dev.to/2unde/it-passes-on-my-machine-and-fails-in-ci-a-practical-checklist-and-where-ai-actually-helps-5d4n"&gt;tests that pass on your machine and fail in CI&lt;/a&gt;. I listed five usual suspects: speed, leftover state, environment, the network path and parallel runs.&lt;/p&gt;

&lt;p&gt;Then the comments arrived, here and on Reddit, LinkedIn and the Ministry of Testing Club. Testers shared failures that took them a day, a week, and in one case over a year to solve. Most of them weren't on my list.&lt;/p&gt;

&lt;p&gt;So this is part 2: the causes I learned from you, each with the fix that worked, and then a small CI setup that makes the next mystery quicker to solve.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. The mouse pointer was in the wrong place
&lt;/h2&gt;

&lt;p&gt;One reader had tests that always passed locally and always failed in CI. The logs and screenshots showed an empty field. After a day of debugging: the mouse pointer.&lt;/p&gt;

&lt;p&gt;On the CI runner, the pointer happened to sit over an element that opened a popup menu. The menu covered the field the test was filling, so the text went nowhere. The field was optional, so nothing failed there. The test blew up much later, at a step that had nothing to do with the cause.&lt;/p&gt;

&lt;p&gt;Two habits would have saved most of that day:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Make the starting state explicit instead of "wherever the pointer is"&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mouse&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;move&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByLabel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Reference'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fill&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'ABC-123'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;// Assert right after you act, so the failure points at the real step&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;expect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByLabel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Reference'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toHaveValue&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'ABC-123'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Lesson:&lt;/strong&gt; assert where you act. A failure three steps later sends you looking in the wrong place.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. File and glob order
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;fs.readdir&lt;/code&gt;, glob and friends don't promise any order. The order you get depends on the filesystem. Locally it looks alphabetical every time, so nobody notices that the test depends on it. On the runner, it isn't.&lt;/p&gt;

&lt;p&gt;Any test that takes "the first file" or compares a list without sorting is waiting to break.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;files&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;fs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;promises&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readdir&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dir&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sort&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The same applies to database queries without an &lt;code&gt;ORDER BY&lt;/code&gt;, API results without an explicit sort order, and rows in a table that your test reads as a list. Locally they usually come back in insertion order by coincidence; a fresh CI database with a different disk layout or query plan can flip it.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Shallow clones
&lt;/h2&gt;

&lt;p&gt;Most CI checkouts fetch only the latest commit and no tags. Anything that calls &lt;code&gt;git describe&lt;/code&gt;, reads the last tag for a version number, or diffs against &lt;code&gt;main&lt;/code&gt; works on your machine and fails, or quietly returns nonsense, in CI.&lt;/p&gt;

&lt;p&gt;In GitHub Actions, fetch full history for the one job that needs it, not every job:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/checkout@v4&lt;/span&gt;
  &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;fetch-depth&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;   &lt;span class="c1"&gt;# full history and tags
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  4. Dependency drift: both builds green, and broken anyway
&lt;/h2&gt;

&lt;p&gt;This one was the reverse of the usual story. The build succeeded locally and in CI, but one component broke at runtime on a developer's machine.&lt;/p&gt;

&lt;p&gt;The cause: CI couldn't read the lockfile in the repo, so it regenerated it on every run. CI and local ended up with different versions of a worker package. The team followed what CI did step by step and saw no difference, because the difference was in what got installed, not in the steps.&lt;/p&gt;

&lt;p&gt;Two fixes make this kind of drift fail loudly at install time instead of at runtime:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Pin the Node version, for example in &lt;code&gt;.nvmrc&lt;/code&gt; or the &lt;code&gt;engines&lt;/code&gt; field of &lt;code&gt;package.json&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Use &lt;code&gt;npm ci&lt;/code&gt; in CI, not &lt;code&gt;npm install&lt;/code&gt;. It installs exactly what's in the lockfile and fails if the lockfile is out of sync with &lt;code&gt;package.json&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  5. Docker isn't the whole environment
&lt;/h2&gt;

&lt;p&gt;A reader writing unit tests for kernel flags had dockerized builds and still got caught. Containers share the host's kernel. The newer kernel on their laptop supported flags that the CI runner's kernel didn't.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Print &lt;code&gt;uname -r&lt;/code&gt; at the start of the CI job, so the kernel version is in every log&lt;/li&gt;
&lt;li&gt;Check for the feature at runtime and skip with a clear reason, instead of assuming it's there&lt;/li&gt;
&lt;li&gt;If the feature is required, pin the runner OS so the kernel is known&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  6. Shared data gives false greens, not just false reds
&lt;/h2&gt;

&lt;p&gt;I wrote last week that tests sharing data can fail when run in parallel. A reader on X pointed out the scarier half: they can also &lt;strong&gt;pass&lt;/strong&gt; when they shouldn't.&lt;/p&gt;

&lt;p&gt;Two workers share a basket. One test passes because it finds the item the &lt;em&gt;other&lt;/em&gt; test added. A false green is worse than a failure, because nobody looks at it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'adds an item to the basket'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;testInfo&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="c1"&gt;// Each test gets its own basket, so nothing else can make it pass&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;basketName&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sb"&gt;`basket-${testInfo.testId}`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="c1"&gt;// ...create basketName, add the item, then assert on that basket only&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Assert on the specific record you created, not on "an item exists".&lt;/p&gt;

&lt;h2&gt;
  
  
  7. "CI quietly decides it lives in UTC"
&lt;/h2&gt;

&lt;p&gt;Timezone was in part 1, but readers added the sneakiest version: a test that fails for only an hour or two each day, when local time and UTC fall on different dates. Everyone assumes it's flaky until someone notices it always fails late in the evening.&lt;/p&gt;

&lt;p&gt;Set the timezone explicitly, and control the clock in tests that care about dates:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// playwright.config.ts&lt;/span&gt;
&lt;span class="n"&gt;use&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;timezoneId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'Europe/London'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;locale&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'en-GB'&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// in a test that depends on "today"&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;clock&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setFixedTime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nb"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-09T23:30:00Z'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The same goes for &lt;code&gt;localeCompare&lt;/code&gt; and number or date formatting. Minimal CI images often have different locale data, so pass an explicit locale in tests.&lt;/p&gt;

&lt;h2&gt;
  
  
  A small CI setup that makes the next one easier
&lt;/h2&gt;

&lt;p&gt;Several replies came back to the same idea, and &lt;a class="mentioned-user" href="https://dev.to/alexshev"&gt;@alexshev&lt;/a&gt; put it best: &lt;em&gt;CI failures are often environment mismatches disguised as code failures.&lt;/em&gt; The fix is evidence, not guesswork.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Print the environment at the start of every job.&lt;/strong&gt; It costs seconds, and it means local versus CI becomes two lists you can compare instead of two memories:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Environment&lt;/span&gt;
  &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
    &lt;span class="s"&gt;node -v&lt;/span&gt;
    &lt;span class="s"&gt;npx playwright --version&lt;/span&gt;
    &lt;span class="s"&gt;uname -r&lt;/span&gt;
    &lt;span class="s"&gt;date&lt;/span&gt;
    &lt;span class="s"&gt;echo "TZ=$TZ LANG=$LANG"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;2. Keep the evidence when a run fails.&lt;/strong&gt; Save the Playwright report and traces as artifacts, even when the tests fail:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/upload-artifact@v4&lt;/span&gt;
  &lt;span class="na"&gt;if&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ !cancelled() }}&lt;/span&gt;
  &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;playwright-report&lt;/span&gt;
    &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;playwright-report/&lt;/span&gt;
    &lt;span class="na"&gt;retention-days&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;14&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;3. Save the generated input, not just the seed.&lt;/strong&gt; This was &lt;a class="mentioned-user" href="https://dev.to/alexshev"&gt;@alexshev&lt;/a&gt;'s point, and it changed how I think about test data. He records the seed, generator version, locale, timezone and fixture version on every run. For a failed CI run, he also keeps the actual generated input (scrubbed if needed) alongside the trace for a short time. In his words, a seed alone is not enough once a dependency, clock, locale or generator behaviour changes. Once the bug is fixed, that exact input becomes a regression fixture.&lt;/p&gt;

&lt;p&gt;In Playwright, attaching the input to the failing test takes one line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;testInfo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;attach&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'generated-input'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;stringify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;input&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;contentType&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'application/json'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;4. Decide who looks first.&lt;/strong&gt; A question I got on Reddit: when tests break after a merge, is it the tester's job or the SDETs'? It depends on the team, but a common default works well. If the tests fail on your pull request, you look first. If something breaks after merge, whoever's change caused it investigates, and whoever owns the pipeline handles runner and config problems. The worst option is a red build that everyone assumes someone else is fixing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where AI fits in
&lt;/h2&gt;

&lt;p&gt;Same view as part 1, sharpened by the comments. AI assistants are good at reading a long CI log, explaining an unfamiliar error and suggesting theories. Give one the environment printout from a passing local run and a failing CI run, and it will often spot the difference faster than you.&lt;/p&gt;

&lt;p&gt;What it can't do is see your runner. Nobody's AI would have guessed the mouse pointer or the kernel version without that information in front of it. The evidence from the setup above is what makes its suggestions worth anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  Keep them coming
&lt;/h2&gt;

&lt;p&gt;Twelve causes so far between the two posts, and I suspect we're nowhere near the end. What's yours? I'll keep collecting them, and I plan to turn the best ones into hands-on practice challenges.&lt;/p&gt;

&lt;p&gt;Thank you to everyone who shared a story, and to &lt;a class="mentioned-user" href="https://dev.to/alexshev"&gt;@alexshev&lt;/a&gt; for letting me quote him.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm building &lt;a href="https://locatorleague.dev/learn/fix-the-flaky-test?utm_source=devto&amp;amp;utm_campaign=ci_part2" rel="noopener noreferrer"&gt;Locator League&lt;/a&gt;, browser-based Playwright challenges where you write the test and it runs for real. I plan to turn some of the stories above into a "Works on my machine" practice track.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>playwright</category>
      <category>testing</category>
      <category>cicd</category>
      <category>devops</category>
    </item>
    <item>
      <title>It passes on my machine and fails in CI: a practical checklist (and where AI actually helps)</title>
      <dc:creator>Babatunde Hammed</dc:creator>
      <pubDate>Thu, 05 Nov 2026 13:18:27 +0000</pubDate>
      <link>https://dev.to/2unde/it-passes-on-my-machine-and-fails-in-ci-a-practical-checklist-and-where-ai-actually-helps-5d4n</link>
      <guid>https://dev.to/2unde/it-passes-on-my-machine-and-fails-in-ci-a-practical-checklist-and-where-ai-actually-helps-5d4n</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F30myzkbp908uui2hst2c.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F30myzkbp908uui2hst2c.png" alt=" " width="800" height="336"&gt;&lt;/a&gt;&lt;br&gt;
If you work in test automation long enough, you get a very specific kind of message from your pipeline. Red build. You pull the branch, run the same test locally, and it's green. You run it again. Green. You run it ten times. Still green.&lt;/p&gt;

&lt;p&gt;Then CI runs it again and it fails.&lt;/p&gt;

&lt;p&gt;This week I asked other testers for their strangest "only fails in CI" bugs, and the replies were a mix of comfort and horror. One person described a Selenium failure on a VM that devs, DevOps and testers had been chasing for over a year. Nobody had cracked it.&lt;/p&gt;

&lt;p&gt;So this isn't a "five easy fixes" post. Some of these are genuinely hard. But most CI-only failures I've seen fall into a handful of buckets, and knowing the buckets makes the search a lot shorter.&lt;/p&gt;
&lt;h2&gt;
  
  
  The pattern behind most of them
&lt;/h2&gt;

&lt;p&gt;CI usually isn't broken. &lt;strong&gt;Your machine is quietly giving the test something for free, and CI isn't.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Once you think about it that way, the question changes from "why is CI flaky?" to "what does my laptop have that the runner doesn't?"&lt;/p&gt;
&lt;h2&gt;
  
  
  1. Speed
&lt;/h2&gt;

&lt;p&gt;Your laptop is probably faster than the shared runner, and nothing else is competing for its CPU.&lt;/p&gt;

&lt;p&gt;So any check that doesn't wait will pass locally by luck and fail in CI. The classic one in Playwright:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Checks once, right now. No waiting, no retry.&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Saved'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;isVisible&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// Retries until it passes or times out.&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;expect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Saved'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toBeVisible&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Same goes for reading text straight after a click, or a fixed &lt;code&gt;waitForTimeout(2000)&lt;/code&gt; that happens to be long enough on your machine.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Quick check:&lt;/strong&gt; run the test many times locally, ideally while your machine is busy with something else. If it starts failing, you've found your bucket.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx playwright &lt;span class="nb"&gt;test&lt;/span&gt; my.spec.ts --repeat-each=20
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  2. Leftover state
&lt;/h2&gt;

&lt;p&gt;Locally, you've been logged in to that test account for weeks. The test data already exists. Some feature flag got switched on months ago and nobody remembers.&lt;/p&gt;

&lt;p&gt;CI starts from nothing.&lt;/p&gt;

&lt;p&gt;If your test only passes because something already happened before it ran, it'll pass on your machine forever and fail on a clean runner. The fix is boring but reliable: every test sets up what it needs (seed data, log in through an API, saved storage state) and doesn't assume anything else.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Environment differences
&lt;/h2&gt;

&lt;p&gt;The small stuff:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A &lt;code&gt;.env&lt;/code&gt; file that only exists on your machine&lt;/li&gt;
&lt;li&gt;Timezone: runners are very often on UTC. If your machine isn't, "today" and "yesterday" can land on different dates and assertions break&lt;/li&gt;
&lt;li&gt;Locale: number and date formatting&lt;/li&gt;
&lt;li&gt;Screen size: a smaller default window in CI can push elements off-screen or under a sticky header&lt;/li&gt;
&lt;li&gt;Browser version drifting between local and CI&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In Playwright you can pin a lot of this in the config, so local and CI share it by default instead of hoping they match:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// playwright.config.ts&lt;/span&gt;
&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="nf"&gt;defineConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;use&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;locale&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'en-GB'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;timezoneId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'Europe/London'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;viewport&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1280&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;height&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;720&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;trace&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'on-first-retry'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One of the replies I got put it well: the most useful habit is reproducing the CI environment as closely as possible (same OS, headless, browser version, locale, timezone, clean state) &lt;em&gt;before&lt;/em&gt; you start guessing. If your CI runs in a container, run that same container locally.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. The network path
&lt;/h2&gt;

&lt;p&gt;This one is the most invisible, and someone in the thread raised it better than I would have.&lt;/p&gt;

&lt;p&gt;Your laptop may be on a corporate VPN, behind a proxy, using split DNS, or trusting a company certificate authority. The CI image may have none of those.&lt;/p&gt;

&lt;p&gt;And here's the trap: a &lt;code&gt;curl&lt;/code&gt; or Node health check passing in CI doesn't prove the &lt;strong&gt;browser&lt;/strong&gt; is using the same proxy settings. They can take completely different routes.&lt;/p&gt;

&lt;p&gt;To tell a network problem apart from a slow element:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Check DNS resolution from inside the runner&lt;/li&gt;
&lt;li&gt;Check the browser's proxy settings, not just the shell's&lt;/li&gt;
&lt;li&gt;Find the &lt;strong&gt;first&lt;/strong&gt; failed network request in the trace, not the last error message&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And please don't "fix" a certificate error by turning on &lt;code&gt;ignoreHTTPSErrors&lt;/code&gt; and moving on. The test goes green, but you've hidden exactly the difference you were looking for. Get the CA into the CI image properly.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Parallel runs
&lt;/h2&gt;

&lt;p&gt;Tests that pass alone and fail together. Two workers editing the same user, the same basket, the same row in a table. Locally you're often running just the one spec you're debugging, so you never see the collision.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Quick check:&lt;/strong&gt; run the failing test on its own in CI. If it passes alone and fails in the full run, look for shared data.&lt;/p&gt;

&lt;h2&gt;
  
  
  Before you add a retry
&lt;/h2&gt;

&lt;p&gt;Retries have their place. But as one reply put it: &lt;em&gt;retries can hide the cause, but they rarely explain it.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;My order of operations now:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Get the trace, console log, network log and screenshot &lt;strong&gt;from the failed CI run&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Run locally with the same inputs and settings, and get the same artefacts&lt;/li&gt;
&lt;li&gt;Compare them side by side. Where do they first differ?&lt;/li&gt;
&lt;li&gt;Only then decide if it's a real bug, a test bug or an environment problem
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx playwright show-trace path/to/trace.zip
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One caution: traces capture network requests, headers and page content. Check them for tokens and internal hostnames before attaching one to a public ticket.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where AI actually helps (and where it doesn't)
&lt;/h2&gt;

&lt;p&gt;I use AI assistants for this kind of debugging most days now, and they've changed how fast I get through the boring parts. But they're better at some bits than others.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Where it helps:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Reading the wall of log output.&lt;/strong&gt; Paste in a long CI log and ask "what's the first real error here?" It's good at spotting that the timeout at the bottom was caused by a failed request 200 lines earlier.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Explaining unfamiliar errors.&lt;/strong&gt; Certificate errors, DNS failures, odd browser launch messages. You get a plain-English explanation and a list of likely causes in seconds.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Comparing two runs.&lt;/strong&gt; Give it the passing and failing output and ask what's different.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Writing the boring setup.&lt;/strong&gt; Seed scripts, API login helpers, a config that pins locale and timezone.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Git and PRs&lt;/strong&gt;, if that's not your strong side yet. It can create the branch and raise the PR, but read the diff yourself and ask it to explain each command. Your name is on that PR.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Where it struggles:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It can't see your environment.&lt;/strong&gt; It doesn't know your VPN, your proxy or what's baked into your CI image unless you tell it. A lot of CI-only bugs live exactly there.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Its first fix is often a band-aid.&lt;/strong&gt; Ask an assistant to "make this test pass" and you'll quite often get a longer timeout, a retry, or &lt;code&gt;force: true&lt;/code&gt; on a click. That makes the build green without explaining anything. Ask it "why does this fail?" before "fix this".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It sounds just as confident when it's guessing.&lt;/strong&gt; Treat its answer as a hypothesis, then prove it with the trace.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The way I think about it: AI shortens the time between "the build is red" and "I have three good theories". Proving which theory is right is still your job, and the trace is still the evidence.&lt;/p&gt;

&lt;h2&gt;
  
  
  Over to you
&lt;/h2&gt;

&lt;p&gt;The year-long Selenium VM mystery is still bugging me. What's the strangest CI-only failure you've tracked down, and what turned out to be the cause? Drop it in the comments. I'm collecting them.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm building &lt;a href="https://locatorleague.dev/learn/wait-for-dynamic-content?utm_source=devto&amp;amp;utm_campaign=ci_only" rel="noopener noreferrer"&gt;Locator League&lt;/a&gt;, browser-based Playwright challenges for practising this kind of thing (waiting properly, mocking, network checks) without setting up a project.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>cicd</category>
      <category>testing</category>
      <category>playwright</category>
    </item>
    <item>
      <title>How to Review AI-Generated Playwright Tests (Before They Break in CI)</title>
      <dc:creator>Babatunde Hammed</dc:creator>
      <pubDate>Wed, 21 Oct 2026 12:27:39 +0000</pubDate>
      <link>https://dev.to/2unde/how-to-review-ai-generated-playwright-tests-before-they-break-in-ci-4g5m</link>
      <guid>https://dev.to/2unde/how-to-review-ai-generated-playwright-tests-before-they-break-in-ci-4g5m</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp1ax7okptf26kwigt325.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp1ax7okptf26kwigt325.png" alt=" " width="800" height="450"&gt;&lt;/a&gt;&lt;br&gt;
I asked an LLM to write a Playwright test for a simple login flow. It passed the first time. Then I changed the button text from “Sign in” to “Continue” and re-ran it. The test failed.&lt;br&gt;
The model had used a CSS selector tied to a class name and a fixed sleep. Classic brittle patterns — just generated faster than a junior engineer would have written them by hand.&lt;br&gt;
This is becoming normal. Teams are using AI to draft Playwright tests, and the same failure modes from the pre-AI era are still showing up. The difference is volume and speed. If you don’t review the output, you’re shipping flaky tests into CI on purpose.&lt;br&gt;
Here’s what to look for and how to fix it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What AI Usually Gets Wrong
&lt;/h2&gt;

&lt;p&gt;Most models default to patterns they see a lot in training data:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Position or CSS-based locators — page.locator('.btn-primary'), nth child, or long XPath paths. These break the moment the DOM shifts or a class is renamed.&lt;/li&gt;
&lt;li&gt;Fixed sleeps — await page.waitForTimeout(2000) or similar. The model is trying to “wait for the page to settle” without knowing how Playwright’s auto-waiting works.&lt;/li&gt;
&lt;li&gt;Weak or missing assertions — The test clicks through the flow but never checks the outcome the user cares about (e.g. “Order confirmed” is visible).&lt;/li&gt;
&lt;li&gt;No resilience thinking — It optimizes for “this works on the page I see right now,” not “this will still work after a small redesign.”&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The generated code often looks plausible and even passes once. That’s what makes it dangerous.&lt;/p&gt;

&lt;h2&gt;
  
  
  A Practical Review Checklist
&lt;/h2&gt;

&lt;p&gt;Before you accept an AI-written test, run it through these questions:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Locator strategy — Does it prefer getByRole, getByLabel, or getByText? Or is it using CSS / XPath / nth?&lt;/li&gt;
&lt;li&gt;Waiting — Any waitForTimeout, waitForSelector with a hard timeout, or other fixed delays? Prefer web-first assertions (expect(...).toBeVisible()) that retry automatically.&lt;/li&gt;
&lt;li&gt;What it actually asserts — Does the test check the user-visible outcome, or does it only confirm that clicks happened?&lt;/li&gt;
&lt;li&gt;Redesign resilience — If the button text changes, a wrapper div is added, or elements are reordered, does the test still pass? If you’re not sure, it probably doesn’t.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;You don’t need to rewrite the whole test. Spot the weakest locator or wait and fix that first. One solid change often removes the flakiness.&lt;/p&gt;

&lt;h2&gt;
  
  
  Side-by-Side Example
&lt;/h2&gt;

&lt;p&gt;Here’s a typical AI-generated version of a login test:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;test('user can log in', async ({ page }) =&amp;gt; {&lt;br&gt;
  await page.goto('https://example.com/login');&lt;br&gt;
  await page.waitForTimeout(2000);&lt;br&gt;
  await page.locator('#username').fill('test@example.com');&lt;br&gt;
  await page.locator('div.form-group:nth-child(2) input').fill('password123');&lt;br&gt;
  await page.locator('.btn-submit').click();&lt;br&gt;
  await page.waitForTimeout(3000);&lt;br&gt;
  await expect(page.locator('.success-message')).toBeVisible();&lt;br&gt;
});&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Problems: fixed sleeps, CSS and positional locators, and a weak assertion.&lt;br&gt;
Revised version:&lt;br&gt;
&lt;code&gt;test('user can log in', async ({ page }) =&amp;gt; {&lt;br&gt;
  await page.goto('https://example.com/login');&lt;br&gt;
  await page.getByLabel('Email').fill('test@example.com');&lt;br&gt;
  await page.getByLabel('Password').fill('password123');&lt;br&gt;
  await page.getByRole('button', { name: 'Sign in' }).click();&lt;br&gt;
  await expect(page.getByText('Welcome back')).toBeVisible();&lt;br&gt;
});&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;The button locator still depends on its label, but that's the right trade-off. If the label changes, the failure is one clear line to update, not a flaky timeout.&lt;/p&gt;

&lt;h2&gt;
  
  
  Practice
&lt;/h2&gt;

&lt;p&gt;Reading a checklist is one thing. Catching these mistakes in real code is another. &lt;a href="https://locatorleague.dev/" rel="noopener noreferrer"&gt;Locator League&lt;/a&gt; has a free Foundations track where you write Playwright locators against real pages and get instant pass/fail feedback, plus a video of the run.&lt;/p&gt;

&lt;p&gt;Challenges where you review and fix AI-written tests are coming next. If you want to be told when they open, follow along here.&lt;a href="https://locatorleague.dev/" rel="noopener noreferrer"&gt;Locator League&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Your turn
&lt;/h2&gt;

&lt;p&gt;What's the worst locator or wait an AI has written for you recently? Drop it in the comments. The patterns are usually familiar once you see a few of them.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>automation</category>
      <category>playwright</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Why Your Playwright Selectors Keep Breaking (And How to Fix Them)</title>
      <dc:creator>Babatunde Hammed</dc:creator>
      <pubDate>Wed, 14 Oct 2026 12:18:56 +0000</pubDate>
      <link>https://dev.to/2unde/why-your-playwright-selectors-keep-breaking-and-how-to-fix-them-4egp</link>
      <guid>https://dev.to/2unde/why-your-playwright-selectors-keep-breaking-and-how-to-fix-them-4egp</guid>
      <description>&lt;p&gt;You've seen this before. The test passes on your machine, fails in CI, and the error is something like "locator.click: Timeout 30000ms exceeded." You re-run it. It passes. You re-run it again. It fails. Congratulations — you've met a flaky test, and the culprit is almost never the assertion. It's the locator.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Three Failure
&lt;/h2&gt;

&lt;p&gt;Modes nth-child selectors- `&lt;code&gt;page.locator('div.container &amp;gt; div:nth-child(3) &amp;gt; button')&lt;/code&gt;&lt;br&gt;
works today because the button happens to be the third child. Tomorrow a designer adds a banner above it, and the selector silently points at the wrong element — or nothing at all. The test doesn't error; it clicks the wrong thing, and your suite lies to you.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fixed sleeps&lt;/strong&gt;- &lt;code&gt;await page.waitForTimeout(3000)&lt;/code&gt; is the most common anti-pattern in Playwright codebases. It assumes the page loads in three seconds. On a fast machine it wastes time; on a slow CI runner it times out. Either way, you're not waiting for the right thing — you're waiting for a clock.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;CSS paths&lt;/strong&gt;. &lt;code&gt;.btn-primary-v2&lt;/code&gt; breaks the moment someone renames the class during a redesign. And if the class was generated by a CSS-in-JS library, it changes on every build.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;p&gt;Prefer getByRole and getByText. These locators target what the user sees and what assistive tech reads — accessible names, roles, visible text. A button's role doesn't change when its styling does.&lt;/p&gt;

&lt;p&gt;// Fragile&lt;br&gt;
&lt;code&gt;await page.locator('#submit-btn-47').click();&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;// Resilient&lt;br&gt;
&lt;code&gt;await page.getByRole('button', { name: 'Submit' }).click();&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Replace fixed sleeps with web-first assertions. Playwright's &lt;code&gt;expect&lt;/code&gt; auto-retries until the condition passes or the timeout hits, so you never guess at timing:&lt;/p&gt;

&lt;p&gt;// Fragile&lt;br&gt;
&lt;code&gt;await page.waitForTimeout(3000);&lt;/code&gt;&lt;br&gt;
&lt;code&gt;await expect(page.getByText('Order confirmed')).toBeVisible();&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;// Resilient&lt;br&gt;
&lt;code&gt;await expect(page.getByText('Order confirmed')).toBeVisible();&lt;/code&gt;&lt;br&gt;
`&lt;code&gt;&lt;/code&gt;&lt;br&gt;
The assertion itself is the wait. No sleep needed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Practice
&lt;/h2&gt;

&lt;p&gt;Reading about this is one thing. Writing locators under pressure is another. I built LocatorLeague — a free playground with twenty-nine browser challenges where you write a Playwright locator, the platform runs it in a real Chromium instance, and you get pass/fail feedback plus a video of the run. No install, no setup. The Foundations track is free at &lt;a href="https://locatorleague.dev" rel="noopener noreferrer"&gt;https://locatorleague.dev&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Closing
&lt;/h2&gt;

&lt;p&gt;What's the flakiest test in your suite? Drop it in the comments — half the time the fix is one locator swap away.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>tutorial</category>
      <category>beginners</category>
      <category>playwright</category>
    </item>
  </channel>
</rss>
