DEV Community

xg704863664
xg704863664

Posted on

Your automation says it worked. Three ways it is lying to you.

#automation #debugging #javascript #testing

I spent a day chasing a problem that did not exist, because my scripts kept reporting success.

Here are the three failure modes I hit, all in the same week, all silent.

1. The check that matched the wrong element

I had a script that clicked a button, then verified the result by searching the page for a confirmation phrase.

The verification matched. The action had not happened.

Why: the page contained the confirmation phrase in help text elsewhere on the page. My check was:

if (pageText.includes('already requested')) return 'DONE';
Enter fullscreen mode Exit fullscreen mode

But the page also contained a sentence like 'If you have already requested this, ...' in a tooltip. So the check passed on every run, whether or not anything happened.

The fix is to assert on a mutually exclusive state, not a substring:

if (pageText.includes('This page is not indexed')) return 'NOT_INDEXED';
if (pageText.includes('URL is on Google')) return 'INDEXED';
return 'UNKNOWN';
Enter fullscreen mode Exit fullscreen mode

Substring checks are fine for reading. They are dangerous for asserting.

2. The element that was never in the viewport

Scroll-then-click is a classic. But the order matters more than people expect:

// WRONG - coordinates are captured before the scroll settles
const box = await getBox(selector);
await scrollIntoView(selector);
await click(box.x, box.y);   // clicks whatever is now at those coordinates
Enter fullscreen mode Exit fullscreen mode

The fix is boring: scroll, wait, then read the bounding rect.

I clicked the wrong element four times in a row because of this, and each time the script reported that it had clicked the target.

3. The state that persisted between runs

This one is my favourite, because the tool worked perfectly and still gave me the wrong answer.

I was reading a dashboard. It reported zero. I assumed the data had been reset.

It had not. The dashboard persists your date-range filter server-side. A previous run had set it to Today. Every subsequent run was faithfully reporting today's data - which was legitimately zero - while I compared it against last month's totals from memory.

The tool was correct. The context was wrong.

Now every scraper I write logs the state it is reading under, not just the values:

console.log('period:', period, '| value:', value);
// period: Today | value: 0      <-- instantly obvious
Enter fullscreen mode Exit fullscreen mode

The common thread

All three are the same mistake in different clothes: treating a successful execution as evidence of a correct outcome.

A click that dispatches is not a click that lands. A text match is not a state assertion. A number without its context is not a fact.

The practical rule I now follow: for every action that matters, find an independent signal that it happened - a fetch, a timestamp, a status from a different system. If the only evidence is the thing that did the action, you do not have evidence.

I write practical growing and preserving guides at Herbal Home Remedies and Small-Space Growing - the automation above exists to run that site without me watching it.

Top comments (0)