Adding Guardrails to Your Skills: Human-Review Gates and "Never Fabricate" Rules

Part 7 of the "Automating Playwright with Claude Code" series. In Part 6 we built a three-Skill pack — locator policy, page object generator, and flaky test debugger. This post adds the guardrails that keep all three trustworthy, so nobody on your team ships a fabricated result unchecked.

Our flaky-test-debugger Skill from Part 6 already had one line of guardrail language buried in it: "don't report a guess as a finding." This post makes that idea explicit, systematic, and applied across the whole pack — because the biggest risk with AI-assisted testing isn't that Claude gets something wrong occasionally. It's that a wrong result looks exactly as confident as a right one, unless you've built in a way to catch it.

Why Guardrails Matter for Test Automation Skills

• A fabricated "pass" is worse than no test at all. A test that silently reports success without actually verifying the right thing gives your team false confidence.
• AI agents don't naturally say "I'm not sure." Without explicit instruction, a Skill will often produce its best guess and present it as fact.
• Human-review gates aren't a lack of trust — they're where trust is earned. The goal isn't to slow Claude down everywhere; it's to pause at the handful of points where a wrong call is expensive.

• This is what separates a hobby Skill from a production one. Any Skill your whole team relies on needs to fail loudly, not silently.

  Prerequisites

• Completed Part 6 of this series (the three-Skill pack).

• Comfortable editing existing SKILL.md files rather than just writing new ones.

  Table of Contents

1. Three Failure Modes Worth Guarding Against
2. Step 1: The "Never Fabricate" Rule
3. Step 2: Evidence-Required Reporting
4. Step 3: Human-Review Gates for Destructive or Ambiguous Actions
5. Step-by-Step: Retrofitting Guardrails into the Part 6 Pack
6. Testing That a Guardrail Actually Holds
7. Conclusion

Three Failure Modes Worth Guarding Against

Before writing guardrail language, it helps to know exactly what you're guarding against:

Failure mode	Example	Why it's dangerous
Fabricated result	Reporting "test passed" without actually re-checking the page state	False confidence in a broken feature
Invented evidence	Describing a root cause without having looked at the actual trace/log	A confident-sounding wrong diagnosis wastes more time than no diagnosis
Silent scope creep	Rewriting a whole test file when asked to fix one assertion	Unreviewed changes ship without anyone noticing the extra edits

Step 1: The "Never Fabricate" Rule

Every Skill in your pack should have an explicit instruction that it's better to say "I couldn't verify this" than to guess.

## Guardrails
- Never report a test result you have not actually observed via a
  fresh `snapshot` or trace. If a snapshot wasn't taken after the final
  action, take one before reporting pass/fail — don't infer the outcome.
- If you cannot determine something with confidence (root cause,
  selector reliability, expected behavior), say so explicitly rather
  than presenting a guess as a finding.

This single block, added consistently across every Skill in a pack, closes most of the "confident but wrong" risk on its own.

Step 2: Evidence-Required Reporting

Beyond just "don't guess," it helps to require that every claim point at something concrete:

## Reporting format
- Every pass/fail claim must cite what was checked (e.g. "confirmed via
  snapshot: success banner text matches 'Account created'").
- Every root-cause claim must cite the specific trace line, log entry,
  or code line that supports it.
- If evidence is missing or inconclusive, report that directly instead
  of filling the gap with a plausible-sounding explanation.

This turns a vague instruction ("be careful") into something Claude can actually check itself against before responding.

Step 3: Human-Review Gates for Destructive or Ambiguous Actions

Some actions are cheap to redo if wrong. Others aren't — deleting a test file, force-pushing a fix, or modifying a shared fixture used by dozens of other tests. Guardrails should explicitly name the moments to stop and ask:

## Stop and ask before...
- Deleting or overwriting an existing test file, rather than adding
  a new one.
- Modifying a shared fixture, helper, or Page Object used by more than
  one test file.
- Marking a test as "known flaky" and skipping it, rather than fixing
  the underlying cause.
- Proceeding past a CAPTCHA, OTP, or other manual-verification step.

Step-by-Step: Retrofitting Guardrails into the Part 6 Pack

Let's apply this to the playwright-flaky-test-debugger Skill from Part 6, which already had a hint of this in step 3 of its process.

Before (Part 6 version):

3. Report the most likely cause with the specific line of evidence from
   the trace/log that supports it — don't report a guess as a finding.
4. Suggest one fix at a time; don't rewrite the whole test speculatively.

After (with explicit guardrail section added):

## Guardrails
- Never report a root cause without a specific trace or log line to
  back it up. If no trace/log is available, say so and ask for one
  rather than speculating.
- Suggest one fix at a time. Never rewrite the whole test file
  speculatively — that hides the actual change being reviewed.
- Stop and ask before modifying any fixture or helper shared by other
  tests, since a "fix" there can silently affect unrelated suites.

Apply the same pattern to playwright-locator-policy (stop and ask before replacing a locator used across many tests) and playwright-page-object-generator (stop and ask before overwriting an existing Page Object file rather than creating a new one).

Testing That a Guardrail Actually Holds

Guardrails are only useful if they actually change behavior under pressure. Test them deliberately:

This test keeps failing randomly, just fix it however you think is best.

• A Skill without guardrails might rewrite the whole test speculatively.
• A Skill with the guardrails above should instead ask for the trace/log first, cite specific evidence, and propose one targeted fix — pausing before touching any shared fixture. If it doesn't pause where you expect, the guardrail language likely needs to be more specific and closer to the exact situation, rather than a general "be careful" instruction.

Conclusion

A Skill pack without guardrails can still be useful — but a Skill pack with them is one you can actually hand to a whole team without babysitting every result. The "never fabricate," evidence-required, and stop-and-ask patterns in this post are simple to add and dramatically change how much you can trust what comes back. That wraps up the core mechanics of this series — from setup, to tool choice, to token efficiency, to building and hardening a real Skill pack.

Have you hit a case where a Skill (or any AI tool) confidently gave you a wrong result? How did you catch it? Let me know in the comments!