DEV Community

郭帅 for Skillπ

Posted on Fully Autonomous

AI says the bug is fixed. Here are three checks to run

When an AI coding assistant says it fixed a bug, ask for three pieces of evidence: a reproduction, a regression test that failed before the patch, and fresh test output. Three skills from obra/superpowers can help.

1. Reproduce and trace the bug

systematic-debugging starts with the error, reproduction steps, and recent changes. Ask the assistant where the actual behavior first diverges from what you expected.

2. Make the regression test fail first

With test-driven-development, capture the expected behavior in a test. For a date parser, try rejecting February 29 in a non-leap year. Run it before the patch and confirm it fails for the expected reason. Apply the smallest fix and run it again.

3. Check fresh output before handing off

verification-before-completion calls for running the relevant checks and inspecting their output. Ask for the commands, exit status, and untested paths. A regression test covers the behavior it asserts. You still need a runnable test environment and a review of the patch.

Try this prompt: “Reproduce the bug, show me the failing test, then report what passed after the fix and what remains untested.”

Original Chinese article on Juejin · Skillπ · Infinite Skills. Infinite Possibilities.

Carousel from the October 2026 editorial archive. This English adaptation was generated with AI.

Skillπ development checklist, card 1 of 5

Skillπ development checklist, card 2 of 5

Skillπ development checklist, card 3 of 5

Skillπ development checklist, card 4 of 5

Skillπ development checklist, card 5 of 5

Top comments (1)

Collapse
 
nexusverify profile image
Nexus Verify •

These three are the right floor. The one I'd add as a fourth: make sure the verdict doesn't come only from the same hands that made the fix.

In practice that means re-running the reproduction against the original failing input — the exact one, not a paraphrase the assistant generates afterward — and separately confirming the regression test actually fails on the pre-fix code (check it out, run just that test, watch it go red). A test that was only ever seen green after the patch proves it exists, not that it guards anything.

Same reason a checklist helps: "fixed" is a claim, and claims want evidence produced by someone — or something — that wasn't trying to make the claim true.