DEV Community

The AI Leverage Weekly
The AI Leverage Weekly

Posted on

How to Debug a Failing Test with AI (Step-by-Step)

#testing #ai #debugging #programming

How to Debug a Failing Test with AI (Step-by-Step)

A test is red. The error message is vague. You've stared at the diff for ten minutes and nothing jumps out. This is exactly where AI earns its keep — not by guessing, but by systematically surfacing the class of failure you're dealing with before you've wasted an hour going in the wrong direction. Here's the exact workflow I use, with copy-paste prompts you can drop in today.

Step 1: Collect the Full Failure Context Before You Prompt

AI can only help as much as the context you give it. Before you open a chat window, gather:

  • The full test output (not just the last line — the whole stack trace or assertion diff)
  • The test code itself
  • The function or module under test
  • Any recent changes to either (a one-line git diff summary is fine)

Don't summarize yet. Raw output is better than your interpretation at this stage.

Step 2: Use the Diagnostic Prompt

Paste everything you collected into this prompt. The goal of this first message is triage — classify the failure before you start fixing it.

You are a senior software engineer helping me debug a failing test.

Here is the failing test:
[PASTE TEST CODE]

Here is the function/module it tests:
[PASTE IMPLEMENTATION CODE]

Here is the full test output:
[PASTE FULL ERROR / STACK TRACE]

Recent changes (if any): [BRIEF DESCRIPTION OR GIT DIFF SNIPPET]

Do the following:
1. Identify the category of failure (assertion mismatch, unexpected exception, setup/teardown issue, flaky timing, wrong mock, etc.)
2. State the most likely root cause in one sentence.
3. List up to three hypotheses, ranked by probability.
4. Suggest the smallest possible code change to confirm hypothesis #1 before fixing anything.
Enter fullscreen mode Exit fullscreen mode

That last instruction — smallest change to confirm, not fix — is the key. AI will default to handing you a fix. You want a diagnostic step first, because if hypothesis #1 is wrong, the fix is wrong too.

Step 3: Confirm Before You Fix

Run the diagnostic the AI suggests. It's usually something like adding a console.log, asserting an intermediate value, or temporarily hardcoding input to isolate the boundary. Once you know which hypothesis is correct, go back:

Hypothesis #1 was correct. The root cause is [what you confirmed].

Now give me the minimal fix — only change what is necessary to make this test pass without altering the contract of the function. Explain each change in one line.
Enter fullscreen mode Exit fullscreen mode

Asking for a minimal fix and requiring explanations keeps the AI from over-engineering a solution that touches things you didn't break.

Step 4: Handle Flaky Tests Differently

If the test passes sometimes and fails sometimes, the diagnostic prompt above still works — but add this line before your closing instructions:

"This test is intermittent. Focus specifically on timing issues, shared state, test ordering dependencies, and non-deterministic data."

Flakiness almost always traces back to one of those four categories. Telling the AI upfront prevents it from chasing assertion logic that isn't the real problem.

Step 5: Validate the Fix Doesn't Just Pass — It Tests the Right Thing

Once the test is green, run one more prompt:

Here is the original test and the fix we applied:
[PASTE BOTH]

Does the test still meaningfully verify the intended behavior, or did the fix change what is being tested in a way that reduces coverage? Point out any assertions that became weaker or any edge cases that are no longer exercised.
Enter fullscreen mode Exit fullscreen mode

This is the step most people skip, and it's where AI adds real value beyond just making CI green. A test that passes because you changed the assertion threshold isn't a fixed test — it's a hidden regression.

This pattern — triage, confirm, fix minimally, validate coverage — is one of the ones I've packaged into The AI Leverage Playbook: 50 Prompts & Workflows for Engineers — but the version above is enough to get value on its own.

A Note on What AI Gets Wrong Here

AI will occasionally misread a stack trace and chase a symptom instead of the cause. Two guardrails help:

  1. Never skip Step 2's confirmation step. If you jump straight to the fix, you lose the ability to catch a misdiagnosis cheaply.
  2. If the AI's hypothesis doesn't match what you see in the code, say so explicitly. "That hypothesis doesn't fit because X. Revise." Models course-correct well when you push back with evidence.

Putting It Together

The full flow takes about five minutes of prompting for most failures. The payoff is that you stop debugging by vibes — scrolling, tweaking, re-running — and start debugging by elimination. AI doesn't replace the engineer; it just makes the triage loop faster and more systematic.

Copy the two main prompts above into a scratch doc or a snippet manager and reach for them the next time a test goes red.

I break down one workflow like this every week in The AI Leverage Weekly — practical, no fluff, free. It's one concrete AI workflow per issue, aimed at working engineers who don't have time to wade through hype. Subscribe: https://theaileverageweekly.beehiiv.com/subscribe?utm_source=devto&utm_medium=article&utm_campaign=long_w10

Top comments (0)