The most dangerous sentence in an AI-built app is often:
The bug is fixed.
That sentence can mean several different things:
- the original symptom disappeared;
- the error message was hidden;
- the failing path was bypassed;
- the code now handles one example but not the workflow; or
- the change fixed one screen while quietly breaking another.
I have learned to treat an AI-generated bug fix as a claim that needs evidence, not as a conclusion.
The beginner-friendly rule is this:
A bug is fixed only when the original failure is gone, the intended behavior works, and the nearby behavior still works.
That is a small regression test. You do not need a large QA department to run it. You need a clear before-and-after record.
If you are staring at a blank prompt box, my AI App Builder Starter Prompts are free and include prompts for scope, debugging, QA, deployment, and launch. The useful part is not asking AI to sound confident. It is making the proof request specific.
First, Capture The Bug Before Changing Code
Before asking AI to fix anything, write down the smallest reproducible version of the failure.
Use this format:
Starting state:
[what must already be true]
Actions:
1. [action]
2. [action]
3. [action]
Expected result:
[what a user should see or be able to do]
Actual result:
[what happens instead]
Evidence:
[error text, screenshot, console output, or saved record]
For example:
Starting state: I am signed in and have one saved task.
Actions: open the task, edit its title, press Save, refresh the page.
Expected result: the new title remains after refresh.
Actual result: the screen says Saved, but the old title returns.
Evidence: the network request succeeds, but the database record is unchanged.
That description is much more useful than “Saving is broken.” It gives AI a path, an expected outcome, and a place to look for false success.
The Four Proofs Of A Real Fix
When AI proposes a patch, I want four separate proofs.
1. The original reproduction now passes
Run the exact same starting state and actions. Do not paraphrase the steps because the patch may only work under a slightly different condition.
If the original reproduction cannot be run anymore, ask why. Sometimes the code change genuinely changed the workflow. Sometimes the test data disappeared. Sometimes the error was hidden. “I cannot reproduce it” is not the same as “it is fixed.”
Record the result plainly:
- original steps: pass or fail;
- expected result: observed or not observed; and
- evidence: screenshot, saved record, test output, or visible state.
2. The data or state is correct underneath
Screens can lie by being polite.
A toast that says “Saved” does not prove that the record was saved. A blank error area does not prove that an invalid request succeeded. A loading spinner that stops does not prove that the correct result arrived.
Check the state behind the screen using the safest visible tools available to your project:
- refresh the page;
- close and reopen the app;
- sign out and back in when ownership matters;
- inspect the visible record in the database dashboard if that is part of your workflow; or
- run the project’s existing test or logging command.
The question is: did the promised state change persist in the place where the product actually owns it?
3. The failure is still honest when it should fail
A real fix does not turn every input into a success message.
Try one invalid input, one missing value, one offline or timeout condition, or one unauthorized account when the bug involves those boundaries. The exact test depends on the feature, but the principle is stable: the app should fail clearly and safely.
If AI changed a validation error into an empty result, it may have hidden the bug. If it catches every exception and returns a generic success object, it may have hidden several bugs at once.
Ask:
When this request is invalid or unavailable, what does the user see, what state is written, and how can they recover?
4. The neighbor workflow still passes
Most bug fixes touch shared code. That means the original screen is not the only thing worth testing.
If AI changed a shared form component, test another form. If it changed an authentication check, test both an owner and a non-owner. If it changed a database query, test an empty list and a list with several records. If it changed mobile layout code, test a smaller screen and a keyboard-open state.
You are not trying to test the entire universe. You are testing the closest behavior that could have been affected by the same change.
The Before-And-After Table I Use
For a small app, this is enough:
| Check | Before patch | After patch | Evidence |
|---|---|---|---|
| Original reproduction | Fails | Passes | visible result or test output |
| State after refresh | Old value returns | New value remains | refreshed screen or record |
| Invalid input | Misleading success | Clear correction path | error state |
| Neighbor workflow | Passes or fails | Still behaves as expected | second scenario |
| Logs and secrets | Error or unsafe data | Useful, non-sensitive evidence | log/test review |
The table is deliberately boring. Boring is good. It stops a confident explanation from replacing a working product.
A Debugging Prompt That Requests Proof
I would give AI this prompt instead of “fix the bug”:
Treat this as a regression investigation, not a request for a confident explanation.
Bug reproduction:
[starting state, exact actions, expected result, actual result]
Constraints:
[files or behavior that must not change]
Before editing code:
1. Explain the most likely cause and the evidence that would confirm it.
2. List the smallest files or functions that may need to change.
3. Name one way a patch could hide the symptom instead of fixing the cause.
After proposing the patch, provide:
1. The exact original reproduction to rerun.
2. One persistence or underlying-state check.
3. One invalid-input or failure-path check.
4. One nearby workflow that could regress.
5. A short explanation of what evidence would make us reject the patch.
Do not remove an error merely to make the screen look successful. Do not broaden the change beyond the stated bug without explaining why.
The phrase “what evidence would make us reject the patch” is important. It gives you a way to disagree with AI before the code becomes too large to understand.
When The Fix Is Not Ready
I would keep the bug open if any of these are true:
- the original steps changed without an explanation;
- the UI says success but refresh loses the result;
- the error disappeared and no replacement behavior exists;
- the fix works only with one hard-coded example;
- the patch changes unrelated screens without a clear reason;
- no failure-path test was run; or
- nobody can say what evidence proves the underlying state is correct.
That is not perfectionism. It is a boundary between a demo and a product.
The deeper field manual behind this workflow is AI App Builder From Zero. It walks through the larger path from idea and scope to architecture, prompting, QA, deployment, and launch. Use it when you need the whole build system, not just one debugging prompt.
The Short Version
When AI says it fixed a bug, ask it to help you prove four things:
- The exact original reproduction passes.
- The underlying state is correct after refresh or relaunch.
- Invalid or unavailable inputs still fail honestly.
- The nearest shared workflow did not regress.
If you can show those four proofs, you have more than a plausible patch. You have a defensible fix.
If you want a free place to start, AI App Builder Starter Prompts gives you practical prompts for turning a rough idea into scoped, testable work. The prompts are free. When you are ready for the deeper build-along process, AI App Builder From Zero is the next step.
You can also find me here:
Medium: https://medium.com/@marcusykim
DEV.to: https://dev.to/marcusykim
Website: https://marcusykim.com/
X: https://x.com/marcusykim
LinkedIn: https://www.linkedin.com/in/marcusykim/
Top comments (0)