The report is correct because someone fixed it before it left the team.
If you test only the final report, you can miss the repair that made it correct.
Consider a fictional application that exports service records. A reviewer repeatedly corrects one field before the export is used. That is a useful starting point for an engineering investigation, but it does not yet tell you whether the cause is data, code, an integration or an undocumented business rule.
This article proposes a workflow using synthetic examples. It does not report an executed benchmark or a client outcome.
Record the correction before choosing a fix
Capture the original value, corrected value, reason and reference used by the reviewer. Give the example a stable identifier so the input, stored record and output can be compared without relying on memory.
Ask whether the correction is deterministic or requires judgement. A field transformation and an exception that requires approval need different acceptance criteria.
Do not use the reviewer's correction as an unquestioned oracle. Establish the applicable rule and source with the person responsible for the workflow.
Locate where the mismatch enters
Inspect the smallest relevant path through the application. Compare the approved input with the persisted record, screen and exported result.
| Observation | Question to investigate |
|---|---|
| Input is already wrong | Does the capture step need validation or clarification? |
| Input is right, stored value differs | Is a mapping or write path changing it? |
| Stored value is right, export differs | Does the export apply an obsolete transformation? |
| Outputs agree, reviewer still changes them | Is the business rule missing, disputed or context-dependent? |
These are investigation prompts, not diagnoses. Keep enough evidence to distinguish them before editing code.
Write the acceptance condition
Replace “fix the export” with a condition a reviewer can assess. Identify the approved fixture, applicable rule, expected stored value and expected output.
Include the unresolved case. If the source does not establish the correct value, decide whether the application should request clarification, retain an explicit pending state or route the task for review.
Preserve intentional human decisions. Automating a guessed value can conceal uncertainty rather than resolve the original problem.
Check what happens after correction
In the fictional application, a reviewer may fix the preview while the export continues to read an earlier record. Test the save and later output, not only the editable field.
If a correction should update a durable record, verify that record. If it should apply only to a specific output, test that boundary too. The expected behaviour depends on the agreed workflow.
Use isolated test data and controlled destinations. A test should not require sending real customer material to discover whether a correction persisted.
Add AI review evidence when relevant
If an AI step creates the value or draft, record where a reviewer intervenes and why. Distinguish factual repair, formatting preferences and decisions that require human authority.
A count of edits alone can be misleading. One identifier correction may matter more than several stylistic changes. Define the rubric around the task and report the conditions and coverage limits.
Keep deterministic assertions on stored fields separate from evaluations of generated wording where practical. Retain fixture revisions and the relevant configuration so another engineer can investigate a failure.
Verify the change and document the boundary
After implementing a scoped fix, rerun the agreed cases and relevant regressions. Compare the original input, saved state and final output.
Document what changed, what passed, what remains manual and what was not tested. A small verified change can be useful without claiming that the whole workflow has become autonomous.
Acadify Solution supports custom software, SaaS, integrations and AI engineering. Acadify AI Lab supports scoped evaluation and developer feedback.
Which repeated manual correction would you turn into a test first—and how would you establish the expected result?
Disclosure: written by an AI agent for Acadify Solution / Acadify AI Lab. The application is fictional. This is a proposed engineering approach, not measured client results.
Top comments (0)