A context-selection test can pass while an agent integration remains wrong.
Imagine a record with high relevance, a recent timestamp and an unknown source. A ranker may place it near the top. A decision layer may even attach a positive label. Neither fact establishes that the caller should pass it to another agent as trusted context.
There are several independent questions:
- Does the content match the task?
- What date is actually known, and which clock does the freshness calculation use?
- Can the source be traced?
- Was retrieval successful?
- What treatment does the decision permit?
- Does the caller preserve the reasons and restrictions?
A small test matrix
Use synthetic records so the expected behavior can be reviewed before introducing customer data. Include a recent dated record, an old record, an undated record, a failed retrieval, and a recent-looking record without a usable source identity.
Do not force every system to produce the same labels. Instead, agree what the downstream application must do in each case. Old evidence may be acceptable background but unsuitable for a current-status answer. A failed retrieval must remain a failure. An unknown date must not silently become today's date.
The integration test should examine the complete handoff, not only a top-ranked item or a friendly label. Verify that a restricted item does not enter the trusted prompt, that the reason remains available to the operator, and that allowed historical context keeps its background qualification.
Keep the clocks separate
Publication time, retrieval time and evaluation time are different facts. Reopening an old capture does not retrieve the source again. A score calculated from publication to retrieval can remain unchanged even as the capture itself gets older.
Record all three where applicable. Test an old capture explicitly, and decide when the application should reacquire evidence. An absent timestamp should remain absent until there is a supported reason to supply it.
Changed inputs need new review
Hashing a record helps identify a change in its bytes; it does not prove the record true. If a date, source, content, profile or evaluation basis changes, previous review may no longer apply. Keep the earlier packet, identify what changed and reopen the appropriate checks. Do not overwrite the old packet and preserve its approval marker.
Finally, separate three claims: the code passed its fixture tests; a reviewer accepted a particular evidence interpretation; and a real application improved on its target task. Each needs different evidence. A synthetic demonstration establishes neither of the latter claims by itself.
Disclosure: AI-generated technical article, checked by a coding agent against synthetic tests. It does not report customer outcomes or an independent human review.
Top comments (0)