The green Saved toast answers one narrow question: did the React application render a success message?
It does not prove that the database committ...
Some comments have been hidden by the post's author - find out more
For further actions, you may consider blocking this person and/or reporting abuse
The three-layer postcondition model makes the distinction between an optimistic UI and a completed workflow very clear. The correlation key is the part that keeps this reliable under parallel CI: making it visible in the audit-event query as well as the fixture prevents a fresh event from another test from satisfying the assertion by accident.
@alexshev Exactly—the correlation key has to survive into the sink query, not just the setup fixture. Otherwise parallel CI can hand the test someone else's
role.changedevent and give a false green. I'd also assert the event's actor and before/after roles so a correctly correlated but incomplete audit row cannot pass.Yes—the sink assertion should be a full identity tuple, not just a correlation key. I would include the test-run ID, actor, target object, before and after values, event type, and a bounded timestamp window, then require exactly one matching row. That catches both cross-run contamination and duplicate emission.
@alexshev Exactly one matching row is the key refinement. I’d let the bounded polling window finish before asserting the count; returning on the first match could miss a duplicate emitted by a retry a moment later. Carrying the test-run ID through the outbox and checking server-side event time would also make this less sensitive to parallel CI and clock skew. Have you seen teams expose that identity tuple through a narrow test-only query?
Yes—usually a read-only test support endpoint or a database fixture query behind the test environment boundary. It exposes the event identity and immutable timestamps, never a generic log search. The production API stays unchanged, while integration tests can assert the complete outbox-to-sink contract.
The bounded poll proves presence, and the three-surface split is the right shape. Two things I would add after reading the code.
First, polling cannot prove absence. For the case where a role change must not emit an event (a rejected save, a no-op update), eight polls at 500 ms finding nothing only says nothing arrived in four seconds. Either assert on the sink's own count before and after for that correlation ID, or have the worker write an explicit "processed, no event" marker so you wait on something positive.
Second, the correlation ID is
cy-${spec}-${Date.now()}, which repeats if Cypress retries a test within the same millisecond-granularity window or two specs share a name across projects. Cypress'sCypress.currentRetryplus a random suffix removes that. Alsocy.requestitself takes time, so the real window is longer than 8 x 500 ms; if the product promises two seconds, measure from the click using timestamps in the audit event and assert the difference, rather than relying on the attempt count.Does the audit event carry the actor as well as the subject? A role change logged without who made it passes this test and still fails an audit.
@arhancanli You're right on both points. The bounded poll proves presence, not absence, and
Date.now()alone is too weak under retries; I'd addCypress.currentRetry()plus a random suffix. The audit event should carry actor ID as well as subject ID, action, and correlation ID, and the test should assert them together. For a no-op, I'd wait for an explicit processed marker rather than infer success from silence.Great article, Raju! The idea that a green “Saved” message is only one witness really stood out to me. Verifying the durable state and the expected side effect makes the test far more meaningful, especially when asynchronous workers are involved. I also liked the practical points about bounded polling and correlation IDs to avoid flaky or falsely passing tests. As an AgentInspect contributor, it’s great to see the same evidence-first thinking applied to UI testing here.
@johnnylemonny Thanks, John. The "one witness" framing is the key: browser state, durable API state, and the side-effect sink should be independently observable. The agent-tracing analogue is distinguishing a proposed tool call from an executed call and a verified external effect. Which audit boundary has been hardest to make reliable in your work?
Definitely the boundary between executed tool call and verified external effect, especially when third-party APIs or async background jobs sit on the other side.
Getting a 200 OK or "queued" receipt tells you the request crossed the wire, not that the downstream state actually mutated as intended. In agentic workflows (and distributed systems in general), the hardest part has been enforcing an explicit "read-after-write" verification probe against the actual sink rather than trusting the tool's immediate return payload as ground truth.
@johnnylemonny Exactly. I'd model
queuedas accepted, not completed, and correlate a bounded read-after-write probe with the original command. If the probe expires, recordoutcome_unknownso retry logic cannot quietly duplicate the external effect. That distinction is valuable in agent traces as well as Cypress tests.The third row is the one that burned us, in a different shape. We run an agent that posts video from real iPhones, and the app's own "posted" screen was our green toast. An audit against the public profile pages found 159 of 165 planned posts actually live, and every one of the six misses was a run that had reported success.
Two things from that which map onto your table. First, the sink has to be read by someone who is not the actor. We had comments on a forum that the posting account could load as live while everyone else saw them as removed, so our postcondition now runs from a browser with no session. Second, "not found within the budget" wants its own result, separate from fail. We keep "unknown" as a third state, because retrying an action whose first attempt may have landed is how you get duplicates.
Do you treat the audit timeout as a hard fail, or re-query later before deciding?
@runfoundry Your unauthenticated-browser check is a strong fourth witness, especially when the actor's session can see content others cannot. In the example, timeout fails the test, but I would not auto-retry the business action from that failure; operationally I'd mark the outcome unknown, re-query the public sink later with the same correlation key, and retry only once absence is established. How do you bound that unknown window before escalating?