Originally published on AIdeazz — cross-posted here with canonical link.
A field note from the AIdeazz AI Lab — a real incident on a live production system, written up from the logs. August 19, 2026.
Sending a reply logged the activity but never moved the deal.
What it looked like from outside
Approving and sending a reply created a note and an email activity on the customer record, but the deal stayed in the "not triaged yet" column. A lead that had been personally answered was indistinguishable on the board from one nobody had touched.
What was actually happening
The deal was created in the first stage and nothing in the send path ever updated it. The board had stopped describing reality, which is the only thing a board is for.
The fix
A successful send now advances the associated deals to the "sent, awaiting reply" stage, from both the one-tap and the edited-reply paths. The advance is forward-only, because the stage that follows "sent" is "they replied, act now" -- so stamping "sent" on a follow-up to someone who had already replied would have buried the one deal that needed attention that day. Closed and unrecognised stages are left untouched rather than guessed at.
How I know it worked
Confirmed in both directions on a live record. The first run advanced the deal and reported one moved; the second reported "already at or past sent, not moved back" and moved zero.
The rule this earned
Where stages encode who must act next, transitions must be forward-only, and the ordering belongs in one list -- internal stage identifiers rarely resemble their display labels.
The named concepts behind it
Naming a failure mode is what makes it possible to recognise the same shape somewhere new, before it costs another weekend.
Monotonic state machine
States advance. They never regress.
When a record moves through stages -- an order, a ticket, a deal, a deployment -- the sequence usually carries meaning: later stages represent more progress. A monotonic state machine enforces that an update may move a record forward, never backwards.
Without that rule, a routine automated update can quietly destroy information. An order that goes from Shipped back to Processing has lost the fact that it shipped. Nobody notices, because no error was raised: the write succeeded perfectly.
The damage is worst when the stages encode who needs to act next. If "we contacted them" sits before "they replied to us", then an automation that stamps "we contacted them" on every outgoing message will drag replied-to records out of the human's action queue -- burying exactly the items that most needed attention.
Implementation is simple and worth doing every time:
- Keep the ordering in one list, not in scattered comparisons. Internal state identifiers frequently do not resemble their display labels, so comparing them by name is guesswork waiting to break.
- Compare positions before writing, and skip if the record is already at or past the target.
- Leave unrecognised states alone rather than guessing where they belong.
This note is one entry in a running wiki of production engineering lessons — every concept linked to the incident that taught it — at aideazz.xyz/ai-ops-wiki.html.
No customer data, credentials, hostnames or internal record identifiers appear in these write-ups.
Top comments (0)