In my pipeline the zero deviations on main thread stuck out for the same reason. I found out my tool orchestrator was defaulting to Opus on every retry; I'd only specified the model for the main agent and forgot that other components could drift. Ran for about three weeks before I caught it by accident while looking at something else in the logs. If you break out time-to-detection by deviation class, I'm curious what the gap looks like.
Good question, and I cannot answer it the way you asked. That turns out to be
the answer.
Time-to-detection did not vary by class, because detection was not a per-class
event. All five classes surfaced in one run, on one day. What varies is how long
each had been running before that day:
class
first seen
ran undetected
cost
workflow-subagent on Opus 4.8
Jun 2
32 days
$489.40
general-purpose on Opus 4.8
Jun 3
31 days
$92.17
workflow-subagent on Fable 5
Jun 10
24 days
$656.83
general-purpose on Fable 5
Jun 11
23 days
$9.70
general-purpose on Haiku 4.5
Jun 11
23 days
$0.03
23 to 32 days, and the spread is explained entirely by when each class first
appeared. Your three weeks and my month are the same measurement: the gap until
somebody happened to look.
The part I would flag is that the most expensive class was not the longest
running one. Fable on workflow-subagent ran eight fewer days than Opus on
workflow-subagent and cost $167 more, $27 a day against $15. Ranking by how long
something hid sends you after the wrong one.
Your retry case is one mine would have missed, by the way. My grain is (unit, component), and a retry is not a separate component. It inherits inside
the same one, so the deviation and the compliant call collapse into a single
verdict. Same defect, invisible to my method. If your orchestrator writes the
model per attempt rather than per step, that is the thing worth keeping.
Following up on the retry case. I said my grain would miss it, and I have not changed that. (unit, component) still collapses a retry into the enclosing component, so per-attempt drift stays invisible on my side.
What I did change is the fall-through: no applicable rule is now its own verdict rather than a default, which is a different hole in the same table.
If you want to point the checker at your own logs, I am happy to hand it over. It reads Claude Code JSONL and writes a per-decision table, you write the policy file, and nothing leaves your machine. Or if a sample of your orchestrator's records is easier, that would tell me whether per-attempt grain is worth adding. Right now I have exactly one corpus and it is my own, which is a poor basis for a schema.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
In my pipeline the zero deviations on main thread stuck out for the same reason. I found out my tool orchestrator was defaulting to Opus on every retry; I'd only specified the model for the main agent and forgot that other components could drift. Ran for about three weeks before I caught it by accident while looking at something else in the logs. If you break out time-to-detection by deviation class, I'm curious what the gap looks like.
Good question, and I cannot answer it the way you asked. That turns out to be
the answer.
Time-to-detection did not vary by class, because detection was not a per-class
event. All five classes surfaced in one run, on one day. What varies is how long
each had been running before that day:
23 to 32 days, and the spread is explained entirely by when each class first
appeared. Your three weeks and my month are the same measurement: the gap until
somebody happened to look.
The part I would flag is that the most expensive class was not the longest
running one. Fable on workflow-subagent ran eight fewer days than Opus on
workflow-subagent and cost $167 more, $27 a day against $15. Ranking by how long
something hid sends you after the wrong one.
Your retry case is one mine would have missed, by the way. My grain is
(unit, component), and a retry is not a separate component. It inherits insidethe same one, so the deviation and the compliant call collapse into a single
verdict. Same defect, invisible to my method. If your orchestrator writes the
model per attempt rather than per step, that is the thing worth keeping.
Following up on the retry case. I said my grain would miss it, and I have not changed that. (unit, component) still collapses a retry into the enclosing component, so per-attempt drift stays invisible on my side.
What I did change is the fall-through: no applicable rule is now its own verdict rather than a default, which is a different hole in the same table.
If you want to point the checker at your own logs, I am happy to hand it over. It reads Claude Code JSONL and writes a per-decision table, you write the policy file, and nothing leaves your machine. Or if a sample of your orchestrator's records is easier, that would tell me whether per-attempt grain is worth adding. Right now I have exactly one corpus and it is my own, which is a poor basis for a schema.