Level 0 of Arc Ops made a retried tool call safe to repeat. Level 1 stopped the retries amplifying an outage. Neither bounds the bill you run up on yourself, and max_cost = 5.00 is a number without a place. The place is what decides the outcome.
The reference workload is a month-end close: 6 steps, 8 calls, 72 billing units, 741 cents - and 400 of those 741 sit in one indivisible unit, a bulk reprice that bills once and cannot be stopped after it starts. That single attribute drives the whole level, and it is not an accident: in real agents the cheap calls stream and the expensive ones are single purchases.
PYTHONPATH=src python -m arc_ops.levels.l2_cost_caps.demo # every number below
python -m pytest -q # no network, no model
Repo: https://github.com/dev48v/arc-ops - PUBLIC, MIT, dependencies = [], 369 pytest at the time, and a page that fetches nothing off itself: https://dev48.infy.uk/arcops/level2-cost-caps.html
A spend dashboard is an observer that reads the meter once, at the end: at a ceiling of 250 it bills the full 741. The best observer money can buy - poll every tick, zero lag, publish per chunk - bills 521 and still overruns, because the 400-cent call had already started.
| check at | spend at ceiling 250 | mean overrun over 741 ceilings | worst |
|---|---|---|---|
| run end (the control) | 741 | 370.00 | 740 |
| per step | 581 | 163.65 | 459 |
| per call | 521 | 129.96 | 399 |
| per chunk | 521 | 108.66 | 399 |
Per-chunk checking is genuinely worth 21.30 cents per ceiling - and it is byte-identical to per-call at 445 of the 741 ceilings, sharing its worst case to the cent, because the expensive call is the uninterruptible one. Fine granularity buys the cheap region and nothing at the top.
Then the unit problem. Hold the lag at exactly one tick and sweep the ceiling across all 741 places it could sit: the marginal cost of that identical delay runs from 3 cents to 400 - a 133x spread with ticks_late == 1 in every row. "Our alert fires within thirty seconds" is not a budget guarantee, it is a sentence in the wrong unit.
The inversion, which I did not see coming
The control is no cap at all: 200 tasks delivered, 209,244 cents, 1,046.22 each. Sweep the ceiling and not one value improves cost per delivered task. Seven make it worse.
| ceiling | completed | wasted | cost per delivered task |
|---|---|---|---|
| 400 | 24 | 90% | 3,917.96 (3.74x) |
| 1,000 | 119 | 50% | 1,461.47 |
| 2,000 | 195 | 5% | 1,065.83 |
| none | 200 | 0% | 1,046.22 |
At cap 400, ninety per cent of what the fleet spent bought nothing. A cap does not reduce cost. It reclassifies it, from delivered work into waste - and whoever absorbs that is the person who asked for the task that died at ninety per cent.
The wreckage is worse than the abort. Restarting the 58 aborted runs under the same ceiling finishes 0 of 58: 696 attempts, 891,924 cents, every attempt billing identically, so more attempts is not slow progress. A checkpointed resume finishes 58 of 58 at 1.08x the uncapped price.
206 in-page assertions, recomputed on load. Next is L3, approval gates: a cap decides how much an agent may spend, and nothing here decides what it is allowed to do. Nine levels, one at a time: https://dev48.infy.uk/arcops.php
Top comments (0)