On July 17, 2026, Fortune reported OpenAI CFO Sarah Friar's proposed lens: "useful intelligence per dollar." For developer teams shipping agents, that is the measurement shift that matters: stop treating token usage as the main ROI signal, and start asking whether the agent produced work a reviewer accepted.
Cost per successful task is the metric I would put next to every agent run in CI.
If tokens go down while accepted work stays flat, you optimized inference, not the workflow.
Why tokens are the wrong denominator
Token cost is useful for capacity planning. Seat licenses are useful for access control. Neither tells you whether the agent produced dependable output.
That distinction matters for developer teams building automated SDRs, copy agents, media-buying orchestrators, or content pipelines like Vanaxity, Van Data Team's review-gated SEO, GEO, and AEO agent. These systems can create visible activity all day: sent messages, draft pages, generated variants, scheduled bids, and handoff artifacts.
Activity is not success.
A low token bill can still hide an expensive workflow if humans reject the output, rerun prompts, patch data errors, clean up unsupported claims, or rebuild the artifact downstream.
Start with accepted tasks
The important shift behind useful intelligence per dollar is not a new dashboard widget. It is moving measurement from model consumption toward dependable completed work.
For an engineering team, I would translate that into one rule:
Only count a task after it clears a stable quality bar.
That bar should be explicit enough that two reviewers can apply it the same way. For a code agent, maybe the patch builds, tests pass, and review comments are addressed. For an SDR agent, the message might need compliant targeting, correct company context, and no fabricated personalization. For an AI copywriter, accepted work could mean the draft matches the brief, uses sourced facts, and needs only normal editing.
Those marketing and sales examples are my application of the accounting pattern, not an OpenAI recommendation. The broader principle is simple: measure completed work against the full cost required to get it accepted.
Charge the whole path
The simple formula:
cost_per_successful_task = full_workflow_cost / accepted_tasks
The hard part is the numerator.
Full workflow cost should include more than inference. Count model spend, orchestration, software licenses, retrieval or data services, human review, failed runs, retries, rework, and cleanup. If the same task takes multiple attempts and a reviewer pass before acceptance, all of that belongs to the accepted task.
A concrete tracking chain helps:
- Usage: tokens, compute, seats, API calls.
- Attempts: every agent run that tried to complete a task.
- Completions: tasks the agent claimed were done.
- Accepted work: completions that passed the quality gate.
- Business value: verified revenue, savings, risk reduction, or other measured impact.
Those are separate because they answer separate questions. Token cost asks how expensive inference was. Attempts ask how busy the automation was. Accepted work asks whether the system produced usable output. Business value asks whether the output mattered.
Build it into the agent
Do not leave this as a finance exercise at the end of the month. Put task accounting into the workflow.
Emit a task id when the agent starts. Record each attempt, reviewer decision, retry cause, and acceptance status. Store the quality bar version with the result, because definitions drift. Capture human time in a coarse but consistent way. Keep rejected work visible instead of dropping it from the report.
A minimal event shape can be enough to start:
{
"task_id": "uuid",
"task_type": "agent_task_name",
"quality_bar": "review_gate_name",
"attempt_id": "uuid",
"attempt_status": "accepted|rejected|retry",
"reviewer_decision": "accepted|changes_requested|rejected",
"retry_cause": "missing_context|quality_gap|policy_issue|null",
"costs": {
"model": "decimal",
"tools": "decimal",
"human_review": "decimal",
"rework": "decimal"
}
}
For review-gated content systems like Vanaxity, the same model applies across research, writing, illustration, publishing, and syndication. The accepted blog post is not just the final draft. It is the cost of every stage required to get quality-approved work through the pipeline.
Tradeoffs worth accepting
This metric is slower than watching a token dashboard. It also creates uncomfortable visibility. A team may discover that the cheapest model produces more rejected work, or that a high-quality agent is bottlenecked by review policy rather than generation.
There is also a risk of overfitting the quality gate. If acceptance rules become too narrow, the agent learns to satisfy the checklist instead of the business need. If the rules are too loose, cost per successful task becomes vanity math.
The answer is to report two layers honestly. Use production efficiency when you can verify accepted output but not downstream value yet. Use business ROI only when there is verified value attached to the work.
The developer takeaway
If you are shipping agents, do not stop at "the run succeeded." Add the accounting chain beside your logs.
A practical first version can be small: define one task type, one acceptance rule, one reviewer outcome, and one full-cost calculation. Then compare models, prompts, tools, and agent designs by accepted work per dollar instead of usage per dollar.
What is the toughest part to instrument in your agent stack today: human review time, retry causes, or accepted-task quality gates?
📖 Read the full guide → AI Agent ROI: How to Measure Cost per Successful Task
Top comments (0)