Most teams learn what an agent costs from the provider's invoice at the end of the month, with no way to tie a number to the task that produced it. AgenticOS puts a run's model, token usage, reported cost and tool events together in Activity. That lets you connect a cost figure to the task that produced it. This guide reads an existing demo record, then explains how to use those fields when testing an agent and setting a budget in the platform we build at Vstorm.
This article was drafted and revised with AI assistance. The amounts shown are from an existing demo capture. They are not provider invoices, current price recommendations or a benchmark of a representative workload.
Open one run
The record shows Support Copilot answering "how do I rotate a provider key?" through Slack. It names OpenRouter and anthropic/claude-opus-4.5, displays 5,397 input tokens and 170 output tokens, and reports $0.0312. A completed search_documents call appears in the timeline.
Read those fields together. The amount without the task says little about whether the run was useful. The token counts without the resolved model are not enough to interpret pricing. The successful status tells you the run completed, but the source passage and answer still need a quality check.
For this example, the screen helps you understand where to look. It does not establish what another document question will cost or how much a model switch would save. Use your own fixed task and inputs for that comparison.
Keep a comparison tied to the task
Suppose you want to shorten a support agent's prompt. Choose a small set of sample questions and preserve the expected source material. Run the current configuration and the proposed revision with those same cases.
Keep the configured model, answer, retrieved sources, status and total recorded usage. If fallbacks are enabled, the documented run row can retain the primary profile even when another model answers. Use a no-fallback test profile or separate provider evidence for a controlled model comparison. Check whether the shorter prompt preserves the instructions the task depends on. If the answer drops an important exception, lower usage does not make it an equivalent result.
Include retries and unsuccessful attempts when evaluating the complete task. Selecting only the cheapest successful run can hide the work needed to obtain that answer. If a model or provider setting changes as well as the prompt, record that change rather than attributing the difference to the prompt alone.
This is a proposed comparison procedure. No result from that experiment is reported here.
Include work performed before failure
A run can fail after it has already made billable model requests. AgenticOS keeps failed runs in history with their recorded usage. That makes failures relevant when reviewing the cost of a workflow, even if the user never received a final answer.
When inspecting a failed run, locate the stage that stopped. A tool error after model work is different from a request refused before execution. The record can help you connect usage with the progress made before the failure.
Do not assume that every failure has the same cost or that failures are necessarily more expensive than successful runs. Use the recorded events and usage for the actual task. If you retry, keep the earlier attempt beside the successful one when assessing the resources needed to complete it.
Read the Spend view's coverage
The existing Spend capture shows a warning about unpriced runs. A missing price is not a zero price: the displayed total can be a floor for the usage in that window.
Before comparing the dashboard with a provider account, check which model identifiers were priced and whether the provider or proxy adds charges outside that lookup. Also compare the same reporting period. A calendar-month figure and a rolling window can describe different sets of runs even when both appear on the same page.
Model usage is only one cost component of an agent workflow. External tool services, sandbox compute and storage can have separate bills. Keep those components visible when estimating the cost of operating the whole task rather than treating the model-cost field as the entire invoice.
Set the budget on the intended configuration
AgenticOS checks accounted spend before model requests. If the applicable limit has already been reached, the next request can be refused. Use the published configuration when checking which limit a run is meant to use; changing a draft setting is not the same as publishing it.
A useful trial is to use a disposable configuration with a deliberately small limit and inspect the refusal path. Check the run's status and recorded usage, then confirm which setting caused the stop. This establishes the behavior of that trial rather than a universal upper bound on every possible workflow.
The budget is not a reservation for the full future bill. A request can incur cost after passing its check, and concurrent or already committed work may not yet be reflected in the accounted total. Pricing gaps add another reason that a displayed amount may not describe all usage.
Keep those conditions beside the cap when deciding how much traffic to allow. A spending limit and a request-rate limit answer different questions, and external-service charges need their own controls.
Look at the input behind the number
Input usage can include instructions, tool definitions, retrieved material and conversation history. The actual request also depends on runtime behavior such as selection, caching and history processing. Counting the words visible in the chat does not reproduce that request.
If input grows during a long task, inspect what is being retained. Compaction can summarize history, but the summary may omit details the next step needs and producing it can require work of its own. Compare both the task result and total usage before deciding that the change helped.
The same approach applies to a smaller model: keep the source inputs and success criteria fixed, inspect the answer and record the whole task's usage. A cheaper individual request is useful only if it still does the work you need or reduces the complete workflow's cost at an acceptable quality level.
Keep a useful run record
For your first trial, retain the task input, published configuration, resolved model, output and usage together. Add unsuccessful attempts and identify any external costs the platform does not count. That gives a later prompt or model change something concrete to compare with.
The governance guide explains budgets and their accounting limits. The demo screen above shows how to read one run; a workload estimate needs observations from the tasks and traffic you actually intend to support.
Which of your agents would you want to see a cost for first, and what would you compare it with?
I'm Kacper, Principal Engineer and Open Source Tech Lead at Vstorm. I build AI agents and open-source tools in the Pydantic AI ecosystem. Find me on LinkedIn or GitHub.
Top comments (0)