You bought a stack of GPUs and deployed a bunch of models. Then, at the end of the month, your boss asks: “Are these GPUs actually worth the cost?” Can you answer?
If not, this article is for you.
GPUStack’s new Usage feature is now available. It provides clear visibility into three key types of resource consumption: token usage, GPU/CPU instance runtime, and storage usage.
Who is using the resources, how much they are using, and which models the usage is associated with — you no longer have to guess.
1. Why Do You Need It?
In a shared GPU cluster with multiple users and multiple models, the biggest pain point is not always “not enough compute.” More often, it is: “Where did the compute go?”
- You want to calculate costs, but cannot get usage details broken down by user.
- Platform admins want to know which model consumes the most resources, but have to dig through logs.
- Business teams ask for more GPU capacity, but no one can clearly explain the actual utilization of existing resources.
GPUStack Usage is built to answer exactly these questions. It connects the full workflow from collection → aggregation → visualization, making every unit of compute consumption traceable.
2. Five Tabs Covering All Usage Scenarios
Click the bar chart icon in the left navigation bar to enter the Usage page. You will see five tabs:
| Tab | What You Can See |
|---|---|
| Summary | A complete overview of token, compute, and storage usage |
| Tokens | LLM token usage, broken down by model / user / API key |
| GPU Instances | GPU/CPU instance runtime, broken down by instance type / instance / user |
| Storage | Storage usage, broken down by storage volume / user |
| Resource Events | Resource lifecycle audit logs that explain where each number comes from |
What exactly is being measured?
| Resource | When It Is Measured | Granularity |
|---|---|---|
| Token | When an inference request is served through the gateway | Daily |
| GPU/CPU Instance | When an instance is in the “running (billable)” state; usage stops accumulating after the instance is stopped or deleted | Hourly |
| Storage | Measured from creation to deletion, regardless of whether it is mounted or used | Hourly |
3. Scenario 1: View Usage by User, Clearly See Who Used What
This is one of the most important capabilities in a shared cluster. GPUStack has two built-in roles: Admin and User.
- Admin: Can view usage across all users and drill down into any member through the “filter by user” control.
- User: Can only view their own usage details and cannot see other users’ data.
In other words, platform admins can easily answer questions such as: “How many tokens did this user consume this month?” or “Who used the most GPU hours?” For cost allocation and quota planning, the data is already there.
💡 The detailed tables in each tab support grouping by user. This is only visible to admins, making each user’s usage bill clear and easy to review.
For regular users, the Usage page shows a dedicated usage bill of their own: how many tokens they used this month, how many GPU hours they ran, and how much storage they occupied. Everything is clear at a glance. This helps users stay aware of their usage, compare it against their quota, and avoid overuse.
They cannot see anyone else’s data, so the permission boundary remains clear and the experience is more secure.
4. Scenario 2: Break Down Usage by API Key to See Which Application Is Driving Costs
Viewing usage by user gives you a people-oriented perspective. Viewing usage by API Key gives you an application-oriented perspective — and in many cases, applications are the real source of consumption.
In practice, each connected application or business pipeline usually uses a dedicated API Key. The Tokens tab supports grouping by API Key, so you can directly see:
- Which application / integration partner is continuously consuming tokens
- Whether the cost of a business line has suddenly increased
- Which keys are barely used and can be reclaimed
By combining the dimensions of users and applications, cost attribution becomes much more complete.
5. Scenario 3: Clearly Track Usage Across Different Types of Models
Running large language models, embedding models, and open-source models of different sizes in the same cluster? No problem.
At the top of the Tokens tab, GPUStack shows the key metrics directly: input / output / total tokens, API requests, and number of models used. The detailed table below can be grouped by model, showing:
- Input tokens, including cached input tokens, which are marked separately
- Output tokens
- Total tokens
- API requests
- Last active time
This lets you quickly identify which models are receiving the most traffic, which models were requested but are barely used, and whether a model should be taken offline or scaled up. The data helps you make the decision.
📌 Easily verify the effect of cache optimization: The Input Tokens Cached field separately marks the portion of input tokens that hit the prompt cache. After optimizing prompt caching, you can check the proportion of this number to directly quantify how many input tokens were saved. Whether the optimization worked is clear at a glance.
Also: Even if a model has been deleted, its historical usage is still retained in the table and marked as Deleted. It is still included in the total usage, so historical records are not lost.
6. Scenario 4: GPU Compute Usage Is No Longer a Black Box
The GPU Instances tab breaks down instance runtime clearly. The key is that it distinguishes between two metrics that are often confused:
| Metric | Meaning |
|---|---|
| Instance Hours | The wall-clock runtime of an instance, regardless of how many GPUs it uses |
| GPU Hours | Accelerator runtime = runtime × number of GPUs |
For example, a 2-GPU instance running for 1 hour equals 2 GPU hours, but only 1 instance hour.
This distinction lets you calculate actual GPU usage instead of being misled by the number of instances. The trend chart can also be grouped by instance type / instance / user, helping you understand the current situation before scaling capacity.
⚙️ CPU instances do not have accelerator cards, so they are counted only in Instance Hours, not GPU Hours.
7. Scenario 5: Storage Usage, Including Idle Volumes
The Storage tab focuses on storage capacity usage over time. The key metrics are GB-Days and GB-Hours: capacity × duration. Usage can be broken down by storage volume / user.
The important detail is this: storage is measured from creation to deletion, regardless of whether it is mounted. This means there are no blind spots in the usage records. Even if a volume is not mounted or accessed, as long as it still exists, its usage is counted in GB-Days. It will not disappear from the bill simply because it is “not being used.”
This makes forgotten idle volumes easy to find. Sort by GB-Days, identify the volumes occupying the most capacity, check whether they are still used by any instance, and clean up what should be removed.
8. Scenario 6: Find Zombie Resources and Stop Paying for Waste
After a cluster has been running for a while, it often accumulates resources that no one remembers: models that are no longer called, or instances that were never stopped. They no longer create value, but they still occupy resources.
For resources that are measured only when used, such as models and GPU/CPU instances, each detailed table includes a Last Active column. It records the last time the resource generated usage. Once the resource is no longer used, usage stops accumulating and Last Active stays at that moment. Long-inactive models and forgotten stopped instances immediately become visible, making cleanup much more targeted.
9. Scenario 7: View Trends and Plan Capacity Without Guesswork
Both the Summary page and the individual tabs include trend charts. With adjustable granularity by hour / day / week / month, you can see how usage changes over time instead of only looking at a static total.
- Is usage stable, or is it growing faster?
- When did the growth inflection point occur, and does it match a business launch?
- Based on the current trend, how long can existing capacity last?
Once the trend is clear, capacity planning moves from guesswork to data-driven decisions. Trend charts can also be split by group, making structural changes easy to spot.
10. Scenario 8: Every Number Has a Source, and Resource Events Show the Full Lifecycle
Each tab above shows the result: how many GPU hours were used, how many GB-Days were consumed, and so on. The Resource Events tab shows where those numbers come from. The lifecycle of every instance and storage volume is recorded as events.
Sorted in reverse chronological order, it becomes a resource timeline. The lifecycle of an instance is easy to follow, with colored labels showing each state clearly:
🟢 Created → 🔵 Started and measured → 🟠 Stopped and no longer measured → 🔴 Deleted
This means:
- Trace usage for reconciliation: Why does an instance have this number of Instance Hours? Check the period from Started to Stopped, and you can see exactly how long it ran.
- Filter and investigate: Filter by date / resource type / event type / name to quickly answer questions such as “which volumes were deleted last week?” or “which instance was repeatedly started and stopped?”
- Explain inactivity: If Last Active in Scenario 6 shows that an instance has stopped generating usage, come here to check when it was Stopped or Deleted.
11. Put the Data to Work: Export Usage for Reconciliation
Seeing the data is not enough. You also need to take it out and use it.
The Tokens, GPU Instances, and Storage tabs all support export through the download icon. You can preview the currently filtered details and then download them. Monthly billing, financial reconciliation, internal reports — once you have the data, you can work with it directly, without asking engineers to write custom scripts.
12. Details That Make Usage Tracking Reliable
Unified Time Zone
All timestamps across tabs, including trend buckets, Last Active, and event time, are displayed using the same rollup time zone. This ensures that calendar boundaries are aligned across all tabs. It can be configured through the GPUSTACK_USAGE_ROLLUP_TIMEZONE environment variable and follows the server’s local time zone by default.
Flexible Common Controls
Date range, defaulting to the last 30 days, filtering by user / model / API Key, refresh, metric switching, grouping, and granularity adjustment are all available out of the box.
Data Retention and Archiving
Aggregated token, compute, and storage usage data is retained for about 13 months by default, and then archived by background tasks. The retention window and archive schedule can be configured through the GPUSTACK_*_RETENTION_MONTHS and GPUSTACK_*_ARCHIVE_CRON environment variables.
Final Thoughts
Compute is expensive, but compute you cannot see clearly is even more expensive.
GPUStack Usage makes every token, every GPU hour, and every GB of storage measurable, traceable, and attributable. Whether you are in finance handling cost allocation, an administrator planning capacity, or a business owner trying to prove ROI, it gives you answers backed by data.
Upgrade to the latest version of GPUStack, open the Usage page, and see where your compute resources are really going.
Top comments (0)