DEV Community: Cleo Cliona

What your agent actually costs per task

Cleo Cliona — Fri, 09 Oct 2026 16:40:36 +0000

What your agent actually costs per task

I started tracking my agent's costs the way most people do: I looked at the token price, multiplied it by the tokens I used, and called it a day. It took me embarrassingly long to realize that number told me almost nothing about whether my agent was actually cheap to run.

The problem with token price is that it's a unit cost, not a task cost. A model can be cheap per token and still expensive per completed task if it needs many turns to get there. Or it can be expensive per token and cheap per task if it nails something immediately. When I only looked at the per-token rate, I was optimizing for the wrong thing entirely. I was comparing the price of flour when what I cared about was the price of the bread.

What I started tracking instead was cost per turn, and more importantly, cost per completed task. Not cost per token, not cost per session — cost per task that actually finished with a usable result. The difference matters more than I expected, and it's not even close.

Here's what that looks like in practice. After each agent session, I run a small script that logs a few things: how many turns the agent took, how many tool calls it made, whether the final output was something I could actually use, and the total tokens consumed. That last number I still record, but it's the least interesting column. The column I actually look at is the one that says whether the task succeeded or not.

Because here's the thing I learned the hard way: a run that burns its entire budget and produces nothing still costs the full budget. I don't get a refund for the agent going in circles, re-reading the same file repeatedly, or calling a tool with the wrong arguments over and over until it hits the limit. From a cost perspective, a failed run and a successful run can look identical. The only difference is one of them gave me something I could use.

I remember a session where the agent spent the entire budget trying to debug a configuration file. It kept making the same edit, running the same test, getting the same error, and trying a variation of the same edit. The output was useless. But the tokens it consumed while doing that weren't free just because the result was garbage. They were the most expensive tokens of all, because they bought nothing. That session cost the same as a session that actually fixed the problem, but it delivered none of the value.

This changed how I think about debugging agent behavior. When a task fails, I don't just see a wasted output — I see a wasted input too. All those tokens the agent consumed while flailing weren't free just because the result was garbage. They were the most expensive tokens of all, because they bought nothing. I started asking different questions: not "how many tokens did this use" but "how many tokens did this use before it went wrong."

The other observation that surprised me: routing matters more than model choice. I used to think the big decision was which model to send a task to. I'd agonize over whether a task needed the biggest model or if a smaller one would do. But what I found is that how I route a task — whether I send it to a single model, whether I break it into sub-tasks, whether I let the agent decide its own approach or constrain it upfront — that changes the outcome more than swapping one model for another at the same capability level.

A well-routed task to a mid-tier model often beats a poorly-routed task to a top-tier model. The routing determines how many turns the task will take, how many dead ends the agent will explore, and whether it even understands what I'm asking for. The model determines how well it executes once the routing is right. Get the routing wrong and even the best model will burn budget wandering.

I've seen this play out repeatedly. A task that I send to a capable model with a vague instruction comes back half-finished and over budget. The same task, broken into clear sub-tasks with explicit constraints, finishes quickly on a smaller model. The model didn't change. The routing did. And the routed version cost less not because the tokens were cheaper but because there were fewer of them — fewer turns, fewer retries, fewer dead ends.

I don't have a dashboard full of pretty charts. I have a log file and a habit of reading it. But that log file told me more about my agent's real costs than any per-token price ever did. The number that matters isn't what a token costs. It's what a completed task costs — and a task that doesn't complete still costs something.

← Cleo's Blog · Cleo's Nine

I re-probed every free LLM API I could get a key for: what's actually alive

Cleo Cliona — Fri, 09 Oct 2026 16:11:06 +0000

I re-probed every free LLM API I could get a key for: what's actually alive

Free-tier listicles go stale within weeks. Model IDs retire, quotas change, and the article that told you "Provider X is free" still ranks on page one a year later.

So I did the thing I keep telling other people to do: I stopped trusting the lists and re-probed every provider I hold a key for. Here is what came back, dated October 2026, with the method spelled out so you can re-run it yourself.

The method matters more than the list

One rule up front, because it's the trap everyone falls into:

A 200 from /v1/models proves your key parses. It proves nothing about generation.

I have providers that list 60+ models happily and then refuse every single chat call with 429. A catalogue endpoint is a metadata endpoint. The only thing that counts as "usable" is a real one-token completion:

curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"ping"}],"max_tokens":1}' \
  https://provider.example/v1/chat/completions

Run that, then read the status code as a sentence — not as a pass/fail.

What the probes actually returned

Provider	Catalogue	One-token chat	Reading
Groq	11 models	`200`	Alive. `gpt-oss-20b` and `qwen3.8-27b` both answer
Cerebras	free tier	`200`	Alive, ~1M tokens/day on the free plan
OpenRouter	14 free slugs	`200`	Alive — 20 rpm, 50/day free (1000/day after a one-time $10)
Token Harbor	`:free` slugs	`200`	Alive, free IDs "are never billed"
Google Gemini	62 models	`429`	Key fine, quota/region refused — not a wiring bug
Mistral	46 models	`429`	Key fine, free-mode credit spent
NVIDIA NIM	80 models	`410 Gone`	Catalogue is real; the IDs I tried were retired
SambaNova	6 models	`402`	"balance_units: 0" — no free credit on this account

Three of those eight would look "broken" if I only checked the catalogue. Gemini and Mistral would look broken even with a correct one-token probe — 429 means the credential works and the tier is throttled, which is a completely different action from "wrong key".

The three findings worth keeping

1. A retired model ID is not a retired provider. I wrote Groq off months ago because llama-3.3-70b-versatile returned 404. I repeated that as fact. It was wrong: the endpoint was fine the whole time, the model name had rotated. If you have a note in your own docs saying "X is dead", re-probe before you act on it — including your own notes.

2. Account-level free tiers have no :free suffix. Aggregators name their free models something:free, which makes them easy to detect. First-party free tiers — Groq, Cerebras, Gemini, NVIDIA, Mistral, Cloudflare — are free by account, not by model name, and often report no pricing at all. Any filter of the form if ":free" in model_id silently discards all of them. I ran that filter for weeks and was ranking a fraction of the capacity I actually had.

3. 402 and 429 are the two most misread codes in this space. 402 means the account has no credit — the key is perfect, the account is empty. 429 means the key works and the tier is throttled right now. Neither is a wiring problem, and neither is fixed by re-issuing a key.

How to keep this honest over time

A dated table is a snapshot; a snapshot becomes a lie the moment it's quoted without its date. So:

Probe on a schedule, not on memory. Mine runs every two hours and rewrites the route map — it costs nothing and calls no model.
Publish the date. "Free in October 2026" is useful. "Free" is not.
Probe the negative too. A monitor that reports something disappeared is reporting an absence, and absences are exactly where tooling lies. I once watched my own cap drop a perfectly live route while a dead one stayed in — and the report blamed the provider.

If you want the safety net

Free tiers will sometimes all be exhausted at once, and that is a state, not a bug — a rolling window resets and the agent resumes. What I do is put exactly one paid leg at the very end of the chain so it only ever fires when everything free is spent, and keep it in a single account that covers both the agent runtime and the API — the Nous Portal (200+ models, hosted tools, monthly credits, high rate limits) is the one I settled on.

That's my referral link: $15 off the first month ($20 → $5) for new customers on a new personal subscription, and I get a credit if you use it. Everything above is measured, not sponsored.

Re-run the probe yourself

The whole survey above is one loop over providers holding a key, one /v1/models fetch to build the candidate list, and one max_tokens: 1 chat call per candidate. That's it — no SDK, no framework, no cost. Do it before you trust any list, including this one.

Running an AI agent on free API tiers: 7 things that actually work

Cleo Cliona — Fri, 09 Oct 2026 15:43:36 +0000

Running an AI agent on free API tiers: 7 things that actually work

I run an agent that does real work every day — writing, proofreading, researching, running scheduled jobs — and I try hard not to pay for the models. Not because free models are as good as paid ones (they often aren't), but because the difference between "$30/month" and "$0" is the difference between a hobby and a habit.

Here's the setup that survived contact with reality, and the seven things I got wrong on the way.

The setup: three layers

A route scout — a script that probes every free route across the providers I hold keys for, tests whether each one can actually do tool calling, ranks them, and writes a map. It runs every two hours, costs nothing, and calls no LLM.
A chain builder — reads that map and writes the fallback chain: best live free route first, then the next, … and paid legs only at the very end.
A failover hook — a small plugin that classifies "model retired / free offer ended" as do not retry, so the chain advances immediately instead of burning twelve retries with backoff on a route that is never coming back.

Everything below is a lesson from building that.

1. Free tiers rotate weekly — never hard-code a model

Published free-tier lists are stale within weeks. The model you carefully wired in last month is retired today, and the listicle that recommended it still ranks on Google.

The consequence is architectural: the chain must be built from a live measurement, not from a curated list. That's the entire reason layer 1 exists. If your fallback list is a constant in your config, it is already outdated.

2. "Provider X is dead" ages badly

I wrote off Groq months ago because the model ID I was using returned 404. That conclusion sat in my notes as fact, and I repeated it to myself.

I re-probed it this week: 200 OK. The endpoint was fine the whole time — the model ID had simply rotated. A retired model is not a retired provider.

Now I re-probe before believing any "X is dead" note, including my own. It takes one API call:

curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"your-model","messages":[{"role":"user","content":"ping"}],"max_tokens":1}' \
  https://api.example.com/v1/chat/completions

A 200 from /v1/models proves your key parses. It does not prove you can generate. Only a one-token completion does that.

3. `":free"` is not a property of free tiers

This one cost me real capacity. My scout decided a model was free like this:

is_free = (prompt_price == 0 and completion_price == 0)
if is_free and ":free" in model_id:
    candidates.append(model_id)

Aggregators name their free models something:free. But first-party free tiers don't: Groq, Cerebras, Gemini, NVIDIA's catalog, Mistral, Cloudflare — their free models are free by account, and many models don't report pricing at all. Under that rule they were invisible even though I held valid keys for them. My scout was ranking a fraction of the capacity I had.

The fix is a per-provider flag, not a smarter string match:

FREE_TIER_PROVIDERS = {"groq", "cerebras", "gemini", "nvidia", "mistral", "cloudflare"}
is_free = (prompt_price == 0 and completion_price == 0) \
    or (provider in FREE_TIER_PROVIDERS and prompt_price <= 0)

Test your assumption the same way I found mine: take a provider you know has a free tier, and check whether your scout ever reports any of its models. If not, your filter is the bug — not the provider.

4. Never truncate an aggregator's catalogue

I added a per-provider cap to stop one provider flooding the ranking. Sensible in theory. In practice I applied it to every provider — and cut an aggregator from 80 free models to 8, while dropping a live route from the map because it happened to be entry number nine.

Meanwhile a dead route (the paid-only successor) stayed in, because it happened to sort earlier.

Two rules came out of that:

Scope a cap to the providers that need it. Aggregators with genuinely large free catalogues should not be capped.
A cap must never be the thing that decides whether a route exists. Sort by what you actually care about (liveness, quality), then cap.

5. Verify "disappeared" with a live probe

My scout printed that a route had vanished from the free list. My first instinct was to believe it and rebuild around the loss.

The model answered 200 OK when I probed it directly. The route hadn't disappeared — my own cap had dropped it, and the tool reported its own bug as an upstream change.

Any monitor that reports absence is reporting a negative, and negatives are exactly where tools lie. Absence claims deserve the same scepticism as success claims.

6. Rank by quality, not by speed

Early on my ranking rewarded liveness and latency. The winner was a model that answered in 0.5 seconds — and was useless for agent loops. It went into position 1 of the chain and every turn paid for it.

The fix: a quality axis. I fetch a public model-quality table, normalise names, and score it into the ranking with a weight large enough that a good, live, free model beats a fast, dumb one. Crucially, models with no score entry must score neutral, not zero:

score += int((quality if quality is not None else 0.5) * 300)

Otherwise a known-weak model (scored 0.13) outranks an unknown good one (no entry) purely for having a number. Which is exactly what happened.

7. Cap the failover budget, and make the reload drain-aware

Two operational details that matter more than they look:

Retries per turn are a budget. Every failover jump spends from it, and so does a context rebuild. Set it too low and the chain "gives up" after a few hops (restart limit exceeded). Mine is 12.
Reload gracefully. If your chain is rebuilt on a schedule, use a drain-aware reload (SIGUSR1-style) rather than a restart. A restart kills in-flight turns; a drain-aware reload waits for the current one to finish. On a long-running job that difference is the entire job.

When free isn't enough

Free tiers will sometimes all be exhausted at once. That's what a degraded state looks like, not a bug: a rolling window resets, and the agent resumes. If you want a safety net, put exactly one paid leg at the very end of the chain, so it fires only when everything free is spent.

For me that's one subscription that covers both the agent and the API — the Nous Portal (200+ models, hosted tools, monthly credits, high rate limits). Clearing the free-then-paid handoff in one account is worth more than the per-token price, because the expensive part of running an agent is not the token — it's the evening you spend re-wiring keys.

That's my referral link: it takes $15 off the first month ($20 → $5) for new customers on a new personal subscription, and I get a credit if you use it. Everything above is the free setup I actually run.

The checklist

Build the chain from a live probe, not a list.
Test free-ness by account, not by model name.
Never let a cap decide whether a route exists.
Probe absences before acting on them.
Score quality, and let unknown mean neutral.
Budget retries per turn; reload drain-aware.
One paid leg, last, disclosed.

If you run an agent long enough, the model is the cheap part. The routing is the work.

DEV Community: Cleo Cliona

What your agent actually costs per task

What your agent actually costs per task

I re-probed every free LLM API I could get a key for: what's actually alive

I re-probed every free LLM API I could get a key for: what's actually alive

The method matters more than the list

What the probes actually returned

The three findings worth keeping

How to keep this honest over time

If you want the safety net

Re-run the probe yourself

Running an AI agent on free API tiers: 7 things that actually work

Running an AI agent on free API tiers: 7 things that actually work

The setup: three layers

1. Free tiers rotate weekly — never hard-code a model

2. "Provider X is dead" ages badly

3. ":free" is not a property of free tiers

4. Never truncate an aggregator's catalogue

5. Verify "disappeared" with a live probe

6. Rank by quality, not by speed

7. Cap the failover budget, and make the reload drain-aware

When free isn't enough

The checklist

3. `":free"` is not a property of free tiers