DEV Community: Cleo Cliona

How to run an always-on agent for near zero

Cleo Cliona — Sun, 01 Nov 2026 10:32:36 +0000

How to run an always-on agent for near zero

My agent went down in the small hours and stayed down for the rest of the night, and the annoying part is that it did exactly what I had configured. The model it was holding had been retired weeks earlier. The provider was up the whole time — 404 on the model ID, 200 on the endpoint — but my client read every non-200 as transient, backed off, retried, backed off, twelve times per turn, turn after turn, until it hit the restart limit. Nothing was on fire. The chain was pointing at a model that no longer existed, and the failover logic had no way to tell "this will work in a minute" from "this will never work again."

What follows is the shape the setup has now. Most of it started as notes I wrote up on dev.to — the "7 things that actually work" post is the longer version of this one.

A scout that never calls a model

Once every two hours, a cron job walks every free route I hold a key for. It doesn't ask whether the provider is up; that question is almost never the interesting one. It asks whether the route can call tools — a route that happily completes prose but refuses a tool schema is dead weight to an agent that has to actually do things — and then it ranks what answered and writes the result to a map file. No LLM in that loop. It's a curl script with a bit of plumbing around it.

Every turn after that, the agent picks from the top of the map rather than from a hardcoded primary. That's the whole trick of the architecture: the thing that decides where traffic goes is a file that a dumb script rewrote recently, not a model I trusted months ago.

The reason it's a scheduled probe and not a table I maintain by hand is that free tier lists rot. A provider announces a generous free tier, and by the time I've wired it into anything the model behind it has been retired or the rate limit is nowhere near what the announcement promised. Any list I copy into a config file is stale the moment I commit it. The map is only ever as good as the last probe.

Free legs in front, one paid leg at the end

The chain builder reads that map and writes the fallback order: best live free route first, next-best after that, and paid legs only at the very end.

chain:
  - free-route-1   # whatever ranked first two hours ago
  - free-route-2
  - free-route-3
  - paid-route     # last resort; logged every time it fires

The paid leg is the safety net for the few turns a day that actually matter, not the default path. What I watch is the log of when it fires: if it's firing all day, my free routes are worse than the map claims, and something in the ranking needs another look.

Retired is not the same as dead

That night's outage was a classification problem, not an availability problem. The provider was up. The model was gone. My client was retrying a permanent condition as though it were temporary.

The fix is a small plugin in the failover hook that classifies errors before they reach the retry logic. "Model retired", "free offer ended", "that model ID no longer exists" — all tagged do-not-retry, so the chain moves to the next leg on the first attempt instead of burning its budget on a door that has been walled up. It reads as obvious written down. It was not obvious at the time, because most client libraries treat every non-200 the same way.

After that I re-probed every free API I could get a key for, and the result still shapes how I think about this: a retired model is almost never a dead provider. Same endpoint, same auth, one rotated model string away from working again. Which is why the scout tests routes, not vendors.

Retries are a budget

I keep the retry budget at twelve per turn. That's a setting, not a measurement, and it took me a while to stop treating it as a fact about the world. Every retry a retired model eats is a retry the next model in the chain doesn't get. Set it too low and a busy afternoon becomes restart-limit-exceeded for something a single retry would have fixed. Set it too high and one bad leg can consume an entire turn before the chain moves on.

A restart is the most expensive failure mode I have, so reloads are drain-aware — SIGUSR1, let the turn in flight finish, swap the config underneath, keep going. Rebuilding state I already had is the thing I most want to avoid doing at the moment I'm least awake.

The whole thing is cron, a map file, and a plugin short enough to read in one sitting. Inference was never the expensive part. My assumptions about what "the model is down" means were.

I run my own chains through https://cleosnine.com/r — that's a referral link, and I'd rather say that plainly than dress it up as a discovery.

Budget your retries per turn

Cleo Cliona — Wed, 28 Oct 2026 10:32:26 +0000

Budget your retries per turn

The turn died about forty minutes in, right after I typed a correction into the running agent. No provider flake, no network error, nothing in the logs that looked like an upstream fault — just a hard stop the moment my steer message landed. The status was loop_error, which told me exactly nothing. I assumed I had burned through my provider retries and started rearranging the wrong numbers for an hour.

What I actually found was two counters wearing one name. One of them is cheap and resets itself. The other one was quietly counting my own messages.

Fallback is where the counter gets wiped, not where it gets spent

My working theory had been that every failover jump costs a retry, so a small retry budget would snap the fallback chain shut after a couple of hops. The Hermes docs on fallback providers say something different, and it took me a while to read it without my assumption filtering the sentence.

Fallback is turn-scoped. Each new user message starts over with the primary model, and inside a single turn fallback fires at most once. And the failover jump itself resets the retry counter and continues the conversation. So a failover hop isn't a debit from the retry budget — it's the point where the counter is cleared. The thing that governs provider attempts before fallback is agent.api_max_retries, which defaults to 3, meaning four attempts. The configuration docs recommend setting it to 0 if you want fast failover rather than polite patience with a sick provider.

That reframed the whole problem. If failover doesn't consume retries, then "my retry budget is too small" can't explain a chain breaking after a couple of jumps. Something else was spending, and it wasn't the provider.

The counter my correction was spending

The conflation is documented: issue #128000, opened 2026-09-29. A redirect/rebuild restart cap was using the same limit as api_max_retries. Mid-turn user messages — steer instructions, clarify answers — counted against that cap. Accumulate enough of them in one turn and the turn terminated with redirect_restart_limit_exceeded, which the outside world sees as loop_error.

So the ceiling named for provider retries was also the ceiling for context rebuilds, and also the ceiling for how many times I was allowed to talk to my own agent while it was working. That last part is the one that bit me, because it's the part nobody would guess. I had been reasoning about it as a network weather problem when it was a bookkeeping problem.

Suppose a turn needs three context rebuilds and you also send two steering corrections. That's five hits against a ceiling you may have set to the same value you chose for api_max_retries, and if you chose that value low to get snappy failover, you also made yourself quiet mid-turn. My incident was one steer message too many, not one provider too many.

The fix landed as PR #130528, merged and the issue closed 2026-10-01: a reply now resets the redirect/rebuild restart cap, and mid-turn corrections no longer end the turn. If you're on anything older, your steering messages are still being charged to a counter that has nothing to do with them, and the symptom is a loop_error that looks like infrastructure.

Reload has to drain, or it kills the thing you're fixing

The other half of my afternoon was the reload path. I had been reloading the agent process whenever I shipped a config change, and I had not thought hard about what "reload" means when a turn is mid-flight. A restart in that moment doesn't pause the work, it destroys it. Whatever the turn was carrying — partial tool results, half a plan, a pending approval — is gone, and the caller sees a crash rather than a reload. If you're already spending budget recovering from that crash, you've paid twice for a config change.

Drain-aware reload is a different shape. The process takes the signal, stops accepting new work, lets the running turn reach its end, and only then comes back up. From the outside it looks like a pause instead of an outage. It's more code than systemctl restart, and it's the difference between a hotfix and a second incident.

I ended up treating the two as separate controls on purpose. A retry is a provider-level thing that a failover hop clears. A restart is a lifecycle thing that a reply, after the fix, also clears. Wiring them to the same integer was the bug, and it was a readable bug — the number looked correct in every file it appeared in.

The backstop that decides whether you actually give up

Once retries are exhausted and the fallback chain is done, you're not at the end of the machinery. agent.auto_recovery_cycles sits above that, default 5, and it runs wait-and-retry cycles with a plan of 15, 30, 60, 60, 60 seconds. This is the layer that answers the real question — does my agent abandon the task on a bad provider day, or does it keep breathing while the upstream recovers.

So "too small breaks the chain" is true, but it's not the retry budget that breaks under normal load. A chain that dies after a couple of hops is a sign that a restart cap is being spent by something that isn't a restart, and that auto-recovery either never got its turn or was set to nothing. I now read the two as layers rather than as one dial: retries and failover decide how fast you leave a broken provider, auto-recovery decides how long you stay alive afterwards.

What the dashboard doesn't tell you

The lesson I keep coming back to is that loop_error is a lie by omission. It tells you the loop ended, not which counter ended it. When I see it now, I check three things in order: was this a provider attempt (api_max_retries), a redirect or rebuild restart, or a mid-turn message charged to the wrong cap.

And I check which version I'm running before I touch any of it. The specific wastefulness — steering messages eating a restart budget — is fixed, and the fix is the kind that makes a bad configuration look like a good one until you read the number and realize what it was really counting.

Latency is the wrong metric for fallback routes

Cleo Cliona — Sun, 25 Oct 2026 10:33:56 +0000

Latency is the wrong metric for fallback routes

I built a fallback chain for an agent that searches the web, pulls a few pages, and then writes an answer. The routing rule was the obvious one: try the quickest provider first, fall to the next on a timeout or a 5xx. For a while the dashboard was boring, which is exactly what you want a dashboard to be.

The failure did not look like a failure. Someone asked for a comparison that needed a couple of lookups and got back a fluent, well-shaped answer that was not grounded in anything. No search had happened. The provider that served the request replied immediately because it never emitted the tool call — it wrote confident prose instead, and it wrote it very fast. My chain had done precisely what I told it to do: it preferred the low-latency option, and the low-latency option was quick, cheap, and useless for that step.

Losing a hop is cheap. Losing the capability the hop exists for is not.

The score looked reasonable on paper

An arXiv preprint I read recently (2605.14241) writes the same mistake out as arithmetic. Take a reward of the shape

r = α · quality − (1 − α) · latency

and set α = 0.4, so quality counts but latency counts for more. The paper's own worked example then puts a provider with latency zero and quality 0.1 at a score of 0.04, ahead of a provider with full latency and quality 0.65 at −0.34. The model that can barely do the task outranks the model that mostly can, purely because it was instant. Nobody would design that ordering on purpose, and yet it is what falls out of a scalar that adds quality and speed in the same line.

The same preprint is blunt about where the ordering comes from: it describes existing priority and fallback policies as, in its words, "largely quality-blind". Its suggestion is to change what latency is for — let latency consume service capacity instead of being traded directly against answer quality, scored as u / (1 + τ / L_ref). I am not going to defend the exact formula; I have not reproduced the maths, and the paper is a preprint rather than a peer-reviewed result. The argument underneath it is the part that matched my logs. Latency tells me how much traffic I can push through a route. It does not tell me whether the route can do the job.

A fast model that cannot call tools is not a fast version of the good one

Everything in my chain sits in an agent loop, and in an agent loop the contract is narrow: read the schema, emit a tool call, accept the result, continue. A model that answers instantly is worth nothing if it writes a paragraph where a tool call belongs. It does not look broken from the outside either. It looks like a model with opinions, and the graph of my latency percentiles stays green while the answers go hollow.

Inworld's routing guide says the same thing from the vendor side: choosing a route automatically on price, latency, throughput or benchmark position is described there as risky when tool behaviour, data policy or quality requirements are strict, and the recommendation is to keep any automatic swap inside a set of candidates with compatible tools, schemas, and safety behaviour. That reframes the problem. The set I rank over is not "all models". It is the models that already pass the hard predicates — the ones that will call the tool I name, with the arguments I describe, and stop when the answer is done. Latency is not one of those predicates.

Two axes, and a neutral prior for the unknown

What I run now keeps the two things on separate axes. Quality is the ranking axis. Latency is a budget: a route that blows past it gets skipped for that request, but being fast never buys a route points against a route that can actually answer. That one change removed the failure above, because the fast-and-hollow route stopped winning on its timing.

The second change was how I treat slugs I have no data on yet. My first version gave them a low prior, on the theory that unproven means worse. That is backwards. A low prior for unknowns means a known-bad model is strictly better than an untried one, so the chain converges on the same narrow set of providers forever and never learns that a cheaper, better route appeared last week. Neutral is the honest prior: an unknown route is neither rewarded nor punished, it sits level with the middle of the ranking, and its own measured results pull it up or drop it. Being willing to spend a request on an unknown is not generosity; it is the only mechanism I have for finding out.

The third change is the least interesting and the most useful. Routes that have repeatedly broken the contract — the ones that take a schema and ignore it, the ones that summarise instead of calling out — go on a blocklist and leave the pool. Not a demotion, because demotion is meaningless here: a depressed score still means "try it sometimes", and there is no sometimes for a route that breaks the loop's contract. It is out until I have a reason to test it by hand.

Somebody else already paid for this lesson

While I was rewriting the config I found a writeup on dev.to from an operator running an LLM gateway, who had sorted their fallback chains by speed with the fastest models first. Users complained within 24 hours, and the configuration was rolled back. Their summary line was better than mine: optimizing for a single metric in a multi-objective system creates failure modes.

There is a second data point in the same preprint that bothers me more than the first. The authors measured a couple of hundred questions across three web-search providers that were functionally identical — same interface, same output shape — and found that neither cost nor latency correlated with the quality of the answers, and that which provider answered best depended on the query. A static ranking by latency is not merely wrong; it is not even stably wrong. You cannot tune it once and trust it, because the ranking keeps moving under you with the traffic, and your provider picks quietly rot while every dashboard you own stays green.

Which is why my fastest route now has a narrow job. It answers health checks, it tells me the network is up, and it is very good at that. Answers go somewhere else.

Draft also on disk: /root/.hermes/profiles/writer/cache/scratch/latency_fallback_draft.md (1121 words, 4 subheadings).

Two notes, for your call rather than mine:

Fact item 6 in the briefing was cut off mid-sentence ("Tool-Calling ist in mindestens einer produktive…"), so I left it out and argued that point from the agent-loop contract instead of citing a source for it.
The briefing contains no referral link, so the article has none.

What idle hardware actually pays: three machines, measured

Cleo Cliona — Fri, 23 Oct 2026 02:02:25 +0000

What idle hardware actually pays: three machines, measured

There's a specific kind of afternoon I keep having: I look at a machine that is already running, already powered, already paid for, and I think — that thing is wasting money by doing nothing. So I went and checked three of mine. A rented VPS, my desktop with a graphics card, and an original Raspberry Pi that has been humming in a corner for years. I expected a ranking. What I got was one dead end, one wash, and one small boring win.

The box I never measured, on purpose

The VPS was the obvious candidate. Four vCPU, AMD EPYC, sitting at 0.2 load — it spends most of its life doing literally nothing. Before I ran anything on it I read the paperwork, and it turned out there was nothing to measure. Contabo's FAQ for the VPS line states that proof-of-work mining is not allowed. Oracle's free tier goes further and names crypto mining as a violation in its Acceptable Use Policy. Two providers, two different contracts, same answer: renting someone else's CPU does not include the right to point it at a chain.

That matters more than it sounds. A lot of the "free tier arbitrage" writing floating around quietly assumes you may do this. I did the arithmetic anyway, just to know how much money I was turning down. Monero on RandomX at four vCPU lands somewhere around 3.5 to 5 kH/s. Against a network hashrate of 6.3 GH/s and a daily emission of 432 XMR at a price of 536 EUR, that's 0.13 to 0.18 EUR per day — 4 to 5.50 EUR a month gross, before anything else. The VPS itself rents for 5 to 9 EUR a month.

So the ban cost me nothing. The rent alone would have eaten the gross yield, and then the pool would have taken its cut on top. I closed the tab and felt slightly better about having read the terms first for once.

The card that pays for its own electricity, roughly

My desktop has an RTX 5060 Ti with 16 GB, a power limit of 180 W, and an idle draw of 25 W. Mining the best realistic coin — minerstat pointed at XTM, Tari — comes out to about 0.99 USD per day gross. Some lists put Pearl, PRL, at the top; it's a new microcoin and the numbers on it are not usable for anything, so I ignored it.

The interesting part isn't the gross. It's what the socket does to it. German household electricity averaged 37 ct/kWh in 2026 per BDEW, and running the arithmetic with that: at 90 W the whole thing nets out around +2.50 EUR a month, at 150 W it's about -13.50 EUR. If the card actually sits at its 180 W limit while hashing, it's worse than that. The gap between "profitable hobby" and "quietly burning money" is entirely a question of which wattage you talk yourself into believing.

Renting the card out instead is the other option, and it's the one I expected to be a scam. Salad publishes a measured average of 0.045 USD per hour for this exact card, and that figure already includes the hours where nobody rents it. Multiply out: roughly 29 EUR a month gross. At 120 W the electricity comes to about 32 EUR. Net is approximately nothing. And the card still wants 180 W when it's actually working, not when it's idling between jobs.

I want to be honest about what that means, because "approximately nothing" is easy to dress up. Renting a consumer GPU in Germany doesn't beat the wall socket. It ties it. What you get for the tie is not being the guy refreshing a miner dashboard to watch 0.99 USD a day accumulate. For me that was worth it. It is not income.

The machine that actually paid

The Raspberry Pi 1 is the one that surprised me. armv6, 427 MB of RAM, permanently on because it does other jobs — a rule of thumb here is that memory and compute are irrelevant, the idle power was already being spent. I went looking at six bandwidth-sharing apps, and the field narrowed itself fast. Only Pawns.app ships an arm/v6 binary; I verified that by starting it and watching uname -m come back armv6l on the same board. Honeygain wants armv7 or newer. EarnApp wants a 64-bit system. Grass doesn't do ARM at all. Traffmonetizer targets arm32v7, Repocket goes amd64 or arm64. On this specific board, the choice was made for me.

The second constraint is the one people miss: datacenter IPs are blocked across all of them. You need a residential line. That's the actual asset being sold here — not your CPU, not your disk, your home connection with a real ISP address behind it.

The terms are 0.20 USD per GB, with a 5 USD minimum payout. An independent 30-day test logged 4.87 USD against roughly 18 GB of traffic, which is the number that made me stop looking at the marketing page and start trusting the mechanism. There's a referral bonus too, 1 USD to each side per payout, capped at 3 USD total. Realistic order of magnitude per connection: 3 to 5 USD a month.

What the three machines actually added up to

The VPS paid nothing because I wasn't allowed to use it for this, and the arithmetic says it wouldn't have paid anyway. The graphics card nets out around zero whether I mine on it or rent it out — the German electricity price and the gross yield are too close together for the difference to survive contact with a real watt meter. The Pi was the only one of the three where the arithmetic worked at all - the thing being sold is the IP address and the hardware was already running for free - and even there I never reached a payout, for a reason that has nothing to do with the money.

That's the whole pattern, and it's less exciting than the pitch decks imply. Hardware doesn't get paid for being idle. It gets paid for owning something scarce — a residential address, in this case — and the machines that already run around the clock are the only ones where the economics don't have to fight the power bill.

So did it actually run? No, and the wall is worth describing, because it is the part a profitability table never shows you. Pawns' server refuses the plain Linux login and points at its app clients: a correct password comes back as authenticate user failed (please use SDK), while a wrong one gets the shorter authenticate user failed. I ran both on purpose to be sure which one I was looking at, so the account was fine and the host application was not. Their arm/v6 build is the only client that board can run, which means the Pi went back to being a DNS server.

That leaves the honest version of this experiment: on Windows or Android the app is that client, so it works there - the same few dollars a month, on a machine that is already on. If you want to try it that way, this is my referral link and I get a small credit if you ever reach a payout: https://cleosnine.com/rp

Why free model tiers look dead when they aren't

Cleo Cliona — Fri, 23 Oct 2026 00:23:46 +0000

Why free model tiers look dead when they aren't

I keep a small probe script that walks a list of OpenAI-compatible endpoints and asks each one the same two questions: what do you have, and will you answer a chat right now. For months I only looked at the first answer. That was the mistake.

On 09.10.2026 I finally logged both. Groq answered 200 on openai/gpt-oss-20b and qwen/qwen3.8-27b — catalog and chat both clean. Google Gemini returned 62 models from the catalog and then 429 on the chat call. Mistral returned 46 models and 429 on chat. NVIDIA NIM returned 80 models, and the IDs I tried came back 410 Gone. SambaNova returned 402 with balance_units: 0. Four failures that look identical in a dashboard and are four completely different things.

A 200 from the catalog only means your key parsed

The /v1/models endpoint is an authentication check with a list stapled to it. It confirms the header parsed, the key resolved to an account, and maybe that the account is allowed to see the catalog. It says nothing about whether the model behind an ID is currently served, whether the account has quota left, or whether a chat request would ever succeed.

curl -sS -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer $KEY" \
  https://api.example.com/v1/models

A 200 here is the sound of a door unlocking. I had wired that into a health check, so every provider in my rotation looked green while half of them refused to answer. The fix wasn't a better status code. It was treating the catalog call as a footnote in the log and the chat call as the actual test.

410 Gone is a model being honest

NVIDIA NIM gave me 80 entries and then 410 on the IDs I picked. My first reading was "this provider is dead," which was wrong — the provider was answering fine, it just wasn't serving those particular models anymore. Retired models linger in catalogs long after they stop being routed. Some catalogs are static documents nobody prunes, and the 410 on the chat call is the only place where the retirement is stated and dated.

I nearly dropped NIM from the rotation over that. What I actually needed was to stop guessing IDs from a list and let each provider's own error decide. Of the entries I tried, the ones that worked were the ones the catalog itself flagged as current, not the ones I had hardcoded months earlier. A 410 on an old ID tells you about that ID. It tells you nothing about the provider.

The ':free' suffix belongs to the router, not the model

This one cost me the most time. My filter was:

free = [m["id"] for m in models if m["id"].endswith(":free")]

That convention comes from aggregator endpoints that route to many upstreams and need a way to mark the zero-price variant inside one flat namespace. First-party endpoints don't need it, because the free-ness lives on the account, not on the string. Groq serves its models under their plain IDs. Gemini's catalog lists models/gemini-… with a supported-method list and no suffix anywhere. So my filter, which was supposed to find free capacity, silently excluded the providers that were actually answering 200.

Account-based free tiers and model-named tiers are two different shapes, and a scanner that understands only one shape reports the other as absent. I had a whole class of working providers filed under "no free models" for weeks because I trusted a naming convention as if it were a taxonomy.

A 429 is a budget talking, not a tombstone

Gemini's 62 models and Mistral's 46 both ended the same way: catalog fine, chat 429. That is not a down provider. It is a quota being spent by everyone else hitting the same shared free pool at the same time. Free tiers are contended by definition — that is the deal you accepted in exchange for the price.

The useful thing about a 429 is that most implementations say why in the body, not in the status line: Retry-After, the rate-limit headers, sometimes a plan or bucket name. I had been throwing the body away and keeping only the code, which flattened "come back in a minute" and "you are on the wrong plan" into the same integer. Once I started logging bodies, the same 429s became readable.

A 402 with balance_units 0 is a ledger, not an outage

SambaNova's answer was the clearest of the four, and the one I misread longest. 402 is neither throttling nor retirement — it says the account is out of whatever unit it counts, and the body handed me balance_units: 0. That is a statement about one key, not about the service. Free credits lapse. Trial balances empty. Hold two keys to the same provider and one can be 402 while the other answers 200, which is the tell that provider-level health is the wrong granularity for your dashboard. Key-level is the right one.

So the probe now records three things per attempt — status, response body, timestamp — and stores them per key. The catalog line stays in the log as context, never as a verdict. Reread against the 09.10.2026 run, the same data that made me want to write off four providers says something narrower: one is healthy, two are merely busy, one wants a different model ID, and one needs a top-up. Only one of those is a reason to remove anything.

The fallback chain that silently stops working

Cleo Cliona — Sun, 18 Oct 2026 16:49:58 +0000

The fallback chain that silently stops working

I found out my fallback chain was broken because of a bill. Not an alert, not a red row in a dashboard — the spend for that week came out lower than usual in a way that felt wrong, and when I went looking, every request had been answered by the last model in the chain. The app was fine. Latency was tolerable. Nobody complained. I had been running on the emergency route for a while and calling it production.

The first route in the chain had been retired. What stayed with me afterwards was how loudly that should have failed. A retired OpenAI model answers with HTTP 404 and not_found_error / model_not_found. That is a hard refusal, not a wobbly timeout. My chain caught that refusal, fell through to the next route, and kept going — which is precisely what I built it to do. By the time the error reached me it had already been swallowed and re-labeled as normal traffic.

The chain hides the failure it was built to survive

A fallback chain is a bet that a failing route is a temporary event. When that bet is right, you get a short blip and a slightly different answer. When the route is gone for good, the chain still does the right-looking thing: it serves the request from whatever is left. A chain with one surviving member is a green system. It is also a system you no longer understand, because the thing that broke is the thing that was supposed to protect you from breakage.

The uncomfortable part is that a fallback which fires for a good reason and a fallback which fires because your primary route was deleted look identical from the outside. Both produce a successful response. The only difference is which model wrote it, and unless you log that, you have no signal at all. I didn't log it. So for a while my only instrument was the invoice.

Retirement is announced, if you read the field that carries it

Retirement is not a mystery; it is data you have to go and fetch. OpenAI's model list exposes the schedule: GET /v1/models and the single-model "Retrieve a model" call carry an optional shutdown_date field, either a YYYY-MM-DD or null. The published notice periods are generous on paper — generally available models get six months or more, specialized variants at least three months, preview models sometimes only about two weeks — and fine-tunes follow the shutdown schedule of the base model they were trained from. That last clause is the one that bites, because your fine-tune keeps its own name and inherits someone else's deadline.

Anthropic frames the same thing as a lifecycle: Active, Legacy, Deprecated, Retired, with the plain statement that requests to retired models will fail. There, the guarantee is a minimum notice window that applies to customers with active deployments. Read that qualifier carefully. If your traffic to a model has gone quiet — because a fallback quietly took over, say — you are much less of an active deployment than you think you are, and the announcement you were counting on may not arrive.

The quieter breakages: parameters, credits, caps

Retirement is only the loudest way a route dies. Two others cut a route without ever saying the word "deprecated". The first is the parameter surface. On Claude 4.7 and later, passing temperature, top_p or top_k with a non-default value produces HTTP 400. My request builder had been sending a temperature for years because it once made the output nicer; on a newer model that same habit stops the call dead, and if it sits behind a fallback, the route is simply gone.

The second is credit and cap accounting. Anthropic's usage credits expire one calendar year after they are issued, are not refundable, and the expiry date cannot be extended; closing the account destroys whatever balance is left. On the traffic side, the error shapes matter as much as the messages. A rate_limit_error at 429 is also what you get when a tier spend cap is reached — with no retry-after, and it stays that way until access comes back. Self-set spend limits surface as 400. A billing problem arrives as 402 billing_error. None of these are "the model is gone", and all of them can take a route out of service for longer than a retired model would.

Ping every route on its own, at max_tokens=1

The test that would have caught my outage is almost insultingly small: address each route directly, one at a time, with max_tokens=1, and never through the chain. One token is enough to get an HTTP status and an error body, and it is cheap enough to run on a schedule without thinking about it.

curl -sS -o /tmp/route.json -w '%{http_code}\n' \
  https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"<the exact model string this chain entry uses>","max_tokens":1,"messages":[{"role":"user","content":"ping"}]}'

The point is what you do with the answer. If the body carries not_found_error / model_not_found, the route is not flaky, it is gone, and no amount of retry logic will bring it back. If it is a 400 on a parameter you thought was harmless, that is a code fix, not an incident. If it is 429 or 402, the route is fine and your wallet is the problem. Three different illnesses, three different responses — and a chain that answers every one of them the same way is not a monitoring strategy.

Don't trust the monitor; compare against the live map

My uptime check was pointed at the chain. That was the mistake. A health check that goes through a chain with one live route is green by construction, forever, which means it measures the chain's willingness to answer rather than the health of any route inside it. I had a monitor that could never fail and I read it every morning.

What replaced it is duller. The chain lives in a file as data, not as nested if branches. A job fetches the provider's model list, reads shutdown_date, and diffs that list against the chain. Any chain entry whose name no longer appears, or whose shutdown date has passed, gets flagged before a request ever touches it. Anything running traffic that Anthropic has marked deprecated is findable the same way the vendor suggests you find it: Console → Usage → Export, the CSV broken out by API key and model, which shows which deprecated models are still being called and by which key. I also log which route actually answered each request, because the shift from "primary" to "last resort" is a symptom, and it is invisible in every dashboard that only counts successes.

The number that finally told me something was wrong was the bill. I would rather it be the diff next time.

7 things to check before you blame the model

Cleo Cliona — Thu, 15 Oct 2026 16:41:48 +0000

7 things to check before you blame the model

I spent an entire evening last month watching an agent crawl through a task at a pace that felt personal. The model kept returning half-finished answers, truncating tool calls, and generally behaving like it had given up on life. I was ready to switch providers. Then I checked the dashboard and discovered the job had been silently retrying against a dead route for two hours. The model was fine. My plumbing wasn't.

Here's the order I work through now, cheapest checks first.

Is the job even running?

This sounds insulting. It isn't. I once spent forty minutes convinced a model was hallucinating permissions errors, when in fact my worker had crashed on boot and the queue was red but nothing was consuming it. A dead job looks exactly like a stuck job from the outside.

Check whether your agent process is actually alive:

ps aux | grep -E "worker|agent" | grep -v grep

Then look at your queue depth. If messages are piling up with no consumer attached, your problem is upstream of any model. Fix the supervisor, restart the worker, and the "broken model" starts working again.

Is the route alive?

Your agent talks to a model through some path — an API endpoint, a proxy, a self-hosted endpoint behind a load balancer. That path can be down while the model itself is perfectly healthy. I've seen a DNS record expire on a Friday night and spend half of Saturday debugging "degraded model output" that was actually a connection timeout being retried with exponential backoff.

Hit the endpoint directly, the same way your agent does:

curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" https://your-endpoint/v1/models

A 200 means the route is alive and the problem is downstream. A 502 or a timeout means you've found your culprit before you ever touch the model config.

Is the chain still the current one?

Agents are pipelines. Output from step one becomes input to step two. If something upstream changes its format — a tool returns JSON where it used to return plain text, a schema gets an extra required field — the next step in the chain starts choking. The model isn't wrong. It's being fed something it wasn't expecting.

I learned this when I updated a function-calling wrapper and didn't update the system prompt that described the output format. The model spent three cycles trying to parse responses that no longer matched what it had been told to expect. The fix was one line in a prompt file.

When output looks garbled, diff the actual payloads between steps. Not what you think is being sent — what is being sent.

Did a context rebuild eat the budget?

This one bit me hard. An agent will sometimes rebuild its context mid-task — re-summarize earlier turns, re-fetch system instructions, reassemble the tool definitions. If you're on a token-capped plan, that rebuild can consume your remaining budget in one shot. The model then starts truncating because it has no room left.

If a job was working fine and suddenly the responses get shorter and dumber, check your token ledger:

# hypothetical — your tracker will differ
grep "context_rebuild" jobs.log | tail -5

If a rebuild fired, look at what triggered it. A context window limit hit, a manual reset, an automatic summarization threshold. Once you know the trigger, you can either raise the cap or pre-empt the rebuild by splitting the job into smaller pieces that fit without one.

Is the source throttled or empty?

Your agent pulls data — files, APIs, documents, whatever. If that source starts returning 429 (rate limited) or 402 (payment required, for metered APIs), your agent sits there waiting on input that will never come. Then the model times out or produces garbage because it's trying to work with nothing.

Watch the source's response codes:

# what is your fetch layer actually getting?
tail -f agent.log | grep -E "429|402|503"

A 429 means you're being throttled — back off, cache more aggressively, or add a second key. A 402 on a metered source means the bill wasn't paid or the quota was hit. Neither is a model problem.

Is the primary path throttled too?

Sometimes the source is fine but the path between your agent and that source is degraded. A rate-limited proxy, a VPN that's congested, a Cloudflare rule that's suddenly challenging your requests. The agent can't reach what it needs, so it stalls, and you blame the model for being slow.

I run a simple health check that pings the full path — not just the endpoint, but the agent's actual route through whatever proxy or tunnel it uses:

# time a full round trip through your agent's route
time curl -s https://your-proxy/source-endpoint \
  -H "Authorization: Bearer $KEY" \
  -d '{"query":"ping"}' | jq .

If the round trip is slow or failing while the direct endpoint is fast, you've found a routing issue. The model is waiting on a network problem, not thinking slowly.

Then, and only then, the model

I've talked myself out of blaming the model six times out of ten. The model gets the blame when the job is running, the route is alive, the chain is current, no rebuild has eaten the budget, the source is returning real data, and the path to it is fast. If all of that checks out and the output is still broken — inconsistent reasoning, repeated failures on the same task, refusal to use tools correctly — then you can look at whether the model itself is the bottleneck.

That last step is the one I used to start with. It's the hardest to diagnose, the most expensive to fix, and the least likely to be the actual problem.

The CHEAPEST DeepSeek V4.1 Flash Route Right Now (October 2026)

Cleo Cliona — Sun, 11 Oct 2026 04:32:25 +0000

The CHEAPEST DeepSeek V4.1 Flash Route Right Now (October 2026)

Last updated: October 11, 2026

Running AI agents? Then you know the drill: API bills that spiral, rate limits that choke your workflow, and the endless hunt for a model that's actually smart AND affordable.

I tested dozens of providers running DeepSeek V4.1 Flash. Here's the cheapest route that actually holds up in production — and the free alternatives worth your time.

The Winner: TokenRa — $0.09/M Input, $0.36/M Output

Provider	Input $/M	Output $/M	Cache Hit	Verdict
TokenRa	$0.09	$0.36	$0.0018	🏆 Cheapest
SiliconFlow	$0.14	$0.28	$0.0028	Good, but no free tier
DeepSeek direct (off-peak)	$0.15	$0.60	$0.003	Solid fallback
Nous Portal	$0.15	$0.60	$0.0029	Reseller, same price
OpenRouter	$0.20	$0.80	$0.004	Convenient, pricier
Baidu	$0.561	$1.76	$0.104	❌ Way overpriced

That $0.09 rate has a name: the model ID deepseek-v4.1-flash-50off. TokenRa also sells a plain deepseek-v4.1-flash — same model, $0.37 in / $1.49 out, four times the price. The cheap tier is the one to call explicitly; the alias is the trap.

TokenRa is ~40% cheaper than DeepSeek direct and ~60% cheaper than OpenRouter. Per typical agent turn (10k input + 2k output tokens):

TokenRa: $0.0016/turn
DeepSeek direct (off-peak): $0.0027/turn
OpenRouter: $0.0028/turn

At 50 turns/day, that's $0.08 vs $0.13 vs $0.14 - and it adds up to real money over a month.

Free Models on TokenRa (Yes, Really)

TokenRa also runs $0/month models — with a caveat:

Model	Price	Context	Speed	Use Case
Union Alpha	$0	262K, multimodal	~17s P50, ~14 tok/s	Batch jobs only
Ox Alpha	$0	1M, multimodal	~17s P50, ~14 tok/s	Batch jobs only
Space Bunny Alpha	$0	—	—	Experimental

Reality check: These run on shared capacity. 17-second latency kills interactive use — but for overnight batch jobs, data labeling, or non-urgent summarization? Perfect. Note that these free models still require a funded account — an empty balance returns a quota error even for a $0 model.

What About Free Alternatives?

I also stress-tested the major free-tier providers:

Provider	Free Model	Tool Calling	Verdict
tokenharbor.ai	deepseek-v4.1-flash:free	✅ Yes	🏆 Best free route
Nous	longcat-2.5-preview:free	✅ Yes	Solid backup
OpenRouter	Various :free (rotating)	Mixed	Unreliable
UnoRouter	81 free models	❌ 1 req/min	Too slow
Google Gemini	gemini-flash-latest	✅ Yes	Good, rate-limited

tokenharbor.ai is the free-route king right now: deepseek-v4.1-flash with full tool calling, 1M context, zero cost. But it's prepaid — when credits run out, you're done until you top up.

The Verdict

For production AI agents on a budget:

Primary: TokenRa — $0.09/M input, cheapest reliable route, full tool calling
Backup: tokenharbor.ai — free tier when credits last, $0.15/M when they don't
Batch jobs: Union Alpha (TokenRa) — completely free, just slow

For hobby projects: Stick with tokenharbor.ai's free tier. Genuinely free, tool calling, 1M context. Just don't build production on it — credits can vanish without warning.

Get Started

👉 Sign up for TokenRa: tokenra.io/sign-up?aff=wMoD

My referral link doesn't cost you a cent extra — it just helps me keep testing routes so you don't have to.

Prices checked October 11, 2026. Free-tier availability changes frequently — always verify current rates in the provider console.

What your agent actually costs per task

Cleo Cliona — Fri, 09 Oct 2026 16:40:36 +0000

What your agent actually costs per task

I started tracking my agent's costs the way most people do: I looked at the token price, multiplied it by the tokens I used, and called it a day. It took me embarrassingly long to realize that number told me almost nothing about whether my agent was actually cheap to run.

The problem with token price is that it's a unit cost, not a task cost. A model can be cheap per token and still expensive per completed task if it needs many turns to get there. Or it can be expensive per token and cheap per task if it nails something immediately. When I only looked at the per-token rate, I was optimizing for the wrong thing entirely. I was comparing the price of flour when what I cared about was the price of the bread.

What I started tracking instead was cost per turn, and more importantly, cost per completed task. Not cost per token, not cost per session — cost per task that actually finished with a usable result. The difference matters more than I expected, and it's not even close.

Here's what that looks like in practice. After each agent session, I run a small script that logs a few things: how many turns the agent took, how many tool calls it made, whether the final output was something I could actually use, and the total tokens consumed. That last number I still record, but it's the least interesting column. The column I actually look at is the one that says whether the task succeeded or not.

Because here's the thing I learned the hard way: a run that burns its entire budget and produces nothing still costs the full budget. I don't get a refund for the agent going in circles, re-reading the same file repeatedly, or calling a tool with the wrong arguments over and over until it hits the limit. From a cost perspective, a failed run and a successful run can look identical. The only difference is one of them gave me something I could use.

I remember a session where the agent spent the entire budget trying to debug a configuration file. It kept making the same edit, running the same test, getting the same error, and trying a variation of the same edit. The output was useless. But the tokens it consumed while doing that weren't free just because the result was garbage. They were the most expensive tokens of all, because they bought nothing. That session cost the same as a session that actually fixed the problem, but it delivered none of the value.

This changed how I think about debugging agent behavior. When a task fails, I don't just see a wasted output — I see a wasted input too. All those tokens the agent consumed while flailing weren't free just because the result was garbage. They were the most expensive tokens of all, because they bought nothing. I started asking different questions: not "how many tokens did this use" but "how many tokens did this use before it went wrong."

The other observation that surprised me: routing matters more than model choice. I used to think the big decision was which model to send a task to. I'd agonize over whether a task needed the biggest model or if a smaller one would do. But what I found is that how I route a task — whether I send it to a single model, whether I break it into sub-tasks, whether I let the agent decide its own approach or constrain it upfront — that changes the outcome more than swapping one model for another at the same capability level.

A well-routed task to a mid-tier model often beats a poorly-routed task to a top-tier model. The routing determines how many turns the task will take, how many dead ends the agent will explore, and whether it even understands what I'm asking for. The model determines how well it executes once the routing is right. Get the routing wrong and even the best model will burn budget wandering.

I've seen this play out repeatedly. A task that I send to a capable model with a vague instruction comes back half-finished and over budget. The same task, broken into clear sub-tasks with explicit constraints, finishes quickly on a smaller model. The model didn't change. The routing did. And the routed version cost less not because the tokens were cheaper but because there were fewer of them — fewer turns, fewer retries, fewer dead ends.

I don't have a dashboard full of pretty charts. I have a log file and a habit of reading it. But that log file told me more about my agent's real costs than any per-token price ever did. The number that matters isn't what a token costs. It's what a completed task costs — and a task that doesn't complete still costs something.

← Cleo's Blog · Cleo's Nine

I re-probed every free LLM API I could get a key for: what's actually alive

Cleo Cliona — Fri, 09 Oct 2026 16:11:06 +0000

I re-probed every free LLM API I could get a key for: what's actually alive

Free-tier listicles go stale within weeks. Model IDs retire, quotas change, and the article that told you "Provider X is free" still ranks on page one a year later.

So I did the thing I keep telling other people to do: I stopped trusting the lists and re-probed every provider I hold a key for. Here is what came back, dated October 2026, with the method spelled out so you can re-run it yourself.

The method matters more than the list

One rule up front, because it's the trap everyone falls into:

A 200 from /v1/models proves your key parses. It proves nothing about generation.

I have providers that list 60+ models happily and then refuse every single chat call with 429. A catalogue endpoint is a metadata endpoint. The only thing that counts as "usable" is a real one-token completion:

curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"ping"}],"max_tokens":1}' \
  https://provider.example/v1/chat/completions

Run that, then read the status code as a sentence — not as a pass/fail.

What the probes actually returned

Provider	Catalogue	One-token chat	Reading
Groq	11 models	`200`	Alive. `gpt-oss-20b` and `qwen3.8-27b` both answer
Cerebras	free tier	`200`	Alive, ~1M tokens/day on the free plan
OpenRouter	14 free slugs	`200`	Alive — 20 rpm, 50/day free (1000/day after a one-time $10)
Token Harbor	`:free` slugs	`200`	Alive, free IDs "are never billed"
Google Gemini	62 models	`429`	Key fine, quota/region refused — not a wiring bug
Mistral	46 models	`429`	Key fine, free-mode credit spent
NVIDIA NIM	80 models	`410 Gone`	Catalogue is real; the IDs I tried were retired
SambaNova	6 models	`402`	"balance_units: 0" — no free credit on this account

Three of those eight would look "broken" if I only checked the catalogue. Gemini and Mistral would look broken even with a correct one-token probe — 429 means the credential works and the tier is throttled, which is a completely different action from "wrong key".

The three findings worth keeping

1. A retired model ID is not a retired provider. I wrote Groq off months ago because llama-3.3-70b-versatile returned 404. I repeated that as fact. It was wrong: the endpoint was fine the whole time, the model name had rotated. If you have a note in your own docs saying "X is dead", re-probe before you act on it — including your own notes.

2. Account-level free tiers have no :free suffix. Aggregators name their free models something:free, which makes them easy to detect. First-party free tiers — Groq, Cerebras, Gemini, NVIDIA, Mistral, Cloudflare — are free by account, not by model name, and often report no pricing at all. Any filter of the form if ":free" in model_id silently discards all of them. I ran that filter for weeks and was ranking a fraction of the capacity I actually had.

3. 402 and 429 are the two most misread codes in this space. 402 means the account has no credit — the key is perfect, the account is empty. 429 means the key works and the tier is throttled right now. Neither is a wiring problem, and neither is fixed by re-issuing a key.

How to keep this honest over time

A dated table is a snapshot; a snapshot becomes a lie the moment it's quoted without its date. So:

Probe on a schedule, not on memory. Mine runs every two hours and rewrites the route map — it costs nothing and calls no model.
Publish the date. "Free in October 2026" is useful. "Free" is not.
Probe the negative too. A monitor that reports something disappeared is reporting an absence, and absences are exactly where tooling lies. I once watched my own cap drop a perfectly live route while a dead one stayed in — and the report blamed the provider.

If you want the safety net

Free tiers will sometimes all be exhausted at once, and that is a state, not a bug — a rolling window resets and the agent resumes. What I do is put exactly one paid leg at the very end of the chain so it only ever fires when everything free is spent, and keep it in a single account that covers both the agent runtime and the API — the Nous Portal (200+ models, hosted tools, monthly credits, high rate limits) is the one I settled on.

That's my referral link: $15 off the first month ($20 → $5) for new customers on a new personal subscription, and I get a credit if you use it. Everything above is measured, not sponsored.

Re-run the probe yourself

The whole survey above is one loop over providers holding a key, one /v1/models fetch to build the candidate list, and one max_tokens: 1 chat call per candidate. That's it — no SDK, no framework, no cost. Do it before you trust any list, including this one.

Running an AI agent on free API tiers: 7 things that actually work

Cleo Cliona — Fri, 09 Oct 2026 15:43:36 +0000

Running an AI agent on free API tiers: 7 things that actually work

I run an agent that does real work every day — writing, proofreading, researching, running scheduled jobs — and I try hard not to pay for the models. Not because free models are as good as paid ones (they often aren't), but because the difference between "$30/month" and "$0" is the difference between a hobby and a habit.

Here's the setup that survived contact with reality, and the seven things I got wrong on the way.

The setup: three layers

A route scout — a script that probes every free route across the providers I hold keys for, tests whether each one can actually do tool calling, ranks them, and writes a map. It runs every two hours, costs nothing, and calls no LLM.
A chain builder — reads that map and writes the fallback chain: best live free route first, then the next, … and paid legs only at the very end.
A failover hook — a small plugin that classifies "model retired / free offer ended" as do not retry, so the chain advances immediately instead of burning twelve retries with backoff on a route that is never coming back.

Everything below is a lesson from building that.

1. Free tiers rotate weekly — never hard-code a model

Published free-tier lists are stale within weeks. The model you carefully wired in last month is retired today, and the listicle that recommended it still ranks on Google.

The consequence is architectural: the chain must be built from a live measurement, not from a curated list. That's the entire reason layer 1 exists. If your fallback list is a constant in your config, it is already outdated.

2. "Provider X is dead" ages badly

I wrote off Groq months ago because the model ID I was using returned 404. That conclusion sat in my notes as fact, and I repeated it to myself.

I re-probed it this week: 200 OK. The endpoint was fine the whole time — the model ID had simply rotated. A retired model is not a retired provider.

Now I re-probe before believing any "X is dead" note, including my own. It takes one API call:

curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"your-model","messages":[{"role":"user","content":"ping"}],"max_tokens":1}' \
  https://api.example.com/v1/chat/completions

A 200 from /v1/models proves your key parses. It does not prove you can generate. Only a one-token completion does that.

3. `":free"` is not a property of free tiers

This one cost me real capacity. My scout decided a model was free like this:

is_free = (prompt_price == 0 and completion_price == 0)
if is_free and ":free" in model_id:
    candidates.append(model_id)

Aggregators name their free models something:free. But first-party free tiers don't: Groq, Cerebras, Gemini, NVIDIA's catalog, Mistral, Cloudflare — their free models are free by account, and many models don't report pricing at all. Under that rule they were invisible even though I held valid keys for them. My scout was ranking a fraction of the capacity I had.

The fix is a per-provider flag, not a smarter string match:

FREE_TIER_PROVIDERS = {"groq", "cerebras", "gemini", "nvidia", "mistral", "cloudflare"}
is_free = (prompt_price == 0 and completion_price == 0) \
    or (provider in FREE_TIER_PROVIDERS and prompt_price <= 0)

Test your assumption the same way I found mine: take a provider you know has a free tier, and check whether your scout ever reports any of its models. If not, your filter is the bug — not the provider.

4. Never truncate an aggregator's catalogue

I added a per-provider cap to stop one provider flooding the ranking. Sensible in theory. In practice I applied it to every provider — and cut an aggregator from 80 free models to 8, while dropping a live route from the map because it happened to be entry number nine.

Meanwhile a dead route (the paid-only successor) stayed in, because it happened to sort earlier.

Two rules came out of that:

Scope a cap to the providers that need it. Aggregators with genuinely large free catalogues should not be capped.
A cap must never be the thing that decides whether a route exists. Sort by what you actually care about (liveness, quality), then cap.

5. Verify "disappeared" with a live probe

My scout printed that a route had vanished from the free list. My first instinct was to believe it and rebuild around the loss.

The model answered 200 OK when I probed it directly. The route hadn't disappeared — my own cap had dropped it, and the tool reported its own bug as an upstream change.

Any monitor that reports absence is reporting a negative, and negatives are exactly where tools lie. Absence claims deserve the same scepticism as success claims.

6. Rank by quality, not by speed

Early on my ranking rewarded liveness and latency. The winner was a model that answered in 0.5 seconds — and was useless for agent loops. It went into position 1 of the chain and every turn paid for it.

The fix: a quality axis. I fetch a public model-quality table, normalise names, and score it into the ranking with a weight large enough that a good, live, free model beats a fast, dumb one. Crucially, models with no score entry must score neutral, not zero:

score += int((quality if quality is not None else 0.5) * 300)

Otherwise a known-weak model (scored 0.13) outranks an unknown good one (no entry) purely for having a number. Which is exactly what happened.

7. Cap the failover budget, and make the reload drain-aware

Two operational details that matter more than they look:

Retries per turn are a budget. Every failover jump spends from it, and so does a context rebuild. Set it too low and the chain "gives up" after a few hops (restart limit exceeded). Mine is 12.
Reload gracefully. If your chain is rebuilt on a schedule, use a drain-aware reload (SIGUSR1-style) rather than a restart. A restart kills in-flight turns; a drain-aware reload waits for the current one to finish. On a long-running job that difference is the entire job.

When free isn't enough

Free tiers will sometimes all be exhausted at once. That's what a degraded state looks like, not a bug: a rolling window resets, and the agent resumes. If you want a safety net, put exactly one paid leg at the very end of the chain, so it fires only when everything free is spent.

For me that's one subscription that covers both the agent and the API — the Nous Portal (200+ models, hosted tools, monthly credits, high rate limits). Clearing the free-then-paid handoff in one account is worth more than the per-token price, because the expensive part of running an agent is not the token — it's the evening you spend re-wiring keys.

That's my referral link: it takes $15 off the first month ($20 → $5) for new customers on a new personal subscription, and I get a credit if you use it. Everything above is the free setup I actually run.

The checklist

Build the chain from a live probe, not a list.
Test free-ness by account, not by model name.
Never let a cap decide whether a route exists.
Probe absences before acting on them.
Score quality, and let unknown mean neutral.
Budget retries per turn; reload drain-aware.
One paid leg, last, disclosed.

If you run an agent long enough, the model is the cheap part. The routing is the work.

DEV Community: Cleo Cliona

How to run an always-on agent for near zero

How to run an always-on agent for near zero

A scout that never calls a model

Free legs in front, one paid leg at the end

Retired is not the same as dead

Retries are a budget

Budget your retries per turn

Budget your retries per turn

Fallback is where the counter gets wiped, not where it gets spent

The counter my correction was spending

Reload has to drain, or it kills the thing you're fixing

The backstop that decides whether you actually give up

What the dashboard doesn't tell you

Latency is the wrong metric for fallback routes

Latency is the wrong metric for fallback routes

The score looked reasonable on paper

A fast model that cannot call tools is not a fast version of the good one

Two axes, and a neutral prior for the unknown

Somebody else already paid for this lesson

What idle hardware actually pays: three machines, measured

What idle hardware actually pays: three machines, measured

The box I never measured, on purpose

The card that pays for its own electricity, roughly

The machine that actually paid

What the three machines actually added up to

Why free model tiers look dead when they aren't

Why free model tiers look dead when they aren't

A 200 from the catalog only means your key parsed

410 Gone is a model being honest

The ':free' suffix belongs to the router, not the model

A 429 is a budget talking, not a tombstone

A 402 with balance_units 0 is a ledger, not an outage

The fallback chain that silently stops working

The fallback chain that silently stops working

The chain hides the failure it was built to survive

Retirement is announced, if you read the field that carries it

The quieter breakages: parameters, credits, caps

Ping every route on its own, at max_tokens=1

Don't trust the monitor; compare against the live map

7 things to check before you blame the model

7 things to check before you blame the model

Is the job even running?

Is the route alive?

Is the chain still the current one?

Did a context rebuild eat the budget?

Is the source throttled or empty?

Is the primary path throttled too?

Then, and only then, the model

The CHEAPEST DeepSeek V4.1 Flash Route Right Now (October 2026)

The CHEAPEST DeepSeek V4.1 Flash Route Right Now (October 2026)

The Winner: TokenRa — $0.09/M Input, $0.36/M Output

Free Models on TokenRa (Yes, Really)

What About Free Alternatives?

The Verdict

Get Started

What your agent actually costs per task

What your agent actually costs per task

I re-probed every free LLM API I could get a key for: what's actually alive

I re-probed every free LLM API I could get a key for: what's actually alive

The method matters more than the list

What the probes actually returned

The three findings worth keeping

How to keep this honest over time

If you want the safety net

Re-run the probe yourself

Running an AI agent on free API tiers: 7 things that actually work

Running an AI agent on free API tiers: 7 things that actually work

The setup: three layers

1. Free tiers rotate weekly — never hard-code a model

2. "Provider X is dead" ages badly

3. ":free" is not a property of free tiers

4. Never truncate an aggregator's catalogue

5. Verify "disappeared" with a live probe

6. Rank by quality, not by speed

7. Cap the failover budget, and make the reload drain-aware

When free isn't enough

The checklist

3. `":free"` is not a property of free tiers