Everyone's RAG tutorial starts with "just use a free embedding API." So I took the same workload — embed 5,000 document chunks (~2.1M tokens) with a bge-small-class model — and ran it three ways: Hugging Face Serverless free tier, Google Colab free GPU, and Ollama on my own M-series laptop. Same model class, same chunks, wall-clock timed.
The free tiers weren't free. They were paid for in my time.
The numbers (5,000 chunks, bge-small-en-v1.5 class)
| Runner | Wall time | Setup friction | Failure mode | Real cost |
|---|---|---|---|---|
| HF Serverless (free) | 3h 40m | none | 429s after ~800 requests, retry hell | $0 + my patience |
| Colab free (T4) | 22 min | runtime died once | idle disconnect mid-run, lost session | $0 + one restart |
| Ollama (local, CPU+GPU) | 31 min | one ollama pull
|
none | $0, my electricity (~$0.02) |
Hugging Face's free tier rate-limits you into a retry loop — I wrote a backoff wrapper and it still took 3.5x longer than Colab. Colab won on speed but killed my kernel for idling while I made coffee. The laptop just... ran.
The code that survived all three
# Ollama — the one that didn't need a retry wrapper
import ollama
def embed_batch(texts):
return [ollama.embed(model="bge-m3", input=t)["embeddings"][0]
for t in texts]
Versus HF's free tier, which needed this just to not die:
for attempt in range(6):
try:
return client.feature_extraction(text)
except Exception: # 429, 503, cold start...
time.sleep(2 ** attempt)
The uncomfortable conclusion
"Free API tier" and "free to use" are different products. HF's free inference is a demo tier wearing a production costume. Colab free is genuinely fast but punishes you for looking away. If your pipeline is batch-shaped and under ~10M tokens, a local model on hardware you already own beats both — no rate limits, no sessions, no vendor deciding your throughput today.
I prototyped all three pipelines with MonkeyCode — free, runs against the local Ollama endpoint, so the whole experiment loop cost $0: https://ly.cyberserval.tech/iIETXiF
Have you actually timed a "free" API tier end-to-end, or are you trusting the pricing page? Curious if HF's paid inference endpoints close the gap enough to be worth it.
Top comments (0)