DEV Community

AI Pulse
AI Pulse

Posted on

How I Built a $0 LLM Production Stack with 46 Free APIs

#llm #ai #freellm #opensource

How I Built a $0 LLM Production Stack with 46 Free APIs

And why every "free LLM API list" goes stale within a week

The Problem

I needed LLMs for a side project but couldn't justify $50-500/month for API costs. Every "free LLM API" list I found had the same problems:

  1. Stale within days — providers change limits, add credit card requirements, deprecate models
  2. Markdown tables — not machine-readable, can't plug into code
  3. No verification — no way to know if a provider actually works today
  4. No deployment path — just a list, no config for Hermes/LiteLLM/Portkey

So I built free-llm-atlas: 46 free LLM API providers, auto-probed daily, structured JSON + gateway configs.

The Zero-Cost Stack

After testing 40+ endpoints, my production fallback chain:

Layer 1 (Speed):     Groq — 300+ tok/s, 14.4K req/day
Layer 2 (Multimodal): Google AI Studio — 2M context, vision/audio, 1.5K req/day
Layer 3 (Reasoning): NVIDIA NIM — Nemotron Ultra 1M ctx, function calling, 40 RPM
Layer 4 (Chinese):   Z.AI GLM-4 — 1M ctx, strong Chinese, 60 RPM
Enter fullscreen mode Exit fullscreen mode

Covers 95% of workloads at $0.

The 17 Permanent Free Providers (No Credit Card)

Provider Best For Rate Limit Context
Groq Speed 14.4K req/day 131K
Google AI Studio Multimodal / Long context 1.5K req/day 2M
NVIDIA NIM Reasoning / Function calling 40 RPM 1M
Cerebras Extreme speed 2.6K tok/s 1M
Cloudflare Workers AI Edge inference 10K neurons/day 128K
Cohere RAG / Embeddings 1K req/month 16K
Mistral EU data residency 1 RPS 32K
HuggingFace Model variety $0.10/mo credits Varies
GitHub Models GPT-5, o4-mini free 150 req/day 128K
OpenRouter 14 free models 50 RPD 1M (Nemotron)
Nebius New free tier 100 RPM 128K
OVHcloud EU GDPR Anonymous 2 RPM 4K
Inference.net New permanent free Unknown Unknown
LLM7.io Claude/GPT access 30 RPM 200K
Requesty Router 200 RPM Varies
Z.AI Chinese 60 RPM 1M
Coze Agent platform 100/day 128K

Why Daily Probing Matters

Static lists rot. Providers:

  • Change rate limits (Groq dropped from 14.4K → 1K RPD in 2026)
  • Add credit card requirements overnight
  • Deprecate models without notice
  • Shut down endpoints silently

free-llm-atlas runs GitHub Actions daily at 06:00 UTC — every provider, every endpoint, every day.

probe.py → test /models + /chat/completions → measure latency, success, tokens/sec, context
→ update providers.json → commit if changed → Git history = uptime dashboard
Enter fullscreen mode Exit fullscreen mode

Usage: 30 Seconds to Production 

git clone https://github.com/happyyboxx/free-llm-atlas
cd free-llm-atlas

# Find free providers needing NO credit card
python3 -c "
import json
d = json.load(open('data/providers.json'))
for p in d['providers']:
    if p['tier']=='permanent_free' and not p.get('requires_card'):
        print(f'✅ {p[\"name\"]}: {p.get(\"rate_limit\",\"N/A\")}')
"

# Probe all 46 providers
pip install httpx pyyaml
python3 scripts/probe.py --all

# Export gateway config
python3 scripts/probe.py --export-config litellm > config.yaml
Enter fullscreen mode Exit fullscreen mode

Gateway Configs: Drop-in Replacement 

# Auto-generated litellm.yaml
model_list:
  - model_name: llama-3.1-70b-groq
    litellm_params:
      model: groq/llama-3.1-70b-versatile
      api_base: https://api.groq.com/openai/v1
      max_tokens: 8192
    fallback: [gemini-flash, nim-nemotron, z-ai-glm]
Enter fullscreen mode Exit fullscreen mode

One command switches entire stack. Zero code changes when a provider fails.

The Hidden Limits Nobody Talks About

Everyone compares daily request limits. The real bottleneck is tokens/minute (TPM):

Provider Daily Req TPM Real Limit
Groq 14,400 6,000 Concurrency: ~12 msg/min
Together 100 100,000 Daily limit hits first
NIM ~2,400 40 RPM Batch-friendly
Google AI Studio 1,500 1,000,000 Effectively unlimited
OpenRouter (free) 50 20 RPM Very low

TPM determines concurrency, not daily requests. Match provider to YOUR bottleneck.

GitHub Actions = Free Infra

The probe runs on GitHub Actions (free tier):

  • 0 infrastructure cost
  • Runs daily at 06:00 UTC
  • Updates providers.json with live status
  • Generates gateway configs
  • Commits changes → Git history = uptime dashboard

Zero maintenance. Zero cost.

Contributing

The project aggregates from:

PRs welcome for new providers, probe fixes, or doc improvements.

Try It 

git clone https://github.com/happyyboxx/free-llm-atlas
cd free-llm-atlas
python3 scripts/probe.py --all
python3 scripts/probe.py --export-config litellm
Enter fullscreen mode Exit fullscreen mode

Star ⭐ if this saves you money on LLM inference.

Built because I was tired of paying for inference during development. Now my entire LLM stack costs $0.

Top comments (0)