The latest articles on DEV Community by AI Pulse (@happyyboxx). https://dev.to/happyyboxx https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4066699%2Ff486d335-5689-4309-adb8-5ee41d334a7f.jpg https://dev.to/happyyboxx en AI Pulse Fri, 07 Aug 2026 06:25:22 +0000 https://dev.to/happyyboxx/how-i-built-a-0-llm-production-stack-with-46-free-apis-4fa0 https://dev.to/happyyboxx/how-i-built-a-0-llm-production-stack-with-46-free-apis-4fa0 <h1> How I Built a $0 LLM Production Stack with 46 Free APIs </h1> <p><em>And why every "free LLM API list" goes stale within a week</em></p> <h2> The Problem </h2> <p>I needed LLMs for a side project but couldn't justify $50-500/month for API costs. Every "free LLM API" list I found had the same problems:</p> <ol> <li> <strong>Stale within days</strong> — providers change limits, add credit card requirements, deprecate models</li> <li> <strong>Markdown tables</strong> — not machine-readable, can't plug into code</li> <li> <strong>No verification</strong> — no way to know if a provider actually works <em>today</em> </li> <li> <strong>No deployment path</strong> — just a list, no config for Hermes/LiteLLM/Portkey</li> </ol> <p>So I built <strong>free-llm-atlas</strong>: 46 free LLM API providers, auto-probed daily, structured JSON + gateway configs.</p> <h2> The Zero-Cost Stack </h2> <p>After testing 40+ endpoints, my production fallback chain:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Layer 1 (Speed): Groq — 300+ tok/s, 14.4K req/day Layer 2 (Multimodal): Google AI Studio — 2M context, vision/audio, 1.5K req/day Layer 3 (Reasoning): NVIDIA NIM — Nemotron Ultra 1M ctx, function calling, 40 RPM Layer 4 (Chinese): Z.AI GLM-4 — 1M ctx, strong Chinese, 60 RPM </code></pre> </div> <p><strong>Covers 95% of workloads at $0.</strong></p> <h2> The 17 Permanent Free Providers (No Credit Card) </h2> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Provider</th> <th>Best For</th> <th>Rate Limit</th> <th>Context</th> </tr> </thead> <tbody> <tr> <td><strong>Groq</strong></td> <td>Speed</td> <td>14.4K req/day</td> <td>131K</td> </tr> <tr> <td><strong>Google AI Studio</strong></td> <td>Multimodal / Long context</td> <td>1.5K req/day</td> <td>2M</td> </tr> <tr> <td><strong>NVIDIA NIM</strong></td> <td>Reasoning / Function calling</td> <td>40 RPM</td> <td>1M</td> </tr> <tr> <td><strong>Cerebras</strong></td> <td>Extreme speed</td> <td>2.6K tok/s</td> <td>1M</td> </tr> <tr> <td><strong>Cloudflare Workers AI</strong></td> <td>Edge inference</td> <td>10K neurons/day</td> <td>128K</td> </tr> <tr> <td><strong>Cohere</strong></td> <td>RAG / Embeddings</td> <td>1K req/month</td> <td>16K</td> </tr> <tr> <td><strong>Mistral</strong></td> <td>EU data residency</td> <td>1 RPS</td> <td>32K</td> </tr> <tr> <td><strong>HuggingFace</strong></td> <td>Model variety</td> <td>$0.10/mo credits</td> <td>Varies</td> </tr> <tr> <td><strong>GitHub Models</strong></td> <td>GPT-5, o4-mini free</td> <td>150 req/day</td> <td>128K</td> </tr> <tr> <td><strong>OpenRouter</strong></td> <td>14 free models</td> <td>50 RPD</td> <td>1M (Nemotron)</td> </tr> <tr> <td><strong>Nebius</strong></td> <td>New free tier</td> <td>100 RPM</td> <td>128K</td> </tr> <tr> <td><strong>OVHcloud</strong></td> <td>EU GDPR</td> <td>Anonymous 2 RPM</td> <td>4K</td> </tr> <tr> <td><strong>Inference.net</strong></td> <td>New permanent free</td> <td>Unknown</td> <td>Unknown</td> </tr> <tr> <td><strong>LLM7.io</strong></td> <td>Claude/GPT access</td> <td>30 RPM</td> <td>200K</td> </tr> <tr> <td><strong>Requesty</strong></td> <td>Router</td> <td>200 RPM</td> <td>Varies</td> </tr> <tr> <td><strong>Z.AI</strong></td> <td>Chinese</td> <td>60 RPM</td> <td>1M</td> </tr> <tr> <td><strong>Coze</strong></td> <td>Agent platform</td> <td>100/day</td> <td>128K</td> </tr> </tbody> </table></div> <h2> Why Daily Probing Matters </h2> <p>Static lists rot. Providers:</p> <ul> <li> <strong>Change rate limits</strong> (Groq dropped from 14.4K → 1K RPD in 2026)</li> <li> <strong>Add credit card requirements</strong> overnight</li> <li> <strong>Deprecate models</strong> without notice</li> <li> <strong>Shut down endpoints</strong> silently</li> </ul> <p><strong>free-llm-atlas runs GitHub Actions daily at 06:00 UTC</strong> — every provider, every endpoint, every day.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>probe.py → test /models + /chat/completions → measure latency, success, tokens/sec, context → update providers.json → commit if changed → Git history = uptime dashboard </code></pre> </div> <h2> Usage: 30 Seconds to Production </h2> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>git clone https://github.com/happyyboxx/free-llm-atlas <span class="nb">cd </span>free-llm-atlas <span class="c"># Find free providers needing NO credit card</span> python3 <span class="nt">-c</span> <span class="s2">" import json d = json.load(open('data/providers.json')) for p in d['providers']: if p['tier']=='permanent_free' and not p.get('requires_card'): print(f'✅ {p[</span><span class="se">\"</span><span class="s2">name</span><span class="se">\"</span><span class="s2">]}: {p.get(</span><span class="se">\"</span><span class="s2">rate_limit</span><span class="se">\"</span><span class="s2">,</span><span class="se">\"</span><span class="s2">N/A</span><span class="se">\"</span><span class="s2">)}') "</span> <span class="c"># Probe all 46 providers</span> pip <span class="nb">install </span>httpx pyyaml python3 scripts/probe.py <span class="nt">--all</span> <span class="c"># Export gateway config</span> python3 scripts/probe.py <span class="nt">--export-config</span> litellm <span class="o">&gt;</span> config.yaml </code></pre> </div> <h2> Gateway Configs: Drop-in Replacement </h2> <div class="highlight js-code-highlight"> <pre class="highlight yaml"><code><span class="c1"># Auto-generated litellm.yaml</span> <span class="na">model_list</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">model_name</span><span class="pi">:</span> <span class="s">llama-3.1-70b-groq</span> <span class="na">litellm_params</span><span class="pi">:</span> <span class="na">model</span><span class="pi">:</span> <span class="s">groq/llama-3.1-70b-versatile</span> <span class="na">api_base</span><span class="pi">:</span> <span class="s">https://api.groq.com/openai/v1</span> <span class="na">max_tokens</span><span class="pi">:</span> <span class="m">8192</span> <span class="na">fallback</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">gemini-flash</span><span class="pi">,</span> <span class="nv">nim-nemotron</span><span class="pi">,</span> <span class="nv">z-ai-glm</span><span class="pi">]</span> </code></pre> </div> <p>One command switches entire stack. Zero code changes when a provider fails.</p> <h2> The Hidden Limits Nobody Talks About </h2> <p>Everyone compares <strong>daily request limits</strong>. The real bottleneck is <strong>tokens/minute (TPM)</strong>:</p> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Provider</th> <th>Daily Req</th> <th>TPM</th> <th>Real Limit</th> </tr> </thead> <tbody> <tr> <td>Groq</td> <td>14,400</td> <td>6,000</td> <td>Concurrency: ~12 msg/min</td> </tr> <tr> <td>Together</td> <td>100</td> <td>100,000</td> <td>Daily limit hits first</td> </tr> <tr> <td>NIM</td> <td>~2,400</td> <td>40 RPM</td> <td>Batch-friendly</td> </tr> <tr> <td>Google AI Studio</td> <td>1,500</td> <td>1,000,000</td> <td>Effectively unlimited</td> </tr> <tr> <td>OpenRouter (free)</td> <td>50</td> <td>20 RPM</td> <td>Very low</td> </tr> </tbody> </table></div> <p><strong>TPM determines concurrency, not daily requests.</strong> Match provider to YOUR bottleneck.</p> <h2> GitHub Actions = Free Infra </h2> <p>The probe runs on GitHub Actions (free tier):</p> <ul> <li><strong>0 infrastructure cost</strong></li> <li><strong>Runs daily at 06:00 UTC</strong></li> <li><strong>Updates <code>providers.json</code> with live status</strong></li> <li><strong>Generates gateway configs</strong></li> <li><strong>Commits changes → Git history = uptime dashboard</strong></li> </ul> <p>Zero maintenance. Zero cost.</p> <h2> Contributing </h2> <p>The project aggregates from:</p> <ul> <li> <a href="https://github.com/cheahjs/free-llm-api-resources" rel="noopener noreferrer">cheahjs/free-llm-api-resources</a> (28K+ ⭐)</li> <li> <a href="https://github.com/tashfeenahmed/freellmapi" rel="noopener noreferrer">tashfeenahmed/freellmapi</a> (17K+ ⭐)</li> <li> <a href="https://github.com/mnfst/awesome-free-llm-apis" rel="noopener noreferrer">mnfst/awesome-free-llm-apis</a> (6K+ ⭐)</li> <li><a href="https://github.com/nejib1/Free-LLM" rel="noopener noreferrer">nejib1/Free-LLM</a></li> </ul> <p>PRs welcome for new providers, probe fixes, or doc improvements.</p> <h2> Try It </h2> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>git clone https://github.com/happyyboxx/free-llm-atlas <span class="nb">cd </span>free-llm-atlas python3 scripts/probe.py <span class="nt">--all</span> python3 scripts/probe.py <span class="nt">--export-config</span> litellm </code></pre> </div> <p><strong>Star ⭐ if this saves you money on LLM inference.</strong></p> <p><em>Built because I was tired of paying for inference during development. Now my entire LLM stack costs $0.</em></p> llm ai freellm opensource