DEV Community: Ram Chandra Giri

What I learned making 20 free LLM APIs behave like one

Ram Chandra Giri — Wed, 14 Oct 2026 02:47:28 +0000

In September I shared a few small Node packages. One of them was llm-free-cascade. The idea is simple: send a chat request to Groq. If Groq says no, try Gemini. Then Cerebras. And so on, through every free tier you have a key for.

The idea was simple. Getting it right was not. Here is what I learned.

1. Don't keep retrying the same key

My first version always started with key #1. Once key #1 ran out for the day, every request still tried it first, failed, and only then moved on. More keys made it slower, not faster.

Fix: take turns between keys. When a key is rate-limited, let it rest and skip it until it's ready again.

2. Limits are usually per model

A 429 rarely means "this key is done." It usually means "this model is done for this key." So try another model on the same key before giving up on the provider.

new LLMCascade({
  keys: { groq: process.env.GROQ_API_KEY },
  models: { groq: ['openai/gpt-oss-120b', 'openai/gpt-oss-20b'] }, // 2nd is tried if the 1st is limited
});

3. Know how long to wait

A fixed cooldown is wrong. It's too long for a per-minute limit and too short for a daily one.

Providers tell you when to come back, each in their own way:

a Retry-After header
a message like "try again in 2m59s"
words like "per minute" or "per day"
a known daily reset time (Gemini resets at midnight Pacific)

The cascade reads all of these. A per-minute limit waits about a minute. A daily limit waits until that provider resets.

4. Not every error means the provider is broken

Error	What the cascade does
Prompt too long	Skips that provider for this one request only
429 / quota	Rests that key and model until the reset
401 / 403	Stops using that one key
404 (model removed)	Stops using that model
Timeouts, 5xx	After 2 in a row, skips the provider for a minute

That last one matters. Without it, a provider that is down makes every request wait for the full timeout.

5. One trick I chose not to use

You could send the same request to two providers at once and keep the faster answer. On free tiers that spends twice the quota for one answer, so I left it out.

6. Free tiers change all the time

This year alone, Cerebras' free tier became a 30-day trial, SambaNova started asking for a card, and GitHub Models shut down. Model names change even more often.

So the provider list lives in one data file, providers.json. Each entry says when it was last checked and where the numbers came from. It also warns you about things like "non-commercial use only" or "needs a phone number". A small script checks every model name against each provider's live list.

One note: many providers don't allow several free accounts to get around their limits. Check each provider's terms.

7. Use it from any language

Someone opened an issue asking to use it like the OpenAI API, from their own tools. So now you can run it as a local server:

npx llm-free-cascade serve        # http://127.0.0.1:8787/v1

Then point any OpenAI client at it, for example from Python:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8787/v1", api_key="unused")
reply = client.chat.completions.create(
    model="cascade",  # or "groq" to use only Groq
    messages=[{"role": "user", "content": "Explain photosynthesis in one sentence."}],
)
print(reply.choices[0].message.content)

It works with streaming too. The server only listens on your own machine by default, since it holds all your keys.

8. A local backup with Ollama

If every free tier is busy at the same time, Ollama can answer from your own computer. No limits, just slower.

new LLMCascade({
  custom: {
    ollama: {
      baseUrl: 'http://localhost:11434/v1',
      model: 'llama3.2',
      apiKey: 'ollama', // any text works, but it can't be empty
      tier: 5,          // try it last
    },
  },
});

9. You can see what it's doing

npx llm-free-cascade keys     # which providers you're missing, and where to sign up
npx llm-free-cascade status   # your providers, keys and limits
npx llm-free-cascade probe    # quick live test of every key

In code, cascade.stats() shows calls, failures, speed and cooldowns for each provider and key.

llm-free-cascade is free (MIT), has zero dependencies, and runs on Node 18+.

GitHub · npm · Guide

Found a model name that no longer works? Fixing it is a one-file PR. Want tool calling in the server? Open an issue and tell me what you're using it with.

Stop launching Chromium for every PDF — a tiny pool for Puppeteer HTML-to-PDF

Ram Chandra Giri — Mon, 12 Oct 2026 19:28:40 +0000

My exam platform, mcqplex.com, turns HTML templates (reports, certificates) into PDFs with Puppeteer. Like many tutorials, my code launched and closed a full Chromium for every PDF:

const browser = await puppeteer.launch();
// ...render...
await browser.close();

It worked, but it was slow, and memory jumped when many downloads came in at once.

Keeping one browser warm

const browser = await puppeteer.launch(); // once, at startup

async function renderPdf(html) {
  const context = await browser.createBrowserContext(); // separate cookies/storage
  const page = await context.newPage();
  try {
    await page.setContent(html, { waitUntil: 'domcontentloaded' });
    return await page.pdf({ format: 'A4' });
  } finally {
    await context.close();
  }
}

On my 2-core laptop, a render went from about 1.9 s to about 0.66 s. Your numbers will depend on your machine.

Images still delayed printing

My templates load a school logo from a remote URL. I used domcontentloaded so a slow image host wouldn't block the render, but page.pdf() still waited for the pending image. It waits for fonts to be ready by default, and that didn't finish while the request was hanging.

Calling window.stop() before printing stopped the wait, but it also removed logos that loaded only a little slowly. A short grace period worked better for me:

await page.waitForNetworkIdle({ idleTime: 100, timeout: 3000 }).catch(() => {});
await page.evaluate(() => window.stop());

Slow logos still show up, and a broken one is left blank after 3 seconds. I only found both issues by testing with real Chromium. My mocked tests didn't catch them.

The package

In production I also needed a concurrency limit, a queue limit, timeouts and crash recovery, so I put it all into a small package, pdf-render-pool:

const { createPdfRenderPool } = require('pdf-render-pool');
const { renderPdf } = createPdfRenderPool({ concurrency: 2, maxQueue: 50 });

GitHub: https://github.com/rcgiri-physics/pdf-render-pool

If you try it, I'd be glad to hear how it works for you, especially on Linux servers.

I open-sourced 3 small Node utilities I pulled out of building an edtech platform

Ram Chandra Giri — Sat, 19 Sep 2026 20:08:57 +0000

I've spent the last while building MCQplex, an exam-prep
platform for Nepal's NEB curriculum. Along the way I built a few pieces of
infrastructure that had nothing to do with exams specifically, so I pulled
them out into their own packages:

llm-free-cascade

Free-tier LLM APIs (Gemini, Groq, Cerebras, and others) are individually
rate-limited, but between them they add up to a lot of free inference if you
just fail over from one to the next. This cascades a chat completion across
however many you've got keys for, with automatic multi-key rotation (if
you've made more than one free account on a provider) and cooldown for a
provider that's structurally broken. Zero dependencies.

const { LLMCascade } = require('llm-free-cascade');
const cascade = LLMCascade.fromEnv();
const { text, provider } = await cascade.generate({ system, user });

pdf-render-pool

Launching a fresh headless Chromium per PDF (the naive way to use Puppeteer)
costs 1-2 seconds and ~200MB of RAM every time. This keeps one browser warm
and gives each render its own page, with a concurrency cap so a burst of PDF
requests queues instead of taking the server down.

mongo-job-lock

If you run node-cron jobs on more than one instance (PM2 cluster, several
servers), every instance fires the job unless you do something about it.
This is a tiny MongoDB-backed advisory lock — steal-on-expiry, so a crashed
holder can't deadlock the job forever — for whatever Mongo connection you
already have.

All three: zero or optional-peer dependencies, MIT licensed, tested, on npm.
Feedback / PRs welcome, especially on llm-free-cascade since free-tier
model names drift constantly and providers.json is set up so a one-file PR
fixes a stale one.