ArshTechPro

Posted on Sep 4

FreeLLMAPI: One OpenAI-Compatible Endpoint for 34 Free LLM Providers

#ai #llm #opensource #agents

Almost every AI lab now hands out a free tier. Google, Groq, Cerebras, Mistral, Cohere, NVIDIA, Cloudflare, OpenRouter, and a couple dozen more. Each one on its own is small. A few million tokens a month, a few thousand requests a day. Stacked together, they turn into something you can actually prototype against.

The problem is stacking them by hand. Thirty-four SDKs, thirty-four sets of rate limits, thirty-four places your request can fail at 2 AM.

FreeLLMAPI is an open-source router that collapses all of that into a single /v1 endpoint. You point any OpenAI client at your own machine, and it routes across whatever providers you've added keys for.

First, the question everyone asks: is it actually free?

Short answer: yes, the software is free and stays free. There is a paid tier, but it does not gate the router.

Here is the honest breakdown.

What is free, permanently

The router itself is MIT-licensed. Self-host it with Docker, npm, or the desktop app. No account with the project required to run it.
Every feature works: routing, failover, rate tracking, the dashboard, analytics, the playground, all the API surfaces.
The inference is free because you bring your own free-tier provider keys. FreeLLMAPI does not resell tokens. It never sees your prompts on a remote server.

What costs money

One thing only: the live model catalog feed. $19/year or $49 once for lifetime.
Free installs get the same signed catalog, but from a monthly snapshot. A new model reaches free installs about 30 days after it hits the live feed. The project says free builds currently sit roughly 303 models behind.
Nothing expires, nothing gets crippled, no request caps are added. Updates just arrive later.

So the mental model is: the software is free, the tokens are free (they're your own free tiers), and the $19/yr is a convenience subscription for same-day model catalog updates. If you're fine being a month behind on newly-launched free models, you never pay anything.

The catch that isn't about money

The repo is blunt about this, and so am I: this is for personal experimentation and learning. Not production. No frontier models, variable latency, no SLA, and the effective quality of the endpoint drops late in the day as the best free models hit their daily caps (they reset at UTC midnight). Your relationship with each upstream provider is still governed by the terms you agreed to when you signed up for them. Ship something real, swap in a paid API first.

So how does the "aggregator" actually work? Do I get an API key?

Yes, but probably not the way you're imagining. There is no signup page where FreeLLMAPI hands you a key to a hosted service. It is local-first and single-user by design.

The flow is two layers of keys:

Layer 1: your provider keys (inbound). You go and get free-tier API keys yourself from Google AI Studio, Groq, Cerebras, Mistral, and so on. You paste them into the FreeLLMAPI dashboard on the Keys page. They get AES-256-GCM encrypted and stored in a local SQLite database, then decrypted in memory only for the duration of a request.

Layer 2: your unified key (outbound). The router generates a single bearer token that looks like freellmapi-.... That is the only credential your applications ever see. Your app never touches the provider keys.

Your app  ──[freellmapi-xxx]──►  Local router  ──[real provider keys]──►  Groq
                                      │                                   Google
                                      │                                   Cerebras
                                      └─ picks, tracks limits, fails over  ...

What the router does per request:

Looks at your fallback chain and picks the highest-priority model that has a healthy key and is currently under all of its rate limits.
Decrypts that provider key in memory and calls the provider.
On a 429 or 5xx, it puts that key on cooldown and immediately retries the next model in your chain.
Tracks RPM/RPD/TPM/TPD per (provider, model, key) so it stays under every free-tier cap instead of discovering the cap by getting rejected.
Returns an X-Routed-Via: <provider>/<model> header so you can see who actually served the request.

There is also sticky sessions (a conversation stays on one model for 30 minutes so replies stay coherent), unified model entries when the same model exists on several providers, and named routing profiles you can switch per request with auto:<profile>.

Simple steps to get running

Step 1: Install

The one-liner needs Docker. It creates ~/freellmapi, generates an encryption key, pulls the image, and starts the container.

curl -fsSL https://freellmapi.co/install.sh | bash

If piping to bash makes you uncomfortable, the script is readable at the same URL first. Re-running it is safe: it keeps your existing .env and encryption key.

On Windows or macOS you can skip Docker entirely and grab the desktop installer from the GitHub Releases page. It runs the whole router and dashboard from your tray, and there is no password to set up.

Step 2: Open the dashboard

http://localhost:3001

Step 3: Add your free provider keys

Go to the Keys page and paste in whatever free-tier keys you have. Start with two or three, you don't need all thirty-four. Good ones to begin with:

Google AI Studio for Gemini models
Groq for very fast inference
Cerebras for very fast inference
Mistral

Each key shows a status dot and when it was last health-checked, so you'll know immediately if you pasted a bad one.

Step 4: Set your fallback chain

On the Models page, drag your preferred models into order. That order is literally the routing priority. Top of the list gets tried first, and everything below it is your automatic failover.

Step 5: Copy your unified key

It's in the header of the Keys page, starting with freellmapi-. This is the key your code uses.

Step 6: Point your code at it

Nothing new to learn. It's the OpenAI SDK with a different base_url.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",   # let the router choose
    messages=[{"role": "user", "content": "Explain database indexes in two sentences."}],
)

print(resp.choices[0].message.content)

Node is the same idea:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:3001/v1",
  apiKey: "freellmapi-your-unified-key",
});

const resp = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Explain database indexes in two sentences." }],
});

console.log(resp.choices[0].message.content);

And plain curl:

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"hello"}]}'

For the model field you can pass:

"auto" — router picks the best available
"auto:fast" or "auto:smart" — bias toward speed or capability
"auto:<your-profile>" — a named chain you built in the dashboard
an explicit model id — pin it to one model
"fusion" — fan the prompt out to several free models in parallel and have a judge model synthesize one answer

Step 7 (optional): Wire up your coding agent

This is where it gets genuinely useful. Most CLI agents configure themselves with one command:

npx freellmapi setup-claude --url http://localhost:3001 --api-key <unified-key>

There are generators for Codex CLI, Cline, Continue, Aider, OpenCode, Goose, Qwen Code, Roo, Kilo, Crush, DeepSeek Harness, and more. Every one supports --dry-run and backs up your existing config before touching it.

What surfaces you get

Beyond /v1/chat/completions, the router implements:

/v1/responses (what Codex CLI needs), /v1/completions for editor ghost-text
/v1/embeddings, /v1/models
/v1/images/generations, /v1/videos/generations, /v1/audio/speech, /v1/audio/transcriptions
/v1/messages — Anthropic's wire format, so Claude Code and the Anthropic SDKs work against your free pool
/v1beta — Gemini's native surface for Gemini CLI
Optional Ollama emulation for Zed and JetBrains AI
/mcp — an MCP server so agents can introspect available models and provider health mid-session

Tool calling and structured outputs round-trip across providers, including a nice touch where plain-text tool calls from weaker models get rescued into proper tool_calls.

You can also add a custom provider pointing at any OpenAI-compatible endpoint: llama.cpp, LM Studio, vLLM, a local Ollama, or a remote gateway. So your local models sit in the same fallback chain as the cloud free tiers.

Should you use it?

Use it if you are: prototyping, building side projects, running a coding agent on your own machine, learning how routing and failover work, or just tired of managing eleven different .env variables.

Do not use it if you are: shipping to customers, need an SLA, need frontier-model quality, or need predictable latency.

It runs on anything with Node 20+, including a Raspberry Pi, at around 40 MB RSS idle.

Repo: github.com/tashfeenahmed/freellmapi

DEV Community