DEV Community

Jamse Bao
Jamse Bao

Posted on

Tried `tashfeenahmed/freellmapi`: One `/v1` Gateway for 635 Free Models

#ai #opensource #tutorial #programming

Tried tashfeenahmed/freellmapi: One /v1 Gateway for 635 Free Models

tashfeenahmed/freellmapi is gaining serious attention—+612 GitHub stars today—because it tackles a common AI developer problem: provider fragmentation.

The project aggregates 34 free LLM providers and 635 free model endpoints, reportedly handling around 7.4 billion tokens per month. Instead of rewriting integrations for every provider, you connect to one OpenAI-compatible /v1 endpoint and let the gateway handle routing, failover, and encrypted API keys.

The practical value is not “free models” alone. It is avoiding brittle application code when a provider starts returning 429, changes availability, or silently drops a model. Smart routing and automatic failover can keep experiments running—although I would still add request timeouts, retries, and context-length checks on the client side.

The primary gateway model in this test drive is claude-fable-5:

curl https://your-freellmapi-host/v1/chat/completions \
  -H "Authorization: Bearer $FREELLMAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-fable-5",
    "messages": [
      {"role": "user", "content": "Summarize this incident log in five bullets."}
    ],
    "temperature": 0.2
  }'
Enter fullscreen mode Exit fullscreen mode

The same pattern works with a custom OpenAI-compatible backend:

from openai import OpenAI

client = OpenAI(
    base_url="https://b-lost.com/v1",
    api_key="B_LOST_API_KEY",
)

response = client.chat.completions.create(
    model="claude-fable-5",
    messages=[{"role": "user", "content": "Explain this stack trace."}],
)
Enter fullscreen mode Exit fullscreen mode

For long prompts, B-Lost’s native Anthropic /v1/messages Prompt Caching can reduce repeated-input cost, with cache hits advertised at a 90% discount. That is useful for agents carrying large system instructions, repositories, or tool schemas.

My recommendation: use freellmapi for personal experimentation and routing experiments, but monitor provider health, token limits, and terms carefully before treating it as production infrastructure.

Repo: tashfeenahmed/freellmapi

Top comments (0)