DEV Community: shiva prasad

Free LLM APIs for coding assistants

shiva prasad — Thu, 12 Nov 2026 18:00:04 +0000

Free LLM APIs for coding assistants

Yes, you can point an editor plugin, a CLI coding agent, or a small script at an OpenAI-compatible base URL and a free-tier key. The setup is the same pattern as any other compatible API: a published base URL, the provider's key in the environment, and a model id copied from that provider's official docs.

What changes is the volume. A chat playground sends one request when you press enter. A coding assistant may send many requests for a single edit, plus retries, plus context from your files. That is why published rate limits matter more here than for a one-line test, and why you should think carefully about relays before you index a private repository through a third-party host.

The figures below come from the 12 November 2026 snapshot on Free LLM API Tokens. I will not invent quality scores, context-window sizes or latency numbers. When a quota is not published, I will say so.

What to configure in the tool

Most OpenAI-compatible coding tools ask for three values:

An API key, which should come from the environment or the tool's secret store, not from a file inside the repo
A base URL, copied from the provider's docs or from the card on the provider list
A model id, copied from the same provider, not from another catalogue

If the tool has a "provider" dropdown, only use a preset if it matches the official host you intend. A preset that points at an unknown relay is still a relay. If the tool lets you set a maximum number of requests or a monthly budget, set one before the first large run.

Do not commit the tool's local config if it contains the key. Add those files to .gitignore.

Why free tiers run out faster during coding

Coding prompts include source files, diffs and logs. Token counts grow with context, not just with the size of the visible answer. Agents may re-read a file, call tools, and retry on a failed patch. If the provider counts input and output tokens toward a daily cap, a long debugging session can exhaust it even when the published number looked large in a table.

Look at more than tokens per day. Requests per minute, requests per day, concurrent connections and maximum context size are all separate limits. A coding agent that fires parallel calls is often the first way people discover an unpublished RPM cap.

A few official figures from the 12 November snapshot, to show how different the units can be:

Groq publishes 200,000 tokens per day and 1,000 requests per day on listed models, with no card required. Limits vary by model. The documented base URL is https://api.groq.com/openai/v1.
NVIDIA Build / NIM publishes a free trial on hosted models, with 40 requests per minute and 10,000 requests per day, and no card required. The documented base URL is https://integrate.api.nvidia.com/v1.
SambaNova Cloud published 200,000 tokens per day per model as of 3 October, but only 20 requests per day, and a card is required. A low request cap will bind a chatty agent long before the token cap does.
OVHcloud AI Endpoints publishes keyless free access at 2 requests per minute per IP per model, with no card required.
Gemini API (Google AI Studio) has a free tier on selected models, with per-model RPM and RPD on Google's own rate-limit page. Tokens per day are not published in the tracker. The notes also record that free-tier prompts may be used to improve Google products — read that policy before you send a private repo.
Cloudflare Workers AI publishes 10,000 Neurons per day, which is not a token count, and its base URL needs a real account id in place of {ACCOUNT_ID}.

If tokens per day are marked "Not published", you still have a cap; you just cannot plan from a number the tracker recorded. Watch the official dashboard and stop on the first 429 response. Tight auto-retries will make the situation worse.

Phone checks, cards and trial end dates also matter. A trial with a high daily cap and a card on file is not the same as an ongoing no-card free tier. Cerebras, for example, publishes 1,000,000 tokens per day during a $5, 30-day trial, with a card required.

Official provider, documented gateway, or relay?

If the repository is private, prefer an official provider whose privacy terms you accept, or a model you run locally. Some official free tiers disclose that prompts may be used to improve the product. "Official" is not a synonym for "will not train on my code."

Documented gateways can be convenient when you want to switch models through one base URL. They may also log prompts, and the model host behind a free route may log them too. OpenRouter is recorded in the snapshot as a gateway (not official), OpenAI compatible, with :free models, a published limit of 50 requests per day until credits are bought, no card required, and a note that free-model providers may log prompts. Fifty requests per day is easy for a coding agent to spend on a single stubborn edit.

Unofficial relays are a poor default for source code. The 12 November snapshot flagged 47 of 112 listed providers with a relay warning, including some of the largest advertised daily caps. NaraRouter and Flexlora were both recorded at 7 million tokens per day, both as relay-flagged gateways. A large cap does not make a relay safer for a private codebase. Hide those rows when you are choosing a host for work code.

The coding-specific checklist, including rate-limit and key-safety notes, is in Free LLM APIs for coding.

Keep a coding key from leaking

Issue a key that is only used by the assistant, so you can revoke it without rotating everything else. Keep it out of shell history if you can, out of screenshots of errors, and out of issue trackers when a command fails. Do not paste source that contains .env files into a prompt on a relay.

If the assistant can run terminal commands, it may accidentally echo the environment or print a debug HTTP dump. Treat that as a leak and revoke the key if a log might have been shared.

A small script that is easy to retarget

For your own scripts, construct the client once and keep the model id in a variable or environment value. That way you can switch providers without rewriting the rest of the program.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["API_KEY"],
    base_url=os.environ["API_BASE_URL"],
)

def complete(prompt: str) -> str:
    resp = client.chat.completions.create(
        model=os.environ["MODEL_ID"],
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.choices[0].message.content or ""

Set API_BASE_URL to the published base URL, including any /v1 suffix the provider documents. Do not hard-code a host you saw in an unofficial list if the tracker says the endpoint is not published — look it up on the official site instead.

Add a clear failure path for HTTP 429 and 401. A script that ignores those errors can sit in a retry loop until the day cap is gone.

How to evaluate "good for coding" without inventing benchmarks

A public tracker can tell you whether a compatible base URL is published, whether the allowance is a trial or an ongoing tier, whether a card is required in the public terms, and whether a relay has been flagged. It cannot honestly tell you which model is "best at Rust" or how many tokens of context you will get.

If you need quality or latency, measure them on a public sample, or read the official model card. Start with one official provider, one short file, and a request cap in the tool. If that loop is stable, then consider more context or a second model id. If the official terms do not fit a private repository, do not try to solve that with an unofficial free relay.

Re-check the official page before you depend on a number from any snapshot. Offers end. The same November list still carries rows for ended Hugging Face monthly credits, a removed Fireworks signup credit, and an Ollama Cloud plan that is no longer a free cloud tier.

Official vs relay LLM gateways: risks and how to spot them

shiva prasad — Thu, 12 Nov 2026 17:53:06 +0000

Official vs relay LLM gateways: risks and how to spot them

An OpenAI-compatible chat.completions call looks the same whether you are talking to a first-party API, a documented multi-model router, or an unofficial relay. The SDK will still return a string. The operator, the privacy policy, the logging and the way a free tier ends all belong to the service behind the base URL, and those are not the same.

This article uses the classifications and published fields from the 12 November 2026 snapshot on Free LLM API Tokens. I will not invent a logging policy for a site that does not publish one, and I will not treat a large tokens-per-day figure as a security control.

Three kinds of service, not two

Official provider, in that tracker's data, means a model maker or a cloud that offers the API under its own name and documentation. You create an account with that organisation and, typically, call a base URL on its domain. Groq, Google's Gemini API, Cloudflare Workers AI, Mistral, SambaNova and Cerebras are official examples in the current list.

Gateway is broader. A gateway sits in front of one or more models — sometimes its own, often someone else's — and gives you a single key and a single request format. Both official providers and gateways can be OpenAI compatible. Both can have a free allowance. They are not the same kind of dependency.

Relay warning is a separate flag. It marks unofficial resellers or unclear operators. Do not treat every gateway as a relay, and do not treat every official row as a permanent free plan. Of the 112 providers in the 12 November snapshot, 37 were classified official and 75 as gateways, and 47 carried a relay warning.

The comparison is written up at more length in the site's guides on official providers vs gateways and relay and gateway risks.

Why the distinction matters if the code is identical

With an official API you can usually find a legal entity, a status page and a written data policy. You still have to read that policy. "Official" is not a synonym for "will not train on my prompts" or "will stay free." The Gemini API entry, for example, records that free-tier prompts may be used to improve Google products. Official free credits also end: the same snapshot records that Hugging Face's free monthly inference credits had ended, and that Fireworks AI's signup credit had been removed.

With a documented gateway you must also ask where the model runs and who may store the prompt. A well-known example in the list is OpenRouter. It is recorded as a gateway (not official), OpenAI compatible, with a documented base URL of https://openrouter.ai/api/v1. The published free allowance is :free models, with a limit of 50 requests per day until credits are bought, and no card required. The notes say free-model providers may log prompts. That is a documented trade-off, not a hidden one, and it is a reason to keep private data out of :free routes even without a relay flag.

With an unofficial relay you may not get a clear answer at all. The site may present a long model list and a generous free allowance without saying where the request actually goes. A model name can be mapped to a different model than the one you expected. A free cap can disappear. A domain can change. Billing disputes and abuse reports are harder when the operator is unclear.

What a relay can see

Your prompt, system instructions and any files you attach may be visible to the relay operator, and may be logged or reused. The operator may also send that text to a third-party model host with its own logging policy. If you pasted customer data, private source code, or an API key inside the prompt, you have now shared it with at least one extra party.

A relay can also collect your API keys if you are asked to paste a key from another provider into their dashboard so they can "bring your own key." That is a different, higher risk than using a key they issued for their own endpoint: you are trusting them with a credential that can spend money or read projects on the first-party account. A tracker should never ask you for a key. If a site does, stop.

How to read the public signals

You do not need a private investigation to do a first pass. Look at:

Who is named as the operator, and whether a company or legal entity is published
Whether the docs live on that organisation's domain
Whether the product is described as a router in front of other APIs
Whether routing and logging are explained, including for free routes
Whether a card, a phone check or a trial end date is published
Whether the base URL is a typo-squatting lookalike of a well-known host

Then read the tracker's badges as a set, not as a recommendation: official or gateway, relay warning or not, OpenAI compatibility, the written allowance, and whether a card is required. No single badge means "safe." Absence of a warning is not a certification.

A large tokens-per-day number does not offset the risk

A published cap is a usage number, not a security control. In the 12 November snapshot, some of the highest daily figures belonged to relay-flagged gateways. NaraRouter and Flexlora were both recorded at 7 million tokens per day, both as gateways, both with a relay warning. APInex was recorded at 3 million tokens per day, also a relay-flagged gateway. Those rows are useful if you are comparing advertised allowances. They are a poor default for private data.

By contrast, an official no-card tier can look small next to those numbers and still be the better default for learning or for private prompts. Groq's published allowance in the same snapshot was 200,000 tokens per day and 1,000 requests per day on listed models, with no card required. "Smaller advertised cap, clearer operator" is often the right trade.

Card requirements appear in both categories. A card-backed official trial can be more constrained than a no-card official free tier with a smaller published number. Cerebras, for example, publishes 1,000,000 tokens per day during a $5, 30-day trial, with a card required. Compare the sentence in the allowance field, not only the category badge.

If the card field is "Unspecified" or "Not published", do not treat that as "no card."

What you should never send through a flagged relay

Do not send anything you would not be willing to post in a public forum. In particular:

API keys, passwords, cookies, or private tokens, including keys for other providers
Customer records, health, financial or government identifiers
Confidential source code, unreleased product plans, or internal documents
Personal data about other people
Content that the upstream provider would forbid

If your workflow needs those inputs, use an official provider whose policy you accept, a contract that matches your requirements, or a model you run locally.

If you still want to try a gateway

Use a throwaway prompt and a key that exists only on that site. Do not reuse a production key. Do not point a production app at a new base URL until you have read the terms and privacy policy. Prefer HTTPS base URLs copied from official docs or from a list that only repeats published endpoints. Revoke the key when you are done experimenting.

A first "hello world" request is enough to test compatibility. It is not a reason to point your main repository at that base URL.

If you later migrate from a gateway to an official API, and both are OpenAI compatible, keep the application code and change the environment: key, base URL, and model id. Test streaming, tools and JSON output, because they may differ even when a basic chat call works.

The safe default is the same in every case: treat undocumented routes as public, keep keys in environment variables, and confirm terms on the official page before you rely on a free tier.

Free OpenAI-compatible LLM APIs in 2026: how to switch `base_url` in Python and JS

shiva prasad — Thu, 12 Nov 2026 17:52:16 +0000

Free OpenAI-compatible LLM APIs in 2026: how to switch `base_url` in Python and JS

You do not need a new SDK for every free LLM API. If a provider documents an OpenAI-compatible endpoint, the official Python and JavaScript openai clients will talk to it after you change two things: the base URL and the API key. The model id is a third value, and it has to come from that same provider's docs or dashboard.

This walkthrough uses only figures and endpoints that were published on the providers' own pages, as recorded in the 12 November 2026 snapshot on Free LLM API Tokens. When a quota is not published, I will say so rather than invent one. Always re-check the official page before you plan work around a daily cap; these numbers change.

What "OpenAI compatible" actually promises

Compatibility means the server accepts the same style of HTTP request that OpenAI's API uses — typically a chat.completions call with a model id and a list of messages. It is not a promise that streaming, tool use, vision, batch jobs or embeddings all exist, or that they use the same names.

Trackers that are careful about this will mark a provider as yes, partial, or not published. "Yes" means they document a compatible endpoint. "Partial" means only some of the surface works that way. "Not published" means there is no clear public statement, not that the API is incompatible.

In the 12 November 2026 snapshot, the tracker listed 112 providers. Of those, 105 were marked OpenAI compatible, two were partial, one was not compatible, and four had no public statement recorded. That is a snapshot, not a ranking, and it is not a reason to skip the provider's own docs.

Pick a provider before you write any code

Start with an official provider if you are still learning, or if the prompt might include anything you would not post in public. Official, in the tracker's terms, means a model maker or a cloud that offers the API under its own name. Groq, Google's Gemini API, Cloudflare Workers AI, Mistral, SambaNova and Cerebras are examples in the current list.

A gateway sits in front of one or more models and gives you a single key and a single request format. Some gateways are well documented. Some are unofficial relays. A relay warning on a card means you should not send private data through that site, however large the advertised token cap looks. The 12 November snapshot flagged 47 of 112 entries with that warning, and classified 37 as official and 75 as gateways.

Use the filters for "OpenAI compatible" and "No card needed" on the provider list, then open the official pricing or rate-limit page from the card. If tokens per day are marked "Not published", do not assume a number.

A few published official figures, as of that snapshot:

Groq publishes 200,000 tokens per day and 1,000 requests per day on listed models, with no card required. The documented base URL is https://api.groq.com/openai/v1. Limits vary by model.
Gemini API (Google AI Studio) has a free tier on selected Gemini models, with per-model RPM and RPD on Google's rate-limit page, and no card required. The documented compatible base URL is https://generativelanguage.googleapis.com/v1beta/openai/. The tracker's notes also record that free-tier prompts may be used to improve Google products.
Cloudflare Workers AI publishes 10,000 Neurons per day — that unit is not tokens — and its base URL includes an {ACCOUNT_ID} placeholder that you must replace. Do not treat the placeholder as a hostname.
SambaNova Cloud published 200,000 tokens per day per model as of 3 October, with 20 requests per day, and a card required.
Cerebras publishes 1,000,000 tokens per day during a $5, 30-day trial, with a card required. That is a trial, not an ongoing no-card tier.

Keep the key out of the source

Store the key in an environment variable and read it at runtime. Do not paste it into a chat, a screenshot, a ticket, or a Git repository. If a key leaks, revoke it on the provider's dashboard and create a new one.

# current shell session only
export API_KEY="paste-the-provider-key-here"

On Windows PowerShell the equivalent is $env:API_KEY = "paste-the-provider-key-here". A .env file is fine only if that file is listed in .gitignore.

Python: construct the official client with `base_url`

Install the official openai package, then pass base_url and api_key. Replace the base URL with the exact string from the provider's docs (or from the tracker card, if it repeats a published URL). Replace MODEL_ID with a model id copied from that same provider — not from this article, and not from a different catalogue.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["API_KEY"],
    base_url="https://api.example.com/v1",
)

resp = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Reply with the word ok."}],
)
print(resp.choices[0].message.content)

To point this at Groq, for example, you would set base_url to the published value https://api.groq.com/openai/v1 and copy a model id from Groq's docs. Some base URLs already end in /v1 or /openai/v1. Copy them as published. Do not add a second /v1 unless the official docs say to.

JavaScript: the same two fields, on a server

The official Node SDK uses baseURL (camel case) and apiKey. Do not embed a secret key in a browser bundle; call the provider from a server or a function you control.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.API_KEY,
  baseURL: "https://api.example.com/v1",
});

const resp = await client.chat.completions.create({
  model: "MODEL_ID",
  messages: [{ role: "user", content: "Reply with the word ok." }],
});
console.log(resp.choices[0].message.content);

If you already have application code written for OpenAI, this is often the only change you need, besides picking a model id that exists on the new host.

A raw HTTP request, if you would rather skip the SDK

A compatible server usually accepts a JSON POST to {base URL}/chat/completions with a bearer token. If the published base URL already includes a version suffix, you still append /chat/completions unless the official docs say otherwise.

curl https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"Reply with the word ok."}]}'

If this returns an authentication error, the key or the header name may be wrong. If it returns a model error, the model id is usually wrong or not on the free tier. If it returns a rate-limit error, you have hit a published or unpublished cap.

Why this article will not fill in a model id

Model catalogues change. A name that worked last month may be paid-only, renamed, or removed from a free tier. A tracker can record allowances, compatibility and base URLs without pretending to be a live model directory. Official docs and the provider dashboard are the right source for the string you put in the model field.

The longer, step-by-step version of this setup lives in the site's guide on how to use free OpenAI-compatible APIs.

Test without burning the day's allowance

Start with a one-line prompt and a low max_tokens value if the API supports it. Read the dashboard, or any remaining-quota headers, before you run a loop. Free tiers often count both input and output tokens, and a retry loop can exhaust a daily cap in minutes.

The common failures are practical: a typo in the base URL, a second /v1 on a URL that already has one, an unset environment variable, a model id copied from a different provider, or an offer that has ended since the last snapshot.

Once a test request works, keep client construction in one module, log the provider name and model id (never the key), and add basic backoff on HTTP 429. If you outgrow a free tier, switch the base URL, the key and the model id rather than rewriting the rest of the app — that is the point of compatibility.

DEV Community: shiva prasad

Free LLM APIs for coding assistants

Free LLM APIs for coding assistants

What to configure in the tool

Why free tiers run out faster during coding

Official provider, documented gateway, or relay?

Keep a coding key from leaking

A small script that is easy to retarget

How to evaluate "good for coding" without inventing benchmarks

Official vs relay LLM gateways: risks and how to spot them

Official vs relay LLM gateways: risks and how to spot them

Three kinds of service, not two

Why the distinction matters if the code is identical

What a relay can see

How to read the public signals

A large tokens-per-day number does not offset the risk

What you should never send through a flagged relay

If you still want to try a gateway

Free OpenAI-compatible LLM APIs in 2026: how to switch `base_url` in Python and JS

Free OpenAI-compatible LLM APIs in 2026: how to switch base_url in Python and JS

What "OpenAI compatible" actually promises

Pick a provider before you write any code

Keep the key out of the source

Python: construct the official client with base_url

JavaScript: the same two fields, on a server

A raw HTTP request, if you would rather skip the SDK

Why this article will not fill in a model id

Test without burning the day's allowance

Free OpenAI-compatible LLM APIs in 2026: how to switch `base_url` in Python and JS

Python: construct the official client with `base_url`