DEV Community: nRouter

One API key, every language in your stack: notes on nRouter's SDK setup

nRouter — Fri, 13 Nov 2026 07:37:43 +0000

Disclosure: I work on nRouter. This post was drafted with AI assistance and reviewed by me before publishing.

Most teams I talk to don't run one language. There's a Python service doing the data work, a TypeScript API layer, maybe a Go worker for the high-throughput parts, and a mobile app in Swift or Kotlin. Every one of those needs to call an LLM, and without a gateway each one grows its own provider key, its own rotation schedule, and its own little pile of "which secret goes where" documentation.

That's the problem I keep coming back to when I read our own SDKs page, so here's the practical version of what it sets up.

One key, configured once per service

Every official nRouter SDK reads the same environment variable:

NROUTER_API_KEY=<your virtual key>

...and connects to the same endpoint, https://api.nrouter.ai/v1. Python service, Go worker, Swift app — different clients, one credential lifecycle. Issuing access is one action. Rotating is one action. Revoking a key when someone leaves actually works, because there's nowhere else for access to hide.

The clients, per language

Each SDK ships as a native package in its ecosystem's registry:

Python — pip install nrouter-sdk
TypeScript / Node — npm install @nrouter_ai/sdk
Go — go get the sdks/go/v3 module
Rust — cargo add nrouter
Java / Kotlin / Android — Gradle, group ai.nrouter
Swift — Swift Package from the GitHub repo
Dart — dart pub add nrouter
R — the nrouter package

Every row on the page has its own guide, and a few link to runnable demos on GitHub, which is the part I'd copy if I were documenting my own platform. The distance between "installed" and "first successful call" is where integrations quietly die.

Already on the OpenAI client? Don't rewrite anything

The rows I find most honest in the table aren't ours. The standard OpenAI Python client is listed, plus the PHP and Ruby clients, and plain cURL. The deal: point your existing OpenAI or Anthropic client at the nRouter endpoint, keep your call sites exactly as they are. The gateway speaks both wire formats, so the move is a config change, not a refactor.

That ordering matters. Adopt at the configuration layer first, then let the official SDKs earn their way in per service where the typing and guides help.

Why the clients stay thin

One line on the page does the architectural work: every SDK supports automatic cost tracking, server-side guardrails, and centralized fallback routing. Server-side is the operative word. Cost tracking, guardrails and fallback behaviour are gateway properties the clients inherit, not features ten libraries reimplement in ten languages with ten release cadences.

If each client rolled its own retry and accounting logic, "what happens when a provider fails" would have a different answer per service. With thin clients, parity is structural — it can't drift, because there's only one implementation, and it's not in your codebase.

Full table, guides and install commands: https://nrouter.ai/docs/sdks

If you're running a polyglot stack, I'd genuinely like to hear how you handle the key sprawl today — one gateway key, per-service provider keys, a secrets manager pattern? The comments are the interesting part of posts like this.

Before you scale an agent, make every tool call answer these 5 questions

nRouter — Tue, 10 Nov 2026 12:46:20 +0000

A demo agent is easy to trust: you watched it work. A production agent is different because nobody watches the 3 a.m. run that refunds the wrong order, retries the wrong request, or calls a tool with almost-right arguments.

Before an agent is allowed to act at scale, every tool call should be able to answer five questions later, from logs alone:

Who or what authorized this action? A user request, a scheduled job, another agent, or a policy rule should be distinguishable after the fact.
What exactly reached the model? Keep the raw input next to the extracted intent. When the summary and the source disagree, the source is how you debug it.
Which route actually served it? Record the model and provider that served the final attempt, not just the first choice in the config. Failover makes "what we asked for" and "what answered" different facts.
What changed in the world? A tool call that reads is not the same risk as one that sends, pays, deletes, or updates. The log should make the blast radius obvious.
Can this be replayed or undone? If the answer is no, the action needs a tighter gate: confirmation, a value threshold, or a human review queue.

The pattern I keep coming back to: treat the request log as evidence, not exhaust. If a bad action cannot be reconstructed, the team does not really have observability — they have a dashboard and a hope.

What is the one log field you added only after an agent incident taught you the hard way?

We keep collecting these patterns in public — longer notes on Hashnode, events on lu.ma.

What makes the best LLM gateway? The criteria we'd test any gateway against

nRouter — Wed, 04 Nov 2026 13:19:56 +0000

Every "best LLM gateway" list we've read ranks products by feature checklist: routing ✓, caching ✓, guardrails ✓. Checklists are where evaluation goes to die — every gateway claims every feature, and the differences that decide whether your production traffic survives a bad Tuesday live one level down, in behaviour you can only see by testing.

We build nRouter, so apply whatever discount you like to our inclusion in the set below — but the criteria are the point, and they're testable against any gateway, including ours. Here's the test bench we'd run before trusting one with real traffic.

1. Failover you can prove, not a diagram

The test: kill the primary provider mid-request in staging — a hard failure, not a graceful one — and watch three things: does the client see an error, how much latency did the fallback add, and did the fallback go where your policy said it would?

What separates gateways: whether fallback chains are explicit policy (this model, then that provider, then degrade) or a provider's internal behaviour you can't inspect. Silent substitution is the failure mode to hunt: the request "succeeds," but on a different model than the one your evals approved.

2. Retry discipline

The test: force 429s and 500s and count what the gateway actually does. Which statuses does it retry? With what backoff? Does it respect idempotency — or will it happily re-run a non-idempotent call and bill you for the duplicate?

A gateway that retries everything is a cost incident waiting for a trigger. A gateway that retries nothing is a reliability feature that exists only in the brochure. The behaviour table should be documented, and your staging run should match it.

3. Budget enforcement in the request path

The test: set a small cap on one key, then deliberately exceed it. What happens — and when?

Alerts are not enforcement. Dashboards are not enforcement. Enforcement is the request being refused before the spend happens, per key and per team, with a refusal you can distinguish from a provider error (a budget 429 and a provider 429 must never look identical in your logs). If the cap only exists as a chart, a runaway loop will find the difference before you do.

4. A request log that works as an audit trail

The test: pick one request from last week and reconstruct it fully: which model actually served it, what it cost, how long each attempt took, whether a fallback fired, which key made it.

If you can't answer all of that from the log, the gateway is a black box you're renting blind. Two sub-tests: prompts and outputs need redaction controls (logs are where PII goes to be forgotten about), and the log must be exportable — an audit trail you can't take with you isn't yours.

5. Keys and access control that match how teams actually work

The test: one app gets compromised keys at 2 a.m. How fast can you revoke exactly those keys — without touching any other team's traffic?

One shared key for a whole organisation fails this by construction. Look for virtual keys scoped per app, environment and team, with revocation measured in seconds. Then check the boring part: can a key be limited to specific models, or is every key an all-access pass?

6. Guardrails placed in the path, at a known cost

The test: run a prompt-injection and PII suite through the gateway and measure two numbers — what it catches, and the latency it adds per call.

Every serious gateway offers checks of some kind now (OpenRouter documents its own, as do we); the real differences are placement and price in milliseconds. A check bolted on as a separate service adds a hop and a failure mode. A check in the request path adds neither — but should tell you exactly what it costs.

7. An exit you can afford

The test: if you left this gateway next quarter, what breaks?

The honest answer should be "one base-URL change." An OpenAI-compatible surface, standard auth, your logs exportable, your prompts portable. The best gateway is confident enough to be easy to leave — lock-in is a business model, not an architecture.

The set we'd run these against

Run the bench on the gateways you're actually considering. Ours would include: OpenRouter (a marketplace model — broad catalogue, provider choice as the product), Portkey (observability-led, deep analytics heritage), LiteLLM (self-hosted and open source, total control at the cost of running it yourself), and nRouter (a managed gateway built around routing policy, in-path budget enforcement and audit-grade logging). Different architectures optimise for different criteria above — which is exactly why the checklist version of this decision keeps failing people. Run the tests. The behaviour table doesn't care whose logo is on it.

What did we miss? If you've production-tested a gateway, which criterion decided it — and which one on this list would you strike?

Deeper version of the gateway's job description: What is an LLM gateway?

What's the right shape for an API trial: one lump of credit, or a small daily allowance?

nRouter — Mon, 02 Nov 2026 11:40:43 +0000

We've been going back and forth on this while designing a trial for an API product, and I keep flip-flopping — curious how others have settled it.

Option A: the full trial credit up front. Simple to explain, feels generous, and a serious evaluator can run a real workload on day one. Downside: it's a magnet for farming — scripted signups drain it in minutes — and a genuine user can burn it accidentally in one bad retry loop and conclude the product is expensive.

Option B: the same total, released as a small daily allowance. Farming gets slow and boring, real users get a reason to come back several days in a row, and support sees problems while there's still trial left to fix them. Downside: a serious evaluator hits the ceiling mid-benchmark on day one and walks away annoyed.

The middle paths I've seen: a daily allowance that can burst once the account verifies more identity; an allowance that grows after the first successful integration; and separate lanes entirely — a restricted trial model instead of the full catalog.

What I actually want to know:

If you've run a trial like this: which shape survived contact with real users, and what did the abuse look like?
If you've used API trials as a developer: which shape made you take a product seriously, and which made you close the tab?

Asking because we're about to lock ours in, and I'd rather steal the right answer than learn it the expensive way.

What's your rule for when an LLM call should NOT be retried?

nRouter — Tue, 27 Oct 2026 06:16:41 +0000

Everyone has a retry story. Mine: an agent that retried a malformed request eleven times before anyone noticed, each attempt replaying the full conversation. The request was never going to succeed — a 400 doesn't heal — but the retry policy didn't care, because it only looked at "did it throw?"

Our working rules now:

Retry timeouts, 429s and 5xx — with backoff, jitter, and the server's Retry-After winning over our schedule.
Never retry 400/401/403. Fail fast, surface it, fix the request.
A retry budget, not just a count: retries get a slice of the task's total budget, then the task fails honestly.
If it dies mid-stream, restart the message cleanly — never append to a half-generated answer.

What I haven't settled: provider 529s / overloaded errors during peak. Retry in place, or fail over to a different model immediately and accept the quality change?

What's your rule set — and which error taught it to you?

Helicone vs nRouter: The Request-Path Test Every LLM Gateway Evaluation Should Run

nRouter — Thu, 08 Oct 2026 05:28:44 +0000

Helicone vs nRouter: The Request-Path Test Every LLM Gateway Evaluation Should Run

Here's a test I now run on every LLM infrastructure evaluation, and it takes about thirty seconds: can it refuse a request?

Not log it. Not chart it. Refuse it — before the provider call happens, before a token is billed.

Why this test exists

Helicone is an observability product. It offers a request-path proxy and an async logging SDK, and the async path is the one to understand: it records calls after they happen. By construction, nothing off the request path can refuse a request.

That's fine for analysis. It fails the test above. If the guardrail you want lives on the path you're not using, no plan upgrade changes physics.

nRouter is an inline gateway — every request passes through it. Preflight checks (guardrails, hierarchical budget enforcement) run before the provider call. An exhausted budget returns 402/429 before a token is billed; blocked guardrail requests cost zero credits.

from openai import OpenAI

# nRouter: OpenAI-compatible endpoint, one managed key
client = OpenAI(base_url="https://api.nrouter.ai/v1", api_key="nr-...")
chat = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Summarize this thread."}],
)

The migration from Helicone's proxy is a baseURL swap: drop the provider key and the Helicone-Auth header, use one nRouter key.

The plan-ladder test

Second test: which plan are the features on?

On Helicone's published pricing (checked 2026-08-23), guardrails, custom rate limits, evals/scores, prompt management, sessions, and alerts all begin on Pro at $79/month. SOC-2 and HIPAA begin on Team at $799/month. Helicone ships these features — the difference is which plan they're on. The first time you need to block something rather than watch it, the answer is a plan change.

nRouter puts guardrails and hierarchical budgets (org → team → user → key; daily/weekly/monthly/lifetime; block/warn/throttle) on every plan, with a 0% platform fee on credits as the commercial lever. Entry is a $5 minimum credit purchase, card required — no free tier.

The retention test

Third: how long do the logs live? Helicone Pro: 30 days. Team: 90 days. Enterprise: unlimited. A 30-day window plus a 60-calls-per-minute export API means your quarterly cost review needs an export pipeline before the window closes. nRouter keeps a ledgered spend history — good for cost questions, thin for debugging. Neither is a warehouse; plan accordingly.

The BYOK test

Fourth: whose keys pay? Helicone supports BYOK — your provider key keeps your provider billing, so provider credits and negotiated rates stay intact — plus a 0%-markup credits path. nRouter has no BYOK: one managed key, nRouter holds the upstream credentials. Provider-granted credits and negotiated rates cannot be spent through nRouter. This is the architectural position, not a gap that closes. If your finance team negotiated provider rates, this single test decides the shortlist before any feature matrix.

When to stay on Helicone

Be honest about it. Stay on Helicone if you need session replay, HQL trace querying, datasets, BYOK economics, Google models (nRouter doesn't serve them), or certified SOC-2/HIPAA today (nRouter's SOC 2 Type II is in progress). Move to an inline gateway when your next three needs are guardrails, evals, and per-team budgets on every plan.

Every claim on both sides, traced to the vendors' own published pricing and docs, is in the full Helicone comparison.

Originally published as part of nRouter's documented Helicone comparison.