DEV Community: Russell Van Curen

Using Codex and Claude Code through one gateway: setup guide

Russell Van Curen — Thu, 15 Oct 2026 14:57:40 +0000

Plenty of teams don't standardize on one coding agent. Some engineers prefer Claude Code, others Codex, and a few switch depending on the task, or on whose usage limit reset last. That's fine until someone asks what the team spent on coding agents last month, and the answer spans two providers, several shared keys and a spreadsheet.

This guide sets up both tools to go through one gateway. Every engineer gets one key with one budget, and every session lands in one set of logs, whichever agent they ran. We use TokenRouter, which we build, but the tool-side config also shows what to look for in any gateway.

Why this works: two protocols, one gateway

Claude Code speaks the Anthropic Messages API (/v1/messages). Codex CLI speaks the OpenAI Responses API (/v1/responses) and supports custom model providers. A gateway that serves both natively can sit in front of both tools, with no wrappers or forks.

TokenRouter serves /v1/messages natively for Anthropic clients, passing requests to Claude models through verbatim. For OpenAI-shaped clients it serves /v1/responses, plus /v1/chat/completions and /v1/embeddings.

Step 1: connect providers, set budgets, create keys

In the console:

Connect providers. Open Provider Keys → Add Provider Key and add each provider you'll use, for example Anthropic and OpenAI. The Free plan connects 2 providers, and paid plans connect all 13.
Invite the engineers. Under Members → Invite member, add each engineer with the Member role and pick their team.
Set a daily hard cap per engineer. Under Budgets → Create budget, set Scope to Member, Period to Daily, a dollar limit, and the hard-cap switch on. Alerts go out at 50%, 80% and 100% by default. Daily budgets reset at 00:00 UTC.
Have each engineer create their own key under API Keys → Create key, with an RPM and TPM limit. Keys a member creates are tied to them, so their member budget covers every key they own. That means one budget covers both tools, whether they use one key for both or one key per tool.

One key for both tools (say agents-alice) is simplest. One key per tool (claude-code-alice, codex-alice) gives you a clean Claude Code vs Codex cost comparison, under the same person's budget.

When a hard cap would be crossed, the request is refused before it reaches the provider with 429 budget_exceeded. Rate-limit breaches return 429 rate_limit_exceeded with a Retry-After header.

Step 2: Claude Code

export ANTHROPIC_BASE_URL=https://api.tokenrouter.io
export ANTHROPIC_AUTH_TOKEN=tr_your_key_here
claude

There's no /v1 on this base URL. Anthropic clients append /v1/messages themselves. ANTHROPIC_API_KEY=tr_... works too. Add both lines to your shell profile.

Verify:

claude -p "Reply with exactly: routed via TokenRouter"

Step 3: Codex CLI

Add a provider to ~/.codex/config.toml. This base URL does include /v1, the opposite of Claude Code:

model = "openai/gpt-5-mini"
model_provider = "tokenrouter"

[model_providers.tokenrouter]
name = "TokenRouter"
base_url = "https://api.tokenrouter.io/v1"
env_key = "TOKENROUTER_API_KEY"
wire_api = "responses"

Then:

export TOKENROUTER_API_KEY=tr_your_key_here
codex "explain what this repo does"

Codex sends stateless Responses calls and resends the context each turn. That matches TokenRouter's Responses implementation, so no previous_response_id is needed.

Step 4 (optional): opencode

opencode takes any OpenAI-compatible provider. In opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "tokenrouter": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "TokenRouter",
      "options": {
        "baseURL": "https://api.tokenrouter.io/v1",
        "apiKey": "{env:TOKENROUTER_API_KEY}"
      },
      "models": {
        "openai/gpt-5-mini": { "name": "GPT-5 mini" },
        "anthropic/claude-sonnet-4-5": { "name": "Claude Sonnet 4.5" },
        "auto": { "name": "TokenRouter Auto" }
      }
    }
  }
}

Step 5: pick models per tool, or let the router pick

Models are addressed as provider/model across all 13 providers, and the same ID works on every endpoint that takes a model field. Any routed model works in Codex's model field, including Anthropic models such as anthropic/claude-sonnet-4-5. TokenRouter translates Responses calls to each provider's native API, so Codex can run against a Claude model through the same key:

model = "anthropic/claude-sonnet-4-5"

On paid plans you can pass auto with a strategy: auto:cost, auto:latency, auto:quality or auto:balanced. The router picks among providers you have keys for and respects the team's model allowlist. The chosen model comes back in the response's model field and shows up in analytics, so attribution stays exact. On the Free plan, auto returns 403 auto_routing_unavailable, so pass an explicit model.

A reasonable pattern: keep agents on explicit models you've tested, and use auto:cost for side jobs like summarizing logs.

Step 6: one place to see spend

Every request from either tool is tied to a key, and a member-created key also ties it to that engineer and their team. Analytics and CSV exports break spend down by team, member, key, model and provider. With separate keys per tool, that's also a direct Claude Code vs Codex cost comparison for your own team. It's more useful than per-token prices, because the same model can cost very different amounts per task depending on the harness.

A rollout checklist for a team

Start with two volunteers, one on each tool, for a week. Use member budgets with the hard-cap switch off, so nobody gets blocked while you learn what normal looks like.
Find each person's heaviest normal day in analytics, then turn the hard cap on with a limit comfortably above it. The cap is there to stop loops, not to ration work.
Set a TPM limit on every key, so a retry loop gets throttled in minutes instead of discovered at the end of the day.
Once you know which models the team actually uses, add a team model allowlist (Model Access).
Write the two env vars and the Codex provider block into your onboarding doc, including the /v1 difference between them.
Offboarding becomes simple: revoke that person's keys and remove them from Members.

Known issues to test before a team rollout

Both tools change fast, and some open issues affect any gateway or custom provider:

Claude Code: the desktop app's Code tab ignores ANTHROPIC_BASE_URL from settings.json (#97574). Any non-default base URL drops message threads and reduces parallel tool calls in headless runs (#98464). /model on resumed sessions may not show proxied models (#98540).
Codex: several open issues report custom-provider sessions failing to archive, or losing the provider's model catalog after an update (openai/codex#48367, #48961, #49227, #50282). Pin versions for team rollouts and recheck after upgrades.

Troubleshooting

404 from Claude Code: check for /v1 on ANTHROPIC_BASE_URL.
403 model_not_allowed: the model isn't on that team's allowlist.
429 budget_exceeded vs rate_limit_exceeded: the first means the request would go past a hard cap for the current period. The second means slow down and retry after the Retry-After delay.
502 upstream_error: the gateway couldn't get a good response from the provider on your key.

We build TokenRouter. The free plan needs no card. Setup guides for Claude Code, Codex, opencode and zcode are in the docs: tokenrouter.io/docs/codex.

Running LLM-as-a-judge on live traffic without blowing your budget

Russell Van Curen — Tue, 13 Oct 2026 14:59:55 +0000

Offline evals tell you a model did well on the examples you thought of. Live traffic tells you what it's actually doing. The obvious way to watch live traffic is to have a strong model grade every response. That's also the fastest way to make evals cost more than the feature they're watching.

This guide covers how to run an LLM judge on production traffic at a cost you won't notice. The principles apply to any setup. Where it helps, we show how it works in TokenRouter's monitors, which we build.

Principle 1: deterministic checks on everything, the judge on a sample

Some failures don't need judgment. A leaked API key matches a regex. A broken JSON response fails a parser. Those checks are cheap, so run them inline on every request and block when they match.

A judge model is for judgment calls. "Does this answer cite a document we don't have?" "Did it follow an instruction injected through a retrieved page?" Those need a model, and they don't need to run on every request.

In TokenRouter these are two separate features. Rules are expressions evaluated in-process on every request or response, with actions like block, redact, transform, route or alert, and an observe mode for rolling a rule out safely. Monitors are the judge, async and sampled.

Principle 2: sample deterministically

Random sampling is fine for averages and bad for debugging. If you rerun a check, or compare Tuesday with Wednesday, you want to judge the same slice. Hash a stable request ID into the decision, so the same request always gets the same answer.

A minimal version:

import hashlib

def should_sample(monitor_id: str, request_id: str, rate_pct: int) -> bool:
    digest = hashlib.sha256(f"{monitor_id}:{request_id}".encode()).hexdigest()
    return int(digest[:8], 16) % 100 < rate_pct

Including the monitor ID means two monitors at 10% don't judge the identical 10%. TokenRouter monitors sample this way: you set the rate per monitor, and the same request always makes the same sampling decision.

Principle 3: never put the judge in the request path

A judge that runs inline adds latency to every sampled request and gives production a new way to fail. Run it after the response. Monitors run seconds after the response, off the request path, so they can't add latency, block a request, or touch a stream.

Principle 4: make each judgment cheap

Three levers set the cost of each judgment:

Model. Start with a small, cheap judge and move up only if it can't do the job. In TokenRouter the judge has to be a tool-capable model, because verdicts come back through a tool call (more on that below), and the model picker only lists those.
Input size. Truncate what the judge reads. Monitors cap the input at about 4k tokens per judgment.
Rubric shape. One narrow question with a yes/no verdict and a reason. A 1-10 score per response is harder to act on and harder to calibrate.

Principle 5: give the judge its own budget

A judge is just more LLM traffic, so it needs a cap like any other traffic. In TokenRouter, each monitor has a daily judge budget. When it's used up, the monitor pauses itself, and the pause clears at midnight UTC. Leave the field empty and there's no cap, so set one.

Judge calls run on your own provider keys with zero markup and go through the gateway like any request: rate-limited, logged with source=monitor, and counted against your organization's default team. A team budget on that team is a second backstop, and the judge's spend sits in analytics next to the traffic it reviewed.

Our docs give a rough figure: at 100,000 requests a month with 10% sampling, a mini-class judge costs roughly $2 to $10 a month of your own provider spend. Your prompt sizes will move that, so watch the source=monitor spend for the first week.

Principle 6: treat the judge as an attack surface

If the judge reads user content, it can be prompt-injected like any other model. Two defenses help. Present the transcript to the judge as data rather than instructions. And make it return its verdict through a forced tool call, not free text. Monitors do both. Even so, no judge is perfect, so the actions are graded:

Flag (always on): the finding goes to a review queue with a severity, categories and the judge's reasoning.
Notify (opt-in): org admins get an email, at most one per cooldown window, with repeat findings batched into it.
Disable key (opt-in, with a severity gate): the offending API key is disabled, admins are emailed, the action is audit-logged, and re-enabling it is one click on the API Keys page.

Treat auto-disable as a circuit breaker for severe cases, not a content filter.

Principle 7: calibrate before you trust the dashboard

Before pointing a judge at live traffic, hand-label 50 to 100 real exchanges, run the judge on the same set, and read every disagreement. Many turn out to be a vague rubric rather than a bad model. Tighten the rubric and repeat. After launch, keep reading the judge's reasoning on each finding. That's how you notice it drifting.

Principle 8: only capture what you mean to judge

You can't judge what you didn't keep, and keeping everything is its own risk. By default TokenRouter stores no prompt or response bodies. Request logs carry token counts, cost and latency only.

Capture is opt-in for a key, a team or a member, and a request is captured when any scope covering it has capture enabled. Captured bodies are compressed and encrypted before they leave the API process. Capture can never fail, slow down or alter a request. Retention is capped by plan: 7 days on Starter, 30 on Team, 90 on Enterprise. You can choose a shorter window, and admins can purge everything at once.

Monitors only read captured traffic, so start by opting in one team or one key, not the whole organization.

Principle 9: turn findings into tests

A flagged answer is the best eval example you'll get. From a captured transcript, Add to dataset turns it into a test case. Dataset items are snapshots, so retention sweeps and purges never break your evals.

Eval runs grade with exact_match, contains, regex, or llm_rubric, a judge scoring against your rubric. That makes "prove a cheaper model scores the same on our traffic, then switch" something you can measure instead of argue about.

Putting it together in TokenRouter

A reasonable first setup:

Turn on capture for one team. Go to Settings → Data capture and add a capture scope (Team, Member or API key). Or use Enable capture on a single key from its menu on the API Keys page. Changes reach the gateway within a minute.
Create a monitor. Go to Monitors → Create monitor and fill in:
- Name
- Scope: Whole organization, Team, Member or API key, plus a target
- Judge model: a cheap, tool-capable one
- Sample rate (%): start at 5 to 10
- What should it watch for?: one narrow, yes/no rubric
Set the Daily judge budget ($).
Leave notifications and key disables off for the first week. Review findings and tune the rubric.
Add the real failures to a dataset.

Plans matter here. Monitors need capture, which starts on Starter. Starter includes 1 judge monitor, Team includes 5, and Enterprise is unlimited. Datasets and eval runs are on Team and Enterprise. The Free plan includes neither capture nor monitors.

We build TokenRouter. If you need deep tracing, experiments and CI gating, dedicated eval platforms do that well, and plenty of setups use both. Our monitor docs are at tokenrouter.io/docs/monitors.

How to give every engineer a Claude Code key with a hard daily cap

Russell Van Curen — Sun, 11 Oct 2026 16:24:52 +0000

If your team runs Claude Code on a company Anthropic API key, two questions come up sooner or later. Who spent what last week? And what stops an agent stuck in a loop at 2am from spending the rest of the month?

With one shared key, the honest answers are "we don't know" and "nothing". The fix is boring and effective: give each engineer their own key, give each engineer a hard daily cap, and check that cap before the request reaches Anthropic. This guide covers why that works, the options you have, and the exact setup with TokenRouter, the gateway we build.

Why a shared key fails

A shared key has three problems.

No attribution. Anthropic's usage console shows spend per key. If everyone uses the same key, you see one number. Anthropic's own Claude apps gateway docs say it plainly: with one shared upstream credential, "your provider's bill attributes everything to that credential, not to individual developers."

No per-person limit. Any limit on a shared key is a limit on the whole team. One runaway session can use up everyone's budget.

Offboarding means rotating the key for everyone.

What a useful cap looks like

A cap is only useful if it does four things:

It blocks. It doesn't just report. An alert that arrives after the money is spent is a receipt.
It's per person, not per company.
It fails in a way tools can see. An agent should get a clear error, not something that looks like a normal reply. There's an open Claude Code issue (anthropics/claude-code#98441) where a plan cutoff arrives as an ordinary assistant turn, so subagents can't tell it apart from a real answer.
It's checked before the call, not after. Claude Code's --max-budget-usd flag is checked after each call returns, and an open issue (#100111) measured a $1 cap stopping at $1.38. That's fine for a single headless run. It's a weak bound for a whole day of agent traffic.

Your options

--max-budget-usd: a per-run bound for headless runs, not a per-person budget.
Claude Team or Enterprise seats: per-member limits for seat users, but not for raw API-key usage in scripts and CI.
Anthropic's Claude apps gateway: self-hosted, with per-user, per-group and org spend limits, for Claude traffic.
A self-hosted proxy with virtual keys and budgets, such as the open-source LiteLLM.
A hosted gateway with per-person budgets, which is where TokenRouter fits.

Setup with TokenRouter

1. Connect your Anthropic key. In the console, open Provider Keys → Add Provider Key, pick Anthropic, and paste the key. Provider keys are AES-encrypted at rest, decrypted only in memory at request time, and never written to logs. Anthropic keeps billing you directly at your rates. We charge a flat subscription and take nothing per token.

2. Create a team and invite engineers. The structure is organization → teams → members → keys. Create a team under Teams → Create team (for example platform or mobile). Then go to Members → Invite member, enter the engineer's email, leave the role as Member, and pick the team. Seats depend on the plan: Free has 1 seat and 1 team, Starter has 5 seats, and Team has 25.

3. Give each engineer a daily hard cap. Open Budgets → Create budget and set:

Scope: Member, then pick the engineer
Period: Daily
Limit (USD): your number (see "Choosing the number" below)
Alert thresholds: 50%, 80% and 100% are preselected, and you can change them
Hard cap: on. The dialog spells out what that means: "Requests beyond the limit are rejected with a 429 until the period resets."

A member budget covers every key that engineer owns. You can also put a team budget on top as an overall ceiling, and a request has to fit under both.

When does "daily" reset? Budget periods use UTC day boundaries, so a daily budget resets at 00:00 UTC. That's 8 PM US Eastern during daylight time and 7 PM in winter. Your organization's time zone setting changes how the console displays time, not when budgets reset.

4. Each engineer creates their own key. Each engineer signs in, opens API Keys → Create key, picks the team, and names it something like claude-code-alice. Keys a member creates are tied to that member automatically, so their member budget applies and analytics attribute spend to them. Keys an admin creates in the console belong to the team but no particular member, so only team budgets apply to those. For per-person caps, let each person create their own.

5. Add a rate limit as a safety valve. The same dialog has RPM limit and TPM limit fields, plus an optional expiry in days. Going over a rate limit returns 429 rate_limit_exceeded with a Retry-After header. A budget catches the slow burn over a day. A TPM limit catches a retry loop within minutes.

6. Optionally, restrict models. Each team can have a model allowlist (under Model Access, or the team's own page). A request for a model outside it fails with 403 model_not_allowed.

7. Point Claude Code at the gateway. On each engineer's machine:

export ANTHROPIC_BASE_URL=https://api.tokenrouter.io
export ANTHROPIC_AUTH_TOKEN=tr_your_key_here
claude

There's no /v1 on the base URL. Claude Code appends /v1/messages itself, so adding /v1 produces /v1/v1/messages and a 404. ANTHROPIC_API_KEY=tr_... also works. Put the exports in your shell profile.

8. Verify.

claude -p "Reply with exactly: routed via TokenRouter"

If the reply comes back, you're routed through the gateway. The request shows up under Logs within seconds, attributed to that engineer's key. Claude requests pass through verbatim, so extended thinking, tools and vision behave as they do with Anthropic directly.

What happens at the cap

The cap is checked before the request is forwarded. The gateway reserves the request's worst-case cost: an estimate of the prompt plus the request's maximum output tokens, priced at that model's rates. If spend so far, plus other requests still in flight, plus that reservation would go over the limit, the request is refused with 429 budget_exceeded and never reaches Anthropic. Once the response finishes, the reservation is replaced with the actual cost, which is computed from exact token counts and the provider's published pricing.

Two practical consequences:

Close to the cap, large requests are refused first, while small ones may still fit.
The prompt part of the reservation is an estimate. Treat the cap as a tight bound, not to-the-cent accounting.

Gotchas to test before rolling out

These apply to any gateway or proxy, including ours:

The desktop app's Code tab currently ignores ANTHROPIC_BASE_URL from ~/.claude/settings.json, while the terminal CLI honors it (anthropics/claude-code#97574).
An open issue reports that any non-default base URL, even a byte-for-byte pass-through, drops message threads and sharply reduces parallel tool calls in headless runs (#98464). If parallelism matters to your workflow, measure before and after.
Plugins that make their own model calls may not send custom headers (#99857).

Choosing the number

There's no universal right daily cap, and we won't pretend there is. A reasonable way to pick one: run a week with the hard-cap switch off (alerts only), look at each engineer's heaviest normal day, then turn the hard cap on with a limit comfortably above that. The cap's job is to stop runaway loops, not to ration normal work. Keep the TPM limit tight enough that a loop can't burn the whole day in minutes.

Offboarding

Revoke that person's keys on the API Keys page and remove them under Members. Everyone else keeps working, and nobody rotates anything.

We build TokenRouter. The free plan needs no card and includes the same budgets and hard caps as paid plans. The Claude Code setup is in the docs: tokenrouter.io/docs/claude-code.

Build AI Agents in Ruby with Just One Gem

Russell Van Curen — Sat, 02 Aug 2025 12:35:10 +0000

OpenAI’s function calling (now called tool_calls) is one of the most powerful ways to extend GPT. It lets the model reason over structured actions and trigger code — giving you the foundation for real AI agents.

But for Ruby developers, using tools meant writing manual JSON schemas, messy parsing, or gluing Python into the mix.

So I built this: openai-toolable

A tiny Ruby gem to make your methods OpenAI-toolable in seconds.

What It Does

openai-toolable gives you a simple way to:

Annotate any Ruby method as a tool
Auto-generate OpenAI-compatible tool schemas
Handle tool calls from OpenAI response JSON
Return function results cleanly — no boilerplate

It’s perfect for building:

AI agents with Ruby backends
GPT-augmented Rails apps
CLI tools, Discord bots, copilots — you name it

Quick Example

require "openai/toolable"
require "openai"

# Set your API key
OPENAI_API_KEY = ENV["OPENAI_API_KEY"]

# Define a tool with required parameters
weather_tool = Openai::Toolable::ToolFactory.build(
  name: "get_weather",
  type: "function",
  description: "Get the current weather in a given location",
  parameters: [
    { name: "location", type: :string, description: "The city and state, e.g. San Francisco, CA", required: true },
    { name: "unit", type: :string, description: "The unit of temperature, e.g. celsius or fahrenheit", required: true }
  ]
)

# Create a tool handler and register the tool
tool_handler = Openai::Toolable::ToolHandler.new
tool_handler.register(
  name: "get_weather",
  lambda: ->(location:, unit:) { puts "Getting the weather in #{location} (#{unit})..." }
)

Now you can generate the tool spec and call the tool from an OpenAI response:

# Create a client
client = OpenAI::Client.new(api_key: OPENAI_API_KEY)

begin
  # Create a chat completion
  response = client.chat.completions.create(
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "What's the weather like in Boston?" }],
    tools: [weather_tool.to_json],
    tool_choice: "auto"
  )

  # Handle the response
  tool_handler.handle(response: response)
rescue StandardError => e
  puts "An error occurred: #{e.message}"
end

It's a full round-trip:
Tool spec → GPT picks it → You call it → Return result.

Features

Define tools inline with your Ruby logic
Fully OpenAI-compatible tool schemas
Works with any GPT-4 / gpt-3.5 tool_calls
Clean, declarative DSL
Supports multiple tools per class

Installation

Add it to your Gemfile:

gem "openai-toolable"

Or install manually:

gem install openai-toolable

Full Docs & Usage

All source code, examples, and docs here: https://github.com/vancuren/openai-toolable

Why Use This?

With OpenAI’s tool_calls, the future is function-aware models.
This gem lets you build GPT agents using just Ruby, without the mess of wiring schemas and handlers manually.

Instead of this:

{
  "type": "function",
  "function": {
    "name": "calculate_something",
    "parameters": ...
  }
}

You just write a method and mark it tool(...).
Simple, readable, and ready to go.

Future Ideas

Tool chaining for multi-step calls
Streaming + parallel calls
Built-in error handling + retry
Optional Rails generators for service objects

Open to feature suggestions! Drop them in Issues or send a PR.

Help Spread the Word

If you find this useful:

⭐ Star the repo
Share it on Twitter/X
Mention it in r/ruby or dev communities
Link it in your AI projects or Rails agents

Let’s make Ruby a first-class citizen in the AI tooling world

Stay in the Loop

I'm building open-source AI infrastructure and Ruby dev tools.
Follow me on GitHub for updates.