DEV Community: llong

Team-Level LLM API Key Management: The Guide I Wish Someone Gave Me

llong — Thu, 29 Oct 2026 06:05:16 +0000

When your app was a side project, one .env file with an OpenAI key was fine. The moment a second engineer joins, that setup turns into a small disaster: keys leak into git, a junior's experiment blows the budget, and no one knows who spent what.

Here's the system we ended up with after burning two budgets and rotating three keys.

1. Never share one key across the team

The instinct is to put the production key in 1Password and call it a day. The problem:

You can't revoke one person's access without killing everyone's
You can't attribute spend
When that key leaks (and it will), you rotate it for the whole team

Give each developer their own key. Scope it down.

2. Separate environments from users

Two axes to think about:

Axis	Values
Environment	dev / staging / prod
Owner	each engineer + CI bots

That's a matrix of keys, not one key. A CI key should never have production write access. An intern's dev key should never work in staging.

3. Budget per key, not per account

OpenAI's account-level limit is too coarse — one runaway loop burns everyone's quota. You want per-key spend caps:

# Pseudocode: what the gateway should enforce
per_key_budget = {
  "dev-llong":      {"monthly": 20,  "models": ["gpt-4o-mini"]},
  "prod-service-a": {"monthly": 500, "models": ["gpt-4o","claude-3-5-sonnet"]},
  "ci-bot":         {"monthly": 5,   "models": ["gpt-4o-mini"]},
}

4. Rotate, but make it cheap

Key rotation should be a 5-minute operation, not a sprint. If every rotation means updating 30 service configs, people stop doing it. A gateway layer with one ANTHROPIC_API_KEY-equivalent that routes to upstream providers lets you rotate the upstream key without touching the app.

5. Audit log everything

You want to know, after an incident:

Which key made the call
Which model and how many tokens
From which service

Without this, "who burned $400 last night?" becomes a he-said-she-said.

6. What we actually use

We ran through Envoy + custom Lua scripts, then a vendor-specific dashboard, before landing on a gateway that does per-key budgets, per-model routing, and a unified audit log out of the box. Agent Token is what we use today — one OpenAI-compatible key, per-key spend caps, team usage dashboard across GPT / Claude / Gemini.

The pattern matters more than the tool, though: one key per person per environment, hard budgets, and an audit log you'd actually check.

How does your team manage LLM keys? Genuinely curious what works for bigger orgs.

Paying for GPT, Claude and Gemini in Southeast Asia: What Actually Works in 2026

llong — Thu, 22 Oct 2026 06:35:29 +0000

If you're a developer in Southeast Asia, getting API keys for OpenAI, Anthropic or Google is only half the fight. The other half is paying for them.

A 2026 survey of 2,719 AI developers across the region found that 81.7% use more than one model platform. Yet local banks' cards routinely fail on OpenAI billing, Anthropic doesn't serve the region at all, and Google Cloud billing needs a corporate entity.

Here's what I've seen actually work — ranked by how much it'll hurt.

Option 1: Use an overseas friend's card

What it sounds like: a favour.

What it is: a permanent, unmonitored liability. Your friend gets surprise charges, has to reconcile what's yours vs theirs, and one disagreement can get the account locked. This works for a weekend prototype. It doesn't work for a product with real users.

Option 2: Virtual / one-shot cards

Services like Wildcard, Dupay and similar virtual card providers market themselves as the answer. In practice:

Cards work for a month or two, then get declined when you try to top up
Top-up fees stack 3-8% on every charge
You end up funding 3 cards in parallel because any single one can die

I burned through two virtual cards before I stopped counting.

Option 3: Go through a reseller / gateway

This is what I landed on. A gateway that bills you in one currency (often with local payment support) and hands out OpenAI-compatible keys upstream:

from openai import OpenAI

client = OpenAI(
    api_key="your-gateway-key",
    base_url="https://api.agentoken.co/v1"
)

# Same SDK. GPT, Claude, Gemini all work behind one key.
resp = client.chat.completions.create(
    model="claude-3-5-sonnet",
    messages=[{"role": "user", "content": "Hi"}]
)

What you trade: a small margin on top of the upstream price. What you get: one bill, one dashboard, local payment, and access to models that are otherwise region-blocked.

Option 4: Build around region-blocked providers entirely

Use only the providers that accept local cards: a few Indonesian and Singaporean cloud LLM offerings, OpenRouter, etc. Cheapest on paper, but your model choice narrows and you still have to wire up each SDK separately.

What I'd actually recommend

If you're solo and just testing: virtual card, expect to switch it every few months.

If you have real users: one gateway that handles billing across GPT, Claude, and Gemini. The small margin pays for itself in hours saved and accounts not getting banned.

Agent Token is the gateway I'm using — OpenAI-compatible, pay-as-you-go, no per-provider card required.

What's your payment setup? Keen to hear what's working for folks in ID, PH, VN, TH.

Designing Failover for LLM Agents: When GPT Goes Down, Claude Shouldn't Notice

llong — Thu, 15 Oct 2026 06:09:58 +0000

Your AI agent just called GPT-4o. OpenAI has an outage. Your agent crashes. Customers notice.

Here's the failover pattern we use in production:

# Simple failover: try provider A, fall back to B
import requests

MODELS = [
    {"provider": "openai", "model": "gpt-4o"},
    {"provider": "anthropic", "model": "claude-3-5-sonnet"},
]

def call_with_failover(messages):
    for provider in MODELS:
        try:
            return call_model(provider, messages)
        except (requests.Timeout, requests.ConnectionError, Exception):
            log.warning(f"{provider} failed, trying next")
            continue
    raise Exception("All providers down")

Why not just retry?

Retries don't help when the entire upstream is down. You need circuit breaking: if OpenAI has failed 3 times in a row, stop calling it for 60 seconds and route traffic to Claude/Gemini instead.

Where to put the logic

You can build this into your app... or you can put it in a gateway in front of your models. Same OpenAI SDK, same code, but the gateway handles:

Health checks on each upstream
Automatic routing to a healthy fallback
Retries with exponential backoff
Per-request logs so you know which model actually responded

from openai import OpenAI

client = OpenAI(
    api_key="your-gateway-key",
    base_url="https://api.agentoken.co/v1"  # gateway handles failover
)

# If gpt-4o is down, gateway auto-routes to claude
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hi"}]
)

Key lesson

Don't build failover into every agent you ship. Put it once in a gateway, and every agent you build after that inherits it for free.

Agent Token runs this kind of routing + failover out of the box, so you don't have to babysit model outages.

Switching from GPT to Claude in 5 Lines: How an OpenAI-Compatible Gateway Makes Model Swaps Trivial

llong — Thu, 01 Oct 2026 06:16:17 +0000

When your production app calls GPT-4o and Anthropic announces a price drop on Claude, you don't want to refactor your whole codebase.

The trick? Point the OpenAI SDK at a different base URL.

from openai import OpenAI

# Before: direct to OpenAI
client = OpenAI(api_key="sk-...")

# After: one key, any model
client = OpenAI(
    api_key="agent-token-key",
    base_url="https://api.agentoken.co/v1"
)

response = client.chat.completions.create(
    model="claude-3-5-sonnet",  # swap to gpt-4o, gemini-1.5-pro, etc.
    messages=[{"role": "user", "content": "Hello!"}]
)

That's it. Same SDK, same request shape, different model behind the scenes.

Why this matters

Most devs build on OpenAI first because the SDK is the most familiar. When you later want to add Claude for long context or Gemini for free tier, you end up maintaining:

3 SDK clients
3 auth flows
3 billing dashboards
3 rate limit handlers

A compatible gateway collapses all of that into one. You change the model string, and the gateway handles routing, retries, and failover.

Try it

Agent Token gives you one key that works across GPT, Claude, Gemini and more, with pay-as-you-go pricing. No per-provider accounts needed — especially useful if you are in SEA where international credit cards are a pain.

Built for devs who just want to ship, not juggle 5 SDKs.

One API Key for Every LLM: Build Once, Call GPT, Claude & More from a Single Endpoint

llong — Sat, 19 Sep 2026 12:11:21 +0000

If you build AI products, you have probably felt this: every model provider ships its own SDK, its own auth, its own billing dashboard. GPT here, Claude there, Gemini somewhere else — and your codebase fills up with provider-specific glue.

It doesn't have to be that way.

In this post I will show the pattern that lets you treat every major LLM as a drop-in replacement behind one OpenAI-compatible endpoint — same code, one key, one bill — and how routing plus automatic failover make it more reliable than pinning your app to a single provider.

Why multi-model is the default now

Most builders do not use a single model anymore. The GenAI Fund 2026 State of AI Builders in Southeast Asia report (2,719 builders) found:

81.7% of builders use more than one AI platform — multi-homing is the default
77.9% use OpenAI, 76.6% use Claude, 61.1% use Gemini — near-parity across the big three
The most common project types: AI agents (18%), automation and workflows (15%), chatbots (10%)

So the real problem is not "which model should I pick" — it is managing five SDKs, five API keys, and five invoices.

The integration mess

Provider	SDK	Auth style	Billing
OpenAI	`openai`	API key	per-token
Anthropic	`anthropic`	`x-api-key` header	per-token
Google	`google-genai`	API key	per-token

Every provider means another dependency, another error-handling path, another rate-limit policy, another team to teach.

The standard answer: OpenAI-compatible endpoints

Most gateways today expose an OpenAI-compatible REST API. If you already use the OpenAI SDK, switching models becomes a two-line change:

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-key",
    base_url="https://your-gateway.example/v1",  # from your gateway dashboard
)

resp = client.chat.completions.create(
    model="claude-sonnet-4-5",  # or gpt-4o, gemini-2.5-pro, ...
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Change the model name, keep the code. Your application does not care which upstream actually answered.

The level-up: routing and automatic failover

A single endpoint that sits in front of multiple upstream accounts gives you two things you rarely get from one provider account:

Automatic failover — if an upstream is rate-limited or down, the gateway retries on another. Your users see no error.
Cost control — usage-based billing with quota limits, plus a dashboard showing exactly what each team or project consumed.

This is the pattern behind Agent Token, an LLM API gateway marketplace: one API key for Claude, GPT, Gemini and more, with smart routing across upstream accounts, automatic failover, pay-as-you-go billing, quota limits, and team usage visibility.

The payment problem (especially in Southeast Asia)

For indie developers and small startups in Southeast Asia, the friction is not only code — it is payments. Many do not have the international credit cards required to open accounts with US-based providers. A unified gateway changes the math: sign up once, top up once, and switch between models without registering each upstream provider separately.

Getting started in 3 steps

Sign up at agentoken.co and create an API key
Point your existing OpenAI-compatible client at the gateway base URL
Set a spending quota so you never get a surprise bill

export OPENAI_API_KEY=sk-xxx
export OPENAI_BASE_URL=https://your-gateway.example/v1   # from your dashboard

Done — the same code now talks to GPT, Claude, Gemini, and more.

A note on responsible use

Before routing heavy production traffic through any gateway, review the upstream providers' terms of service, and keep your own keys and quotas tight. A gateway is a tool for reducing integration overhead — not for bypassing a provider's rules.

When you should NOT use a relay

You need a specific provider's enterprise SLA or DPA → go direct.
You handle sensitive data with strict residency requirements → verify where the gateway routes.
Tiny hobby project with a single model → direct access is fine.

Wrap up

Multi-model is the default. The gateway pattern removes SDK and billing sprawl, adds failover, and is one base_url change away from your current code.

If you are building agents, chatbots, or automation in Southeast Asia and want one endpoint for multiple models, give Agent Token a try — and drop a comment about what you are building. I read them all.

Low‑cost LLM Token Access for Southeast‑Asia Developers

llong — Wed, 16 Sep 2026 11:26:16 +0000

About Agentoken: https://www.agentoken.co

Agentoken is a global LLM token marketplace and unified API gateway built for developers, especially targeting Southeast‑Asia indie developers and small‑size startups.

👉 Website: https://www.agentoken.co

Core capabilities:

Multi-model access in one dashboard, switch models easily
Pay‑as‑you‑go pricing based on actual input & output token consumption
Prompt caching support to reduce costs for multi‑turn conversations and AI agents
Visual dashboard for real‑time token usage, call logs and expense auditing

Who is it built for:

Indie developers building chatbots and AI agent prototypes
Small startup teams running AI services in Southeast Asia
Developers evaluating different LLMs without registering every upstream provider separately

Closing thoughts

For Southeast‑Asia developers, you no longer need to maintain dozens of API accounts to test different LLMs. Agentoken puts Xiaomi MIMO, OpenAI GPT, Grok, Zhipu GLM and more under one compatible endpoint.

If you are building AI‑related projects, feel free to try out Agentoken and leave your feedback in comments.``