DEV Community: AI ROUTER

How to estimate ChatGPT API costs before moving a coding agent to production

AI ROUTER — Tue, 01 Sep 2026 17:20:45 +0000

How to estimate ChatGPT API costs before moving a coding agent to production

Subscription pricing and API pricing are different products. A coding agent can feel inexpensive in a chat subscription and still produce a very different bill when it starts sending long context, tool results, retries, and parallel requests through an API. The safest migration is to measure the workflow first, then choose an endpoint and package that match the observed usage.

This checklist is provider-neutral. It works for a direct provider endpoint or an independent relay that exposes an OpenAI-compatible interface.

1. Measure a real task, not a hello-world request

Capture a small sample of the tasks your agent actually performs:

input tokens, including system instructions and retrieved files
output tokens, including tool arguments and structured responses
model ID and whether the request uses Chat Completions or Responses
number of tool calls and the size of each tool result
retries, timeouts, and requests that were cancelled halfway through a stream

The basic estimate is:

estimated cost = input tokens × input rate
               + output tokens × output rate
               + cache/tool charges, if applicable
               + retry and background-job cost

Do not compare a subscription allowance with a token rate as if they were the same unit. Record the units beside every number in your spreadsheet or dashboard.

2. Treat retries as part of the workload

A failed request is not automatically free. A retry may resend the full conversation, and a tool loop can multiply the context several times. Track a request ID and an attempt number so that you can answer three questions:

How often did the first attempt fail?
How many tokens were sent again?
Did the second attempt return a complete result or only a partial stream?

For streaming clients, persist only safe metadata such as status, duration, model ID, and token counters. Never put API keys or authorization headers in logs.

3. Verify compatibility before changing the base URL

“It returned HTTP 200” is not a complete compatibility test. A useful smoke test checks the exact path your application needs:

authenticated model discovery, such as GET /v1/models
the request shape used by your SDK
streaming delimiters and the [DONE] event, if your client streams
tool calls, JSON arguments, and finish reasons
distinct handling for 401, 403, 404, 408, 429, and 5xx responses
usage visibility in the provider dashboard

Run the same fixture against both endpoints and compare the parsed fields, not just the HTTP status. Keep the model ID account-specific; a model name shown in one account may not be enabled in another.

4. Keep the application change small

For an OpenAI-shaped client, the first experiment should usually change only the key and base URL:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AI_ROUTER_API_KEY"],
    base_url=os.environ.get("AI_ROUTER_BASE_URL", "https://api.ai-router.dev/v1"),
)

response = client.chat.completions.create(
    model=os.environ["AI_ROUTER_MODEL"],
    messages=[{"role": "user", "content": "Run one compatibility smoke test."}],
    timeout=30,
)

print(response.choices[0].message.content)

Keep the key in an environment variable or secret manager. Start with a small quota, set an application-side budget, and make the base URL configurable so that rollback is one deployment setting rather than a code rewrite.

5. Decide whether a relay is useful for your workflow

An independent relay can be useful when a team wants one API-key workflow, usage visibility, package-based spending limits, or access to more than one model family through a consistent integration surface. It is still a separate service: verify its current model catalog, limits, data handling, support path, and prices before sending production traffic.

Disclosure: I work on AI-ROUTER, an independent service that provides a ChatGPT and Claude API relay. It is not OpenAI or Anthropic, and this article is not an endorsement by either provider. Developers can review the current endpoint, package information, and account controls on the ChatGPT and Claude API relay homepage.

6. Use a go/no-go checklist

Before moving a coding agent beyond a small trial, confirm:

the observed input/output mix fits the budget you set
retry and tool-call behavior has been measured
streaming and structured-output fixtures pass
model IDs are available to the account that will run the job
API-key usage and balance can be inspected without exposing secrets
a timeout, retry limit, and rollback URL are configured
the team knows which service owns the endpoint and where to report an outage

This approach gives you a defensible cost estimate and a reversible migration path. It also prevents the common mistake of choosing an API solely because its headline price looks lower while the actual agent workload, retries, and limits remain unknown.

Как быстро проверить ChatGPT API в существующем OpenAI-клиенте

AI ROUTER — Fri, 28 Aug 2026 09:31:52 +0000

Если у вас уже есть приложение или скрипт с OpenAI-compatible клиентом, для первого теста часто достаточно заменить endpoint и ключ, не переписывая всю интеграцию.

Практический маршрут:

Зарегистрировать аккаунт.
Выбрать небольшой дневной или недельный пакет либо пополнить баланс.
Создать собственный API key.
Указать endpoint и key в клиенте согласно документации.
Выполнить тестовый запрос и проверить расход в кабинете.

Начните с небольшого объёма: так проще проверить именно ваш сценарий и фактическое потребление. Документация с примером первого запроса: https://ai-router.dev/ru/docs/getting-started/first-request/?utm_source=devto&utm_medium=community&utm_campaign=ru_activation_2026q3

AI-ROUTER — сторонний сервис-посредник для ChatGPT API с OpenAI-compatible способом вызова. Это не официальный сервис OpenAI. Доступные модели, лимиты, сроки действия пакетов и условия указаны на актуальных страницах сервиса.

OpenAI-compatible API: как проверить endpoint, лимиты и первый запрос

AI ROUTER — Fri, 28 Aug 2026 04:57:49 +0000

Короткий путь к первому запросу

При подключении ChatGPT API к прототипу чаще всего ломается не модель, а конфигурация: лишний /v1, неверный model ID, потерянный заголовок Authorization или неожиданный лимит тарифа.

Ниже — небольшой чек-лист для Python, Node.js, curl и других клиентов с OpenAI-compatible интерфейсом.

1. Зафиксируйте четыре значения

Перед запуском запишите:

базовый URL;
имя переменной с API key;
точный model ID из доступного каталога;
квоту и срок действия выбранного тарифа.

Если библиотека сама добавляет /chat/completions и /models, ей обычно нужен корневой адрес:

https://api.ai-router.dev

Если клиент ожидает OpenAI SDK Base URL и сам формирует /v1/chat/completions, используйте:

https://api.ai-router.dev/v1

Не добавляйте /v1 дважды: путь /v1/v1/chat/completions даст 404 даже при правильном ключе.

2. Проверьте каталог и авторизацию

Храните ключ только в переменной окружения или секрет-хранилище:

export AI_ROUTER_API_KEY="замените-на-свой-ключ"

curl https://api.ai-router.dev/models \
  -H "Authorization: Bearer $AI_ROUTER_API_KEY"

Смотрите на фактическое поле data[].id. Не угадывайте имя модели по названию тарифа и не вставляйте ключ в репозиторий, issue, URL или frontend bundle.

3. Сделайте минимальный smoke-test

curl https://api.ai-router.dev/chat/completions \
  -H "Authorization: Bearer $AI_ROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4-mini",
    "messages": [
      {"role": "user", "content": "Ответь одним предложением: соединение работает?"}
    ],
    "max_tokens": 64
  }'

В рабочем приложении используйте model ID из каталога, доступного именно вашему ключу и тарифу.

4. Обрабатывайте ошибки по классам

401 — проверьте ключ и заголовок Authorization.
404 — сравните фактический URL с правилом /v1.
429 — проверьте квоту, баланс, RPM/TPM и Retry-After; ограничьте параллелизм, а не запускайте бесконечный retry.
Обрыв stream — не доказывает, что запрос не был принят. Повторяйте только операции, для которых replay действительно безопасен.

После первого запроса откройте usage, баланс и срок действия в кабинете. Начинайте с небольшого дневного или недельного объёма, чтобы проверить именно свой workflow.

5. Что проверить перед production

Каталог возвращает ожидаемый model ID.
Короткий запрос завершается успешным ответом.
401/404/429 попадают в понятные метрики.
Таймауты и обрывы не вызывают небезопасные повторы.
Для разных приложений используются отдельные ключи.
Квота и срок тарифа соответствуют реальной нагрузке.

AI-ROUTER в этом примере — независимый ChatGPT API relay с OpenAI-compatible способом вызова, а не официальный сервис OpenAI. Актуальные русскоязычные примеры endpoint и первого запроса: https://ai-router.dev/ru/docs/getting-started/first-request/?utm_source=devto&utm_medium=content&utm_campaign=ru_api_first_request_202608

Если у вас есть другой совместимый клиент, полезно начать с тех же двух проверок: какой URL он формирует и какой model ID реально видит ваш API key.

Safe Retries for OpenAI-Compatible APIs: HTTP Errors, Retry-After, and Partial SSE Output

AI ROUTER — Tue, 28 Jul 2026 11:17:56 +0000

Retrying an API request sounds simple until the request generates text, invokes a tool, or consumes billable model capacity. A disconnected POST /v1/chat/completions is different from a failed GET /v1/models: the server may have accepted and executed the generation before the client saw an error. A stream that has already emitted output is different again, because replaying the whole request can duplicate visible text, tool calls, side effects, or cost.

This guide builds a conservative error and retry boundary for OpenAI-compatible APIs in Node.js and TypeScript. It covers four questions:

What kind of failure occurred?
What diagnostic data can be logged without retaining request or response bodies?
Did a streaming response already produce meaningful output?
Does the caller know that replay is safe within its attempt and time budgets?

The goal is not to build another API client. It is to make the decision inputs explicit before any retry loop runs.

The examples use the published @ai-router/openai-compatible-errors@0.1.0. Its source, tests, and release workflow are public on GitHub.

Why `retryable: true` is incomplete

Most retry logic starts with status codes:

Retry 429 because it indicates rate limiting.
Retry 502, 503, and 504 because an upstream service may recover.
Retry a transport timeout because the network may recover.
Do not retry 400 or 401 because the request or credential needs correction.

Those are useful error classifications, but they do not establish replay safety.

Consider a generation POST that times out while awaiting response headers. Three different events can look identical to the caller:

The request never left the process.
The request reached a proxy, but the proxy never sent it upstream.
The model completed the request, while the response was lost on the return path.

Only the first case is obviously safe to replay. A generic network error often cannot distinguish the three. Returning retryable: true collapses “the failure may be transient” and “repeating the operation is safe” into one dangerous bit.

A safer model separates them:

type Transience = "transient" | "permanent" | "unknown";
type ReplaySafety = "safe" | "unsafe" | "unknown";
type RetryAction = "retry" | "do_not_retry" | "manual_decision";

An automated retry should require both a transient error and explicit replay safety. Unknown evidence should stay unknown.

Classify the error before planning a retry

OpenAI-compatible services can expose failure evidence in several places:

HTTP status: 401, 403, 408, 413, 429, 5xx.
JSON body: error.message, error.type, error.code.
Headers: Retry-After, x-request-id, request-id, correlation IDs.
SDK error: status, statusCode, request_id, responseHeaders.
Transport exception: AbortError, timeout, DNS, connection reset.
SSE event: { "error": ... } or a Responses-style response.failed event inside an HTTP 200 stream.

Status alone is not always enough. A 429 can mean a temporary request-rate limit or an exhausted quota that will not recover after a two-second delay. Check structured error codes and types before assigning the category.

A useful taxonomy can remain small:

Category	Examples	Transience
Authentication	401, invalid API key	Permanent
Permission	403, access denied	Permanent
Rate limit	429 + rate-limit signal	Transient
Quota	insufficient quota, billing/credit signal	Permanent
Validation	400/409/422	Permanent
Timeout	408, timeout exception	Transient
Network	DNS, connection reset, Fetch failure	Transient but replay may be unknown
Upstream	502/503/504	Transient but replay may be unknown
Stream	premature EOF	Transient only before output and when replay is safe
Unknown	insufficient evidence	Unknown

The category should be a stable application contract. Keep raw vendor strings out of that contract.

Reproduce the decision without an API key

The following fixture runs entirely on loopback. It returns a synthetic 429 with Retry-After: 2, then evaluates the same error twice: once when replay is explicitly safe, and once when replay safety is unknown.

npm install @ai-router/openai-compatible-errors@0.1.0

import { once } from "node:events";
import { createServer } from "node:http";
import {
  decideOpenAICompatibleRetry,
  normalizeOpenAICompatibleResponse,
} from "@ai-router/openai-compatible-errors";

const server = createServer((_request, response) => {
  response.writeHead(429, {
    "content-type": "application/json",
    "retry-after": "2",
    "x-request-id": "req_loopback_demo",
  });
  response.end(JSON.stringify({
    error: {
      type: "rate_limit_error",
      code: "rate_limit_exceeded",
      message: "Synthetic fixture; excluded from the result by default.",
    },
  }));
});

server.listen(0, "127.0.0.1");
await once(server, "listening");
const { port } = server.address();

try {
  const startedAt = Date.now();
  const response = await fetch(`http://127.0.0.1:${port}/v1/demo`, {
    method: "POST",
  });
  const error = await normalizeOpenAICompatibleResponse(response);
  if (!error) throw new Error("Expected a synthetic API error");
  const context = {
    method: "POST",
    phase: "http_error",
    attempt: 1,
    elapsedMs: Date.now() - startedAt,
  };

  console.log({
    error,
    safePlan: decideOpenAICompatibleRetry(error, {
      ...context,
      replaySafety: "safe", // Safe only for this side-effect-free fixture.
    }),
    unknownPlan: decideOpenAICompatibleRetry(error, {
      ...context,
      replaySafety: "unknown",
    }),
  });
} finally {
  await new Promise((resolve) => server.close(resolve));
}

The safe result is retry with a 2,000 ms server-directed delay. The unknown result is manual_decision, even though both plans are based on the same transient 429. A complete, defensively checked copy of this fixture is available in the repository examples.

Build diagnostics that are safe by construction

Logging the original SDK error is convenient, but it can retain more than a status and message. Depending on the SDK and wrapper, the object may contain full headers, request body values, response text, nested causes, or a stack whose message includes provider text.

Redaction after the fact is fragile. A stronger default is to construct a new error object that never stores those fields:

interface SafeApiError {
  name: "OpenAICompatibleError";
  message: string; // A fixed message owned by the normalizer.
  category: string;
  source: "http" | "fetch" | "openai_sdk" | "ai_sdk" | "sse" | "unknown";
  status?: number;
  code?: string;
  type?: string;
  requestId?: string;
  retryAfterMs?: number;
}

The default message should be something predictable such as Rate limit exceeded, not a copy of error.message. Provider messages can echo an API key, prompt fragment, file name, customer identifier, or upstream HTML. If a provider message is genuinely needed, make it an explicit opt-in, redact common credential forms, truncate it, and document that secret redaction is not PII anonymization.

For arbitrary diagnostic objects, use a bounded sanitizer:

Never invoke user-defined getters or toJSON hooks while traversing unknown objects.
Detect circular references.
Limit depth, node count, array items, object keys, and string length.
Replace oversized strings rather than preserving a prefix that might contain a partial secret.
Redact authorization, API-key, cookie, password, token, prompt, message-list, and request/response body fields.
Redact Bearer/Basic values, known key shapes, JWTs, URL userinfo, and sensitive query parameters in free text.

Even a strong sanitizer is defense in depth. The best way to protect a prompt is not to send it to the logger.

Honor `Retry-After` without shortening it

HTTP defines Retry-After as either a delay in seconds or an HTTP date. Some SDKs and gateways also expose a millisecond hint such as retry-after-ms.

function parseRetryAfter(value: string, nowMs = Date.now()): number | undefined {
  const trimmed = value.trim();
  if (/^\d+$/.test(trimmed)) {
    const seconds = Number(trimmed);
    return Math.round(seconds * 1_000);
  }

  const targetMs = Date.parse(trimmed);
  if (Number.isNaN(targetMs)) return undefined;
  return Math.max(0, targetMs - nowMs);
}

Do not clamp a one-hour server delay down to a local ten-second maximum and retry early. If the server hint exceeds the caller's single-delay or total-time budget, the safe result is to stop retrying. A local maximum is a budget, not permission to ignore the server's lower-bound advice.

When there is no valid server hint, capped exponential backoff with jitter is reasonable:

const cap = Math.min(baseDelayMs * 2 ** (attempt - 1), maxDelayMs);
const delayMs = Math.round(Math.random() * cap); // Full jitter.

Check the attempt budget and elapsed-time budget before returning a retry plan. Count the initial request as attempt one so the policy is unambiguous.

Treat SSE output as a replay boundary

Server-sent events are framed as lines. Multiple data: lines form one event, events end with a blank line, and a UTF-8 character or JSON object can be split across arbitrary network chunks. Searching each chunk for "error" is therefore not a parser.

A stream inspector needs incremental state:

interface StreamState {
  eventsSeen: number;
  malformedEvents: number;
  hasOutput: boolean;
  done: boolean;
  unexpectedEof: boolean;
  error?: SafeApiError;
}

For Chat Completions, meaningful output includes non-empty choices[].delta.content, tool-call deltas, or function-call arguments. For Responses-style streams, output text or function-call argument delta events cross the same boundary.

Once hasOutput is true, a whole-request retry should default to do_not_retry. This remains true even if the terminal error is a normally transient 429 or 503. Transience describes the failure; it does not erase partial output.

An early SSE error can arrive inside an HTTP 200 response:

event: error
data: {"error":{"code":"rate_limit_exceeded","message":"..."}}

The error contract should normalize this event the same way as a non-2xx JSON body while recording source: "sse". It should not retain the raw event.

Premature EOF also needs an explicit state. If the stream emitted data but never sent [DONE] or a completion event, mark unexpectedEof. Whether that is retryable still depends on hasOutput and replay safety.

A fail-closed retry decision

The decision order matters. Evaluate the strongest reasons to stop first:

Caller abort: do not retry.
Partial SSE output: do not retry the whole request.
Completed request: do not retry.
Permanent error: do not retry.
Replay explicitly unsafe: do not retry.
Unknown phase or replay safety: require a manual decision.
Unknown error category: require a manual decision.
Attempt or time budget exhausted: do not retry.
Server delay exceeds the budget: do not retry; never shorten it.
Only now may a transient, replay-safe failure return retry.

In code with @ai-router/openai-compatible-errors:

import {
  decideOpenAICompatibleRetry,
  normalizeOpenAICompatibleResponse,
} from "@ai-router/openai-compatible-errors";

const error = await normalizeOpenAICompatibleResponse(response);

if (error) {
  const plan = decideOpenAICompatibleRetry(error, {
    method: "POST",
    phase: "http_error",
    replaySafety: "unknown",
    attempt: 1,
    elapsedMs: Date.now() - startedAt,
  });

  if (plan.action === "retry" && plan.delayMs !== undefined) {
    console.info("Caller may schedule a retry", { delayMs: plan.delayMs });
  } else {
    console.warn("Do not automatically replay this request", {
      category: error.category,
      action: plan.action,
      reason: plan.reason,
    });
  }
}

The library never schedules the retry. Keeping execution outside the package makes rate budgets, cancellation, observability, and endpoint-specific semantics visible to the application.

What should set `replaySafety`?

The method is evidence, not the complete answer. GET and HEAD are commonly designed to be replay-safe, while a generation POST is often ambiguous. An idempotency key helps only when the endpoint contract says how the key is scoped, retained, and applied to the operation. The mere presence of a header called Idempotency-Key is not proof.

Set replaySafety: "safe" only when the application owns enough evidence, such as:

The operation is read-only and the endpoint contract supports replay.
The service documents idempotent behavior for this exact operation and key.
The failure occurred before the request could be sent, and the request phase is trustworthy.

Use unknown when a proxy, timeout, or connection reset makes acceptance ambiguous. Use unsafe after observable side effects or when replay can duplicate user-visible work.

Validation checklist

Before trusting an error boundary, test behavior rather than descriptions:

429 transient rate limit versus 429 exhausted quota.
Retry-After seconds, HTTP date, invalid value, past date, and value beyond budget.
401, 403, 413, 422, 502, 503, and 504.
Loopback raw Fetch and pinned OpenAI Node SDK client catch paths, plus OpenAI/AI SDK-style structural errors.
A raw Fetch Response whose original body remains unread.
A body that is text or HTML rather than JSON.
Error messages containing synthetic API keys, Bearer tokens, JWTs, URL credentials, prompts, and response fragments.
Circular objects, throwing getters, revoked proxies, deep arrays, and oversized strings.
SSE frames split across bytes inside a UTF-8 character.
An error before output, an error after output, malformed JSON, [DONE], and premature EOF.
Attempt, elapsed-time, and per-delay budget boundaries.
ESM, CommonJS, and TypeScript declaration imports from the packed tarball.

Fixtures should use synthetic canaries, never copied production credentials or customer prompts.

When a new dependency is not justified

Use the error types already provided by the official OpenAI SDK when one SDK and endpoint cover your application. Use a multi-provider normalizer when your primary problem is one taxonomy across OpenAI, Anthropic, Gemini, and other native APIs. Use provider utilities when you are implementing an AI SDK provider and already depend on that stack.

Add a separate boundary only when you need its narrower guarantees: default body-free diagnostics, multiple OpenAI-compatible error shapes, SSE output tracking, and an explicit replay-safety decision. Otherwise, another package increases maintenance without reducing risk.

References

Install the package from npm, inspect the public source and test matrix, or report a suspected security issue through the repository's private advisory form.

AI ROUTER maintains @ai-router/openai-compatible-errors as an independent, provider-neutral developer utility. It is not affiliated with or endorsed by OpenAI.

Point an OpenAI-compatible SDK at a ChatGPT API relay

AI ROUTER — Sun, 05 Jul 2026 10:43:18 +0000

When a project already uses the OpenAI SDK shape, the lowest-friction way to test a relay service is to change two things only:

the API base URL
the API key

AI ROUTER provides an OpenAI-compatible ChatGPT API relay endpoint at:

https://api.ai-router.dev/v1

That means a small prototype, coding-agent workflow, internal tool, or automation script can keep the same request pattern while using a relay account that exposes API key management, usage tracking, quota, balance, and subscription status.

cURL smoke test

curl https://api.ai-router.dev/v1/chat/completions \
  -H "Authorization: Bearer $AI_ROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4-mini",
    "messages": [
      { "role": "user", "content": "Reply with a short API smoke test." }
    ]
  }'

Python

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AI_ROUTER_API_KEY"],
    base_url="https://api.ai-router.dev/v1",
)

response = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[{"role": "user", "content": "Write one test sentence."}],
)

print(response.choices[0].message.content)

Node.js

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AI_ROUTER_API_KEY,
  baseURL: "https://api.ai-router.dev/v1",
});

const response = await client.chat.completions.create({
  model: "gpt-5.4-mini",
  messages: [{ role: "user", content: "Write one test sentence." }],
});

console.log(response.choices[0].message.content);

What to check before scaling

Confirm the model name is available in your account.
Start with a small daily or weekly quota plan.
Watch API key usage, balance, quota, and subscription status from the dashboard.
Keep retry and timeout handling in your application code.
Avoid treating any relay as an official OpenAI service unless the provider explicitly says so.

I also published curl, Python, and Node.js examples here:

https://github.com/airouter-dev/chatgpt-api-relay-examples

Product page:

https://ai-router.dev

DEV Community: AI ROUTER

How to estimate ChatGPT API costs before moving a coding agent to production

How to estimate ChatGPT API costs before moving a coding agent to production

1. Measure a real task, not a hello-world request

2. Treat retries as part of the workload

3. Verify compatibility before changing the base URL

4. Keep the application change small

5. Decide whether a relay is useful for your workflow

6. Use a go/no-go checklist

Как быстро проверить ChatGPT API в существующем OpenAI-клиенте

OpenAI-compatible API: как проверить endpoint, лимиты и первый запрос

Короткий путь к первому запросу

1. Зафиксируйте четыре значения

2. Проверьте каталог и авторизацию

3. Сделайте минимальный smoke-test

4. Обрабатывайте ошибки по классам

5. Что проверить перед production

Safe Retries for OpenAI-Compatible APIs: HTTP Errors, Retry-After, and Partial SSE Output

Why retryable: true is incomplete

Classify the error before planning a retry

Reproduce the decision without an API key

Build diagnostics that are safe by construction

Honor Retry-After without shortening it

Treat SSE output as a replay boundary

A fail-closed retry decision

What should set replaySafety?

Validation checklist

When a new dependency is not justified

References

Point an OpenAI-compatible SDK at a ChatGPT API relay

cURL smoke test

Python

Node.js

What to check before scaling

Why `retryable: true` is incomplete

Honor `Retry-After` without shortening it

What should set `replaySafety`?