Success responses across providers have converged enough that a thin adapter handles them. Errors have not. The status codes differ, the envelopes differ, the machine-readable code lives at a different path in each, and the one piece of code that has to be right during an incident is the one with the least test coverage.
Providers agree on success and diverge on failure
Three shapes, all current at the time of writing, all different. Anthropic’s API returns a top-level object with "type": "error", a nested error object carrying its own type and message, and a request_id at the top level. Its documented type strings map to specific statuses: invalid_request_error at 400, authentication_error at 401, billing_error at 402, permission_error at 403, not_found_error at 404, conflict_error at 409, request_too_large at 413, rate_limit_error at 429, api_error at 500, timeout_error at 504 and overloaded_error at 529.
The OpenAI-shaped envelope that a great many providers and proxies imitate has no top-level type at all: it is an error object holding message, type, param and code, where type is the broad family and code is the specific cause. Google’s APIs use the canonical Google error shape, an error object with a numeric code, a message and a status string such as RESOURCE_EXHAUSTED.
So error.code is a string in one, a number in another, and absent from the third. Any handler that reads a single path is right for one provider and silently wrong for the rest — and “silently” matters, because an unrecognised error usually falls to a generic branch that retries or does not retry uniformly, which is how a 400 gets retried sixteen times and a 529 does not get retried at all.
Every code and status listed above is what the providers documented at the time of writing, and Anthropic’s own errors reference states explicitly that the values inside these objects may expand over time. That is the reason for the exhaustiveness test below rather than a reason to distrust the list.
The normalised type is the contract
Define the thing your application actually branches on, and make it small. If it has more than about six members you have copied a provider’s taxonomy instead of writing your own.
// src/errors.ts
export type Kind =
| "auth" // key wrong, revoked, or missing a scope
| "bad_request" // our fault; never retry
| "rate_limit" // back off and retry
| "overloaded" // provider capacity; retry, prefer failover
| "server" // provider fault, unknown cause; retry once
| "too_large"; // request exceeded a size or context limit
export type NormalisedError = {
kind: Kind;
status: number;
provider: string;
providerCode: string; // the raw code, kept for logs, never branched on
retryable: boolean;
retryAfterMs: number | null;
requestId: string | null;
};
Two decisions in that type are worth defending. providerCode is kept but never branched on, so a log line can name the exact cause while the control flow depends only on kind — that is what stops a provider’s new code string from changing behaviour before anyone has decided what it should do. And retryable is a field you compute rather than something you read off the wire, because no provider tells you whether retrying is a good idea for your workload.
Fixtures, and where to get honest ones
Do not hand-write these bodies from documentation. Error responses carry headers that matter — retry-after, request-id, rate-limit remaining headers — and the documented example is usually tidier than the real thing. Capture them instead: send a deliberately bad key to get a 401, an over-length request to get a 413, a malformed parameter to get a 400, and save the whole response including headers. A 429 and a 529 are harder to provoke on purpose; take those from production logs when they occur and add them to the corpus, which is why the corpus should be a directory anybody can drop a file into.
fixtures/errors/
anthropic/401-authentication_error.json
anthropic/429-rate_limit_error.json
anthropic/529-overloaded_error.json
openai/400-invalid_request_error.json
openai/429-rate_limit_exceeded.json
google/429-RESOURCE_EXHAUSTED.json
// each file:
{
"provider": "anthropic",
"status": 429,
"headers": { "retry-after": "27", "request-id": "req_011CSHoEeqs5C35K2UUqR7Fy" },
"body": { "type": "error", "error": { "type": "rate_limit_error", "message": "..." } },
"expect": { "kind": "rate_limit", "retryable": true, "retryAfterMs": 27000 }
}
Putting the expectation in the fixture file rather than in the test body is deliberate: adding a provider or a newly observed code is then a data change that anyone can review, and the test file stops growing.
The test, including the exhaustiveness case
import { describe, it, expect } from "vitest";
import { readdirSync, readFileSync } from "node:fs";
import { normaliseError } from "../src/errors";
const files = readdirSync("fixtures/errors", { recursive: true, withFileTypes: true })
.filter((e) => e.isFile() && e.name.endsWith(".json"))
.map((e) => `${e.parentPath}/${e.name}`);
const fixtures = files.map((f) => ({ file: f, ...JSON.parse(readFileSync(f, "utf8")) }));
describe("provider error contract", () => {
it("has fixtures for every configured provider", () => {
const covered = new Set(fixtures.map((f) => f.provider));
for (const p of ["anthropic", "openai", "google"]) {
expect(covered, `no error fixtures for ${p}`).toContain(p);
}
});
it.each(fixtures)("$file normalises as expected", (fx) => {
const got = normaliseError(fx.provider, fx.status, fx.headers, fx.body);
expect(got.kind).toBe(fx.expect.kind);
expect(got.retryable).toBe(fx.expect.retryable);
expect(got.retryAfterMs).toBe(fx.expect.retryAfterMs);
// The raw code is preserved for logs, whatever it is.
expect(got.providerCode).toBeTruthy();
// request_id where the provider supplies one; never invented.
expect(got.requestId === null || typeof got.requestId === "string").toBe(true);
});
it("never falls through to an unclassified kind", () => {
const kinds = new Set(fixtures.map((fx) =>
normaliseError(fx.provider, fx.status, fx.headers, fx.body).kind));
expect(kinds.has("unknown" as never)).toBe(false);
});
});
The last case is the one that earns the page. A normaliser almost always ends in a default branch, and a default branch is where a new provider code lands quietly. Asserting that no fixture reaches it converts “we got an error we have never seen” from a production surprise into a red build the moment somebody adds the fixture. Pair it with a scheduled job that scans a day of production error logs for provider codes absent from the corpus and opens an issue — the fixture directory then grows from reality rather than from imagination.
Retryable is a decision, not a field
Getting kind right is only useful if the mapping to behaviour is right, and three of these are commonly wrong. A 429 is retryable but only with the provider’s own delay: honour retry-after when present rather than applying your own backoff curve on top of it. A 529 or its equivalent is capacity rather than quota, so the correct response is usually to fail over rather than to wait — retrying the same provider harder is what turns a degraded provider into an outage for you. And a 400 must never be retried, which sounds obvious until a generic “retry all errors” wrapper multiplies a malformed request by your retry count.
Assert those three as separate behaviour tests, not just as classification tests: feed each fixture through the actual retry policy and assert the number of attempts. Classification that is correct and unused is a comfortable illusion.
If you route across providers, this normalisation is a boundary you either own or delegate, and the failure mode of owning it is that the mapping falls behind whichever provider you use least. Multigrid presents one error envelope across the providers behind it, which means the contract your handler is tested against is the gateway’s rather than three moving targets — the corpus above then covers your own handler, and the per-provider divergence stops being your test matrix.
When a provider announces planned downtime the shape is different again, and worth its own case — see testing maintenance-window handling. For what happens when the codes stay the same and the model behind them changes, see silent model updates.
Top comments (0)