Ten AI tasks behind one function, and each one's retry budget is decided by where the call runs

Nakodo uses a model for ten different jobs. Our privacy notice names them in plain English, because the people on the other end of those emails are entitled to know: "OpenAI, to write briefs and fit notes, label comments, and write and read outreach emails." In the codebase the same list is a union:

export type AiTask = "siteResearch" | "keywords" | "proposal" | "comments" | "rationale" | "outreach" | "reply" | "suggestion" | "support" | "spam";

Every one of those goes through a single function, runAi, in a 175-line file. This post is about that file and nothing else: the instructions each task sends are not the subject, and the one that scores creators is not something I write about at all.

The interesting part of the file is not the call. It is the two tables above it.

A model per task, pinned

const TASK_MODELS: Record<AiTask, { model: string; effort: ReasoningEffort }> = {
  siteResearch: { model: "gpt-6-luna", effort: "low" },
  keywords: { model: "gpt-6-luna", effort: "low" },
  proposal: { model: "gpt-6-luna", effort: "low" },
  comments: { model: "gpt-5.6-luna", effort: "low" },
  rationale: { model: "gpt-6-luna", effort: "low" },
  ...

Nine tasks on one model, one on an older one. The comment above the table says why, and it is the kind of thing that is worth writing down at the site of the decision rather than in a wiki nobody opens:

// Site research, comment labels, fit notes (rationale) and first emails
// (outreach) were picked with `pnpm benchmark` (scripts/benchmark-models.ts,
// results in benchmark/results.md, 2026-10-01). gpt-6-luna labels comments as
// well but returned invalid JSON on 20% of comment batches, so comments use
// gpt-5.6-luna. The other tasks haven't been benchmarked: they use
// gpt-6-luna, which won every written task that was.

That is from the bake-off I wrote up in Four AI jobs, four model choices, and the one we picked for comments was not the most accurate. The headline there was that quality lost to validity: a model that is better at the task but hands back unparseable JSON one batch in five is worse at the job. The table is where that result lives now, and the honesty I care about is the last sentence: six of the ten were never benchmarked, and the file says so instead of implying the whole thing was measured.

The production path cannot choose a model. runAi accepts model, effort and timeoutMs overrides, and the only caller that passes them is the benchmark script:

    // Overrides used by the benchmark script
    model?: string;
    effort?: ReasoningEffort;
    timeoutMs?: number;

A retry budget per task, decided by what is waiting for it

This is the table I would copy into the next app.

const TASK_LIMITS: Record<AiTask, { timeoutMs: number; maxRetries: number }> = {
  siteResearch: { timeoutMs: 75_000, maxRetries: 0 },
  // New keywords for a campaign that searched all of its own, in the
  // background but not in a job: a failed round waits for the next one.
  keywords: { timeoutMs: 30_000, maxRetries: 1 },
  // Also while the user waits, inside a 60-second request.
  proposal: { timeoutMs: 25_000, maxRetries: 1 },
  comments: { timeoutMs: 30_000, maxRetries: 0 },

Almost every task gets maxRetries: 0, which looks wrong until you read what is calling it:

// `maxRetries` is the SDK's own retries, straight after a
// timeout, a rate limit or a server error; a timed-out call may still be
// billed. Site research runs while the user waits and gets one try.
// Comments, fit notes, first emails and replies run in jobs, which already try
// three times with minutes between (MAX_ATTEMPTS in src/lib/jobs.ts), so they
// get one try per attempt: three calls at most, not six, and a rate limit has
// time to clear.

Retries compose by multiplication. A job queue that retries three times, wrapped around an SDK that retries twice, is six paid calls for one piece of work, and all six happen within seconds of each other, which is the worst possible pattern against a rate limit. Deciding the retry count at the call site, from the properties of whatever is already retrying, turns that into three calls spread over minutes.

The same logic sets the timeouts, and each one names the thing it is bounded by:

  // A support answer is read by someone watching a widget, so it gets one try
  // and a short leash: past this, a person is the better answer anyway.
  support: { timeoutMs: 20_000, maxRetries: 0 },
  // The spam check on mail to the catch-all runs inside the inbound webhook,
  // before it answers Resend, so it gets one short try. A check that fails
  // forwards the mail (src/lib/email/spam.ts).
  spam: { timeoutMs: 10_000, maxRetries: 0 },

Site research gets 75 seconds because it runs inside a request whose own budget is 120. The spam check gets 10 because a webhook has to answer the provider before it gives up on us. Nothing in that table is a round number chosen for looking tidy; each one is a smaller number than whatever is above it in the stack.

The call itself is 20 lines

  const response = await openai().responses.parse(
    {
      model,
      instructions: opts.instructions,
      input: opts.input,
      text: { format: zodTextFormat(opts.schema, opts.schemaName) },
      max_output_tokens: opts.maxOutputTokens ?? 8000,
      store: false,
      // Calls for the same task share their instructions, so routing them
      // together lets the cached prefix be reused.
      prompt_cache_key: `nakodo:${task}`,
      ...(supportsReasoning(model) ? { reasoning: { effort } } : {}),
    },
    { timeout: opts.timeoutMs ?? TASK_LIMITS[task].timeoutMs, maxRetries: TASK_LIMITS[task].maxRetries },
  );

store: false because none of this needs to live in a dashboard; we have our own audit log and our own run log. The prompt_cache_key is one per task, which is the natural key: everything that shares a cache key shares its instructions word for word, so the cached prefix is as long as it can be. The schema goes through zodTextFormat, with the caveat I ran into earlier that Structured Outputs has no optional fields, so the schema the model sees and the schema we store are deliberately different types.

And one small regex, which exists because a provider changing a parameter's availability is not a hypothetical:

// gpt-4.x and chat-* models reject the reasoning parameter.
function supportsReasoning(model: string): boolean {
  return /^(o\d|gpt-5|gpt-6)/.test(model) && !model.includes("chat");
}

Counted before it is judged

The ordering here is the one line of this file I am most sure about:

  // Paid for whether or not it can be used.
  await recordUsage(task, usage).catch((e) => console.error("Recording AI usage failed", e));
  if (response.status === "incomplete") {
    throw new AiError(`${task}: output incomplete (${response.incomplete_details?.reason ?? "unknown"})`);
  }
  if (response.output_parsed == null) {
    throw new AiError(`${task}: no structured output (model refused or returned nothing)`);
  }

A truncated response is billed. A refusal is billed. If usage were recorded after validation, our own numbers would quietly undercount exactly the calls we most want to notice, which is the opposite of what a cost counter is for. So usage is written first, and the write is wrapped in .catch because failing to count must never fail the work.

What gets counted is four numbers per task per UTC day:

export const AI_USAGE_PARTS = ["calls", "input", "cached", "output"] as const;

Cached input tokens are inside input and counted again on their own, so the cache hit rate is a division rather than a separate query, and reasoning tokens are inside output because they are billed as output. They land in the same daily counters table the rest of the app uses, with no cap attached: what any one account may spend is set by its plan, and these rows exist so the admin run log can show where the money went.

A failure that is not worth retrying

// A response that came back but can't be used: cut off, refused or empty.
// Asking again with the same input gets the same, so it's `permanent`, and a
// job that hits it gives up rather than paying for the same failure again.
export class AiError extends Error {
  readonly permanent = true;

This is the counterpart to the retry table. A timeout or a 429 comes out of the SDK as its own error and the job's three attempts are the right answer to it. A response that arrived, parsed as JSON and still cannot be used is deterministic: the same input will produce the same refusal, and the queue would pay three times to find that out. One boolean on the error class, read by the queue, is the difference between a bad call costing one unit and costing three.

Ten tasks, two tables and one function. The tables are the design; runAi is just the thing that reads them.

Nakodo is the app this runs in, and the privacy notice is where the ten jobs are described for the people they affect.