A 120 second request, a 35 second crawl and a 75 second model call, and the arithmetic has to leave slack

Most of Nakodo runs in the background. One screen does not: when a brand starts a campaign, it types a domain and waits while we read the site and write a brief from it. That makes it the only part of the product with a user watching a spinner for tens of seconds, and the only part where the timeouts are arithmetic rather than guesses.

Here is the budget, from the outside in.

The route declares the ceiling

export const metadata = { title: "New campaign" };
// Site research takes 10 to 40 seconds.
export const maxDuration = 120;

That is on the page, not on an API route, because the work happens in a Server Action invoked from that page. 120 seconds is not an estimate of the work, it is roughly three times the worst case we are willing to let the user sit through, which gives everything underneath it room to fail politely instead of being killed mid-sentence by the platform. Being killed by the platform is the nastiest failure mode available, because none of our code runs: no error, no status change, no refund. There is a separate mechanism for that one, at the end of this post.

The same number appears as a comment in the module that orchestrates it, which is the one place a caller might forget:

// Creates a campaign and runs site research. Research takes 10 to 40 seconds,
// so callers should allow for that (route maxDuration, loading state).

The crawl gets 35 seconds, and each page gets 10

// Each fetch has its own 10-second timeout; this caps the whole crawl so
// research (crawl plus a 75-second AI call) fits in a 2-minute request.
const CRAWL_MS = 35_000;

35 plus 75 is 110, inside 120, with 10 seconds for everything else: the database writes, the plan check, the audit row. The 10 second per-fetch timeout lives in our HTTP module, and it is deliberately not the only limit, because a site can serve ten pages that each answer in nine seconds.

Inside the crawl there are two further budgets, both for things we would rather have than wait for:

const SITEMAP_MS = 8_000; // the sitemap is a nice-to-have; don't wait long for it
const LOGO_MS = 8_000; // the logo is a nice-to-have too

const logo = withTimeout(findLogo(home.icons), LOGO_MS, () => new Error("Logo timed out")).catch(() => null);
const sitemap = await withTimeout(readSitemap(origin), SITEMAP_MS, () => new Error("Sitemap timed out")).catch(() => null);

Both catch to a null. A brand with no sitemap, or a sitemap served by something slow, gets a brief anyway. The logo promise is started before the sitemap is awaited, so the two overlap, and the campaign simply has no logo beside its name if that race goes badly.

The timeout helper is nine lines and says what it does not do:

// Rejects with `error()` if `promise` hasn't settled within `ms`. The work
// itself isn't cancelled; use it where each step has its own timeout anyway.
export function withTimeout<T>(promise: Promise<T>, ms: number, error: () => Error): Promise<T> {
  let timer: ReturnType<typeof setTimeout>;
  return Promise.race([
    promise,
    new Promise<never>((_, reject) => {
      timer = setTimeout(() => reject(error()), ms);
    }),
  ]).finally(() => clearTimeout(timer));
}

Promise.race does not cancel the loser. That is fine here only because every underlying step has its own AbortSignal.timeout, so the abandoned work dies within its own limit instead of running until the function freezes. A withTimeout wrapped around something with no internal timeout is a leak with a reassuring name.

The model call gets 75 seconds and exactly one try

Every AI call in the app goes through one function, and the per-task limits are a table with reasons in the comments:

const TASK_LIMITS: Record<AiTask, { timeoutMs: number; maxRetries: number }> = {
  siteResearch: { timeoutMs: 75_000, maxRetries: 0 },
  keywords: { timeoutMs: 30_000, maxRetries: 1 },
  proposal: { timeoutMs: 25_000, maxRetries: 1 },
  comments: { timeoutMs: 30_000, maxRetries: 0 },
  rationale: { timeoutMs: 30_000, maxRetries: 0 },
  outreach: { timeoutMs: 30_000, maxRetries: 0 },
  reply: { timeoutMs: 30_000, maxRetries: 0 },
  suggestion: { timeoutMs: 30_000, maxRetries: 0 },
  support: { timeoutMs: 20_000, maxRetries: 0 },
};

The header explains the rule behind the column that looks arbitrary:

// Per-call limits, so one slow response can't outlast the request or cron run
// that made it. `maxRetries` is the SDK's own retries, straight after a
// timeout, a rate limit or a server error; a timed-out call may still be
// billed. Site research runs while the user waits and gets one try.
// Comments, fit notes, first emails and replies run in jobs, which already try
// three times with minutes between (MAX_ATTEMPTS in src/lib/jobs.ts), so they
// get one try per attempt: three calls at most, not six, and a rate limit has
// time to clear.

Two things I would carry to any codebase with an LLM in it.

Retries belong to one layer. The OpenAI SDK retries by default, and our job runner retries by default. Left alone, a transient rate limit became six calls in quick succession, which is both expensive and the worst possible behaviour towards a rate limit. So the tasks that run inside jobs have maxRetries: 0 and let the job runner's three attempts, minutes apart, be the retry policy. The tasks with maxRetries: 1 are the ones with no outer retry at all.

A timeout is a budget, not a hope. 20 seconds for a support answer is not because support answers are easy. It is because somebody is watching a widget, and past 20 seconds the honest product is to offer them a person. The number comes from the deadline of the thing waiting, not from the difficulty of the task.

What arrives at the model is bounded too

Timeouts are only half of a latency budget; the other half is input size. Pages are read short, and the ones already stored are re-bounded on the way back out:

// Any one page as it goes to the AI. Pages are read shorter than this
// (./crawl.ts); it bounds ones kept from before, like a sitemap listing every
// post of a blog, which would otherwise go with every suggestion.
const PAGE_TEXT_MAX = 10_000;

const MAX_KEY_PAGES = 5;
const HOMEPAGE_CHARS = 8_000;
const PAGE_CHARS = 5_000;
const SITEMAP_PATH_SAMPLE = 150;
const MIN_SITE_TEXT = 300; // less readable text than this and the brief would be guesswork

The reason PAGE_TEXT_MAX exists separately from PAGE_CHARS is a bug waiting to happen that we met the slow way: the crawl trims pages when it reads them, but those pages are stored, and they are sent again later when the app writes other things from the same research. Change the crawl's limit and old rows keep their old size forever. The second cap is applied at the moment of use, so a row written months ago cannot blow today's budget.

MIN_SITE_TEXT is the opposite kind of limit. Under 300 characters of readable text, we stop rather than spend 75 seconds turning nothing into a confident brief. The user gets a specific message instead:

case "empty":
  return `${domain} has almost no text we can read. It may need JavaScript to show its content.`;

That is one of eight crawl problems, each with its own sentence: invalid, not_found, blocked, private, timeout, tls, empty, error. A homepage fetch tries https://domain, then https://www.domain, then plain http, and reports the most useful failure rather than the last one:

const worth = failures.find((f) => f.problem !== "not_found") ?? failures[0];
throw new CrawlError(crawlErrorMessage(domain, worth.problem, worth.detail), worth.problem);

"We could not find acme.com, check the spelling" and "acme.com has a security certificate problem" send the user to completely different actions, and collapsing both into "research failed" is how a product earns a support ticket it did not need.

Running out of time has to be free

A plan includes a number of website research runs a month, so a timeout is also a billing event. Every metered action is a row in a ledger, which makes a refund a delete rather than a decrement, and the ordinary failure path spends one:

const outcome = await runCampaignResearch(id, { userId, notes });
if (!outcome.ok) {
  await refund(eventId);
  await deleteCampaign(id);
  ...
  throw new ResearchError(outcome.error);
}

A re-run on an existing campaign is gentler: a failure restores the status the campaign had before, pause state included, so a brand re-running research on a live campaign and hitting a timeout does not lose the campaign it already had.

And when a site blocks us outright, there is still a way through, which is why the crawl error message ends with an instruction:

// Crawls the site and writes a brief. Notes from the user are added as an extra
// page, and are enough on their own when the site blocks us.

The failure with no error: three minutes

None of the above runs if the platform kills the function. The request vanishes, the row says researching forever, and the user is looking at a spinner that will never resolve. That case is handled by a clock instead of a catch:

// Research runs inside the request that started it (up to 2 minutes). A
// campaign still "researching" after this was cut off: it goes back to where
// it was (paused if it had been launched) or to "failed", and the run is refunded.
const STALE_RESEARCH_MS = 3 * 60_000;

Three minutes is the 120 second ceiling plus a minute of slack, which is the only honest way to pick that number: it has to be longer than the longest run that could still be alive, or recovery would fight a request that is about to succeed. The recovery is one UPDATE with the decision written as SQL, so every stuck campaign for that user is sorted in one statement:

.set({
  status: sql`case when ${campaigns.brief} is null then 'failed' when ${campaigns.launchedAt} is not null then 'paused' else 'draft' end`,
  // A launched one comes back paused with everything stopped, for the user to resume.
  pauseMode: sql`case when ${campaigns.brief} is not null and ${campaigns.launchedAt} is not null then 'both' end`,
  ...
})

Then the refund, which is a delete with a one hour window so it cannot reach back into an unrelated earlier run:

export async function refundLatestResearch(campaignId: string): Promise<void> {
  await db.execute(sql`
    delete from ${usageEvents} where id = (
      select id from ${usageEvents}
      where campaign_id = ${campaignId} and kind = 'research' and created_at > now() - interval '1 hour'
      order by created_at desc limit 1
    )`);
}

There is no cron behind this. It runs when the user next looks at their campaigns, because that is exactly when a stuck campaign matters and exactly who it matters to.

The shape of it

One user-facing deadline, split into named constants that add up to less than it, with each nice-to-have allowed to fail on its own and each limit paired with a sentence the user can act on. The numbers will be wrong eventually. The arithmetic being written down is what makes them cheap to change.

The user-facing half of this screen is described in how Nakodo works, where step one is the brief written from your website, and the monthly allowance of research runs that the refund above protects is on the pricing page.