We deleted the hosted job runner, and the two schedules we have left are an hourly ping and a monthly invoice

Munchable has a data pipeline behind it. Scan a barcode, and the app needs every word on that ingredients list to resolve to something its rule sets can reason about. Words arrive faster than any fixed vocabulary covers them, so there is a backlog, a deterministic pass over it, and a model pass behind that for the rows the deterministic layers cannot settle.

For a while that ran on a hosted background-jobs platform: scheduled runs every fifteen minutes, plus per-capture tasks for the tail of each label. Both are gone now. The scheduled curation jobs do not exist, the platform is out of the dependency list, and its two environment variables are out of .env.example. What replaced them is after() for the small work and a CLI for the big work.

The output is public, so the honest test of whether this was a downgrade is whether the data kept moving: 373 ingredient answer pages, each generated from the curated rule sets, and seven condition guides built from the same source.

The question that killed the schedule

Not "is this platform good". It was: what does a scheduled curation run accomplish in the fifteen minutes between captures?

The answer turned out to be almost nothing, because the scan path had been getting better underneath it. It now resolves what it can inline and writes the answer straight into the overlay the engine reads. What is left over after a response is the tail of one label: one overlay read, at most one model call.

That fits in after():

export function dispatchResolveLabel(payload: ResolveLabelPayload): void {
  if (payload.words.length === 0) return;
  try {
    after(async () => {
      try {
        await resolveLabelWords(payload);
      } catch (e) {
        console.warn(`[dispatch] resolve-label failed: ${String(e).slice(0, 200)}`);
      }
    });
  } catch {
    // Outside a request scope (tests, scripts): nothing to schedule on.
  }
}

Three things about that snippet are the actual design.

It runs in the same function invocation, after the response has been sent, bounded by the route's own maxDuration. So any route that calls it has to export a maxDuration long enough for the work, which is a real constraint and is written down where the function is defined.

It is best effort, by design. A failure here loses nothing: the words are in the backlog either way, and the bulk job finishes them later. That property is what makes the whole arrangement safe, and it was true before the deletion too. We were paying a platform to provide durability for work that was already durable.

It swallows its own absence. Called from a test or a script there is no request scope to attach to, and that is not an error.

What was left to delete, once that was true: a 357-line model benchmark harness, a 337-line one-off curation script, two parallel curation pipelines that had grown separately and did nearly the same thing, and the hosted runner's own config. The replacement is one runner, one lock, and three job names.

The big work did not want a schedule either, it wanted a person

The remaining job is the bulk pass: drain the whole naming backlog, review the commonest ingredients no condition yet speaks for. It spends money on model calls and writes curated data that is a nuisance to un-write. A bulk pass like that wants somebody watching it, not a cron expression.

So it is a command:

pnpm job taxonomy-mine
pnpm job taxonomy-curate --max 2000            # deterministic pass plus model
pnpm job taxonomy-curate --no-model            # deterministic pass only
pnpm job taxonomy-curate --until-done          # keep going until it is empty
pnpm job rules-review --allow-general --top 500

It reads .env.local, has no clock of its own, and imports each job body dynamically so running one does not load the other two. The whole file is about 120 lines, which is roughly what the platform's configuration used to be.

--until-done is the flag that made a backlog finishable

Every run stops at a row cap and reports capped: true. That is a good property for a job with a budget. It is a terrible property for a human, because a forty-thousand-row backlog at the default cap is a hundred invocations, and a boolean you have to read and then act on a hundred times is a loop with a person standing in the middle of it.

So the CLI closes the loop, under one lock acquisition rather than a hundred:

async function drain(): Promise<unknown> {
  if (!flag('--until-done')) return body();
  let last: Record<string, any> = {};
  for (let pass = 1; ; pass++) {
    last = (await body()) as Record<string, any>;
    const progressed = (last.accepted ?? 0) + (last.deterministic?.resolved ?? 0) > 0;
    console.log(`[pass ${pass}] ${JSON.stringify(last)}`);
    if (!last.capped) return last;
    if (!progressed) {
      console.error(`[pass ${pass}] still capped but nothing was written; stopping rather than looping`);
      return last;
    }
  }
}

The progressed check is the important half, and it is the thing I would have left out of a first draft. Without it, a batch that fails for the same reason every time becomes an infinite loop that keeps asking a model the same question, with a bill attached. "Still capped but nothing was written" is a complete description of a stuck pipeline, so the loop ends there and the last summary is what stays on screen, rather than being buried under the hundred runs that came after it.

The flag also refuses to apply to the jobs that have no row cap:

--until-done only applies to taxonomy-curate (it is the only job with a row cap)

A flag that silently does nothing on two out of three subcommands is a flag you will eventually believe worked.

The lock stayed, because the hazard was never the scheduler

Moving to a CLI removes overlapping scheduled runs, but it introduces a person with two terminals. Both write the same curated overlay, so the Redis lock survived the migration, and the CLI takes it by the same name the cron routes use. --no-lock exists and has to be typed.

Without Redis, the job does not run at all. These jobs spend money and write curated data, so "the lock is unavailable" fails closed rather than open.

The lock is a renewed lease rather than a long timeout, and that distinction cost us a real outage back when it was a long timeout: a run cancelled by hand never reached its finally, so the key sat there for hours while the fifteen-minute schedule kept firing and skipping, and every skip was recorded by the platform as a successful run. Green dashboard, nothing curated. I wrote that one up separately in Our cron dashboard was green every 15 minutes while nothing ran for three hours.

What is actually still scheduled

Two things, both on Vercel Cron, and neither of them is data curation:

{
  "crons": [
    { "path": "/api/cron/rewards-settle", "schedule": "0 4 8 * *" },
    { "path": "/api/cron/indexnow", "schedule": "0 * * * *" }
  ]
}

A monthly contributor-reward settlement, and an hourly search-engine ping. Both are genuinely time-driven: one is about a calendar month closing, the other about telling an index that pages changed. Neither is "go and look for work".

Both are behind the same guard, and the guard has one opinion worth copying:

export function cronAuthStatus(authHeader: string | null, secret: string | undefined): CronAuth {
  if (!secret || secret.length < 16) return 'unset';
  // ...constant-time compare of the bearer token against the secret
}

A missing secret returns unset, and unset makes the route answer 503 rather than running. A cron endpoint that treats "no secret configured" as "no authentication required" is a public button that spends money, and the difference between those two behaviours is a single early return.

The rule I would keep

A scheduled job is a bet that work will arrive while nobody is looking. Before you accept that bet, ask where the work actually comes from.

• Work that arrives because a user did something should happen in that request, or immediately after it. after() is enough far more often than it looks, provided the work is idempotent and losing a run costs nothing.
• Work that arrives because you decided to do a batch is a command, not a schedule. It wants flags, a progress log and a person who can stop it.
• Work that arrives because a date passed is the only thing that genuinely needs a cron entry, and there are usually one or two of those.

We had a hosted runner, a second deployment target, a vendor's environment variables in production, and a dashboard to check. What we actually needed was a terminal and a lock.

Munchable is at munchable.app. The curated side of all this is visible without an account: the ingredient answers, the condition guides, and 38 recipes checked against the same rule sets.