Nakodo's background work runs on a cron in production: campaigns search, results get enriched and scored, emails go out, data gets expired. Operating that from a laptop needs a toolbox, and ours is one file: scripts/pipeline.ts, 221 lines, run as pnpm pipeline <verb>. Twelve verbs, no framework, no commands directory, no subcommand registry.
The docblock at the top is the help text, and it is the first thing anyone touching this reads:
// pnpm pipeline launch <campaignId>
// pnpm pipeline run [--seconds 240] [--max-searches N] [--once]
// pnpm pipeline status [campaignId]
// pnpm pipeline recontacts re-extract contacts with the current rules
// pnpm pipeline restat work typical views out again from stored posts
// pnpm pipeline icons read the websites of businesses that have no icon yet
// pnpm pipeline analyze [campaignId]
// pnpm pipeline rescore [campaignId]
// pnpm pipeline guess-parts [campaignId] [--reopen]
// pnpm pipeline maintenance the daily cron: retention, refreshes, due searches
// pnpm pipeline claim <email>
// pnpm pipeline plan <email> <plan> set a user's plan (free, pro, business)
The dynamic imports are a load-order fix, not a speed trick
Every branch looks like this:
if (command === "restat") {
const { restatChannels } = await import("@/lib/pipeline/enrich");
console.log(await restatChannels());
return;
}
The reason is three lines at the top of the file:
import { config } from "dotenv";
config({ path: [".env.local", ".env"], quiet: true });
Some app modules read configuration at module scope. Our env accessor is deliberately lazy and checks nothing on import, but the database module is not:
const client = globalForDb.pg ?? postgres(env("DATABASE_URL")!, { prepare: false, max: 5 });
That line runs when @/db is imported. A static import { db } from "@/db" at the top of this script is hoisted above config(), so the env file has not been read yet and the connection string is undefined, over a variable sitting right there in .env.local.
You can fix that with a wrapper that imports the real script after loading dotenv. We tried; it adds a file and a layer of indirection to solve a problem that await import solves inline. So the rule in this file is: nothing from @/ at the top, everything from @/ inside the branch that needs it.
The side effects turn out to be the good kind. pnpm pipeline plan someone@example.com pro loads Drizzle, the schema and the billing module, and nothing else: no DuckDB, no queue client, no HTTP clients for any external API. The pleasant surprise of a flat if-chain over a command registry is that a registry has to import every handler to build itself, which defeats the whole arrangement.
Argument parsing is also not a dependency:
function flag(args: string[], name: string): number | undefined {
const i = args.indexOf(name);
return i >= 0 ? Number(args[i + 1]) : undefined;
}
Three lines, numbers only, and args.includes("--once") covers the booleans. Twelve verbs have never needed more.
The long-running verb budgets its own time
run is the one that does real work, and it is the same function the cron calls. There is no separate development path, which is the only reason I trust what I see locally:
const seconds = flag(args, "--seconds") ?? 240;
let searchesLeft = flag(args, "--max-searches");
while (true) {
const r = await runPipeline({ trigger: "manual", timeBudgetMs: seconds * 1000, maxSearches: searchesLeft });
const s = r.summary;
console.log(`run ${r.runId.slice(0, 8)}: ${r.stopReason}, ${r.units} units, ${r.jobsDone} jobs done, ${r.jobsFailed} failed`);
for (const note of s.notes) console.log(` note: ${note}`);
if (searchesLeft !== undefined) searchesLeft = Math.max(0, searchesLeft - s.searches);
if (r.stopReason !== "time_budget" || args.includes("--once")) break;
}
runPipeline returns a stopReason rather than throwing or looping forever, and the loop condition is a single string comparison: keep going only if the last run stopped because it ran out of time. Every other reason, an empty queue, a daily API budget reached, an error, ends the loop and prints why. trigger: "manual" is recorded on the run row, so a run log that looks odd can be traced to someone at a keyboard rather than a schedule.
The --max-searches budget is decremented across iterations, which is the sort of thing that is obvious in the code and invisible from the outside: without it, a loop of ten runs with a per-run cap would spend ten times the cap.
Backfills have to be safe to run twice
Half the verbs exist because a rule changed and stored rows predate it. restat recomputes typical views from posts already in the database with today's rules, recontacts re-extracts contact details, icons fills in what is missing. None of them fetch anything they already have, and all of them are safe to run again, because the alternative is an operator who is afraid of their own tools.
guess-parts is the one with a real decision in it:
// Briefs whose guesses the owner has already been through are left alone:
// the panel is gone for good, so there is nothing left to mark. --reopen
// takes those too and asks once more, for guesses dismissed because they
// pointed nowhere.
if (!row.brief || (row.brief.assumptionsChecked && !reopen)) continue;
Idempotent by default, with an explicit flag to redo work that was deliberately skipped. The flag is not a --force: it has a specific meaning, written down next to the condition it disables.
Exit codes, because a pool keeps the process alive
main()
.then(() => process.exit(0))
.catch((e) => {
console.error(e);
process.exit(1);
});
The explicit process.exit(0) is not decoration. An open Postgres pool and a queue client both hold handles, so the process sits there after the work is done, and a script that never exits is a script you cannot put in a shell loop. Unknown verbs print a one-line usage string and exit 1.
That usage string is duplicated from the docblock, which is a real wart: two copies that can drift, and the short one is already missing guess-parts and plan. It stays because the docblock is what a person reads in their editor and the error line is what they get in a terminal, and merging them means building a help system for twelve ifs.
What it is driving
If you want the non-engineering version of what these verbs operate, how it works walks through the same pipeline in plain language, and our post on what to automate and what to keep human is the editorial line that decides which steps are allowed to be a cron job at all. The plan verb exists because plan limits live in code rather than in Stripe, and those limits are the ones on the pricing page.
The whole file is less code than a CLI framework's config would be, and it has taken every new verb without a structural change. Flat if-chain, dynamic imports, three-line flag parser, explicit exit. That is the entire architecture.
Top comments (1)
tr.ee/dev-to