Six runtime dependencies and no build step, which is why our ledger is a text file

Notifio is a desktop app that watches rental search pages and emails you when something new appears. The paid add-on on the pricing page goes one step further and submits the contact form for you, which means the app has to remember things across restarts that an alert-only product never does:

• has this exact listing already been messaged
• how many replies have gone out in the last hour and the last day

Both are questions where being wrong is expensive in a way the user notices. Messaging the same landlord twice is worse than missing a room.

I wrote about the file format's failure mode in Our ledger is one append-only text file, and a torn last line costs us one row. This post is the decision that came before it, and the growth problem that came after.

The obvious answer was SQLite

Two questions, one of them a lookup by id and the other a count over a time window. That is a database. SQLite ships with Electron ecosystems, the queries are four lines each, and the indices write themselves.

The reason it is not SQLite has nothing to do with the data:

/**
 * NDJSON rather than sqlite on purpose: no native module, so the existing
 * electron-builder pipeline (already fiddly with the bundled Playwright
 * browsers) stays untouched.
 */

Notifio bundles its own Chromium. The scraping runs in a real browser, so the browser has to be in the package rather than downloaded on first run, and that makes the packaging config the most fragile file in the repo. Here is roughly what it currently is:

• 6 runtime dependencies, and not one of them has a native build step
• 5 asarUnpack entries, because a browser cannot be executed from inside an asar archive
• about 30 negated globs in files, each one a toolchain package that would otherwise be shipped to users

I have already written two posts about the ways that package fights back: Two DMGs, because a universal build would ship two Chromiums and fs.cpSync copied the framework and left every symlink pointing at the cache.

Adding a native module to that means prebuilt binaries per platform per architecture, rebuilt against Electron's own Node ABI, unpacked from the asar, and signed. On a two-architecture macOS build plus Windows, that is three more ways the installer can be subtly broken in a way that only shows up on somebody else's machine. The data does not justify it. So the constraint became: whatever this is, it must be writable with node:fs alone.

Append-only, folded on read

Once there is no database, the hard part is not reading, it is writing safely. A JSON file rewritten on every update has a window during which it is half a file, and a monitor that is killed by a laptop shutting down lands in that window eventually.

/**
 * Updates are appended as a new row with the same id and readers fold
 * last-wins, which keeps writes atomic by construction.
 */

"Atomic by construction" is the whole trick. A single appendFileSync of one line under the filesystem block size either lands or does not. There is no partial state that a reader has to interpret, because nothing is ever modified in place. An update is a new row:

export function updateReply(id: string, patch: Partial<ReplyRecord>): ReplyRecord | null {
  const current = fold(readRows<ReplyRecord>(REPLIES_FILE)).get(id);
  if (!current) return null;

  const next: ReplyRecord = {
    ...current,
    ...patch,
    id,
    updatedAt: new Date().toISOString(),
  };
  appendRow(REPLIES_FILE, next);
  maybeCompactReplies();
  return next;
}

And the fold is six lines:

/** Fold append-only rows into current state, last write wins. */
function fold<T extends { id: string }>(rows: T[]): Map<string, T> {
  const byId = new Map<string, T>();
  for (const row of rows) {
    if (!row || typeof row.id !== 'string') continue;
    byId.set(row.id, { ...byId.get(row.id), ...row });
  }
  return byId;
}

Note that it is a spread merge rather than a replace. A row that only carries the fields that changed folds correctly on top of an earlier complete row, which means the format tolerates a future writer that appends partial rows without any reader change.

The reader also tolerates a line that is not JSON:

try {
  rows.push(JSON.parse(trimmed) as T);
} catch {
  // A torn final line (power loss mid-append) must not poison the read.
}

One bad line costs one row, not the file. The alternative, JSON.parse over a whole array, loses everything to the same event.

The cost, and where it gets paid

The honest downside: every read is the whole file, and every update is a read. countReplies folds the file. isListingHandled folds the file. Each of those is called while a reply is being considered, so a few hundred rows is irrelevant and a few hundred thousand would not be.

Which is why the file is bounded:

// Compaction thresholds. Rewrite the file once it grows past this many rows.
const REPLIES_MAX_ROWS = 8000;
const REPLIES_KEEP = 2000;

function maybeCompactReplies(): void {
  const rows = readRows<ReplyRecord>(REPLIES_FILE);
  if (rows.length <= REPLIES_MAX_ROWS) return;
  const folded = Array.from(fold(rows).values())
    .sort((a, b) => (a.createdAt < b.createdAt ? 1 : -1))
    .slice(0, REPLIES_KEEP);
  rewrite(REPLIES_FILE, folded);
}

The two numbers are a four-to-one ratio rather than a trim, on purpose. Compaction is the one operation that is not atomic by construction, so it should happen rarely and amortise well. Cutting 8,000 rows down to 7,000 would mean a rewrite roughly every thousand appends; cutting to 2,000 means one every six thousand. With a daily reply cap in the tens, that is an operation most installs will never perform.

The rewrite is the one place a temp file is needed:

function rewrite(file: string, rows: unknown[]): void {
  ensureDir();
  const tmp = `${file}.tmp`;
  try {
    fs.writeFileSync(tmp, rows.map((r) => JSON.stringify(r)).join('\n') + '\n', 'utf8');
    fs.renameSync(tmp, file);
  } catch (err) {
    console.error(`[ledger] Failed to compact ${path.basename(file)}:`, err);
  }
}

Write beside, then rename over. rename within a filesystem is atomic, so a reader either sees the old complete file or the new complete file. A crash between the two leaves an orphan .tmp that the next compaction overwrites.

Also worth noting: 2,000 rows is a retention policy as well as a performance one. These rows describe messages sent on the user's behalf to real people, they live on the user's own disk, and there is no reason for the app to hold five years of them.

The status that is deliberately in the wrong-looking place

The idempotency check is one line:

export function isListingHandled(listingId: string): boolean {
  const existing = getReplyForListing(listingId);
  return existing ? TERMINAL_STATUSES.has(existing.status) : false;
}

And the set it checks against contains something surprising:

/**
 * Statuses that mean "this listing has been dealt with, never touch it again".
 * `failed` is deliberately included: a submit that errored may still have gone
 * through, and messaging a landlord twice is worse than missing one.
 */

A failed reply is treated as handled. This is the single most important line in the module and it reads like a bug. The reasoning is that "failed" means we did not see a confirmation we recognise, which is not the same as "nothing was sent". A form can accept a submission and then time out, redirect somewhere unexpected, or render a thank-you page in a layout we have never seen. Retrying on failure converts an uncertain single message into a certain double message, and the person on the other end is a landlord deciding whether to reply to you.

The full status list and which of them mean what is in 21 reply statuses, and the 8 that mean never touch this listing again.

The same caution shows up in the counting defaults:

const statuses = new Set<ReplyStatus>(
  options.statuses ?? ['sent', 'probably_sent', 'in_progress', 'awaiting_approval']
);

in_progress and awaiting_approval count against the rate caps. A reply that has not been sent yet still occupies a slot, because otherwise a queue of pending replies could each individually pass a check that they collectively fail.

What is not in here

Found listings are not in this file at all. The alert email is the record of those, and the in-app history is a separate bounded file with its own reason for existing, which I wrote about in The app found the listing, and then had nothing to show you. Keeping them apart matters because the per-search diffing snapshots are deleted whenever a search is re-pointed at a different URL, and that is exactly the moment a user would least like their history to vanish.

You can see which sites the reply feature is honest about supporting on the per-site pages; each one has an auto-reply line stating where it stands rather than a tick. SpareRoom and Kamernet are good ones to compare. The app itself is on the download page.