Our ledger is one append-only text file, and a torn last line costs us one row

Notifio is a desktop app that watches rental search pages, emails you when a new listing appears, and, if you have the auto-reply upgrade, replies to the listing for you by replaying a reply you once demonstrated yourself.

That last part needs memory that survives a restart. Not for the user's benefit, for the engine's. It has to answer two questions before it touches anything:

1. Have we already messaged this listing?
2. How many replies have gone out in the last hour, and the last day?

Get the first one wrong and you message a landlord twice, which is worse than missing the room entirely. Get the second one wrong and you blow through the per-site limits that keep the account usable.

The obvious answer is SQLite. We used a text file.

Why not SQLite

better-sqlite3 is a native module. In an Electron app a native module means a rebuild against the exact Electron ABI, a separate build per architecture, and a path through asarUnpack so the binary is reachable at runtime.

Our electron-builder pipeline already spends all of its goodwill on exactly that kind of problem, because the app bundles its own Chromium: we ship two separate macOS DMGs rather than a universal build, and the copy step that puts Chromium in the bundle has had its own symlink adventure.

Adding a second native dependency to that pipeline, to store a few thousand rows of our own JSON, is a bad trade. So the ledger is newline-delimited JSON in the app's user-data directory, and the only API it needs is fs.appendFileSync.

I went through that particular trade on its own in We picked NDJSON over SQLite, and the reason was the build pipeline. What this post adds is what the format then costs you downstream: how a half-written line is tolerated, why compaction hangs off the update path and never the create path, and why an append-only file quietly removes your freedom to delete a value from an enum.

Appending is atomic by construction

This is the part that actually sold it. There is no update in place:

function appendRow(file: string, row: unknown): void {
  ensureDir();
  try {
    fs.appendFileSync(file, `${JSON.stringify(row)}\n`, 'utf8');
  } catch (err) {
    console.error(`[ledger] Failed to append to ${path.basename(file)}:`, err);
  }
}

A status change does not rewrite anything. It appends a new row carrying the same id:

export function updateReply(id: string, patch: Partial<ReplyRecord>): ReplyRecord | null {
  const current = fold(readRows<ReplyRecord>(REPLIES_FILE)).get(id);
  if (!current) return null;

  const next: ReplyRecord = { ...current, ...patch, id, updatedAt: new Date().toISOString() };
  appendRow(REPLIES_FILE, next);
  maybeCompactReplies();
  return next;
}

Compare that with the shape you get from a single JSON blob: read the file, parse it, mutate the object, serialise the whole thing, write it back. That read-modify-write has a window in it where the file on disk is neither the old state nor the new one, and the thing that lands in the window is a power cut during a 3am poll.

An append has no window. The previous bytes are never touched.

Reading is a fold, and the last writer wins

/** Fold append-only rows into current state, last write wins. */
function fold<T extends { id: string }>(rows: T[]): Map<string, T> {
  const byId = new Map<string, T>();
  for (const row of rows) {
    if (!row || typeof row.id !== 'string') continue;
    byId.set(row.id, { ...byId.get(row.id), ...row });
  }
  return byId;
}

Two things to notice.

It spreads rather than replaces, so a row does not have to be complete. A partial patch row written by some future code path still folds correctly over the full row that came before it.

And it skips rows without a string id instead of throwing. Which matters, because the file is not guaranteed to be well formed.

A torn last line costs exactly one row

for (const line of raw.split('\n')) {
  const trimmed = line.trim();
  if (!trimmed) continue;
  try {
    rows.push(JSON.parse(trimmed) as T);
  } catch {
    // A torn final line (power loss mid-append) must not poison the read.
  }
}

This is the whole durability story, and it is four lines long.

If the machine dies halfway through writing a line, the file ends in a fragment of JSON. That fragment is one unparseable line in a file of parseable ones. The reader drops it and carries on with every row before it.

The failure mode you are buying out of is the one where a single corrupt byte makes JSON.parse reject the entire history. Line-delimited formats are not just convenient for appending, they are the reason corruption stays local.

Compaction is the only place the file is rewritten

An append-only file grows forever, and every read is a full read. So there is a ceiling:

const REPLIES_MAX_ROWS = 8000;
const REPLIES_KEEP = 2000;

function maybeCompactReplies(): void {
  const rows = readRows<ReplyRecord>(REPLIES_FILE);
  if (rows.length <= REPLIES_MAX_ROWS) return;
  const folded = Array.from(fold(rows).values())
    .sort((a, b) => (a.createdAt < b.createdAt ? 1 : -1))
    .slice(0, REPLIES_KEEP);
  rewrite(REPLIES_FILE, folded);
}

Those two numbers are a read-cost budget, not a retention policy. Nothing in the product cares about a reply from four months ago. What it cares about is that the file the engine reads before every single reply stays small enough that reading all of it is free.

The rewrite is the one operation that cannot be an append, so it is done the only safe way:

const tmp = `${file}.tmp`;
fs.writeFileSync(tmp, rows.map((r) => JSON.stringify(r)).join('\n') + '\n', 'utf8');
fs.renameSync(tmp, file);

Write beside the target, then rename over it. A reader either sees the whole old file or the whole new one, because rename within a filesystem is atomic. A crash leaves a stray .tmp and an intact ledger.

Note also where maybeCompactReplies is called from: updateReply, not createReply. Creating a reply happens on the hot path, right when a listing has just been found and seconds matter. Updating one happens afterwards. The expensive housekeeping is deliberately attached to the call that is not in a hurry.

Append-only makes your status union a migration surface

This is the cost nobody mentions. Here is a real line from our status type:

/**
 * Legacy. Produced by the old dry-run rehearsal, which no longer exists.
 * Kept because the ledger is append-only and older rows still carry it.
 */
| 'awaiting_approval'

Nothing in the app can produce that status any more. It cannot be deleted, because somebody's replies.ndjson on disk still has rows with it, and those rows are still folded and still checked against the set of statuses that mean "this listing has been dealt with":

const TERMINAL_STATUSES: ReadonlySet<ReplyStatus> = new Set<ReplyStatus>([
  'queued', 'in_progress', 'awaiting_approval', 'sent',
  'probably_sent', 'failed', 'needs_review', 'duplicate',
]);

failed being in that set is the deliberate part, and I went through the reasoning in 21 reply statuses, and the 8 that mean never touch this listing again. What this post adds is the file underneath it: a dropped enum value in an append-only store is not a refactor, it is a data migration you have chosen not to write.

Nothing here throws

Every path logs and returns a default. ensureDir swallows its error and lets the append report it. A failed read returns []. A failed compaction leaves the file alone.

That is not laziness, it is the same rule the desktop notifications follow: this code runs inside a background poll loop whose actual job is finding rooms before somebody else does. A disk that has gone read-only is worth a log line and a degraded feature. It is not worth a dead monitor.

The rule worth taking away

If your state is a fold over small events, an append-only log is a database you do not have to ship. You get atomic writes from the filesystem, corruption containment from the line format, and a compaction step you can write in nine lines. You pay with a full read on every query and a type union you are no longer free to shrink.

See it working

The app is on notifio.app/download for Mac and Windows, and the auto-reply upgrade that uses this ledger is described on notifio.app/pricing.

If you want to see what the ledger is protecting you from, the per-site pages under notifio.app/alerts cover what each rental portal does with a message once it arrives, and notifio.app/help has the limits the engine enforces before it sends one.