DEV Community

Daniel Pertu
Daniel Pertu

Posted on

The app found the listing, and then had nothing to show you

Notifio watches rental search pages and tells you when something new appears. Here is a bug that is not a crash, not a wrong answer, and was in the product for months.

You open the app. It has been running all night. It found you four rooms. The activity log says:

3 new listing(s) on Kamernet - Leiden
Enter fullscreen mode Exit fullscreen mode

And that is everything the app can tell you. The links are in an email. If the email failed to send, or went to spam, or you deleted it on your phone while half awake, the app that did the work has no copy.

 * Until now a find existed only as an email. If the user missed it, or the
 * send failed, or they just wanted to look at what turned up this morning,
 * there was nothing in the app to look at.
Enter fullscreen mode Exit fullscreen mode

The fix is a hundred lines. The interesting part is why none of the three stores we already had could hold it.

Three stores, none of them the right home

The per-search snapshots. The monitor keeps, for each search, the listings that page held on the previous check, and diffs against it. That file already contains listing ids, titles and URLs, so it looks like a history. It is not:

 * It is separate from the per-search snapshots on purpose: those are a diffing
 * baseline keyed by URL and get deleted whenever a search is re-pointed, which
 * is exactly when the user least wants their history to vanish.
Enter fullscreen mode Exit fullscreen mode

A baseline is a statement about now: this is what the page held last time I looked. Change the search URL and the old baseline is not just stale, it is wrong, so it gets deleted. Storing history in it means you lose the history at the exact moment the user has narrowed their search and cares most.

config.json. The obvious dumping ground, and the file the monitor reads on every poll:

// The recent-finds history shown in the app. Kept out of config.json so a long
// history can never make the file the monitor reads every poll any bigger.
Enter fullscreen mode Exit fullscreen mode

Appending an unbounded list to a hot file is a performance problem you get to discover slowly, months later, on the machines of your heaviest users.

The email itself. Which is the bug.

So: a new file, one job, bounded.

A ring buffer with a number you can defend

/**
 * How many finds to keep. A busy hunt produces a few hundred a week; this is
 * enough to cover "what did I miss overnight" without the file growing forever.
 */
const MAX_FINDS = 200;
Enter fullscreen mode Exit fullscreen mode

The comment matters more than the constant. 200 on its own is a number someone will "optimise" later. 200 plus the sentence explaining that the feature answers "what did I miss overnight" tells the next person what would have to change for the number to be wrong.

The write is the whole implementation:

export function add(items: Omit<Find, 'foundAt'>[]): Find[] {
  if (items.length === 0) return [];
  const existing = read();
  const seen = new Set(existing.map((f) => f.id));
  const foundAt = Date.now();
  const fresh = items.filter((item) => !seen.has(item.id)).map((item) => ({ ...item, foundAt }));
  if (fresh.length === 0) return [];
  write([...fresh, ...existing].slice(0, MAX_FINDS));
  _listeners.forEach((fn) => {
    try { fn(fresh); } catch { /* a broken listener must not lose the find */ }
  });
  return fresh;
}
Enter fullscreen mode Exit fullscreen mode

Three things in there are load bearing.

It returns what it actually added, not what it was given. The ids are stable per listing, so a listing that is already in the history is filtered out. The caller uses the return value, which means "notify about exactly these, and nothing twice" is enforced by the store rather than remembered by each caller.

Newest first, then truncate. slice after the prepend, so the buffer is bounded by construction and there is no separate eviction path that could be forgotten.

A listener that throws must not lose the write. The file is already saved by the time listeners run, and a subscriber blowing up cannot stop the remaining subscribers or the monitor. A UI bug taking out the thing that finds rooms would be an absurd way to lose a flat.

Fail soft on read, atomic on write

function read(): Find[] {
  if (_cache) return _cache;
  try {
    const parsed = JSON.parse(fs.readFileSync(FINDS_PATH, 'utf8'));
    _cache = Array.isArray(parsed) ? (parsed as Find[]) : [];
  } catch {
    // Missing or corrupt: an empty history is a fine place to start and must
    // never stop the monitor.
    _cache = [];
  }
  return _cache;
}

function write(finds: Find[]): void {
  _cache = finds;
  try {
    const tmp = `${FINDS_PATH}.tmp`;
    fs.writeFileSync(tmp, JSON.stringify(finds), 'utf8');
    fs.renameSync(tmp, FINDS_PATH);
  } catch (err) {
    console.error('[finds] Failed to save:', err);
  }
}
Enter fullscreen mode Exit fullscreen mode

Write to a temp file, then rename, because rename is atomic on both platforms we ship. A crash or a force quit mid-write leaves either the old file or the new one, never half a JSON document. Every local store in this app writes this way, and it costs one extra line.

Note the asymmetry in how failures are treated. A failed read is silently an empty history, because the user's history is nice to have and the monitor must keep running. A failed write is logged, because it means something is wrong with the data directory and the log is where we will look. Neither throws. Nothing in a background poll loop should be able to take the loop down for the sake of a display feature.

Array.isArray(parsed) is the small one. The cast to Find[] is a promise the compiler cannot check, since these bytes came off a disk we do not control. One runtime check at the boundary is the difference between an empty list and a crash on .filter is not a function.

Now that a find is a record, the ordering changes

Once finds are stored, the code that runs when new listings are detected has three channels to fire, and the order is a product decision:

// The local channel goes first. It costs no network round trip, so the
// banner is on screen before the email has left the building, and on a
// rental site the first few minutes are the whole difference.
const fresh = finds.add(
  newListings.map((listing) => ({
    id: listing.id,
    siteName: site.name,
    siteUrl: site.url,
    title: listing.title,
    url: listing.url,
  }))
);
if (fresh.length > 0 && config.notifications?.desktop !== false) {
  notifyFinds(site.name, fresh, log);
}
Enter fullscreen mode Exit fullscreen mode

Store, then banner, then email. The store has to go first because the notification is built from its return value, which is what makes the banner impossible to send twice. And the local channel goes before the network one because a native notification is instant while an email has a send queue in front of it, which is the whole argument the product rests on: Count the hops: why "instant" notifications are never instant.

The same onAdd listener that fires the banner also pushes the find down the app's event stream:

/**
 * Subscribe to newly recorded finds. The local API server uses this to push
 * them straight down the event stream, so a listing appears in the app at the
 * same moment the desktop banner shows rather than on the next refresh.
 */
export function onAdd(fn: Listener): void {
  _listeners.push(fn);
}
Enter fullscreen mode Exit fullscreen mode

One write, one subscription point, three surfaces that cannot disagree: the list in the window, the banner, and the email. Compare with the version where the UI polls a file every few seconds and the banner is fired from somewhere else, and you can see where the "banner said 3, list shows 2" bug would have come from.

One more ordering detail from the same function, which took a real bug to learn: the new baseline is not saved at this point. It is queued and persisted only after the alert is actually delivered. If the process dies between finding a listing and telling you about it, the next run should find it again. That whole family of decisions is in Most of our diff code exists to not send an alert.

The general shape

The question that unlocked this was not "where do I put the history". It was "what is this file a statement about?"

  • A baseline is a statement about what a page holds right now. It is disposable, keyed by search, and must be deleted when the search changes.
  • A find is a statement about something that happened. It is append only, keyed by listing, and must survive everything.

Those are different lifetimes, so they are different files. Storing them together would mean one of the two rules has to lose, and the one that loses is always the user's history, because the poll loop's needs are the ones with a deadline.

Have a look

The finds list is the panel the app opens on now, and the first-run behaviour it implies (a fresh start never alerts, it establishes a baseline silently) is written up at notifio.app/help. The app itself, with live demos of the monitoring it is recording, is at notifio.app, and the installers are at notifio.app/download.

Top comments (0)