DEV Community

Daniel Pertu
Daniel Pertu

Posted on

Our Electron app has no test framework, and the only rule is that testable files may not import electron

Notifio is an Electron app that watches rental search pages and tells you the moment a new listing appears. It has no Jest, no Vitest, no test runner config, and no __tests__ directory. It has three files at the repository root and this in package.json:

"test": "pnpm test:baseline && pnpm test:backoff && pnpm test:scraper",
"test:baseline": "tsx test-baseline.ts",
"test:backoff": "tsx test-backoff.ts",
"test:scraper": "tsx test-scraper.ts",
Enter fullscreen mode Exit fullscreen mode

The marketing site next door in the same repository has a real suite, including a test that reads the rendered title of every generated page. The app deliberately does not, and the reason is not that tests are unimportant here. It is that in an Electron main process, almost nothing is testable by default, and the work that buys you tests is not the runner. It is the import graph.

The thing that kills a test process

Every writable path in the app comes from one module:

import path from 'path';
import fs from 'fs';
import { app } from 'electron';

// Root of all writable app data
export const USER_DATA_DIR    = app.getPath('userData');
export const DATA_DIR         = path.join(USER_DATA_DIR, 'data');
export const BROWSER_DATA_DIR = path.join(USER_DATA_DIR, 'browser-data');
export const CONFIG_PATH      = path.join(USER_DATA_DIR, 'config.json');
Enter fullscreen mode Exit fullscreen mode

That is a module level call to an Electron API. Import anything that reaches this file from a plain Node process and you get this, before a single line of your test runs:

export const USER_DATA_DIR    = app.getPath('userData');
                                    ^
TypeError: Cannot read properties of undefined (reading 'getPath')
    at fs (/app/src/paths.ts:14:37)
Enter fullscreen mode Exit fullscreen mode

Under node or tsx, require('electron') resolves to the npm package, whose export is a path string to a binary rather than the runtime API surface. So app is undefined, and the crash happens at import time, which means no test file that transitively imports paths can even load.

The usual answers to this are a module mock, a fake electron in the resolver, or a runner that boots a real Electron process for the suite. All three work. All three also mean your test for "should this search start over" is downstream of a mocking setup, and when that setup breaks you are debugging the harness rather than the product.

I took the other option, which is to make the logic worth testing not touch Electron at all.

Two files with a comment explaining why they are files

baseline.ts decides whether a search starts fresh or gets compared with what its page held last time. It is the rule that decides whether you get an email at all, and I wrote about the rule itself in the first check after a restart is not allowed to tell you anything. Its header says why it is its own module:

/**
 * Its own module because it is the rule that decides whether the user gets an
 * alert at all, and because keeping it free of Electron means the behaviour
 * below can be run rather than argued about.
 */
Enter fullscreen mode Exit fullscreen mode

backoff.ts decides how long to leave a site alone after it refuses us:

/**
 * Its own module for two reasons. It is the policy most worth being able to
 * read in one place, and it is pure: no browser, no Electron, no clock beyond
 * what is passed in, so the ladders below can be checked by running them
 * rather than by reasoning about them.
 */
Enter fullscreen mode Exit fullscreen mode

"Run rather than argued about" is the entire justification for the file layout. These two modules are the ones where I would otherwise be reasoning in a code review about what happens on the fourth consecutive refusal, and reasoning is exactly what gets that wrong.

Here is the import section of each, in full. baseline.ts has none at all: the file opens with its doc comment and then goes straight into export class BaselineTracker {. Not one import, which is why nothing can creep in behind it. backoff.ts imports exactly one type:

import { BlockKind } from './types';
Enter fullscreen mode Exit fullscreen mode

"No clock beyond what is passed in" is the other half. delayFor takes the current failure count, an optional Retry-After value from the site, and a random source, so the test can pin all three:

// Mid-range jitter, so a step comes back as exactly itself.
const noJitter = () => 0.5;
const NOW = 1_700_000_000_000;
Enter fullscreen mode Exit fullscreen mode

Jittered exponential backoff is normally annoying to test precisely because both of its inputs are ambient. Passing them in is a smaller change than any mocking library, and it also makes the production call sites read honestly, because they have to name the clock they are using.

The nine lines standing in for a framework

let failures = 0;

function check(label: string, ok: boolean, detail = ''): void {
  console.log(`   ${ok ? 'ok  ' : 'FAIL'}  ${label}${detail ? `  ${detail}` : ''}`);
  if (!ok) failures++;
}
Enter fullscreen mode Exit fullscreen mode

and at the bottom of each file:

console.log(failures === 0 ? '\nAll checks passed\n' : `\n${failures} check(s) failed\n`);
process.exitCode = failures === 0 ? 0 : 1;
Enter fullscreen mode Exit fullscreen mode

That is the whole framework. process.exitCode rather than process.exit so any pending output flushes. The output reads as prose, because the labels are sentences about the product and the file is organised by scenario rather than by function:

Starting the app
   ok    a fresh start re-bases a search left over from a previous run
   ok    the next check of that search is a real comparison
   ok    and every check after that stays a comparison
   ok    a search with no stored baseline does not re-base

Leaving it running
   ok    a search already checked this run does not re-base again
   ok    stopping and starting the monitor does not re-base  the tracker is not cleared by stop()
   ok    a restart re-bases even though the previous run had checked it
   ok    and the still-running tracker is untouched by it
...
All checks passed
Enter fullscreen mode Exit fullscreen mode

0.3 seconds, and I can read that output to someone who does not write code and they can tell me whether it is the behaviour they want. That mattered here more than coverage tooling would have, because the behaviour in question is a product rule I have already changed my mind about once.

The scenario headings are load bearing in a way I did not expect. The app's rule is about process lifetime, not elapsed time, so the test models "the app restarted" by constructing a new tracker and "the app kept running" by reusing one:

/**
 * A tracker lives exactly as long as the app's process does, so "the app was
 * restarted" is modelled by building a new one and "the app kept running" by
 * reusing the one already there.
 */
Enter fullscreen mode Exit fullscreen mode

Two of those checks exist only because a reviewer, reasonably, expected the opposite. Minimising to the tray and stopping the monitor from inside the app both keep the process alive, so neither is allowed to start anything over. stopping and starting the monitor does not re-base has the tracker is not cleared by stop() printed next to it, which is the implementation detail that makes it true and the one somebody will delete in six months.

The file on the wrong side of the line, and I left it there

I am not going to pretend the rule is universal. Here is the top of finds.ts, the recent finds history:

import fs from 'fs';
import { FINDS_PATH } from './paths';
Enter fullscreen mode Exit fullscreen mode

Second line, straight into Electron. That module is therefore not testable the cheap way, and I did not restructure it, because it is a bounded ring buffer whose logic is a Set and a slice, and injecting a path just to test it would add a seam to production code in exchange for asserting something I can read.

Which is the actual rule, stated honestly: the Electron free boundary is bought deliberately for the modules where the behaviour is a decision, and not paid for everywhere else. The useful property is that it is trivially visible. You do not audit a module for testability, you read its import list.

The third test is a smoke test for the scraping browser, and it has a different rule again:

/**
 * Runs independently of Electron, so it uses the real stealth and extraction
 * modules rather than a copy of them: a test that duplicates the code it is
 * testing tells you nothing once the two drift apart.
 */
Enter fullscreen mode Exit fullscreen mode

That one launches a real browser against a real page, so it is not something CI runs on every commit. The line that matters is the second half of the comment. The earliest version of that file had its own copy of the launch setup, "just for testing", which is a sentence that should set off an alarm: a test that reimplements the code under test is a test of a fork nobody ships.

Worth taking away

If you have an Electron main process you are afraid to test, the runner is not your problem. Find which of your modules import electron, directly or transitively, and then look at what that set has swallowed. In my case it had swallowed the two decisions that determine whether the product works at all, and pulling just those two out was a smaller job than choosing a mocking strategy.

The app is a free download for Mac and Windows. The behaviour those two test files protect is described in plain English on the help page and per rental site under alerts, for example Kamernet and Rightmove.

Top comments (0)