I sell one thing: a live data stream. Everything that has already finished is free, forever, and that free archive is the evidence the paid stream is measured honestly.
Last week I added a third thing between them — a public page showing today's numbers, deliberately late. It sounds like a small feature. It is actually the most dangerous kind of code in the codebase, because it reads the paid file and publishes part of it, and every bug in it is a bug that gives away the product.
Here is what I got right, what I got wrong, and the two bugs that only showed up in production.
The line: give away timeliness, not a new category
The temptation with a "free tier" is to publish a crippled version of everything. That is hard to explain and harder to defend, because every field becomes a negotiation.
The rule I landed on: the free page introduces no new category of number. Every field on it is already free elsewhere on the site the next morning. What it adds is when — and when is exactly what is metered.
So the paid product keeps the things that are actually different in kind: the per-strike ladder, the second-order series, the internal disagreement between three different ways of computing the same thing, and the path through the day. Not trimmed to a teaser. Absent.
That framing turned out to be worth more than the trim itself. It fits in one sentence, it survives contact with a pricing page, and it never invites "why is this field free and that one not".
One function decides, and everything else is presentation
The route that serves this page loads a payload and hands it to exactly one function. That function returns a plain object or null. Nothing else on the request path sees the payload.
This is not tidiness. My codebase already has a comment recording what happens when two routes serve one payload under two policies: someone finds the other spelling and walks through it. In my case a request path could be normalised after it had been checked, so //data/live.json and /data/LIVE.json both missed the rule that guarded /data/live.json and were then resolved into the very file the rule protected. Two parsers of one string will eventually disagree.
So: one canonicaliser, run once per request, consumed by both the gate and the file read. And one trim, called by one route. The new page is a third door into that file, and it is the reason I wrote the trim as a pure function with its own test file before wiring anything up.
Measure the delay against the stream, not the clock
The obvious implementation of "fifteen minutes behind" is: take the reading from now() - 15min.
Do not do that.
Consider what happens when your writer stalls. Wall-clock arithmetic keeps marching, the newest data stops moving, and the effective delay shrinks toward zero — you begin publishing the freshest thing you have at the exact moment your pipeline is unhealthy and nobody is watching.
Measure against the data instead:
// the newest reading at least DELAY_MIN older than the newest reading we hold
const newest = toMin(frames[frames.length - 1].t);
let chosen = null;
for (const f of frames) if (newest - toMin(f.t) >= DELAY_MIN) chosen = f;
if (!chosen) return null; // the whole session so far is inside the window
Now a stall makes the page older, never fresher. That is the direction a paywall should fail in. As a bonus the function needs no clock at all, which makes it trivially testable — the test suite hands it a synthetic payload and asserts the delay holds, including the stalled case.
The bug that would have given it away
My payload carries the same levels twice, in two shapes. There is a five-minute series, and there are session-level scalars.
For a finished day the scalars are the right answer — they are the published level set, the same numbers the archive page prints, and reading them means the two pages can never quote the same day differently.
For a day in progress the scalars mean "as of the most recent rebuild". Current. Undelayed. Publishing them would have handed over live values while the page's own timestamp claimed a fifteen-minute delay.
Two of my seven fields existed only as scalars. The lazy version of this function would have mixed sources — series where available, scalars where not — and the page would have been simultaneously delayed and not, with no visible symptom.
The fix was to read a live day entirely from the series, and accept that this means the whole row belongs to one moment rather than being a composite. That is also what makes the timestamp on the page true.
The test that pins it sets the scalars to absurd sentinel values:
levels: { flip: 1, cr0: 2, ps0: 3 }, // never publishable while live
and then asserts no live reading ever returns 1, 2 or 3. If someone later "simplifies" the function by falling back to the scalars, that test fails loudly instead of the paywall failing quietly.
Say "at least", because your delay composes
I shipped the page saying "15 minutes behind the live stream".
Then I measured it in production, cache-busted, and the reading was twenty-two minutes old.
Nothing was broken. My function controls the gap to the newest reading in the payload — and the payload has a lag of its own, because the frames are published behind the real-time tail. The two lags compose, and I had only ever reasoned about mine.
The timestamp on the page was never wrong. The sentence next to it was: it made a wall-clock promise the code cannot keep. The page now says "at least 15 minutes", which is true at both ends.
If you publish a delayed anything, this generalises: the stamp is the claim, the delay is a floor, and stating a floor as an equality is a promise your dependencies have not agreed to.
What I would take to the next one
Write the trim before the route. A pure function with a test file, then wire it. The test file becomes the paywall's regression suite rather than coverage-for-its-own-sake, and the two assertions that matter — never fresher than the delay, never reads the live source — are the ones you would otherwise verify by staring.
Pass the gate's own signal in, do not recompute it. My trim takes "is this day still being written" as an argument, from the same helper the paywall uses. Two implementations of "is this live" is the same class of bug as two parsers of a path.
Sentinel values beat mocks for "this must never be published". A magic 1 that shows up in output is unmistakable; a mock that returns plausible data hides exactly the bug you were testing for.
Look at it in production before you believe the copy. Both of my real bugs — the composed delay, and a page linking the new URL from a footnote nobody read — were invisible from the code and obvious from one curl and one log line.
This is from gex.live, which rebuilds SPX dealer positioning from the options tape. The page in question is /now; the research section is where the measurements live, and most of them are null results.
Top comments (0)