A language model asked to summarize an earnings report will, sooner or later, produce a revenue figure that is almost right. The digits look plausible, the sentence reads well, and nothing in the output tells you which numbers were copied and which were reconstructed. On a site that scores stocks, that is the worst failure mode available: a confident wrong number that looks like every correct one.
We work on Stock Expert AI, which scores 21,000+ US stocks from 0 to 100 with a MoonshotScore built from five pillars: quality, safety, valuation, growth and momentum. Our answer to the plausible-wrong-number problem is a rule we call dual-channel rendering: the model is never allowed to produce a figure. This post covers what that means in practice and where it hurts.
Two channels, one page
The rule is simple to state. The AI only writes commentary. Every number on the page is rendered verbatim from the market-data provider (Financial Modeling Prep, FMP). Those are the two channels:
- Data channel: values fetched from FMP and rendered by ordinary code. No model sits between the provider response and the pixel.
- Commentary channel: prose written by a model, which explains, contextualizes and connects things.
The point of splitting them is that you stop asking "is the model accurate?", which you can only answer statistically, and start asking "did a number cross from the commentary channel into the page?", which you can check mechanically. The model can be wrong about emphasis or tone. It cannot be wrong about a figure the page displays, because the page does not take figures from it.
That covers the structured parts of the site. It does not cover the prose itself, where the model will happily write "revenue rose to $46.06B" if you let it. That is where the second half of the design comes in.
The journal pipeline: evidence first, then prose
Our news section (the journal) is the place where the model writes the most text, so it has the most machinery around it.
Stories are not chosen by a model. They are picked from FMP data: the earnings calendar, top movers over two sessions with a $1B market-cap floor, and sector themes. We do not write about the same ticker twice within 7 days.
For each story we build a fact pack from FMP endpoints: statements, estimates, ratings, segments, insider trades and the earnings-call transcript. A mid-size model then writes the draft with that fact pack as its only source.
Then the draft goes through a gate before anything else sees it.
The number gate
The gate extracts every number in the draft and checks it against the fact pack, with rounding tolerance. A draft that says "$46.06B" should match a fact-pack value of 46,061,000,000, because the writer rounded to two decimals. A draft that says "$47B" should not.
The useful trick is deriving the tolerance from how the number was written. If the draft shows two decimals in billions, the allowed error is half a unit in the last place written, scaled up. Here is a simplified, generic version:
interface Parsed {
value: number;
tolerance: number;
}
const SCALE: Record<string, number> = { k: 1e3, m: 1e6, b: 1e9, t: 1e12 };
function extractNumbers(text: string): Parsed[] {
const out: Parsed[] = [];
const re = /\$?(\d[\d,]*)(?:\.(\d+))?\s?([kmbt])?\b/gi;
for (const m of text.matchAll(re)) {
const whole = m[1].replace(/,/g, "");
const frac = m[2] ?? "";
const scale = SCALE[(m[3] ?? "").toLowerCase()] ?? 1;
const value = Number(`${whole}.${frac || "0"}`) * scale;
// half a unit in the last digit the writer chose to show
const tolerance = 0.5 * 10 ** -frac.length * scale;
out.push({ value, tolerance });
}
return out;
}
function findUnsupported(draft: string, facts: number[]): Parsed[] {
return extractNumbers(draft).filter(
(n) => !facts.some((f) => Math.abs(f - n.value) <= n.tolerance),
);
}
// "$46.06B" -> 46.06e9 with tolerance 5e6, so 46_061_000_000 passes
This is deliberately naive. A production gate also has to deal with percentages, years, quarter labels, ranges and words like "double". The shape is what matters: extract everything that looks like a figure, require each one to be explained by evidence, and fail closed. Anything unsupported sends the draft back instead of being patched.
Fail-closed has a useful side effect. The gate does not need to understand the sentence. It only needs to know that a figure appeared and that nothing in the evidence accounts for it.
A second reviewer for what regexes can't see
The number gate catches figures that do not exist. It cannot tell that a real figure is attached to the wrong period, or that a sentence drifts into advice, or that a quote was bent. So a larger model reviews the draft, but only for a narrow list: wrong numbers, wrong periods, advice, and distorted quotes. We do not ask it whether the article is good. A narrow checklist is easier to evaluate and harder for the reviewer to wander off from.
Once an article passes, the site stores a SHA-256 hash of the approved text and a hash of the evidence. An article is indexable only while both hashes still match. If someone edits the text, or the underlying evidence changes, the article drops out of the indexable set until it is approved again. That turns "this was verified at some point" into a property the system checks on every request.
Long transcripts and the raw-source rule
Earnings-call transcripts are long, so a cheap model summarizes them before the writer sees them. That introduces an obvious risk: a summarizer could invent or distort a figure, and the writer would then repeat it with a clean conscience.
The fix is an ordering rule. The number gate always runs against the raw transcript, not the summary. If the summary introduces a figure that is not in the source, the gate has nothing to match it against and the draft fails. A summarization error cannot introduce a number into the published text, because the summary is never the thing we trust.
The same idea applies to other lossy steps: whatever compresses or rewrites the evidence, verification should point at the uncompressed original.
Serving it: HTML for crawlers, Markdown for AI crawlers
Crawlers get server-rendered HTML. Every article also has a plain Markdown copy at the same URL plus .md, intended for AI crawlers. The stack is React + Vite + TypeScript on the frontend, Express + TypeScript on the backend, Drizzle ORM on Neon PostgreSQL, a Cloudflare Worker in front, hosted on a Replit VM. None of that is exotic. The relevant point is that the Markdown copy comes from the same approved text as the HTML, so both stay inside the hash check.
Model calls all run headless on a flat subscription, and we choose the cheapest adequate model per task: a cheap one for summarizing, a mid-size one for writing, a larger one for review.
Trade-offs and what we would do differently
The prose gets plainer. A model that cannot cite a figure unless it is in the fact pack writes fewer vivid comparisons. Some drafts get rejected for a number that was technically true but derived, such as a growth rate the model computed itself. We treat derived figures as the model doing arithmetic, which is exactly what the rule forbids, so the fix is to compute them in code and put them in the fact pack.
The gate is only as good as the fact pack. If FMP is missing a field, a true statement fails the gate. We accept false rejections over false approvals, but it does cost throughput.
Extraction is the fragile part. Regex-based number extraction needs constant care around formats. If we started over, we would define a structured representation of figures earlier, so the writer emits tagged placeholders and code fills in the values, rather than parsing numbers back out of free text. That would make the two channels even stricter than they are now.
Hashing is cheap but unforgiving. Any whitespace-level change to approved text invalidates it. That is the behavior we want, but it means normalization rules need to be decided up front.
Closing
The core idea is small: do not trust the model with the thing you cannot afford to get wrong, and verify against the source rather than against another model's summary. A number gate, a narrow reviewer and a pair of hashes are not sophisticated, but together they make a specific promise we can check: the figures on the page came from the data provider, not from the model.
Top comments (0)