An AI assistant can summarize a company in seconds. The harder engineering problem is making its answers reproducible.
Before adding another prompt, build a pipeline that preserves the evidence behind every financial claim.
Start With Structured Inputs
Use company filings as your starting point. The SEC provides APIs for submission histories and extracted financial data.
Keep the response alongside your normalized records. Each observation should retain its company identifier, reporting period, unit, filing date, and source reference.
A revenue figure without a period is an incomplete input.
Keep Arithmetic Outside the Model
Calculate ratios and growth rates in application code. Let the model explain validated results.
For example, revenue increasing from $100 million to $110 million represents 10% growth. However, that comparison becomes misleading if one figure covers a quarter and the other covers a year.
Validate:
- Matching currencies and units.
- Comparable reporting periods.
- Missing or revised observations.
- Zero denominators.
Return an explicit unavailable result when validation fails.
Treat Citations as Data
Store document references with retrieved passages. Require generated factual statements to reference those passages.
Then check whether the evidence actually supports the claim. A working link alone does not establish accuracy.
Separate reported facts, interpretations, and unanswered questions in the interface.
Test Failure Cases First
Try conflicting figures, missing data, repeated filings, and questions your dataset cannot answer.
The expected behavior should be visible uncertainty, rather than an unsupported conclusion.
For background on interpreting company decisions, I publish investing explainers at BetterThisWorldStocks, which I run.
Start with one narrow question and a reviewable answer. Expand only when your pipeline can show where every number came from.
What would you build first: the ingestion layer, calculation engine, or evidence viewer?
Top comments (0)