The first check we built to measure missing price data was 97% wrong, and nothing in the data said so.
Every backtest starts with an assumption nobody writes down: the price history is complete. Every trading day a
stock was listed, there is a bar.
Ours wasn't. And the first check we built to measure how incomplete it was turned out to be 97% wrong - with nothing in
the data to tell us.
The gap
We compared our own daily price files against the exchange calendar, stock by stock, inside each stock's own
first-to-last trading span since June 2025. 30,581 missing stock-days across 738 listed common stocks.
Most of them were names that had changed ticker or listed recently. The list of symbols our fetcher asked for only knew
names it already held, so a renamed company quietly stopped being fetched. One large bank was missing 86 daily
sessions after its ticker change.
The first check: "is there a bar?"
To separate real gaps from days a stock genuinely did not trade, we asked a market-data vendor one question per
missing day: do you have a bar for this stock on this day?
It said yes for 29,793 of the 30,581. That looked like a clean result: nearly every gap is real, go fill them.
What the bars actually were
Before filling anything, we read a few of the worst cases by hand. The bars were there - with zero volume and zero
trades. The price was simply the previous day's close, carried forward.
The vendor serves a bar for a day nothing traded. 19,066 of the "real" gaps were these placeholders.
Asking a different question - does the vendor's bar have trades? - the real number was 10,727 sessions across 240
stocks. All of them were filled the same morning (11,408 daily rows and 869,391 one-minute rows), and the same count
now reads 0. It runs every night.
Why this matters for a backtest
A placeholder bar is not harmless:
- Volume filters lie. A stock looks listed and priced on a day it had no market at all.
- Returns get smoothed. A flat day carried forward hides the real move, which then lands all at once on the next day that traded.
- "Completeness" checks pass. Any check that asks does a row exist? is satisfied by a row that says nothing happened.
We found the same thing in our own stored files: three sampled days held 189, 467 and 432 zero-volume rows.
The lesson
A check that asks does a record exist? measures the vendor's habits, not the market. Ask the question the record is
supposed to answer: did this stock trade?
What I don't know
- Whether the placeholder bars are documented behaviour, or a quirk of the way we ask for historical data.
- What the right storage policy is. A day with no trades arguably has no bar at all - but many tools expect one.
A question for you
Who has a reliable public rule for "did this stock trade today" that doesn't depend on the vendor's own bar - the
exchange's daily volume files, consolidated trade counts, something else? If you've solved it, I'd like to learn how.
Want this checked on your own data or backtest? Backtest & Data Audit, one-week pilot: https://altdataforge.com/audit
Research and education only - not investment advice.
Top comments (0)