If your backtest joins fundamentals on the fiscal-period-end date, it is trading on numbers that did not exist yet. That's the whole bug, in one sentence. A company's fiscal year ends, and then — weeks later — a 10-K gets filed and the numbers become public. Join on the wrong date and every fundamental data point in your backtest arrives early.
We measured how early. Across the 313,562 point-in-time rows in our dataset (5,214 US companies, built from raw SEC EDGAR filings), on rows with a reliable filing date, fundamentals became public an average of 43 days after the fiscal period ended. The maximum in our sample was 61 days.
Forty-three days of free clairvoyance on the average row, compounding across every rebalance. That is why fundamental strategies so often look great in testing and evaporate live.
This post covers what lookahead bias is, the measurement, how to check your own pipeline for it, and what the honest fixes are — including the free ones.
What is lookahead bias?
Lookahead bias is when a backtest uses information that was not yet publicly available at the simulated decision time. The simulation "knows" something the market didn't, so the strategy's historical results are inflated in a way that cannot be replicated live.
It's the quieter sibling of survivorship bias, and it's more dangerous, because it doesn't require a broken universe or a bad vendor — it hides inside a perfectly innocent-looking merge(). The data is real. The values are correct. Only the timestamps are lying.
Price data mostly can't have this problem: a trade prints and is public within milliseconds. Fundamental data is the worst offender, for a structural reason.
Why fundamentals are the worst offender
Every fundamentals row has two dates, and most datasets only make one of them easy to use:
- Period end — the date the fiscal year closed. This is the date the numbers describe.
- First filed — the date the 10-K hit EDGAR. This is the date the numbers became knowable.
The gap between them is dead air during which the company knows its results and you don't. SEC deadlines let a 10-K be filed up to 60 days after fiscal year-end for large accelerated filers, 75 for accelerated filers, and 90 for everyone else — and late filers can extend beyond that.
Naive datasets index everything by period end, because that's the natural key for the accounting. But your backtest doesn't care what period the number describes; it cares when a trader could have acted on it. Those are different questions with different dates, an average of 43 days apart in our data.
A concrete example: Apple's FY2024
These rows come straight from our dataset (you can verify them in the free sample — no signup):
| FY2024 period end | 2024-09-28 |
| FY2024 10-K filed | 2024-11-01 (34 days later) |
| Revenue as first reported | $391.0B |
| Diluted EPS as first reported | $6.08 |
Now ask the only question a backtest should ever ask: what was the newest annual revenue you could honestly know about Apple on a given date?
| As-of date | Newest AAPL annual revenue honestly knowable | Filed |
|---|---|---|
| 2024-10-15 | FY2023 — $383.3B | 2023-11-03 |
| 2024-11-15 | FY2024 — $391.0B | 2024-11-01 |
Same company, one month apart, a different "latest known" number. A naive period-end join hands your October 2024 backtest the FY2024 figures — a month before the market saw them.
And note: Apple is the best case on timing. Across 12 fiscal years in our data, Apple filed its 10-K 30–37 days after year end, faster than the 43-day cross-sectional average — the average company gives your backtest a bigger peek than this. Even Apple, though, isn't clean on what comes next: run the sample and you'll find five AAPL rows flagged restated. FY2018–19 diluted EPS and share counts were retroactively rewritten by exactly 4× when the 2020 stock split was applied to prior years — the diluted EPS the market first saw as $11.91 is carried in current data as $2.98 — and FY2017 operating cash flow was later revised by about 1%. Which is the second problem.
The second leak: restatements
Fixing the filing date is necessary but not sufficient, because filed numbers don't stay put. Companies restate and revise: 10-K/A amendments, reclassifications, corrections, retroactive split adjustments. In our dataset we found 18,539 rows where a later filing revised a previously reported value by more than 0.5% on the same XBRL tag.
Here's why that leaks even in a dataset that has a filing-date field. Most budget fundamentals APIs store one mutable value per period. When a restatement lands, the historical value is silently overwritten. Your backtest then reads the corrected number stamped with the original filing date — information that genuinely did not exist on that date. The lookahead is smaller and rarer than the 43-day version, but it's concentrated exactly where it hurts: in the companies whose numbers turned out to be wrong.
A genuinely point-in-time dataset has to keep both values: what the market first saw, and what we now believe is true.
How to detect lookahead bias in your own backtest
You don't have to take any of this on faith. Two checks, both cheap.
1. Audit your join dates against real filing dates
Our free sample (40 large caps, 3,212 rows, CSV) carries a first_filed column derived from the actual EDGAR filings. Merge your vendor's data against it and count the leaks:
import pandas as pd
# free sample: github.com/christianpichichero-max/pit-fundamentals
pit = pd.read_csv("data/pit_fundamentals_history.csv",
parse_dates=["period_end", "first_filed"])
pit = pit[pit["filed_reliable"]]
print((pit["first_filed"] - pit["period_end"]).dt.days.describe())
# mean lag ≈ 43 days — on this sample and on the full 313,562-row dataset
# your data: ticker, fiscal_year, and the date your backtest
# currently *sees* the row (whatever you join on today)
m = vendor.merge(
pit[["ticker", "fiscal_year", "first_filed"]].drop_duplicates(),
on=["ticker", "fiscal_year"])
leaked = m["visible_date"] < m["first_filed"]
print(f"{leaked.mean():.0%} of rows were visible to your backtest "
f"before they were public")
If that percentage isn't ~0%, your historical results include information that hadn't been published yet.
2. The lag-shift test
Re-run your backtest with every fundamental data point delayed to period_end + 90 days (a conservative bound on public availability). If your Sharpe or CAGR degrades materially, the difference was never alpha — it was the peek. A strategy that only works when it sees filings early is a strategy that doesn't work.
Softer symptoms worth suspicion: fundamental signals that seem to "time" earnings surprisingly well, value strategies whose turnover clusters near fiscal year-ends, and — the classic — a live track record that decays immediately relative to the backtest.
How point-in-time data fixes it
A point-in-time (PIT) dataset makes knowability the primary index. Every value in ours carries:
-
first_filed— the date the number actually became public (the PIT stamp) -
original_value— the number as first reported: what the market saw. Use this for backtests. -
latest_value— the current post-revision number. Use this for present-day screening. -
restated— flagged when a later filing moved the value >0.5% on the same tag (including 10-K/A amendments) -
qa_status— per-row validation result. Rows we can't fully trust are flagged, never hidden.
The query semantics then enforce honesty server-side. Our API is one endpoint:
GET /v1/fundamentals?ticker=AAPL&as_of=2024-10-15
This returns only rows where first_filed <= as_of — so the 2024-10-15 query above returns FY2023 and earlier, never FY2024, no matter how you write your backtest loop. (One honest edge to know: the comparison is inclusive, so a filing dated exactly on your as-of date is returned. If you want to be strict about same-day tradability, subtract a day.)
One more thing, because trust in a dataset should be earned, not asserted: before launch we had independent agents adversarially re-derive values from the raw EDGAR filings and compare them against ours. They found one real bug in our revision tracking. We fixed it before shipping. The full methodology — tag resolution, duration filters, restatement thresholds, QA rules — is published in the open alongside the sample.
Your options, honestly compared
We sell one of these options, so calibrate accordingly. But the comparison below is real, and for some readers the right answer is not us.
1. DIY from SEC EDGAR — free. EDGAR's companyfacts API is public domain, free, and needs no key. This is what we build from, and you can too. The catch is engineering, not access: companies switch XBRL tags over time (Apple's revenue has lived under three different tags since 2014), stub periods and quarters can leak into annual figures, and reconstructing first-reported values through restatements is genuinely fiddly — that's where our own audit found a bug. If you have the time, our published methodology is a usable blueprint. Budget a few weekends, not a few hours.
2. Budget fundamentals APIs (~$15–100/mo). Many include a filing-date field, which solves the 43-day problem if you remember to join on it. The structural gap is the mutable single value per period described above: restatements silently overwrite history, so the "point-in-time" join can still leak, and you can't reconstruct what the market originally saw.
3. Sharadar and Tiingo — the credible budget incumbents. We will not pretend to be the only affordable PIT-aware option, because we aren't. Sharadar (via Nasdaq Data Link, roughly $550–830/yr for personal use) offers as-reported dimensions, quarterly data, ~20,000 companies including delisted ones, and history to 1998 — if you need quarterly or delisted coverage today, buy Sharadar; it is better than us at those things. Tiingo's asReported fields are also genuinely PIT-relevant. The tradeoffs: you assemble the as-of join yourself from dimension tables, pricing sits behind a login, and Sharadar's non-professional license excludes money-management use.
4. Research-grade institutional PIT ($10k–50k/yr). Compustat Snapshot, LSEG, Calcbench, Intrinio (~$9.6k/yr). These are the genuine article — deep history, full revision archives, survivorship-bias-safe universes — and if someone is paying you to manage money, they are probably the right call. They are also priced for institutions and generally won't sell to an individual with a Stripe card.
5. Tradevo Data — the middle. $49/mo, one documented JSON endpoint with server-side as_of, original_value vs latest_value on every row, restatement flags, per-row QA, 5,000 requests/day, key issued instantly after checkout. We think of it as the honest budget tier of a product that has historically cost $10k–50k/yr — not "better data than everyone," just lookahead-safety as the default query semantic, at a price an individual can justify.
What our data doesn't do (yet)
Stating this up front so you don't find out after paying:
- Annual only — 10-K and 10-K/A. Quarterly (10-Q) is on the roadmap, not shipped.
- US companies only (5,214 of them).
- 7 concepts: Revenue, NetIncome, Assets, StockholdersEquity, OperatingCashFlow, EPSDiluted, DilutedShares.
- Up to 12 fiscal years of history.
- No bulk Parquet download yet — it's a query API.
-
first_filed <= as_ofis same-day-inclusive, as noted above.
If any of those is a dealbreaker, the comparison section above has options that fit better, and we'd rather you use them than be disappointed here.
Check the work yourself
The free sample is on GitHub — 40 large caps, 3,212 point-in-time rows, full methodology, no signup: github.com/christianpichichero-max/pit-fundamentals. Run the audit snippet above against your current vendor. If your pipeline comes back clean, excellent — you lost ten minutes and gained a verified negative.
If it doesn't, and you want the fix as an API instead of a weekend project: the full dataset is $49/mo at tradevodata.com, key issued instantly, docs at tradevodata.com/docs, cancel anytime.
Tradevo Data is a product of Tradevo Technologies Inc. Source data is U.S. SEC EDGAR (public domain). We sell a dataset, not signals — nothing here is investment advice, and no data product can make a bad strategy good. It can only stop a backtest from lying to you.
Top comments (0)