Four ways your backtest lies — and a dependency-free Python kit that catches them

rytjy — Fri, 16 Oct 2026 15:03:28 +0000

My strategy looked fine in the backtest and lost money live.\n\nIf you've done this for a while, you already know the suspects. What surprised me is how mechanical they are — each one is a specific, checkable defect, not a vague "the market changed":\n\n- Look-ahead — signals silently shifted onto future bars.\n- A ledger that doesn't balance — cash/position conservation breaks, and the PnL is fiction.\n- Fees and funding counted zero times (or twice) — the cost model quietly drops out.\n- Exit logic that scans the whole window instead of the current bar.\n\nI packaged those four checks into a small, dependency-free Python toolkit (no numpy, no pandas):\n\n

\npip install backtest-audit-kit\n

\n\n- a look-ahead detector\n- ledger invariants (cash/position conservation)\n- a fee / funding double-count detector\n- mutation testing, to find blind spots in your own test suite\n\n*A worked example.* cases/ ships a real BTC backtest whose cost model was zero — reported PnL came out at roughly 5.7x the principal. But the structural problem was worse than the bug: the take-profit needed +0.08% while a round-trip taker fee is 0.10%, so every winning trade net-loses by construction. No amount of parameter tuning fixes that.\n\nRepo: https://github.com/rytjy/backtest-audit-kit\n\n**What check would you add?** I'm collecting the ones I haven't automated yet.

DEV Community: rytjy

Four ways your backtest lies — and a dependency-free Python kit that catches them