Most retail "strategy guides" show you a beautiful equity curve from a backtest that was fit on the same data it trades on. I got tired of that, so I built a small honest backtesting toolkit for Micro Nasdaq (MNQ) futures and ran four classic day-trading strategies through proper walk-forward validation.
The honest results (2 years of MNQ 1h data, walk-forward):
| Strategy | OOS trades | Net PnL | Profit factor | Max DD | Verdict |
|---|---|---|---|---|---|
| Gap fade | 163 | +$2,145 | 1.14 | $3,116 | fails DD gate |
| EMA pullback | 118 | -$3,884 | 0.73 | ā | fails |
| Donchian breakout | 96 | +$225 | 1.03 | ā | noise |
| ATR trailing trend | 104 | -$3,899 | 0.49 | ā | fails |
0 of 4 passed my fail-closed gates (>=30 OOS trades, PF >= 1.3, net > 0, max DD <= 50% of net profit). That is the real answer for most public strategies: they don't survive out-of-sample.
What the toolkit gives you (plain Python, no exotic deps, pandas only):
- walk-forward split runner with fail-closed gates
- hand-written indicators (no TA-lib, no pandas-ta segfaults)
- intraday risk caps: daily loss limits, max trades/day, entry cutoff
- a paper-trading runner that replays the same logic live each day
I packaged it as the MNQ Backtest Toolkit ($29): https://stephimp.gumroad.com/l/zqciyo
The point isn't "buy this and get rich" ā it's that validation infrastructure is the product. If a strategy can't pass its own out-of-sample gates, no amount of leverage or a prop-firm challenge fixes it.
Happy to answer questions about the walk-forward methodology in the comments.
Top comments (1)
viki, this is the exact kind of ruthless empirical rigor the trading (and broader ml) space desperately needs. most people optimize for the prettiest equity curve, but you optimized for the most honest one.
treating "validation infrastructure as the product" is a massive mindset shift. the fail-closed gates are brilliant because they force you to confront the reality of out-of-sample decay rather than hiding it behind overfitting.
one sharp follow-up on the walk-forward methodology: how does your toolkit handle market regime changes between the in-sample (is) and out-of-sample (oos) periods? for example, if the is period is low volatility and the oos period is high volatility, a strategy might fail the gates not because the logic is flawed, but because the environment shifted. do you incorporate any volatility-adjusted normalization in your gates, or do you intentionally let it fail to prove the strategy isn't regime-dependent?
fantastic, no-hype breakdown. reproducible validation is the only way to separate signal from noise. šÆš