We pre-registered 13 simple strategies, ran them through four fixed gates, and published
the result even when it was unflattering: 13 → 0. Then we repeated it in four more
markets. 65 pre-registered tests, zero survivors in all five markets.
This is the protocol, the numbers after our published correction, and the part most
backtests skip: deciding the kill rule before you see the equity curve.
Educational content: methodology only, no signals, no return claims.
By **Mahdi Goodarzi* (g.dev/mahdigoodarzi), Gueta Quant —
Spanish source guide: validar un backtest sin autoengañarse.*
1. The four gates, fixed before execution
Every candidate had to pass all four, in order:
| Gate | Rule |
|---|---|
| G1 | Out-of-sample net return > 0 and Sharpe > 0 |
| G2 | Walk-forward cumulative result > 0 |
| G3 | Deflated Sharpe Ratio (DSR) ≥ 0.95, deflation family N = 14 |
| G4 | Probability of Backtest Overfitting (PBO) < 0.50 via CSCV with S = 16 |
The thresholds were written down first. That is the whole trick: a rule you can move
afterwards is not a rule.
2. What actually survived the funnel
| Market | Funnel | Best PBO |
|---|---|---|
| EUR/USD | 13 → 8 → 7 → 0 → 0 | 0.5639 |
| Gold | 13 → 9 → 8 → 0 → 0 | 0.1737 |
| S&P 500 | 13 → 10 → 8 → 0 → 0 | 0.5695 |
| Bitcoin | 13 → 5 → 4 → 0 → 0 | 0.2139 |
| Ethereum | 13 → 7 → 4 → 0 → 0 | 0.1557 |
These are the corrected counts from our published addendum (2026-08-16), which fixed an
execution-model error and a PBO convention. The zero survived the correction.
The most instructive number is the one that almost passed: the best DSR on gold was
0.8818 (SMA Cross) after the addendum — previously reported as 0.9444. Both are below
0.95, so the strategy was rejected either way. But note what happened: a correction moved
the "best" candidate substantially and the decision did not move, because the threshold
was fixed in advance. Had we chosen 0.94 after seeing 0.9444, we would have shipped a
strategy on a number we later revised to 0.8818.
3. The deflation is doing the work
SMA Cross, Momentum, Bollinger Reversion and Ichimoku passed G1 in all five markets.
They looked consistent. They still failed G3/G4.
That is the bias most backtests never price in: testing many variants and keeping the
winner. Deflation by the number of trials absorbs the apparent edge, because a search over
14 variants produces a maximum that is expected to be large even when nothing is real.
# Sketch of the gate logic — thresholds are arguments, not constants you edit later.
def passes_gates(trial, *, dsr, pbo, n_family=14, dsr_min=0.95, pbo_max=0.50):
g1 = trial.oos_return > 0 and trial.oos_sharpe > 0
g2 = trial.wfo_cumulative > 0
g3 = dsr >= dsr_min # deflated by the family of trials
g4 = pbo < pbo_max
return all([g1, g2, g3, g4])
The important line is not dsr >= dsr_min. It is that dsr_min and n_family were
declared in the pre-registration file before the run.
4. Why report the zero?
Three reasons, all practical:
- A published zero is auditable. Anyone can rerun the funnel. A published winner invites a search for the parameter that makes it look better.
- It calibrates expectations. If 13 textbook strategies die on four gates in five markets, a strategy that sails through deserves suspicion, not celebration.
- It kills the survivorship story. Our own losing variants stay in the record.
Our research notes are public, including the addendum that corrected our own numbers:
guetaquant-research.
5. Checklist you can copy
- Write the gates and thresholds before running anything.
- Fix the trial family size and record every variant, including the ugly ones.
- Report the funnel counts, not just the winner.
- Publish corrections as addenda, with the old and new numbers side by side.
- Treat "almost passed" as a rejection. It is the most seductive number in the table.
Study material under Colombia SFC Decreto 2555 de 2010 — financial-engineering education,
not investment advice. No signals, no managed accounts. Related:
walk-forward and the Deflated Sharpe ·
Monte Carlo permutation testing.
AI assistance was used for drafting and editing; the numbers, gates and corrections come
from the published research files.
Top comments (0)