Why an i.i.d. bootstrap lies about drawdown

Resampling is the cheapest way to answer "how much of my backtest is luck?" It is also, in the specific case of drawdown survival, biased in a direction that flatters the result — and the fix is not more samples. It is fewer, longer ones.

The claim being tested

Suppose you have a strategy's daily P&L. You want a distribution of outcomes, so you resample: draw days independently, with replacement, and simulate the rulebook on each synthetic sequence. This is the bootstrap, and it is a good tool for a lot of questions.

It assumes the days are exchangeable — that shuffling them changes nothing that matters. Under that assumption, a shuffled series is a valid draw from the same process.

For a drawdown rule, that assumption is false in a way that matters, because the order of the days is the mechanism. A floor is breached by a run. Shuffle the days and you do not preserve the runs; you shred them.

The measurement

One generator, one rulebook, three resampling schemes. The generator is a two-state regime process — a chop state with a 40% win rate and a trend state with a 62% win rate, switching with 4% probability per day. The rulebook is a trailing drawdown that locks at breakeven, on a $50,000 account with a $2,500 drawdown and a $6,000 target. 50,000 simulated accounts per row.

Resampling scheme	Pass rate	vs true path
true path (ground truth)	82.23%	—
i.i.d. days from the same pool	92.52%	+10.29 pp
contiguous blocks, B = 5	89.90%	+7.67 pp
contiguous blocks, B = 10	87.68%	+5.45 pp
contiguous blocks, B = 20	85.85%	+3.62 pp

Every scheme draws from the same 20,000-day pool. They differ only in how they pick from it. And the i.i.d. row is wrong by more than ten percentage points — in the direction of "your strategy is safe."

That number is a statement about the rulebook, not about the market: for a rule that can be breached by a losing streak, destroying the streaks is the error. It is not a minor statistical infelicity. It is modelling a different rule.

Why blocks help, and why block size is the actual decision

A block bootstrap draws contiguous stretches of days rather than single days, so within a block the temporal structure survives. The table shows the error falling monotonically as the block grows:

B = 5   +7.67 pp
B = 10  +5.45 pp
B = 20  +3.62 pp

The reason to stop is not that larger is bad. It is that a block cannot meaningfully exceed the length of the history you are drawing from. With a 20,000-day pool, a 20-day block is already reusing the same stretches across draws, so increasing it further starts to convert sampling error into mere repetition. The honest statement of the rule is: use the longest block your pool supports, and if that is still too short to contain the runs you care about, your pool is the problem.

The code

The whole difference between the correct and incorrect version is the inner loop.

// i.i.d. bootstrap — shreds streaks. Biased for any rule breached by a run.
function iidSample(pool, nDays, rnd) {
  const out = [];
  for (let k = 0; k < nDays; k++) out.push(pool[Math.floor(rnd() * pool.length)]);
  return out;
}

// block bootstrap — keeps runs together.
function blockSample(pool, nDays, block, rnd) {
  const out = [];
  while (out.length < nDays) {
    const start = Math.floor(rnd() * (pool.length - block));
    out.push(...pool.slice(start, start + block));
  }
  return out.slice(0, nDays);
}

And the comparison, with the true path as the reference rather than as an afterthought:

const pool = makeDays(20000, mulberry32(SEED + 55), { switchP: 0.04, bigP: 0.06 });

let truth = 0;
for (let i = 0; i < N; i++) {
  const days = makeDays(80, mulberry32(SEED + 1 + i), { switchP: 0.04, bigP: 0.06 });
  if (simulate(days, 'trailing_lock', CFG).out === 'PASSED') truth++;
}

let iid = 0;
for (let i = 0; i < N; i++) {
  if (simulate(iidSample(pool, 80, rnd), 'trailing_lock', CFG).out === 'PASSED') iid++;
}

// truth = 82.23%   iid = 92.52%

Worth stating plainly: the "ground truth" here is ground truth relative to the generator. It is not a claim about real markets. The experiment measures whether a resampling scheme recovers the process it was given, and i.i.d. does not — it cannot, because the process has persistence and the scheme deletes it.

The part that generalises

This is a member of a family of mistakes that all share a shape: applying a resampling scheme whose null hypothesis contradicts the mechanism you are testing.

• Resampling individual trades to test a drawdown rule assumes the losing run is not the thing that kills you. It is.
• Resampling to test a consistency rule assumes the concentration of profit is not the thing that disqualifies you. It is — and, incidentally, i.i.d. resampling understates concentration risk by averaging the fat days out.
• Resampling with the wrong block length in the presence of volatility clustering reproduces the same error with extra steps, because the thing that clusters is the variance, not the mean.

The cheap check before trusting any bootstrap: ask what property of the ordering your resampling scheme destroys, and whether the rule you are testing depends on it. If yes, single-day resampling cannot be used, and the block length has become a real modelling parameter that deserves to be chosen rather than defaulted.

Where this model is wrong

• The regime process is invented. Two states with symmetric switch probabilities is a caricature of trend/chop. Real persistence is asymmetric — trends decay faster than they build — and that asymmetry would push the numbers further apart, not closer.
• One trade per day, up to 80 days, so "runs" are at most 80 long and the pool is 20,000 days. Both are small enough that the block estimates carry their own sampling error, which I have not quantified.
• Survivorship in the pool. The pool is generated once from the process; it is not refreshed, so all block draws are conditional on that one realisation.
• Symmetric win/loss magnitudes. Every day is drawn from the same payoff family regardless of regime, so the regime affects only the win probability, not the size distribution. Real trend regimes change both.

If you just want the practical version: for a rule that fails on a streak, resample contiguous blocks, report the block length, and sanity-check the estimate against a simulation of the actual process when you have one. A bootstrap that cannot represent your worst run is not a risk estimate.

I keep a prop firm profit calculator that runs the cost side of this arithmetic — expected attempts, expected fees and the break-even pass rate for a given payout — for anyone who wants the numbers rather than the method.