TL;DR: The ordinary bootstrap resamples your observations at random, which silently assumes they carry no information about each other. On a time series that assumption fails, and the price is a confidence interval that is far too narrow: on an autocorrelated series I built with a true mean of exactly zero, the i.i.d. bootstrap's standard error for the mean came out 2.4x too small, and in a repeated coverage study on fresh series from the same process its nominal 90% interval covered the true mean only 49.6% of the time: a "90% interval" that is really a coin flip. Resampling contiguous blocks instead of single points keeps the dependence and lifts that coverage to 79.7%. It does not fully fix it, and by how much is part of the story.
I built a time series whose mean I knew exactly, because I chose it: zero. Then I asked a bootstrap for a 90% confidence interval on that mean, the way you would ask any statistics library, and it answered [-0.51, -0.19]. Read that back. A ninety-percent interval that does not contain zero is a ninety-percent-confident claim that the mean is negative (a real, signed effect) on data I generated to have no such thing. The interval was tight, it was confident, and it was wrong.
Nothing in the call was exotic. The series was an AR(1) process, the simplest autocorrelated series there is: each value is 0.7 times the previous value plus a fresh random shock. Autocorrelation just means today's value carries information about tomorrow's, the thing that makes a time series a time series and not a bag of independent draws. I used the default bootstrap, the one every tutorial reaches for first. And it lied to me by a factor of two, with a straight face.
This is the failure the whole tsbootstrap library exists to fix. Here is where the lie enters, and what to do about it.
What the bootstrap actually does
The bootstrap is one of the most useful ideas in applied statistics: to find out how uncertain a number is, resample your data with replacement many times, recompute the number on each resample, and look at how much it wobbles. You have one dataset, but the bootstrap manufactures thousands of plausible alternate datasets out of it and lets the number's spread across them stand in for the uncertainty you could not otherwise measure. No formula, no parametric model you have to name out loud. It is close to magic, and it usually works.
The usually is doing a lot of work. The ordinary bootstrap (i.i.d., for independent and identically distributed, the label you will see in the figures) draws each resampled observation independently and uniformly from your data: pick a row at random, put it back, pick another. That independence is the entire assumption. Drawing observations independently is a mathematical statement that the order of your data does not matter, that the rows are interchangeable. For a bag of independent measurements, they are. For a time series, they are not, and shuffling them destroys the one property that made the data worth collecting in order.
{/* Figure 1: the animated block re-tiling loop (HyperFrames). Figure.astro serves the /assets/terminal/*.gif reference as a muted-loop
The same series, resampled two ways. The block bootstrap lifts out contiguous chunks, so the resampled series still moves like the original; the i.i.d. bootstrap shuffles single points and shreds the wave into noise. The shredded version looks more ‘random,’ and that is why it reports a smaller standard error (the typical miss you should expect between the sample mean and the true mean) than the truth (grey line).
Watch the two panels. On the right, the i.i.d. shuffle takes my autocorrelated wave and scatters its points at random; the resampled series looks like static. On the left, the block bootstrap lifts out contiguous runs of the series and lays them back down intact, so the resampled version still rises and falls in the same lazy way the original did. That static-looking version is the one that understates uncertainty, the opposite of what your gut says.
Why shredding the wave shrinks the interval
When observations are positively autocorrelated, they move in the same direction for a while. A run of above-average values tends to be followed by more above-average values before the series wanders back. That persistence means your sample does not contain as much independent information as its length suggests: two hundred points that each nudge their neighbor are worth fewer than two hundred points drawn fresh and independent. Fewer effective observations means the sample mean is a shakier estimate of the true mean than the raw count of points implies.
You can put a number on "fewer." For an AR(1) at phi = 0.7 (phi is that 0.7 multiplier from earlier, how strongly each value leans on the last), the effective sample size (the number of independent points that would carry the same information about the mean) is n / ((1 + phi) / (1 - phi)) (the denominator counts how much each point merely repeats its neighbors' information), which for my two hundred points is 200 / 5.67 = 35. Two hundred autocorrelated readings are worth about thirty-five independent ones. The i.i.d. bootstrap computes the uncertainty of a mean built from two hundred independent points; the truth is the uncertainty of a mean built from thirty-five. That shrinkage, two hundred down to thirty-five, is where the factor of 2.4 comes from: uncertainty in a mean shrinks with the square root of the number of independent points, so the inflation is the square root of the ratio, sqrt(200 / 35) = 2.4.
The i.i.d. bootstrap never sees any of this. By drawing points independently, it builds resampled series that have no autocorrelation at all: it manufactures the "two hundred independent points" fiction whole, and it measures the wobble of the mean under that fiction. So it reports the uncertainty you would have if your data were independent, which is smaller, often much smaller, than the uncertainty you actually have. The scattered series on the right of the figure is not a bug in the resampling; it is the resampling doing just what it was designed to do, on data that violates its one assumption.
The block bootstrap fixes this by resampling contiguous blocks (chunks of consecutive observations) instead of single points. Because each block is a run of real, consecutive data, it carries the local ups and downs with it. Glue enough blocks together and the resampled series keeps the autocorrelation of the original, so the wobble of its mean reflects the real, reduced information content of dependent data. It resamples in a way that respects the assumption the i.i.d. bootstrap breaks.
The version in the figure is the moving-block bootstrap: slide a window of fixed length along the series, and resample whole windows with replacement. It is the oldest and simplest of the block methods; the others differ in how they choose and glue the blocks, and they get their own article later in this series.
How wide should the interval actually be?
I do not have to guess at the truth here, because I built the data. For an AR(1) series you can write the honest standard error of the mean in closed form. The standard error is just the typical size of the gap between your sample mean and the true mean, how far off you should expect to be. For independent data it is sigma / sqrt(n), the data's typical spread divided by the square root of the count, where sigma is the standard deviation of the series itself, not of the shocks driving it. For an AR(1) with autocorrelation phi, dependence inflates it by a factor that depends only on phi:
SE_true = (sigma / sqrt(n)) * sqrt((1 + phi) / (1 - phi))
That square-root factor is the whole story. At phi = 0.7 it equals sqrt(1.7 / 0.3) = 2.38. The honest standard error of the mean is nearly two and a half times larger than the one you would get by pretending the data were independent, and the i.i.d. bootstrap, by construction, targets the independent one. It is 2.4x too confident, before you have done anything wrong yourself.
The factor is a property of how autocorrelated the data is, not of which block method you pick. At
phi = 0.3it is only 1.36, so the lie is mild; atphi = 0.9it is 4.36, and a naive 90% interval misses the truth most of the time. Stronger dependence, bigger lie. Myphi = 0.7sits in the range plenty of real economic and sensor series actually occupy.
Put the standard errors side by side and the collapse is obvious. On my one example series, the i.i.d. bootstrap's standard error for the mean is 0.097. Every block method lands far higher, 0.16 to 0.20, and the analytic truth is 0.24. The i.i.d. bar is about two-fifths the height of the truth.
Standard error of the mean, one series, six ways to measure it. The naive i.i.d. bootstrap (orange) collapses to 41% of the true long-run standard error (grey rule). Every block method (blue) reaches most of the way to it, though notice none of them quite gets there, which will matter in a moment. The gaps between the four block bars are single-series noise, not a ranking.
I have shipped this exact mistake
I did not learn this from a textbook warning. I learned it because I once reported a result that was not there.
Years ago I had a run of daily measurements and a question about whether their average had shifted from a baseline of zero. I did the responsible-looking thing: I bootstrapped a confidence interval instead of trusting a formula, the interval excluded zero, and I wrote down that the shift was real. It was not. The measurements were autocorrelated (of course they were, they were daily readings of a slow physical process), and my "responsible" bootstrap had shredded that dependence the same way the figure above shredded the wave. The effect I found was nothing more than the interval's collapse. Nobody caught it, because nothing looked wrong; a tight interval that excludes zero is the most ordinary output in the world. That is the quiet part of this failure that unsettles me most. It does not announce itself.
So how often does it actually lie?
One series and one anecdote are not evidence, so I ran the experiment properly. Generate a fresh AR(1) series with phi = 0.7, length 200, true mean zero. Compute the i.i.d. bootstrap's 90% percentile interval and the moving-block bootstrap's 90% percentile interval for the mean, the same simple interval construction for both, so the only thing that differs between them is the resampler. Check whether each one contains the true mean of zero. Do that two thousand times and count.
This is a coverage study, the honest test of any confidence interval. A method that advertises 90% coverage is making a falsifiable promise: across many datasets, its intervals should contain the truth 90% of the time. Fewer than that and the interval is too narrow; the "confidence" is inflated.
The i.i.d. bootstrap's nominal 90% interval covered the true mean 49.6% of the time.
Not 90%. Not 85%. A coin flip. Half the time the interval you would have reported, and believed, does not contain the answer, and it never tells you which half you are in.
Consider what that does to everyday inference. A 90% interval that excludes zero is the standard way people declare an effect "significant" at the 10% level. If that interval is really covering the truth half the time on autocorrelated data, then every such call on the mean of a series with this much dependence carries a one-in-two error rate, not the one-in-ten it advertises. The analyst did nothing careless; the default resampler shipped them the wrong width. I am not claiming this voids any particular published result; I am claiming the gap between 90% and 49.6% is large enough that the burden is on the interval to prove it earned its confidence, and the ordinary bootstrap on a time series has not.
The moving-block bootstrap, on the same two thousand series, covered the truth 79.7% of the time. Its intervals were about twice as wide on average. That extra width is the uncertainty that was really there all along, the uncertainty the i.i.d. bootstrap deleted.
The 49.6% is not a fluke of the simulation. You can predict the expected i.i.d. coverage from a closed-form normal approximation using the same
phi = 0.7: it comes out to 51.0%. The measured 49.6% lands within 1.5 percentage points of that (consistent with Monte-Carlo noise), so the undercoverage is structural, not a simulation artifact.
The promise against the delivery, on one scale. Both methods advertise the same 90% (grey rule). The i.i.d. bootstrap delivers 49.6%, a coin flip wearing a confidence interval; the moving-block bootstrap delivers 79.7%, most of the way back and measurably not all of it.
But 79.7% is not 90% either. The block bootstrap recovers most of the gap the i.i.d. bootstrap opens, and it stops short. At a series this short, block resampling still understates the truth: you can see it coming in the chart above, where even the tallest block bar fell shy of the grey line. The two shortfalls are not the same kind of wrong: give the block bootstrap a longer series and its coverage climbs toward the 90% it promises, while the i.i.d. bootstrap's gap never closes, because it is baked into the resampler rather than the sample size. Block bootstrapping is the right tool here, and it is honest about its own limits; a piece that sold it as a cure would be committing a smaller version of the same sin I am describing. An interval that delivers half its promised coverage is a different category of wrong.
The fix is contiguity, and it is one argument
Everything above reduces to a single call. The library takes the observed series, the statistic you care about, and a method (a typed description of how to resample) and returns the interval:
from tsbootstrap import IID, MovingBlock, conf_int
# an AR(1) series, phi=0.7, n=200, whose true mean is exactly 0
def ar1(n, phi, seed, burnin=500):
rng = np.random.default_rng(seed)
e = rng.standard_normal(n + burnin)
x = np.empty(n + burnin)
x[0] = e[0]
for t in range(1, n + burnin):
x[t] = phi * x[t - 1] + e[t]
return x[burnin:] # drop the burn-in; true mean = 0
x = ar1(n=200, phi=0.7, seed=42)
# the lie: resample observations independently
lo, hi, point = conf_int(
x, "mean", method=IID(),
alpha=0.10, random_state=0)
# -> (-0.508, -0.188)
# a 90% interval that excludes the true mean, 0
# the fix: resample contiguous blocks, length chosen automatically
lo, hi, point = conf_int(
x, "mean", method=MovingBlock(block_length="auto"),
alpha=0.10, random_state=0)
# -> (-0.556, 0.022)
# now the truth is inside
The only thing that changed between the lie and the fix is the method. That is deliberate. There is a small family of block bootstraps (moving, circular, stationary, tapered) and they differ in how they cut the series into blocks and stitch them back together. Each is the right answer to a slightly different question about your data's structure. The one that chose the block length above did it with a standard automatic rule (Politis and White's spectral plug-in estimator, the textbook default), so you get a defensible answer with no tuning; a hand-picked length could squeeze out more of the residual gap, but that is a refinement.
The headline is the thesis this whole series is built on, and it is worth stating plainly, because every later piece is a variation on it: match the resampler to the assumption your data violates. The i.i.d. bootstrap violates independence on a time series and pays for it in coverage. Later you will meet data that breaks other assumptions (errors whose variance shifts over time, models whose residuals are themselves dependent), and each one has a resampling scheme built to respect what it breaks. The bootstrap is not one tool. It is a question you have to answer honestly: what does my data actually do?
What I now check before I trust an interval
I still bootstrap everything. But I no longer read a tight interval as good news, and I no longer reach for the default resampler on data that has an order.
| what I read as true | what was actually true | the rule it became |
|---|---|---|
| a tight interval means a precise estimate | on autocorrelated data, a tight interval means the resampler deleted the dependence | narrowness is a red flag on a time series, not a comfort |
| the bootstrap is assumption-free | the ordinary bootstrap assumes your observations are interchangeable | name the assumption before you resample, because the default has one |
| a 90% interval covers the truth 90% of the time | this one covered it 49.6% of the time | test coverage against a known truth before you trust a method on an unknown one |
| the block bootstrap fixes it | it recovered coverage to 79.7%, not 90% | the right tool can still undercover; honesty about the residual gap is part of the fix |
The interval I opened with, [-0.51, -0.19], still sits in my terminal history. It is a perfectly ordinary-looking result: a signed effect, a tight band, ninety percent confidence. It is also a statement about data with a mean of zero that the mean is not zero. Nothing about the number looks like a lie, which is why the ordinary bootstrap gets away with telling it. Change one argument (resample the blocks, not the points) and, on this series, the truth climbs back inside the interval, where it was the whole time.
{ q: "What is tsbootstrap?", a: "tsbootstrap is a time series bootstrap library for Python. It resamples a series while preserving the temporal dependence that an ordinary bootstrap destroys, using the methods the statistics literature recommends (block, residual, sieve, and wild resampling) plus a conformal layer for prediction intervals. The public API is one `bootstrap` call configured with a typed method spec, and a `conf_int` helper that runs the bootstrap and reads the interval in one call." },
{ q: "Why does the ordinary bootstrap fail on time series data?", a: "The ordinary (i.i.d.) bootstrap resamples observations independently, which assumes they carry no information about each other. A time series violates that: consecutive values are correlated. Resampling single points at random destroys the autocorrelation, so the resampled series looks more random than the real one and the bootstrap reports a standard error that is too small. On an AR(1) series with autocorrelation 0.7, the i.i.d. standard error of the mean comes out about 2.4 times too small." },
{ q: "What is a block bootstrap and how does it fix this?", a: "A block bootstrap resamples contiguous chunks (blocks) of consecutive observations instead of single points. Because each block is a run of real, adjacent data, it carries the local dependence with it, so the resampled series keeps the autocorrelation of the original. That gives a standard error and confidence interval that reflect the real, reduced information content of dependent data. In tsbootstrap you select it by passing `method=MovingBlock(block_length=\"auto\")` to `bootstrap` or `conf_int`; the `\"auto\"` length comes from Politis and White's spectral plug-in estimator, the textbook default rule." },
{ q: "Does the block bootstrap give perfectly correct confidence intervals?", a: "No, and it should not be sold as if it does. In a 2,000-run coverage study on AR(1) data (*phi* = 0.7, *n* = 200), the moving-block bootstrap's nominal 90% interval covered the true mean 79.7% of the time, versus 49.6% for the i.i.d. bootstrap. It recovers most of the gap but still undercovers at short series lengths. The block bootstrap is the right tool, an honest improvement that falls short of exact coverage." },
]} />
The ordinary bootstrap, pointed at a time series, breaks the coverage promise by a wide margin and without a warning. The next piece takes a different broken assumption (errors whose variance shifts over time) and the resampler built to respect it.
Top comments (0)