Bootstrap: Understanding Sampling with Replacement Through Five Numbers
How can we estimate how much a sample mean might vary when we have only one sample? The bootstrap treats the observed sample as a small empirical distribution that we can sample from repeatedly. Draw the same number of records, calculate a mean, and repeat. The spread of these means gives us information about uncertainty.
One resample from five numbers
Take the teaching example 2, 3, 4, 4, 7. Its mean is 4. Each of the five records has its own position: the two 4s are separate records. Randomly select one of the five positions, leaving that position available for later draws. Draw five times to obtain one bootstrap sample.
For example, one resample might be 2, 2, 4, 7, 7. The original 3 is absent, 2 and 7 each appear twice, and only one of the two records containing 4 is selected. A selected record stays in the pool and can be selected again. After five draws, the sample size is still 5. This resample has a mean of (2 + 2 + 4 + 7 + 7) / 5 = 4.4. The next resample may have a different combination and a different mean.
A statistic for each resample
If the target is a mean, calculate a mean each time. If the target is the correlation between two measurements, resample complete records and then calculate the correlation. Resampling the two measurements independently would destroy the pairing that we want to study.
A histogram of the repeated statistics can show where they concentrate, whether the distribution is skewed, and whether a few extreme results occur. Their standard deviation can estimate the bootstrap standard error. More resamples make the computation more stable; the original sample still contains only the observations we started with.
What should we resample?
The simplest ordinary bootstrap applies when records can be treated as independent and identically distributed observations. Seven diary entries from one person belong to the same person; students in the same classroom may also be correlated. The research design should determine whether to resample people, groups, or time blocks. A row in a spreadsheet is not automatically the right resampling unit.
Software interfaces reflect this distinction. R's boot function takes data, a statistic function, and a number of replications. In ordinary resampling, row indices are passed to the statistic function. Understanding what each index represents matters more than increasing the replication count first.
Report the settings and the limitations
Record the original sample size, target statistic, resampling unit, number of replications, and random seed, then report the results. If constructing a confidence interval, specify the percentile, BCa, or other method used. Different constructions should not all be described simply as a "bootstrap interval."
The bootstrap recombines records that have already been observed. If a sample misses a group of people, repeated resampling cannot create those missing people. For targets that depend strongly on the tails, such as a maximum or a range, check whether the method is suitable. Start with how the sample was obtained and what you want to estimate, then decide how to resample.
Sources
Translated and adapted from the original Chinese article on Metho.
Metho · Research Methods — Rituals can be copied. Understanding cannot be faked.
Discuss this article on EnterScholar — A scholarly community for the age of intelligence.
Top comments (0)