A resampling technique that repeatedly draws samples from data to estimate uncertainty.The bootstrap is a resampling technique used to estimate the uncertainty of a statistic, such as the mean, median, or regression coefficient, without making strong assumptions about the underlying distribution. It works by repeatedly drawing random samples, with replacement, from the original data and calculating the statistic for each resample. The distribution of these bootstrap estimates approximates the sampling distribution of the statistic, allowing you to compute standard errors, confidence intervals, and hypothesis tests.
The bootstrap was introduced by Bradley Efron in 1979. Efron developed it as a general-purpose method for estimating the accuracy of statistical estimates, building on earlier jackknife techniques. He coined the term 'bootstrap' from the phrase 'to pull oneself up by one's bootstraps,' reflecting the idea of using the data itself to infer its own uncertainty.
Given a dataset of size n, the bootstrap creates many 'bootstrap samples' by randomly selecting n observations from the original data, allowing the same observation to appear multiple times. For each sample, the statistic of interest is computed. After thousands of repetitions, the collection of these statistics forms an empirical distribution. The spread of this distribution (e.g., its standard deviation) provides an estimate of the statistic's standard error, and its percentiles give confidence intervals.
The bootstrap is used in machine learning for model validation, in ecology to estimate species diversity, in medicine for analyzing clinical trial data, and in economics for forecasting uncertainty.
The bootstrap can be used to estimate the uncertainty of almost any statistic, even when no analytical formula exists, making it a 'universal' tool for statistical inference.














