
Building the null distribution by shuffling labels of the data themselves.A permutation test is a non-parametric method for testing a statistical hypothesis without making assumptions about the underlying distribution of the data. Instead of relying on theoretical distributions (like the normal distribution), it builds the null distribution directly from the observed data by randomly shuffling the labels or group assignments. This is particularly useful when the sample size is small or the data does not follow standard distributions.
The method was introduced by R. A. Fisher in the 1930s, though it became computationally practical only with the advent of modern computers.
The procedure is straightforward. First, compute a test statistic (e.g., the difference in means between two groups) on the original data. Then, repeatedly shuffle the labels of the data points randomly, each time recomputing the test statistic. These shuffled values form the null distribution — the distribution of the statistic if there were no real effect. The p-value is the proportion of shuffled statistics that are as extreme as or more extreme than the observed statistic. If this proportion is very small, the observed result is unlikely to have occurred by chance.
It is used in genomics to test for differential gene expression, in neuroscience for brain imaging data, and in any field where small samples or non-normal data make traditional tests unreliable.
Permutation tests are exact: they do not rely on asymptotic approximations, meaning the p-value is correct even for very small sample sizes.






