Comparing two distributions by the largest gap between their cumulative curves, without binning.The Kolmogorov–Smirnov (KS) test is a nonparametric method to compare two probability distributions, or a sample against a theoretical distribution, by measuring the largest vertical distance between their cumulative distribution functions. Unlike many tests, it does not require binning the data, so it avoids arbitrary choices of bin width and works well with small samples. It is a powerful tool for checking whether a dataset follows a hypothesized distribution or whether two datasets come from the same underlying distribution.
The test was introduced by Andrey Kolmogorov in 1933 for the one-sample case, and extended by Nikolai Smirnov in 1939 for the two-sample case.
You compute the empirical cumulative distribution function (CDF) from your data — a step function that jumps at each data point. For a two-sample test, you do this for both samples. The test statistic is the supremum (maximum) of the absolute difference between the two CDFs. The null hypothesis that the two distributions are identical is rejected if this statistic exceeds a critical value, which depends on the sample sizes and the desired significance level. The test is distribution-free, meaning its critical values do not depend on the underlying distribution, only on sample sizes.
It is used in astronomy to compare the distribution of galaxy properties across different populations, and in particle physics to validate that a Monte Carlo simulation matches real data.
The KS test is most sensitive to differences in the middle of the distribution and less sensitive at the tails, which can be a drawback when you care about rare events.
