Estimating a smooth probability density from samples without assuming its functional form.Kernel density estimation (KDE) is a non-parametric way to estimate the probability density function of a random variable from a finite sample of data points. Unlike histograms, which depend on bin width and placement, KDE produces a smooth curve by placing a smooth 'kernel' (a symmetric, non-negative function, often a Gaussian) centered at each data point and summing them. The result is a continuous estimate that converges to the true density as the sample size grows, without assuming any particular functional form like normal or exponential.
Introduced by Murray Rosenblatt (1956) and Emanuel Parzen (1962), building on earlier work by Fix and Hodges in the 1950s.
For each data point x_i, you place a kernel function K (e.g., a Gaussian) with bandwidth h (a smoothing parameter) centered at x_i. The estimated density at any point x is the average of these kernels: f_hat(x) = (1/(n h)) Σ K((x - x_i)/h). The bandwidth h controls the trade-off between bias and variance: small h gives a wiggly estimate that follows noise, while large h oversmooths and loses features. Choosing h optimally (e.g., by cross-validation) is key.
Used in data science for exploratory data analysis, in finance for risk estimation, and in astronomy for estimating galaxy distributions from sparse observations.
KDE is the basis for the popular 'violin plot', which combines a box plot with a rotated KDE to show the distribution's shape.
