A stochastic process used as a non-parametric Bayesian method for modeling unknown functions.A Gaussian process is a powerful statistical tool used to model unknown functions in a way that naturally handles uncertainty. Instead of giving a single best guess for a function, it provides a full probability distribution over possible functions that fit the observed data. This makes it a cornerstone of non-parametric Bayesian methods, where the complexity of the model grows with the amount of data, rather than being fixed in advance.
The concept was formalized in the 1940s by the mathematician Norbert Wiener and others, but its modern use in machine learning was popularized by Carl Edward Rasmussen and Christopher K. I. Williams in their 2006 book.
A Gaussian process defines a distribution over functions by specifying a mean function and a covariance function (kernel). The kernel encodes assumptions about smoothness, periodicity, or other properties of the function. When data points are observed, the process updates the distribution, yielding a posterior that can predict values at new points along with a measure of uncertainty. This uncertainty is key: it shrinks near observed data and grows in unexplored regions, allowing principled decision-making.
Gaussian processes are used in Bayesian optimization for hyperparameter tuning, in geostatistics for spatial data interpolation (kriging), and in robotics for learning control policies from data.
Gaussian processes can be seen as an infinite-dimensional generalization of the multivariate normal distribution, where any finite set of points follows a joint Gaussian distribution.











