Bayesian inference is a statistical method that updates the probability of a hypothesis as more evidence becomes available.Bayesian inference is a mathematical method for estimating the probability of a hypothesis based on new observations. It requires three components: a prior probability (initial degree of belief in the hypothesis), a likelihood function (how probable the data are given the hypothesis is true), and a normalization constant (the total probability of the data under all possible hypotheses). The result is the posterior probability—what you believe after considering the evidence. Unlike classical statistics, the Bayesian approach directly answers the question "What is the probability that the hypothesis is true given these data?" rather than "What is the probability of obtaining such data if the hypothesis is true?"
Bayes' theorem was first formulated in the posthumously published work of Thomas Bayes (1763), and later independently rediscovered and disseminated by Pierre-Simon Laplace in 1774. Laplace applied it to estimating the mass of Saturn and to demographic research. In the 20th century, interest revived thanks to the work of Harold Jeffreys, who introduced non-informative prior distributions, and Leonard Savage, who developed the axiomatics of subjective probability. With the advent of powerful computational methods such as the Metropolis-Hastings algorithm (1970) and Gibbs sampling, Bayesian inference became applicable to complex models in data science.
Inference relies on Bayes' theorem: P(H|D) = P(D|H) * P(H) / P(D), where H is a hypothesis, D is data. P(H) (prior) reflects the initial degree of belief, P(D|H) (likelihood) is the probability of obtaining data D if H is true, P(D) is the marginal probability of the data, serving as normalization. In practice, direct computation of the denominator, an integral over all hypotheses, is often impossible, and one resorts to Markov chain Monte Carlo (MCMC) methods to approximate the posterior distribution. Modern packages such as Stan and PyMC automate these calculations.
Spam filters learn from your markings, medical tests refine diagnoses based on new symptoms.
British codebreaker Alan Turing used Bayesian methods to crack the Enigma code during World War II, assigning probabilities to various machine settings based on intercepted messages.
| B | variable | Bayes factor — bayes factor |
| D_k | variable | k-th data point — data |
| H_1 | variable | alternative hypothesis — hypothesis |
| H_0 | variable | null hypothesis — hypothesis |
| N | variable | number of data points — sample size |
| ∏ | operator | multiplication of terms for k=1 to N — product operator |
| Z | variable | Bayesian evidence (marginal likelihood) — bayesian evidence |
| Θ | variable | Parameter vector — parameter vector |
| L(Θ) | variable | Likelihood function — likelihood function |
| π(Θ) | variable | Prior probability density — prior probability density |
| ∫ | operator | Integration over parameter space — integral operator |
| dΘ | operator | Infinitesimal volume element — volume element |
| p(x) | variable | Probability density of distribution p — probability density |
| q(x) | variable | Probability density of distribution q — probability density |
| x | variable | Integration variable (sample space) |
| ∫ | operator | Integration over the sample space — integral operator |
| P(A|B) | variable | Conditional probability of A given B — conditional probability |
| P(B|A) | variable | Conditional probability of B given A — conditional probability |
| P(A) | variable | Marginal probability of A — probability |
| P(B) | variable | Marginal probability of B — probability |















