The probability of the data under a model with parameters integrated out; penalises complexity automatically.Bayesian evidence, also known as the marginal likelihood, is the probability of the observed data under a given model, with all parameters integrated out. It is computed as the integral of the likelihood times the prior over the entire parameter space. This quantity is central to Bayesian model comparison: the ratio of evidences for two models (the Bayes factor) tells you how much more likely one model is than the other, given the data. A key property is that it automatically penalizes model complexity — a model with more parameters does not automatically get higher evidence, because the extra parameter space dilutes the prior probability mass.
The concept dates back to the work of Harold Jeffreys in the 1930s, who introduced the Bayes factor for hypothesis testing. The term 'evidence' was popularized by John Skilling in the context of nested sampling in the 2000s.
To compute the evidence, you integrate the product of the likelihood and the prior over all parameter values. For simple models, this integral can be done analytically; for complex models, it requires numerical methods like nested sampling or thermodynamic integration. The evidence balances fit and complexity: a model that fits the data well but has many parameters will have a lower evidence than a simpler model that fits almost as well, because the prior mass is spread over a larger volume.
Bayesian evidence is used in cosmology to compare models of the universe (e.g., with or without dark energy), in exoplanet detection to decide if a signal is a planet or noise, and in machine learning for automatic model selection.
The Bayesian evidence automatically implements Occam's razor: if two models fit the data equally well, the simpler one will have higher evidence, purely from the mathematics of probability.
| Z | variable | Bayesian evidence (marginal likelihood) — bayesian evidence |
| Θ | variable | Parameter vector — parameter vector |
| L(Θ) | variable | Likelihood function — likelihood function |
| π(Θ) | variable | Prior probability density — prior probability density |
| ∫ | operator | Integration over parameter space — integral operator |
| dΘ | operator | Infinitesimal volume element — volume element |








