The basic Markov chain rule: propose a step, accept it with a probability set by the ratio of posteriors.The Metropolis-Hastings algorithm is a Markov chain Monte Carlo (MCMC) method for sampling from a probability distribution when direct sampling is impossible. It is the workhorse of Bayesian computation, allowing one to generate a sequence of samples that asymptotically follow the target posterior distribution, even when the normalising constant is unknown. This makes it indispensable for parameter estimation and model comparison in complex, high-dimensional problems.
The algorithm was introduced by Nicholas Metropolis in 1953 for a specific case, and generalized by W. Keith Hastings in 1970. It gained widespread use in physics and statistics in the 1980s and 1990s with the rise of computational power.
Starting from an initial point, the algorithm proposes a new point according to a proposal distribution. It then computes the ratio of the posterior densities at the proposed and current points. If the ratio is greater than 1, the proposal is accepted; otherwise, it is accepted with probability equal to the ratio. This acceptance rule ensures that the chain spends more time in high-probability regions, and over many steps, the distribution of visited points converges to the target distribution. The proposal distribution can be tuned to balance exploration and efficiency.
It is used in particle physics for fitting complex models to data, in cosmology for exploring the parameter space of the universe, and in finance for risk assessment and option pricing.
The algorithm was originally developed to compute the equation of state of a hard-sphere gas, and its inventors used one of the first electronic computers, the MANIAC, at Los Alamos.











