statistics

gibbs sampling

Sampling a multidimensional posterior one coordinate at a time from its conditional distributions.
23 related articles

Description

Gibbs sampling is a Markov chain Monte Carlo (MCMC) algorithm for generating samples from a multivariate probability distribution when direct sampling is difficult. It works by updating one variable at a time, drawing each new value from its conditional distribution given the current values of all other variables. This approach is particularly useful when the full joint distribution is complex, but the conditional distributions are simple and easy to sample from. Gibbs sampling is a special case of the Metropolis-Hastings algorithm, but it always accepts the proposed move, making it computationally efficient in many settings.

History

The method is named after the physicist Josiah Willard Gibbs, but it was formalized by Stuart Geman and Donald Geman in 1984 in the context of image restoration, and later generalized by others.

How it works

The algorithm starts with an initial set of values for all variables. At each iteration, it cycles through the variables in a fixed order (or randomly). For each variable, it samples a new value from its conditional distribution, holding all other variables at their current values. These new values are immediately used in the subsequent conditional distributions. Over many iterations, the sequence of samples converges to the joint distribution, provided the chain is irreducible and aperiodic. The key is that the conditional distributions must be known and easy to sample from, which is often the case in hierarchical Bayesian models.

In practice

It is widely used in Bayesian statistics for sampling from posterior distributions, such as in latent Dirichlet allocation (LDA) for topic modeling in natural language processing, and in genetic data analysis.

Fun fact

Despite being named after Gibbs, the algorithm was not developed by him; the name was coined by Geman and Geman because of an analogy to Gibbs distributions in statistical mechanics.

Related articles · 23

A Translator That Can't Keep Up with Speech: How a Neural Network Accelerates Gravitational-Wave Astronomy
Imagine a simultaneous interpreter in a room where two people are speaking: they must keep up with both meaning and intonation, but if the speech is too fast, only a paraphrase remains. That's roughly how scientists feel as the stream of gravitational waves from black hole and neutron star mergers grows, while precise signal calculations lag behind.
Nuclear Theory2026-08-142 minarXiv:2608.14045express
Counting the Participants in a Nuclear Collision
When two heavy nuclei smash into each other at enormous speeds, they don't collide as whole objects like billiard balls. Only part of their matter comes into contact. How many nucleons (protons and neutrons) actually took part in the collision, and how many flew past, is not known in advance. All subsequent measurements depend on this number, but it cannot be observed directly.
Quantum Physics2026-08-041 minarXiv:2608.04290express
Quantum Simulator on Fleeting Qubits
A quantum computer is usually like an assembly line: operations proceed one after another, and each error accumulates. In the new work, physicists tried a different path — an architecture where the computation grows out of the measurement process itself. They took 56 ions in a trap and wove them into a three-dimensional cloud of entangled particles.
The Brain as a Predictor: A New Look at Neural Circuits
The brain doesn't just react to the world: it's constantly guessing what will happen next. This principle is called predictive coding: neurons compare expectations with reality and update their internal picture when an error occurs. But how such a complex system could be wired into the living tissue of the cortex remained unclear.
Nuclear Theory2026-08-011 minarXiv:2608.00529express
Quantum Transformer for 600 Dimensions
In a noisy hall, an interpreter tries to catch every word, but rustles and echoes muddle the meaning. That's roughly how conventional computer methods handle complex quantum systems: the more particles and the stronger their correlations, the faster calculations drown in interference.
Signal Processing2026-07-161 minarXiv:2607.14597express
Self-Taught Algorithm: How to Hear a Melody from Noise
Reconstructing a full picture from fragments is like discerning an orchestra's melody in a noisy hall. Such challenges arise in photometry, where a star's brightness is recovered from faint twinkling. Traditional methods need precise noise information, which is almost always unknown.
Cosmic Earthquakes: Algorithm Reveals the Rhythm of Gamma-Ray Bursts
In the depths of the Universe, when a massive star collapses into a black hole or a magnetar is born, a gamma-ray burst flares up — for an instant, it outshines all the stars in the observable Universe.
Genes Speak Different Languages — A Neural Network Found a Common One
60 mice, 1,980 samples, 234 lung cancer patients. Behind each number lie thousands of genes. Their activity is measured to understand the disease. But the data aren’t uniform: for one patient, you assess a protein level (a continuous number), make a diagnosis (yes/no), and determine the tumor subtype (a category).
Symphony of White Dwarfs: How a Neural Network Learns to Hear LISA's Gravitational Music
The Universe is not silent: it is filled with a continuous gravitational-wave hum. In a decade, the LISA space observatory will tune its "ears" to the millihertz range, and the richest source will be our Galaxy — tens of thousands of white dwarf pairs whirling in a frantic dance.
General Relativity2026-06-052 minarXiv:2606.07330express
How to Hear the Dance of Two Black Holes
Imagine an orchestra where each musician plays their own part, and the conductor tries to catch a faint rhythm common to all. That's roughly how astronomers search for gravitational waves from pairs of supermassive black holes (weighing billions of suns) circling each other in distant galaxies.
High Energy2026-05-251 minarXiv:2605.25980express
Why Black Holes Dance Differently
Pairs of black holes dance in three different styles before merging. The first is a neat waltz: the objects swirl in the same direction, like partners facing the same way. Such pairs are born from binary stars and often have unequal masses. The second is chaotic disco: spins are scattered, masses nearly equal.
Instrumentation2026-05-172 minarXiv:2605.17546express
How to Assemble a Galaxy in One Step
An artist paints a portrait stroke by stroke, and then someone offers them a flash camera — one click, and it's done. That's roughly the shift from multi-step models to one-step models in generating images of galaxies. Typical neural networks for image synthesis work like a watercolorist: they apply a layer, blur it, add details.
math.CO2026-05-171 minarXiv:2605.17594express
Keys That Never Cross
You can feel an unfamiliar object along its length, across its width, or diagonally: each time gaining new information. But what if one way of feeling tells you absolutely nothing about what another would reveal? That's exactly how mutually unbiased bases work in quantum mechanics: measuring in one of them leaves you completely in the dark about the outcome in another.
Statistical Mech2026-05-092 minarXiv:2605.09148express
How to Tune a Molecular Motor Without Losing Your Mind
Imagine a factory where hundreds of tiny assembly lines run nonstop. Each line is a molecular motor that converts chemical fuel into motion. To make the motor go faster or in the right direction, an engineer tweaks the settings — the shape of parts, the strength of interactions.
Cosmology2026-04-202 minarXiv:2604.17981express
The Universe in a Test Tube: How Tiny Slices of the Cosmos Replace Massive Simulations
The Universe is vast, and figuring out how galaxies and dark matter are distributed within it is no easy task. Usually, scientists run computer simulations that "play out" the evolution of entire regions of space billions of light-years across. But such calculations demand colossal resources and time.
Dynamical Systems2026-04-192 minarXiv:2604.17528express
Five Faces of the Same Randomness
In a big city after rain, water collects in the same low spots no matter where the first drops fall. An hour later, the pattern of puddles hardly depends on where the downpour started. That is equilibrium — the outcome a system reaches regardless of its start, once the details are forgotten.
cs.ET2026-04-181 minarXiv:2604.17109express
The Machine That Doesn't Get in Its Own Way
An orchestra where each musician plays their own part but hears only the neighbor. One wrong note, and a wave of errors rolls through the hall. That's how Ising machines work — special devices for solving problems where you need to find the best option among many.
Machine Learning2026-03-151 minarXiv:2603.14161express
Learning from Many Brains
A typical machine learning model looks at one object at a time: one brain, one patient, one engine. But biologists need the big picture: how the brain of a species works as a whole, not just that of a particular fish.
Instrumentation2026-03-041 minarXiv:2603.04535express
Neural Network Clears the Gravitational Lens from Ancient Light
Light born at the moment of the Big Bang has traveled to us for almost 14 billion years. But along the way, it passed through the gravitational fields of galaxies and clumps of dark matter, which bent its rays. To an observer, this looks like an image stretched and smudged by an old window pane.
Medical Physics2026-02-042 minarXiv:2602.05077express
The Spiral That Speeds Up Brain MRI
Imagine a photographer shooting a tall building, but instead of a single frame, they take a series of shots while moving the camera in a spiral from top to bottom. This approach captures more detail in less time, but the images can come out blurry. A similar challenge faces MRI machines when they need to quickly image the brain.
Mesoscale2026-01-192 minarXiv:2601.13229express
Magnetic Disks Instead of Microchips
On a table lies a handful of round magnets. As long as no one touches them, they lie in disarray. But send a short current pulse through them, and each disk flips: the north pole swaps with the south. Neighboring disks push each other with their fields, like people in a crowded queue, and the whole handful arranges into a pattern that depends on which way the magnetic wind blew and how warm it is around.
stat.ME2023-06-082 minarXiv:2306.04966
Hybrid Spectral Analysis: How a Bayesian Method Combines the Precision of Parametrics with the Flexibility of Nonparametrics
A rough sketch of a portrait: a few confident lines define the shape, but the details are not yet refined. That is how a parametric model, such as vector autoregression (VAR), describes a time series: it captures the main structure, but if the real data deviate from its assumptions, the portrait comes out distorted.
stat.ME2019-04-052 minarXiv:1904.02855
How to Make Your Model Pay for Its Mistakes
A translator who jumps at every rustle in the hall starts mixing up words by the end of the meeting. Their confidence in being right grows, while their accuracy drops. That is roughly how poorly calibrated probabilistic forecasts behave: the model claims a high probability for an event, but reality disagrees.