statistics

permutation test

Building the null distribution by shuffling labels of the data themselves.
19 related articles

Description

A permutation test is a non-parametric method for testing a statistical hypothesis without making assumptions about the underlying distribution of the data. Instead of relying on theoretical distributions (like the normal distribution), it builds the null distribution directly from the observed data by randomly shuffling the labels or group assignments. This is particularly useful when the sample size is small or the data does not follow standard distributions.

History

The method was introduced by R. A. Fisher in the 1930s, though it became computationally practical only with the advent of modern computers.

How it works

The procedure is straightforward. First, compute a test statistic (e.g., the difference in means between two groups) on the original data. Then, repeatedly shuffle the labels of the data points randomly, each time recomputing the test statistic. These shuffled values form the null distribution — the distribution of the statistic if there were no real effect. The p-value is the proportion of shuffled statistics that are as extreme as or more extreme than the observed statistic. If this proportion is very small, the observed result is unlikely to have occurred by chance.

In practice

It is used in genomics to test for differential gene expression, in neuroscience for brain imaging data, and in any field where small samples or non-normal data make traditional tests unreliable.

Fun fact

Permutation tests are exact: they do not rely on asymptotic approximations, meaning the p-value is correct even for very small sample sizes.

Concept graph →

Related articles · 19

Quantum Physics2026-08-191 minarXiv:2608.18666express
Quantum Communication on 16 Parallel Channels
Quantum communication is like sending a secret letter that tears apart at any attempt to peek. So far, such a letter can only be sent along one road, and that's slow. Scientists have figured out how to send 16 couriers along 16 parallel roads, different frequencies of light, and verified that the letters arrive intact.
cs.HC2026-08-032 minarXiv:2608.01820express
Smartwatches vs. Sugar: Why the Accuracy Was a Mirage
A glucose meter that doesn't prick your finger but just looks at your wrist: that's the dream for millions of people with diabetes. It's the promise of algorithms that extract glucose levels from the ordinary pulse signal, the photoplethysmogram (the same one that measures heart rate in smartwatches).
eess.SY2026-07-141 minarXiv:2607.12343express
How a Smart Algorithm Learns Not to Crash
A driver on a slippery road doesn't know exactly how the car will respond to the steering wheel, and learns on the go, leaving room for error. A smart control algorithm behaves in a similar way.
econ.EM2026-07-013 minarXiv:2607.01429
The Data Watershed: How a Simple Yes/No Switch Reveals Multidimensional Models
On a topographic map, raindrops choose their path: falling on one slope, they flow into River A; on the other, into River B. An observer sees only the winding rivers (the response variables Y) and knows which slope the water came from (the instrument Z), but cannot see the drops themselves (the unobserved factors ε).
Quantum Physics2026-06-291 minarXiv:2606.30468express
Entanglement That Can't Be Broken: Photons in Invulnerable Quantum Communication
Quantum entanglement often behaves like a synchronized dance visible from only one angle. Change the way you observe it, and the particles appear disconnected. This limits secure communications. Now physicists have created a pair of photons whose 'dance' is flawless from all perspectives.
stat.ME2026-06-242 minarXiv:2606.26061express
A New Way to See the Hidden in Data Chaos
A lab robot counts molecules, and gets it wrong. One tube shows 0, another 12, a third 3, though in reality they all hold roughly the same amount. Such 'raw' numbers are common in biology: DNA sequencing, counting species in a water sample, gene activity. There's plenty of data, but it's sparse and more scattered than simple randomness would predict.
Quantum Physics2026-06-191 minarXiv:2606.21768express
Quantum Twins and the Secret of the Common Cause
Twins enter a room from different doors. Each gets a box with a button, and each time they press it, either a red or a green light flashes. If the twins are perfectly identical, you'd expect their lights to follow the same internal rules. But what if the rules differ even though the twins are exact copies? That's precisely the scenario the authors explored.
q-bio.QM2026-06-151 minarXiv:2606.16726express
How Many Mice Does It Take to Understand the Microbiome?
When you're planning a renovation, you don't buy paint by eye: you measure the wall area. In science, deciding how many samples to collect is a similar guess, but the stakes are higher. Skimp on samples, and your study becomes a lottery: you might miss a real effect, or you might mistake noise for a discovery. Overshoot, and you're wasting tubes, months, and, in animal experiments, lives.
Chemical Physics2026-05-242 minarXiv:2605.24895express
NMR Radar for Invisible Molecules
Many chemical transformations resemble assembling furniture in complete darkness: you see the starting parts and the finished cabinet, but the assembly itself remains a mystery. The intermediate states, like awkward positions of the craftsman's hands, vanish too quickly to be discerned.
High Energy2026-05-212 minarXiv:2605.23083express
Why Black Holes in a Pair Look So Alike
In space, merging black hole pairs are common, and often one is noticeably more massive than the other. This mass difference isn't just a number: it's a key to their shared history. Astrophysicists studied the stable mass transfer channel — a process where matter flows smoothly from one star to another without disrupting the system.
Quantum Physics2026-05-142 minarXiv:2605.14644express
Quantum Map Detectives Find Hidden Connections
Two dice always land on the same number, no matter how far apart you toss them. In quantum physics, this behavior is called entanglement, and it's the key resource for future computers and secure communication. The trouble is, not all entanglement is easy to spot.
Statistical Mech2026-04-182 minarXiv:2604.17035express
The Mystery of Hidden Temperature
Hot tea cools down, and its temperature is the same throughout the cup. But in many systems (for example, in the rarefied gas between stars or in plasma) temperature is not uniform: in one place particles are faster, in another slower. Physicists call such systems "superstatistical": it's as if the tea consisted of many small regions, each with its own temperature.
Quantum Physics2026-03-211 minarXiv:2603.20612express
Unusual sensors win in noisy labs
In a lab, you measure a very weak signal — say, a tiny change in a magnetic field. The instrument trembles from passing cars, from the experimenter's breathing, from wires heating up. This technical noise can't be fully eliminated, and it sets a limit on precision. An ordinary sensor works like a scale that sways with any draft.
Quantum Physics2025-12-081 minarXiv:2512.07307express
Diamond Sees Through the Cell
Every cell is a miniature tuning fork. A sharp magnetic pulse makes it 'ring', and the decay of that sound depends on what's inside: viscosity, density, composition. A tiny diamond with a structural defect acts like a hypersensitive microphone, picking up magnetic echoes from individual cells.
Quantum Physics2025-12-061 minarXiv:2512.06333express
Quantum Test of Falling Bodies
All bodies fall the same: this was first observed by Galileo. Einstein elevated the observation to the equivalence principle: gravity and acceleration are indistinguishable, giving birth to curved spacetime.
stat.ME2023-06-082 minarXiv:2306.04966
Hybrid Spectral Analysis: How a Bayesian Method Combines the Precision of Parametrics with the Flexibility of Nonparametrics
A rough sketch of a portrait: a few confident lines define the shape, but the details are not yet refined. That is how a parametric model, such as vector autoregression (VAR), describes a time series: it captures the main structure, but if the real data deviate from its assumptions, the portrait comes out distorted.
stat.ME2019-04-052 minarXiv:1904.02855
How to Make Your Model Pay for Its Mistakes
A translator who jumps at every rustle in the hall starts mixing up words by the end of the meeting. Their confidence in being right grows, while their accuracy drops. That is roughly how poorly calibrated probabilistic forecasts behave: the model claims a high probability for an event, but reality disagrees.
stat.ME2017-06-182 minarXiv:1706.05661
An Orchestra Where Every Instrument Enters on Its Own Schedule
An orchestra plays a complex piece, and each group of instruments enters on its own schedule. The violins might slow down while the woodwinds keep the same tempo, and the percussion might suddenly burst into a roll. The conductor hears everything at once, but pinpointing the exact moment when the flutes, not the cellos, changed rhythm is nearly impossible.
Forecasting Without a Model: Nonparametric Prediction of Chaos and Climate
A translator learns not from a dictionary but from speech. They listen to native speakers and pick up rules that are written in no textbook. Similarly, a new method for forecasting complex systems such as climate or turbulence does not require knowing their equations. Instead, it looks at data and learns to predict their behavior.