statistics

null hypothesis testing

Asking how improbable the data would be if there were no effect — the frame in which discoveries are declared.
22 related articles

Description

Null hypothesis testing is a statistical framework used to decide whether an observed effect in data is real or could plausibly be due to random chance. The 'null hypothesis' typically states that there is no effect, no difference, or no relationship — for example, that a new drug works no better than a placebo. The test asks: if the null hypothesis were true, how unlikely would it be to see data at least as extreme as what was actually observed? If that unlikelihood falls below a pre-set threshold (commonly 5%), researchers declare the result 'statistically significant' and reject the null hypothesis in favor of an alternative.

History

The framework was formalized in the early 20th century by Ronald Fisher, who introduced the concept of significance testing in the 1920s, and later extended by Jerzy Neyman and Egon Pearson with the alternative hypothesis and error types.

How it works

The procedure begins by assuming the null hypothesis is true. From this assumption, one derives a probability distribution for a test statistic — a number summarizing the data, like a difference in means. The observed data yields an actual value of that statistic. The test then computes the probability of obtaining a value as extreme or more extreme than the observed one, under the assumed distribution. This probability is the p-value. If the p-value is small enough, the result is deemed inconsistent with the null hypothesis, and the null is rejected.

In practice

It is the default tool in scientific research, from clinical trials to psychology experiments, for deciding whether an intervention or effect is worth attention.

Fun fact

The 5% threshold for significance was popularized by Fisher, who chose it as a convenient round number in his 1925 book 'Statistical Methods for Research Workers'.

Concept graph →

Related articles · 22

Vortices in the Ocean of Phases: How Visibility Nulls Reveal the Asymmetry of Distant Sources
Imagine an interferometer as a giant ear listening to the radio whisper of a distant galaxy. Three antennas spread across continents capture the signal, and the difference in their phases combines into a quantity that is immune to atmospheric jitter. This quantity is called the closure phase.
Quantum Physics2026-08-092 minarXiv:2608.08778express
How to Test Whether Gravity Is Quantum
Two tiny diamonds float in a vacuum, separated by a distance that light cannot cross during the experiment. Inside each, like a frozen spark, resides an entangled spin — a quantum state linking them into a single whole. Now the diamonds begin to move, but not just any motion: each one seems to split, existing in two places at once.
A Bias That Needs No Teacher
In a warehouse, there are boxes of apples and pears. There are many apples and few pears. But when the stock clerk says "a fruit with smooth skin," we are somehow more likely to recall the rare pear than the common apple. This is the inverse base-rate effect — a persistent glitch in how the brain resolves ambiguity between categories.
cs.HC2026-08-032 minarXiv:2608.01820express
Smartwatches vs. Sugar: Why the Accuracy Was a Mirage
A glucose meter that doesn't prick your finger but just looks at your wrist: that's the dream for millions of people with diabetes. It's the promise of algorithms that extract glucose levels from the ordinary pulse signal, the photoplethysmogram (the same one that measures heart rate in smartwatches).
How Confirmation Bias Helps You Think Faster
We're used to thinking that noticing only confirmations of our guesses is a mistake. But recent calculations, inspired by wave behavior, flip that view. It turns out that the habit of seeking the familiar is the best way to reduce information noise (scientists call it entropy).
Quantum Physics2026-06-192 minarXiv:2606.21664express
Quantum Games Crack a Classic Math Theorem
In 1953, mathematician Alexander Grothendieck proved a theorem that became a cornerstone in the theory of tensor products. It states that for ordinary function spaces, two natural norms, the projective and the injective, are equivalent up to a universal constant. In other words, two ways of measuring the 'size' of combinations of vectors give similar results.
Cosmology2026-05-261 minarXiv:2605.27230express
Is Dark Energy Changing? A Fresh Look
The Universe is expanding faster and faster thanks to mysterious dark energy, discovered in the late 1990s by Adam Riess, Saul Perlmutter, and Brian Schmidt using supernovae.
Quantum Physics2026-05-131 minarXiv:2605.13154express
Quantum Debate: What Are Particles Hiding?
Two particles behave like a pair of unusual dice: thrown at opposite ends of the universe, they always land on the same face. Physicist John Stewart Bell calculated: if the dice had hidden rules, the frequency of matches would have a strict limit.
The Silent String of Gravity: Hunting for κ
The weak equivalence principle, silently tested since the time of Galileo and laid at the foundation of general relativity, states: all bodies fall equally, regardless of their internal composition.
Black Holes: The Perfect Mask of the Abyss
Twenty-first-century astrophysics is deafening with triumphs: we’ve caught the tremor of spacetime from the merger of invisible giants; glimpsed the 'shadow' of a supermassive monster in a distant galaxy; traced the orbits of stars dancing around a compact something at the heart of the Milky Way.
Instrumentation2026-05-031 minarXiv:2605.02969express
Diagnosis for Distant Worlds: How Not to Mistake Life for Noise
The hunt for life on distant planets is like diagnosing a rare disease with a test of unknown accuracy. A single positive result means nothing: maybe it's a glitch? We search the atmospheres of exoplanets (worlds around other stars) for traces of water and other compounds, but any signal could be a false alarm.
Quantum Physics2026-04-272 minarXiv:2604.24496express
Gravity Entangles Light and Matter
Light bends around massive bodies: this has been known since Einstein's time. But what if gravity itself is not a smooth fabric of space but a stream of tiny carrier particles? Then a beam passing near a particle should not just bend but enter into a special quantum relationship with it. This relationship is called entanglement.
Quantum Physics2026-04-092 minarXiv:2604.08663express
How to Catch Quantum Magic with Plain Heat
A quantum system can possess a property called magic. It's not about tricks: it's a measure of how hard it is to reproduce its behavior on a classical computer. Ordinary states, known as stabilizer states, are easy for a computer to handle. But states with magic leave it stumped.
math.OC2026-04-051 minarXiv:2604.04305express
Epidemics as a Game: How Personal Calculations Reshape the Wave of Infections
An epidemic is not just a virus: it's millions of personal decisions. One person keeps riding the subway; another locks themselves at home. Classical models describe the average behavior of a crowd but don't ask why each individual acts that way.
Quantum Physics2026-03-171 minarXiv:2603.17030express
Comparing Scents: A New Type of Bell Inequalities
Comparing two scents is easy even without knowing their ingredients, just tell if they're alike. This everyday principle became the foundation for a new class of Bell inequalities, reimagining tests of the spooky connection between particles. Previously, you had to measure precise properties, like spin direction, but the authors showed that simply comparing outcomes is enough.
A Cold Detector and Extra Signals
In a dark chamber where the temperature sits just four degrees above absolute zero, a sapphire crystal the size of a grain of rice lies waiting. When a neutrino, a ghostly particle that barely notices matter, flies through it, the crystal shudders and warms by a tiny fraction of a degree.
Quantum Physics2026-02-282 minarXiv:2603.00401express
Randomness Without Trust
A coin toss seems random, but in reality, physics decides everything: the force of the throw, air resistance, the surface's elasticity. If you knew all the details, you could predict the outcome. True randomness lives deeper — in the quantum world.
Cosmology2026-01-292 minarXiv:2601.22043express
A Cosmic Ruler Without a Ruler
The universe is expanding, but at what rate is a question that has been troubling astronomers for decades. Two methods of measurement give different answers, and this discrepancy is called the Hubble tension. Some look at the early universe through the cosmic microwave background, others at nearby supernovae and Cepheid variables.
q-bio.GN2026-01-142 minarXiv:2601.09758express
How to Tell a Technical Glitch from a Real Disease
Genetic diagnostics is like comparing two photos taken with different cameras. If one camera slightly shifts the colors, the image will show "defects" that don't exist in reality.
Quantum Physics2025-12-061 minarXiv:2512.06333express
Quantum Test of Falling Bodies
All bodies fall the same: this was first observed by Galileo. Einstein elevated the observation to the equivalence principle: gravity and acceleration are indistinguishable, giving birth to curved spacetime.
The Shadow of a Measured Value: How the EPR Paradox Dissolves in Operators
In 1935, Einstein, Podolsky, and Rosen proposed a thought experiment that, in their view, exposed the incompleteness of quantum mechanics. Two particles fly apart after interacting, and measuring the momentum of one instantly gives exact knowledge of the other's momentum.
stat.ME2020-10-053 minarXiv:2010.02247
A Causal Map Without Intervention
An orchestra without a conductor plays in harmony, and you want to understand who influences whom. The violinist hears the cello, the cellist catches the rhythm of the percussion, and the flute adapts to everyone at once.