method

data cleaning

Process of removing noise or artifacts from data to reveal true signals.
17 related articles

Description

Data cleaning is the process of detecting and correcting (or removing) corrupt, inaccurate, or irrelevant parts of a dataset. In physics, raw measurements are never perfect: detectors have noise, environmental interference creeps in, and systematic errors distort signals. Cleaning separates the true physical signal from these artifacts, making subsequent analysis reliable. It is the unglamorous but essential first step in almost every experimental and observational study, from quantum imaging to cosmology.

History

The term became prominent with the rise of digital data processing in the mid-20th century, but the practice is as old as experimental science. Early astronomers manually removed bad observations; modern automated pipelines emerged with computers in the 1960s–70s.

How it works

Typical methods include filtering (e.g., removing frequency bands dominated by noise), outlier rejection (discarding points that deviate wildly from expected behavior), and calibration (using known references to correct for instrument response). More advanced approaches use statistical models or machine learning to identify and subtract artifacts, such as neural networks that distinguish real events from detector glitches. The key is to remove noise without removing real signal, which requires understanding the noise's characteristics and the physics of the measurement.

In practice

Used in every experimental physics lab, from LIGO's gravitational-wave detectors (cleaning seismic noise) to quantum optics (removing dark counts) and astrophysics (separating foregrounds from cosmic microwave background).

Fun fact

In the LIGO gravitational-wave detectors, data cleaning includes vetoing times when airplanes fly overhead, because their radar signals can mimic gravitational-wave events.

Concept graph →

Related articles · 17

Optics That Fix Themselves
An orchestra of a hundred musicians plays without a conductor. Every instrument is tuned perfectly, but one violin goes slightly off, and the harmony falls apart.
Quantum Physics2026-08-062 minarXiv:2608.05535express
A Quantum Channel Like a Muddy River
Alice prepares a quantum particle in one of several states and sends it to Bob. The channel the particle travels through isn't perfect: it's noisy, scatters, interacts with the environment. Bob receives the signal and tries to figure out what Alice actually sent. This is the classic "prepare-send-measure" scheme, and there's always a risk of losing something along the way.
Quantum Physics2026-08-042 minarXiv:2608.03944express
Quantum Protection vs. Classical Memory
A quantum computer is like a fragile snowflake resting on a warm palm. The slightest breath of heat or a gentle vibration, and it turns into a droplet. To keep the snowflake alive, you constantly adjust it, restoring its original shape. That's quantum error correction.
Stellar2026-07-273 minarXiv:2607.24525
How a Neural Network Learns to Read the Sun’s Magnetic Tapestry
Each morning, dozens of telescopes swing toward the Sun. Their cameras catch a narrow band of red light — the Hα line, discovered by Johann Balmer. The images reveal dark, whimsically curved threads. They look like stitches in a giant tapestry sewn through the blazing corona.
cs.HC2026-06-303 minarXiv:2607.00239
Retouching by the rules: where science ends and deception begins
On the screen is the result of a complex simulation: two planets collided, a shockwave ripples outward. A couple of clicks in an editor, and harsh shadows soften, noise smooths out, the background darkens. This kind of digital retouching (image editing to enhance appearance) has long been part of the scientist’s toolkit.
Instrumentation2026-05-272 minarXiv:2605.28332express
Neural Network Listens to Space Instead of Humans
A rock concert blares at full volume, and you're trying to catch your friend's whisper. That's roughly how radio detectors feel when they pick up signals from cosmic dust particles. A particle slams into the atmosphere and spawns an air shower, a cascade of millions of secondary particles, and this shower emits a short radio pulse.
cs.CE2026-05-012 minarXiv:2605.00730express
How the Brain Takes Out the Trash: A Map of Flows
Imagine a river that doesn't flow in an open channel but through the inside of a sponge, with its currents shifting from place to place. That's a good way to think about the brain's glymphatic system: a network of ultra-fine channels around blood vessels, through which cerebrospinal fluid seeps into the tissue and washes away the waste that has piled up during the day.
Medical Physics2026-04-282 minarXiv:2604.25356express
X-ray detective without cheat sheets
You taste a fruit smoothie through two different straws. One lets through only the big chunks, the other only the liquid. Individually, the sensations are misleading, but together they give you the key: here, it seems, is banana, and here, strawberry.
Quantum Physics2026-04-172 minarXiv:2604.16292express
Instrumentation2026-04-101 minarXiv:2604.09787express
One Galaxy, Two Telescopes: How to Separate Physics from Distortions
If you photograph the same mountain with a film camera and a modern smartphone, you get different images: colors are brighter in one, details are blurrier in the other. The same goes for telescopes: each instrument sees the sky in its own way, adding its own distortions to the true light of galaxies, like a filter through which we look.
Calibration Without a Thermostat
A radiation sensor in the city is like a street musician: it has to hit the right notes even when fingers go numb from cold or the instrument is baked by the sun. Usually, such devices are kept in greenhouse conditions, with heating or cooling, otherwise the energy scale (which distinguishes harmless potassium in concrete from dangerous cesium) starts to lie.
Quantum Physics2026-02-221 minarXiv:2602.19042express
How to Protect Quantum Memory: A Hybrid Approach
Quantum information is fragile: the slightest noise can destroy it. That's why two protection methods have been devised. The first is dynamical decoupling: rapid pulses are applied to qubits to average the noise to zero. It's like shaking a box of fragile items so they don't bump against the walls.
math.NA2026-02-121 minarXiv:2602.11849express
Chemistry Without Hints: Reconstructing Reaction Mechanisms from Data
When chemists see how concentrations of substances change, they need to figure out which reactions are behind those changes. It's like trying to reconstruct a recipe from photos of plates at different stages of a meal. Previously, this often involved derivatives — instantaneous rates of concentration change.
cs.SD2026-01-311 minarXiv:2602.00681express
Forest Sounds Turn into Pictures
An ornithologist in the jungle hears a trill, but the bird is hidden in the foliage. It would be great to get its portrait right away — at least from a photo database. Usually, that requires thousands of pairs of “song recording + photo of the same bird,” and collecting them is expensive and time-consuming. The authors took a detour: they used two ready-made models.
Optics2026-01-101 minarXiv:2601.06448express
Peering into Tight Spots: A Fiber Thinner Than a Hair
Light passing through a thin glass fiber scatters into countless tiny speckles, like ripples on water from a thrown stone. Physicists call this graininess speckle noise. Because of it, images from an ultra-thin endoscope come out blurry, as if you're looking through fogged-up glass. The researchers devised a program that removes this ripple.
Quantum Physics2025-10-291 minarXiv:2510.25982express
Photo Editor for Quantum Computers
Atomic qubits 'snap' computations in microseconds, but capturing the result with a light-sensitive camera stretches out to milliseconds — thousands of times longer.
stat.ME2022-05-192 minarXiv:2205.09252
Breaking Point in the Flow of Functions
A seismograph records underground tremors: the smooth line suddenly jerks and changes its character. Similarly, functional data, such as ocean temperature maps or brain signals, may shift their shape at some point. The new algorithm FSBS catches such breakpoints, even when each curve is measured at only one point and noise drowns the signal.