statistics

feature importance

How much each input variable actually contributes to a trained model's decisions.
42 related articles

Description

Feature importance refers to techniques that assign a score to input features based on how useful they are at predicting a target variable within a trained model. It helps interpret which variables are most influential in the model's decisions, aiding in model understanding, feature selection, and domain insight.

History

The concept has been used informally for decades, but formalized methods like Gini importance were introduced with random forests in the early 2000s by Leo Breiman and Adele Cutler. Permutation importance was also popularized around that time.

How it works

Methods vary by model type. For tree-based models (e.g., random forests, gradient boosting), importance is often calculated by how much each feature decreases impurity (e.g., Gini impurity or entropy) across all splits. For linear models, the magnitude of coefficients can serve as a proxy. Model-agnostic methods like permutation importance measure the drop in model performance when a feature's values are randomly shuffled, breaking its association with the target.

In practice

Used in fields like genomics to identify which genes are most predictive of a disease, in marketing to understand key customer drivers, and in model debugging to ensure the model relies on sensible features rather than spurious correlations.

Fun fact

Feature importance scores are relative and can be misleading if features are correlated; a feature that is truly important may get a low score if a correlated feature is already used in the model.

Concept graph →

Related articles · 42

q-bio.PE2026-08-302 minarXiv:2608.30085
A Biased Race: How Observation Delays Skew Estimates of Variant Advantage
Two runners start a race on a course with changing incline. One starts earlier, the other later, and the judge measures their speeds without knowing about the staggered starts. So it is with virus variants: data arrive with delays, and if those delays are unequal, estimates of competitive advantage can be badly wrong.
The Breathing Orchestra: How a Mat Under the Sheet Hears Pauses in Nighttime Breathing
At night, the bedroom hosts an orchestra, with breathing as the soloist. Under the sheet lies a mat with 512 sensors, each capturing the slightest body movements. When breathing stops for ten seconds, the orchestra falls silent. The challenge is to hear that silence. Previously, it was unclear which notes in this music matter most.
Reward as a Gas Pedal: Why It Gets Harder to Press with Age
A banknote taped to the finish line makes your hand fly faster. In the lab, that banknote was replaced with a 50-pence coin, and the finish line with a target on a screen. The researchers recruited 28 young people and 28 older adults, seated them at a robot handle, and asked them to steer a cursor toward the target. Sometimes they were paid for speed and accuracy, sometimes not.
q-bio.OT2026-08-272 minarXiv:2608.90055
A Neural Report Card: How Organoids Grade Brain Drugs
In a school report card, each grade speaks for itself, but you can only tell how a student is doing overall by looking at the full report. For drugs that act on the brain, such a report card was missing for a long time. Every test on a miniature brain grown in a dish gave its own signal: some genes switched on, others switched off, electrical impulses sometimes sped up and sometimes went quiet.
Movement Alters Decisions
A person flips a coin to make a decision. They stand still — the coin lands heads. They take a few steps, and flip again. It would seem walking shouldn't affect the coin. But what if the walk itself changes how often they decide to simply repeat the previous toss? That's exactly what researchers tested.
Machine Learning2026-08-022 minarXiv:2608.01481express
Reading Thoughts from Magnetic Fields
A magnetoencephalograph, a helmet-like device with hundreds of sensors, picks up the weak magnetic fields generated by active neurons. When a person listens to speech, these signals contain imprints of the heard words. But extracting them is tricky: signals from thousands of sources merge into a tangled mess.
Stellar2026-07-193 minarXiv:2607.17348
SpecFANN: Lightning-Fast Sketch of a Star's Spectrum
An artist makes a quick charcoal sketch, and the face in the portrait is already recognizable. Astrophysicists face a similar challenge: the spectrum of a star is its individual portrait, revealing temperature, composition, and gravity.
eess.SY2026-07-161 minarXiv:2607.14606express
How to read an enemy in the sky in seconds
In combat, a pilot needs to instantly understand what an object is and what its intentions are. Conventional systems wait for complete radar data — like a goalkeeper who only reacts after the shot. The new algorithm works like an experienced goalkeeper: from the attacker's first moves, it predicts the target's maneuver.
Signal Processing2026-07-161 minarXiv:2607.14938express
Where Are You? Radio Waves Know the Answer
Radio signals in a room act just like sound in a concert hall: bouncing off walls and objects, they create a unique pattern — a silent melody for every location. Your phone, like a keen ear, catches this melody amid noise and distortion. It remembers which chord matches the corner of the couch, and which one fits the doorway.
eess.AS2026-07-161 minarXiv:2607.15243express
The Illusion of Accuracy: How AI Testing in Acoustics Misleads
An AI predicting room acoustics is like a student taking an exam with a cheat sheet. If the test points are the same as the training ones, the model simply memorizes the acoustic 'fingerprint' of each position — like a student recognizing a question by the dog-eared corner.
cs.SD2026-07-151 minarXiv:2607.13477express
AI Judges Listen Not with Ears, but with Labels
Automatic speech judges are algorithms that evaluate pronunciation and emotions. However, they often work like a food critic who delivers a verdict after reading the menu but not tasting the dish.
Signal Processing2026-07-152 minarXiv:2607.13583express
An Orchestra of Imperfect Elements: How Manufacturing Irregularities Paradoxically Boost the Radio Signal
Imagine a smart antenna panel, a phased array, that steers radio waves traveling at the speed of light. It works like an orchestra: hundreds of tiny emitters, like musical instruments, shift their phase (the moment they 'join' the overall sound) on command via a variable voltage.
eess.AS2026-07-141 minarXiv:2607.12529express
Hearing Aids Learn to Taste Sound
Traditional hearing aids resemble a chef who cooks from a recipe blindly. They analyze the sound chaos from the microphone and amplify speech based on it. In a noisy café, this approach often errs. The new system behaves like an experienced cook: first processes the signal, then samples the result.
eess.SY2026-07-142 minarXiv:2607.12590express
Coevolution of Agent and Environment: How an Algorithm Learns to Reshape the World Around It
Traditionally, machine learning algorithms operate in a static environment, like water flowing along a pre-cut channel without changing its shape. The new method flips this paradigm: it lets an agent simultaneously master actions and deliberately modify the very environment it inhabits.
Actions are recognized by posture, not by trajectory
Researchers took 16 everyday activities — from washing dishes to squatting — and broke the movements down into elementary components, much like a spectroscope splits light into a spectrum. Comparing three approaches, they found an unexpected key to recognition: a single averaged snapshot is enough. Only nine joints out of dozens carried the crucial information — the rest hardly mattered.
Neurons Reveal Their Secrets in Minutes, Not Hours
A neuron is like a complex musical instrument. To understand how it responds to a touch, you used to have to connect to each string directly — a delicate, slow procedure that only works for individual cells.
cs.HC2026-06-303 minarXiv:2607.00239
Retouching by the rules: where science ends and deception begins
On the screen is the result of a complex simulation: two planets collided, a shockwave ripples outward. A couple of clicks in an editor, and harsh shadows soften, noise smooths out, the background darkens. This kind of digital retouching (image editing to enhance appearance) has long been part of the scientist’s toolkit.
Who Heals Better: General AI or a Narrow Specialist
Every week, doctors ask AI assistants millions of questions at the bedside. But these tools are usually tested on exam-style problems, not on what clinicians actually ask in practice. A team of researchers set out to fix that. They collected 620 real questions from physicians across 30 specialties — from cardiology to oncology.
Stellar2026-06-201 minarXiv:2606.21896express
How AI Makes Solar Forecasts Understandable
Solar flares are powerful explosions on the Sun that can knock out satellites and blackout cities. Predicting them is complicated by the tangled magnetic fields in sunspots.
q-bio.QM2026-06-151 minarXiv:2606.16726express
How Many Mice Does It Take to Understand the Microbiome?
When you're planning a renovation, you don't buy paint by eye: you measure the wall area. In science, deciding how many samples to collect is a similar guess, but the stakes are higher. Skimp on samples, and your study becomes a lottery: you might miss a real effect, or you might mistake noise for a discovery. Overshoot, and you're wasting tubes, months, and, in animal experiments, lives.
Quantum Physics2026-06-021 minarXiv:2606.02986express
physics.data-an2026-05-171 minarXiv:2605.17518express
How chemical logic rescues spectra from noise
A spectrum is like a photograph of a material's inner workings, taken with light. When a beam passes through or reflects off a sample, molecules leave characteristic dark bands, peaks, on that photograph. But if the image is overexposed or veiled in gray, making out details becomes nearly impossible.
Machine Learning2026-05-082 minarXiv:2605.07878express
Why Some Questions Work and Others Don't
You're trying a dish at a restaurant, but you haven't seen the menu. You ask the waiter questions — too salty? what spices?—and from the answers you try to guess what kind of chef prepared it.
q-bio.PE2026-05-052 minarXiv:2605.04339express
The Patchwork Quilt of Evolution
A patchy landscape isn't just a picture from a geography textbook. For microbes, cells, or animals, the world is a mosaic of patches: warmer here, more food there, and behind that rock — almost poison. When a mutant with a new trait appears in such a mosaic, its fate depends on which patch it lands on. This new work ties two factors together.
q-bio.PE2026-04-121 minarXiv:2604.10571express
Evolution by the Same Rules
Engineers improving neural networks act like breeders: they try a variant, check the result, and keep the best. They collected 935 such trials from 161 papers and measured how often a code tweak turns out beneficial, harmful, or neutral. The picture is familiar to any biologist: 68% of changes degrade performance, 19% do nothing, and only 13% yield a gain.
Applied Statistics2026-04-051 minarXiv:2604.03952express
Grip Strength, Breathing, and Running: Three Shields Against Dementia
Brain aging is like the wear and tear on a bridge: cracks appear unnoticed, but if you reinforce the supports in time, the structure lasts longer. Three ordinary things turned out to be such supports — handgrip strength, the ability to move for a long time without breathlessness, and lung capacity.
Machine Learning2026-04-051 minarXiv:2604.04175express
Doctors Don't See Everything: A Model Learns from Gaps
A typical medical neural network sees a patient as a point on a map: there's height, blood pressure, a blood test, and the model places a single label: "healthy" or "sick." But in a real hospital, the map is always torn. One test is done on Monday, another a month later, and the CT scanner is a different brand altogether. A point on such a map is self-deception.
Gait as a Mirror of Health
Gait is not just a way to get from point A to point B. Each step, arm swing, and torso tilt is like a signature the body leaves in space. Doctors have long noticed that disease alters gait, but they considered it merely a symptom of a specific problem: if you limp, your leg hurts. Now it turns out that gait is a whole book about health, and you can read it with an ordinary camera.
q-bio.GN2026-03-221 minarXiv:2603.21201express
46 ways to calculate risk
Geneticists read DNA like a long list of letters. From it, they compute a "polygenic risk score" — a number that shows how much higher or lower a person's chance of getting a disease, like diabetes, is compared to average.
Fluid Dynamics2026-03-191 minarXiv:2603.18615express
A Drop Strikes Differently
A raindrop falling to the ground strikes with a certain force. But what if the drop isn't round but elongated like a cigar or flattened like a pancake? It turns out that shape is everything: the impact force can vary by more than a factor of ten just because of how the drop looks at the moment of impact.
Machine Learning2026-02-011 minarXiv:2602.01045express
Overlapping Features Speed Up Learning
A neural network can be compared to a dormitory where each resident gets their own room. Usually, each feature in the data occupies a separate neuron, and learning proceeds sequentially: first one feature, then another. This is slow and depends on which data you happen to get. But if you allow several features to share a room, overlap them, the picture changes.
q-bio.BM2026-01-311 minarXiv:2602.00660express
How Proteins Find Their Minimum
To tell one protein from another, you need to describe its shape and composition. There are dozens of such descriptions: molecular length, charge, solubility. But not all are equally useful. A method called Differentiable Information Imbalance (DII) can pick out the most informative features while discarding the rest.
stat.ME2026-01-302 minarXiv:2601.22481
Breaking Points: How a Restorer Cleans Data
Old weather records hold surprises: the thermometer suddenly jumps by half a degree, not because of the weather, but because the station switched instruments. Such abrupt shifts in data are called changepoints, and separating them from genuine climate signals is a challenge statisticians grapple with.
A Map in the Mind: How the Brain Transfers Experience
When moving to a new apartment, a person doesn't relearn how to open doors or turn on lights. Experience from the old home transfers automatically: a kitchen is a place with a stove and a sink, a bedroom is a place with a bed. The brain doesn't store an exact map but a flexible schema that it overlays onto unfamiliar space. Scientists have built a computer model of this transfer. It has two levels.
Biological Physics2026-01-212 minarXiv:2601.15106express
How a cell decides whether to survive radiation
A cell hit by a proton beam is like a city after a pinpoint bombing: some blocks are leveled, others just have broken windows. Whether the city survives depends on which buildings are hit and how the strikes are distributed.
q-bio.QM2026-01-161 minarXiv:2601.11148express
How Blood Tells the Story of a Tumor
A drop of blood can reveal how a tumor in the lungs is behaving. Doctors have long noticed that when the disease retreats, some blood markers rise while others fall. But tying these changes into a single coherent picture has been elusive.
Biological Physics2026-01-142 minarXiv:2601.09660express
How a Tendon Tears from Within
The shoulder tendon is like a cable that holds your arm and lets it rotate. When that cable gets injured, its internal structure changes, and it starts to stretch differently. In this study, they took a rat's supraspinatus tendon and pulled it along its fibers while recording how every point inside it shifted.
Machine Learning2026-01-111 minarXiv:2601.06788express
Why neural networks are like black holes: a lesson from 'bald' models
Fine-tuning a neural network is like trying to alter a black hole without touching its event horizon: you change only a tiny fraction of parameters. Researchers looked inside and saw stunning patterns — strong links between pieces of the model, akin to quantum entanglement.
Machine Learning2026-01-011 minarXiv:2601.00192express
The Heart Under a Microscope: A Tiny Model Catches Arrhythmia in Half a Microsecond
An electrocardiogram is like a musical score, where each note is a heartbeat. A doctor hears the melody and spots the off-notes — arrhythmia. But listening to the score for hours is tiring, so a smart monitor is placed by the patient's bedside.
Whisper in the Hall: How to Build a Material That Silences Sound at Two Frequencies at Once
A steel plate with an 8×8 grid of cells dampens elastic waves in two frequency corridors at once. This is not a lucky accident but the outcome of a calculation using a new scheme: first a coarse template for the low frequency, then fine-tuning within free pixels for the high frequency.