statistics

information criterion

Model comparison that trades goodness of fit against the number of parameters, as in AIC and BIC.
37 related articles

Description

An information criterion is a numerical score used to compare competing statistical models for the same dataset. The core idea is to balance two opposing goals: a model should fit the data well, but it should not be so complex that it merely memorizes noise. Information criteria formalize this trade-off by combining a measure of goodness of fit—typically the maximum likelihood—with a penalty term that grows with the number of model parameters. The most famous examples are the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), each derived from different theoretical foundations but serving the same practical purpose.

History

The AIC was introduced by Hirotugu Akaike in 1974, and the BIC was developed by Gideon Schwarz in 1978. Both were motivated by information theory and have since become standard tools in statistics and many scientific fields.

How it works

For a given model, you compute the maximum value of the likelihood function, which quantifies how probable the observed data are under that model. The information criterion then subtracts a penalty that depends on the number of parameters: AIC uses a penalty of 2 times the number of parameters, while BIC uses the natural logarithm of the sample size times the number of parameters. Lower values of the criterion indicate a better model. The difference between two models' criteria can be interpreted as an approximate measure of relative evidence, with a difference of about 10 being strong evidence for the better model.

In practice

In physics, information criteria are used to choose between competing theoretical models—for example, deciding whether a spectral line is better described by a single peak or two overlapping peaks, or whether cosmological data favor a flat universe with a cosmological constant over more exotic alternatives.

Fun fact

The AIC is derived from an information-theoretic measure called the Kullback-Leibler divergence, which quantifies how much information is lost when one probability distribution approximates another.

Concept graph →

Related articles · 37

How to pick a crystal thickness in 50 simulations
A translator at the negotiating table doesn't listen to every word; they grasp the meaning of whole phrases. Electron microscopy poses a similar challenge: you need to figure out the crystal's thickness and its tilt in the beam, and all you have is a diffraction pattern, a mosaic of thousands of bright and dark spots.
Scales for Thoughts: How to Measure Whether a Neural Interface Really Speaks
In the 1980s, the DARPA speech recognition challenge settled the debate between statistical methods and hand-written rules, setting a common standard for the entire field.
A Bias That Needs No Teacher
In a warehouse, there are boxes of apples and pears. There are many apples and few pears. But when the stock clerk says "a fruit with smooth skin," we are somehow more likely to recall the rare pear than the common apple. This is the inverse base-rate effect — a persistent glitch in how the brain resolves ambiguity between categories.
Quantum Physics2026-08-042 minarXiv:2608.03944express
Quantum Protection vs. Classical Memory
A quantum computer is like a fragile snowflake resting on a warm palm. The slightest breath of heat or a gentle vibration, and it turns into a droplet. To keep the snowflake alive, you constantly adjust it, restoring its original shape. That's quantum error correction.
Chemical Physics2026-08-032 minarXiv:2608.02536express
Catching the Moment a Protein Breaks
A mountain pass connects two valleys. In one, the protein is folded; in the other, it is unfolded. To get from one to the other, you must cross the saddle — the highest point on the path. In protein science, this point is called the transition state, and the coordinate that shows the distance to it is the committor.
High Energy2026-07-202 minarXiv:2607.18124express
Neutron Star Twins: When Precision Changes the Picture
A neutron star is the collapsed core of a massive star, squeezed to the size of a city. A teaspoon of its material weighs as much as a mountain. But inside this superdense sphere, a surprise may lurk: under unimaginable pressure, neutrons break apart into quarks, giving the star a new, more compact core.
eess.SY2026-07-161 minarXiv:2607.14606express
How to read an enemy in the sky in seconds
In combat, a pilot needs to instantly understand what an object is and what its intentions are. Conventional systems wait for complete radar data — like a goalkeeper who only reacts after the shot. The new algorithm works like an experienced goalkeeper: from the attacker's first moves, it predicts the target's maneuver.
Machine Learning2026-07-161 minarXiv:2607.14995express
How to Teach AI Not to Confuse Similar MRI Scans
Ordinary AI looks at medical images like puzzles made of unfamiliar pieces: it considers any two unmatched pictures completely different. It's like trying to tell twins apart without knowing they're related. In medicine, however, patients with the same condition look nearly identical on MRI, and the algorithm mistakes 'relatives'.
Signal Processing2026-07-152 minarXiv:2607.13583express
An Orchestra of Imperfect Elements: How Manufacturing Irregularities Paradoxically Boost the Radio Signal
Imagine a smart antenna panel, a phased array, that steers radio waves traveling at the speed of light. It works like an orchestra: hundreds of tiny emitters, like musical instruments, shift their phase (the moment they 'join' the overall sound) on command via a variable voltage.
eess.SY2026-07-151 minarXiv:2607.14245express
How Entropy Helps Robots Find Paths Faster
A search party combs a foggy field. At first, people spread out wide: a high 'temperature' of search means a great variety of attempts. Upon finding a trace, they close in. But the speed of convergence is critical — narrow the circle too early, and the best find stays aside. The coordinator assesses the spread of searchers and commands when to speed up or slow down.
Computer Vision2026-07-141 minarXiv:2607.12375express
AI with a magnifying glass and a spectrometer: image quality assessment
Assessing image quality is a job for a meticulous inspector with a toolbox. The IQA-T1 neural network acts exactly like that: it doesn't just stare at the picture, but applies either a spectroscope or a light meter.
eess.AS2026-07-141 minarXiv:2607.12647express
How a Computer Learns to Hear Everyone in a Noisy Room
In a noisy room, a person picks out a desired voice thanks to two ears capturing the difference in sound arrival. A computer with several microphones does the same: it gets spatial cues — where each voice is coming from. Previously, these cues were used to focus on one direction, like a flashlight beam.
eess.SY2026-07-131 minarXiv:2607.11122express
Float Controller: Stability via an Instant Equation
The float in a toilet tank is an ideal controller: sensing the water level, it instantly brings the system into equilibrium without looking back at the past. Engineers applied this principle to neural networks.
Quantum Physics2026-07-072 minarXiv:2607.06404express
Quantum Code, Carbon Copy
A quantum computer is like an orchestra without a conductor: each qubit is a musician, and the slightest slip throws off the entire symphony. To keep the music from turning into cacophony, quantum error-correcting codes were invented.
econ.EM2026-07-013 minarXiv:2607.01429
The Data Watershed: How a Simple Yes/No Switch Reveals Multidimensional Models
On a topographic map, raindrops choose their path: falling on one slope, they flow into River A; on the other, into River B. An observer sees only the winding rivers (the response variables Y) and knows which slope the water came from (the instrument Z), but cannot see the drops themselves (the unobserved factors ε).
Squealer: When the Model Squeals, Rejecting Your Data
Statistical models are an attempt to stretch a perfect mathematical web over the uneven landscape of data. When a model resists, it emits tension, but we've grown accustomed to ignoring it. Researchers spend hours staring at residual plots, trying to guess where the model is 'coming apart at the seams.' Squealer translates this resistance into sound.
Who Heals Better: General AI or a Narrow Specialist
Every week, doctors ask AI assistants millions of questions at the bedside. But these tools are usually tested on exam-style problems, not on what clinicians actually ask in practice. A team of researchers set out to fix that. They collected 620 real questions from physicians across 30 specialties — from cardiology to oncology.
Materials2026-06-061 minarXiv:2606.08148express
How Clever Sampling Teaches Metal to Predict Strength
A chef doesn't waste ingredients cooking the whole dish to check the taste: they just taste a drop of sauce. The same logic underlies the "information matching" method for predicting metal strength. Instead of expensive calculations of every atom's behavior, scientists "taste" only simple indicators linked to plasticity — a metal's ability to bend without breaking.
cs.CE2026-06-051 minarXiv:2606.07258express
How to Test a Predictor Without Knowing the Answer
A lab sends a robot a list of a thousand candidate molecules for a new drug. The predictor robot, trained on old data, gives each one a 'fit or no fit' score. But when the list comes from an unfamiliar area, a different virus or a rare tumor, the robot starts to get confused. Its confidence stays high, while actual hits drop to zero.
Machine Learning2026-05-082 minarXiv:2605.07878express
Why Some Questions Work and Others Don't
You're trying a dish at a restaurant, but you haven't seen the menu. You ask the waiter questions — too salty? what spices?—and from the answers you try to guess what kind of chef prepared it.
Consciousness as a Thermostat Glitch
A thermostat keeps a room at 20 degrees. Open a window in winter, and it kicks on the heat to restore equilibrium. The brain does something similar: it constantly predicts what will happen a moment ahead and corrects itself when wrong. That's how it learns.
stat.ME2026-04-251 minarXiv:2604.23463express
How Two Doctors Together Make a More Accurate Diagnosis
Two specialists look at the same breast image. One is an experienced doctor, the other is an artificial intelligence program. Sometimes their opinions match, sometimes they diverge. How often they agree determines how accurately they can together distinguish healthy tissue from a tumor.
q-bio.QM2026-04-212 minarXiv:2604.19563express
Why DNA Copies Itself in a Wasteful Way
If you write down a protein recipe on a piece of paper and send it to a print shop, a typo in a single letter can ruin the whole batch.
Why the Second Touch Feels Different
Your hand rests on the table, and two short vibrations arrive at your wrist — one after the other. You need to say which is stronger. The task seems simple, but the answer depends on which vibration came first. If the first is weak and the second is strong, the difference feels larger than when they are swapped.
An Exam for AI Scientists
The best AI assistant for scientists scored worse on this exam than a C student on a pop quiz. Just 21% correct answers, and that's the ceiling for the four strongest models. The problems for the new test were written not by programmers but by graduate students: people who deal with real science every day. The test is called COMPOSITE-STEM.
The Brain Learns by Making Neurons More Alike
A chef's knife and a peeler each handle different tasks, but when a cook learns a new recipe, they often grab the same knife for everything. The brain seems to do the same when mastering a new skill of distinguishing patterns. In the experiment, monkeys trained for weeks to tell apart barely noticeable differences in complex patterns.
q-bio.BM2026-01-311 minarXiv:2602.00660express
How Proteins Find Their Minimum
To tell one protein from another, you need to describe its shape and composition. There are dozens of such descriptions: molecular length, charge, solubility. But not all are equally useful. A method called Differentiable Information Imbalance (DII) can pick out the most informative features while discarding the rest.
stat.ME2026-01-302 minarXiv:2601.22481
Breaking Points: How a Restorer Cleans Data
Old weather records hold surprises: the thermometer suddenly jumps by half a degree, not because of the weather, but because the station switched instruments. Such abrupt shifts in data are called changepoints, and separating them from genuine climate signals is a challenge statisticians grapple with.
Materials2026-01-241 minarXiv:2601.17253express
Neural Network Distinguishes Real Crystals from Fantasies
Porous zeolite minerals work like tiny sponges with holes of a strictly defined size. Through such holes, molecules of one kind pass while others get stuck: this is the basis for water purification, gas separation, and catalysis in petrochemistry.
Medical Physics2026-01-231 minarXiv:2601.17165express
Testing X-rays with sound
A doctor looks at a breast image to find a suspicious lump. If the picture is beautiful and smooth, but the lump has vanished from it, such an image is of little use. This is exactly the problem faced by researchers who developed a new way to test software for photoacoustic tomography.
Physics and Society2026-01-212 minarXiv:2601.14632express
Contact Tracing: Why Missing One Link Breaks the Whole Chain
A translator who jumps at every rustle in the hall turns a one-hour meeting into two. It's the same in epidemic control: contact tracing teams try to reconstruct the chain of infections, but something always slips through. Someone forgets who they met, someone doesn't answer the phone, someone never even comes to the attention of health workers.
Getting the Most Out of Every Photon
A fluorescence microscope is like a photographer shooting in the dark: the more frames, the better the view, but each frame costs photons. In a living cell, the supply is limited, otherwise the light would simply kill the sample. So scientists proposed not to increase the number of photons, but to choose more wisely when to catch them.
math.NA2024-09-103 minarXiv:2409.06797
Noise with Memory: How Colored Noise Alters Climate Causality
Ocean currents intertwine like sticky spider silk: every disturbance leaves a trace that does not vanish instantly. Climatologists have been used to treating random jolts in this web as invisible and memoryless, but new research shows that noise has memory, and that memory strengthens connections between distant nodes.
Calibrating for Small Data: The GOAL Algorithm
An optician turns the frame, adjusts the focus, and a blurry spot becomes a sharp image. The new algorithm GOAL works much the same way: it rotates the data cloud, compresses redundant dimensions, and brings into focus what truly matters.
Climate Lottery Ticket: Why El Niño Warms Three Times More Than It Seems
Winning the lottery changes the winner's life, but for the statistics of all players it means almost nothing: the probability of winning is negligible, and the average winnings per ticket remain tiny.
stat.ME2016-05-022 minarXiv:1605.00346
Where Ice Breaks: Finding Change Points in a Random Flow
The ice cover of a river looks solid, but beneath it lie cracks that alter the flow. So it is with a time series: behind the smooth surface of numbers hide moments when the rules of the game change. These moments are called change points, and finding them is like tapping the ice to locate weak spots.
Finding the Puppet Masters: How Causality Saves Predictions from Computational Blow-Up
The puppet master pulls the strings, and the marionette dances. In forecasting the future from time series, the puppet masters are the few variables that truly control the system. The rest merely echo their movements, creating an illusion of complexity. The challenge is to find these puppet masters and watch only their fingers, not the whole stage.