Model comparison that trades goodness of fit against the number of parameters, as in AIC and BIC.An information criterion is a numerical score used to compare competing statistical models for the same dataset. The core idea is to balance two opposing goals: a model should fit the data well, but it should not be so complex that it merely memorizes noise. Information criteria formalize this trade-off by combining a measure of goodness of fit—typically the maximum likelihood—with a penalty term that grows with the number of model parameters. The most famous examples are the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), each derived from different theoretical foundations but serving the same practical purpose.
The AIC was introduced by Hirotugu Akaike in 1974, and the BIC was developed by Gideon Schwarz in 1978. Both were motivated by information theory and have since become standard tools in statistics and many scientific fields.
For a given model, you compute the maximum value of the likelihood function, which quantifies how probable the observed data are under that model. The information criterion then subtracts a penalty that depends on the number of parameters: AIC uses a penalty of 2 times the number of parameters, while BIC uses the natural logarithm of the sample size times the number of parameters. Lower values of the criterion indicate a better model. The difference between two models' criteria can be interpreted as an approximate measure of relative evidence, with a difference of about 10 being strong evidence for the better model.
In physics, information criteria are used to choose between competing theoretical models—for example, deciding whether a spectral line is better described by a single peak or two overlapping peaks, or whether cosmological data favor a flat universe with a cosmological constant over more exotic alternatives.
The AIC is derived from an information-theoretic measure called the Kullback-Leibler divergence, which quantifies how much information is lost when one probability distribution approximates another.











