How much each input variable actually contributes to a trained model's decisions.Feature importance refers to techniques that assign a score to input features based on how useful they are at predicting a target variable within a trained model. It helps interpret which variables are most influential in the model's decisions, aiding in model understanding, feature selection, and domain insight.
The concept has been used informally for decades, but formalized methods like Gini importance were introduced with random forests in the early 2000s by Leo Breiman and Adele Cutler. Permutation importance was also popularized around that time.
Methods vary by model type. For tree-based models (e.g., random forests, gradient boosting), importance is often calculated by how much each feature decreases impurity (e.g., Gini impurity or entropy) across all splits. For linear models, the magnitude of coefficients can serve as a proxy. Model-agnostic methods like permutation importance measure the drop in model performance when a feature's values are randomly shuffled, breaking its association with the target.
Used in fields like genomics to identify which genes are most predictive of a disease, in marketing to understand key customer drivers, and in model debugging to ensure the model relies on sensible features rather than spurious correlations.
Feature importance scores are relative and can be misleading if features are correlated; a feature that is truly important may get a low score if a correlated feature is already used in the model.















