A committee of decorrelated decision trees, robust and hard to overfit.A random forest is a machine learning method that builds a 'forest' of many decision trees and combines their predictions. Each tree is trained on a random subset of the data and considers a random subset of features at each split, which makes the trees different from each other. The final prediction is the average (for regression) or majority vote (for classification) of all trees. This ensemble approach reduces the risk of overfitting that a single decision tree often suffers from, making the model robust and accurate on new data.
Introduced by Leo Breiman in 2001, building on earlier work on bagging and random subspace methods.
The method creates many decision trees, each trained on a bootstrap sample (random sample with replacement) of the original data. At each node of a tree, only a random subset of features is considered for the best split, not all features. This decorrelates the trees—they make different errors. When predicting, each tree gives its output, and the forest aggregates them. The randomness ensures that no single tree dominates, and the averaging smooths out individual tree noise.
Used in finance for credit scoring, in bioinformatics for gene selection, and in remote sensing for land cover classification.
Random forests can estimate which features are most important for prediction without extra computation, by measuring how much each feature reduces impurity across all trees.














