A committee of simple trees, each fixing the errors of the last; long the standard classifier in particle physics.Boosted decision trees (BDTs) are an ensemble learning method that combines many simple decision trees to produce a powerful classifier. Each tree is trained sequentially, with each new tree focusing on the mistakes made by the previous ones. BDTs have been the workhorse classifier in high-energy particle physics for over a decade, used to distinguish rare signal events from overwhelming background noise in collider data.
The concept of boosting was introduced by Robert Schapire in 1990, and the AdaBoost algorithm was developed by Freund and Schapire in 1995. BDTs became standard in particle physics after the discovery of the Higgs boson in 2012, where they played a key role in separating the Higgs signal from background.
The most common boosting algorithm is AdaBoost: start with a set of weights on training samples, train a shallow tree, then increase the weights of misclassified samples and decrease the weights of correctly classified ones. The next tree is trained on the reweighted data, and the process repeats. The final prediction is a weighted vote of all trees. Gradient boosting is a generalization that optimizes an arbitrary loss function by fitting each new tree to the residual errors of the current ensemble.
BDTs are used in particle physics for event classification, in credit scoring for risk assessment, and in medical diagnostics for disease prediction from patient data.
In the 2012 Higgs boson discovery, BDTs were so effective that they helped extract a signal that appeared in only about one in a trillion proton-proton collisions.








