LogitBoost

In machine learning and computational learning theory, LogitBoost is a boosting algorithm formulated by Jerome Friedman, Trevor Hastie, and Robert Tibshirani.

The original paper casts the AdaBoost algorithm into a statistical framework. Specifically, if one considers AdaBoost as a generalized additive model and then applies the cost function of logistic regression, one can derive the LogitBoost algorithm.

Minimizing the LogitBoost cost function

LogitBoost can be seen as a convex optimization. Specifically, given that we seek an additive model of the form

[math]\displaystyle{ f = \sum_t \alpha_t h_t }[/math]

the LogitBoost algorithm minimizes the logistic loss:

[math]\displaystyle{ \sum_i \log\left( 1 + e^{-y_i f(x_i)}\right) }[/math]

See also

• Gradient boosting
• Logistic model tree