2001 GreedyFunctionApprox

• (Friedman, 2001) ⇒ Jerome H. Friedman. (2001). “Greedy Function Approximation: A gradient boosting machine.” In: The Annals of Statistics, 29(5) doi:10.1214/aos/1013203451

Subject Headings: Gradient Boosted Decision Tree, Partial Dependence Plot.

Cited By

2002

• (Friedman, 2002) ⇒ Jerome H. Friedman. (2002). “Stochastic Gradient Boosting.” In: Computational Statistics & Data Analysis 38, no. 4

Quotes

Abstract

Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions and steepest-descent minimization. A general gradient descent "boosting" paradigm is developed for additive expansions based on any fitting criterion. Specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and multiclass logistic likelihood for classification. Special enhancements are derived for the particular case where the individual additive components are regression trees, and tools for interpreting such "TreeBoost" models are presented. Gradient boosting of regression trees produces competitive, highly robust, interpretable procedures for both regression and classification, especially appropriate for mining less than clean data. Connections between this approach and the boosting methods of Freund and Shapire and Friedman 1996, Hastie and Tibshirani 1998 are discussed.

---

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2001 GreedyFunctionApprox	Jerome H. Friedman			Greedy Function Approximation: A gradient boosting machine		The Annals of Statistics	http://www.salford-systems.com/doc/GreedyFuncApproxSS.pdf	10.1214/aos/1013203451		2001