# A General Boosting Method and its Application to Learning Ranking Functions for Web Search

@inproceedings{Zheng2007AGB, title={A General Boosting Method and its Application to Learning Ranking Functions for Web Search}, author={Zhaohui Zheng and Hongyuan Zha and Tong Zhang and Olivier Chapelle and Keke Chen and Gordon Sun}, booktitle={NIPS}, year={2007} }

We present a general boosting method extending functional gradient boosting to optimize complex loss functions that are encountered in many machine learning problems. Our approach is based on optimization of quadratic upper bounds of the loss functions which allows us to present a rigorous convergence analysis of the algorithm. More importantly, this general framework enables us to use a standard regression base learner such as single regression tree for fitting any loss function. We illustrate… Expand

#### 199 Citations

Adapting boosting for information retrieval measures

- Computer Science
- Information Retrieval
- 2009

This work presents a new ranking algorithm that combines the strengths of two previous methods: boosted tree classification, and LambdaRank, and shows how to find the optimal linear combination for any two rankers, and uses this method to solve the line search problem exactly during boosting. Expand

Ranking, Boosting, and Model Adaptation

- Computer Science
- 2008

We present a new ranking algorithm that combines the strengths of two previous methods: boosted tree classification, and LambdaR ank, which has been shown to be empirically optimal for a widely used… Expand

Ranking function adaptation with boosting trees

- Computer Science
- TOIS
- 2011

A new approach called tree-based ranking function adaptation (Trada) is proposed to effectively utilize data sources for training cross-domain ranking functions and is extended to utilize the pairwise preference data from the target domain to further improve the effectiveness of adaptation. Expand

Multi-task learning for boosting with application to web search ranking

- Computer Science
- KDD
- 2010

In this paper we propose a novel algorithm for multi-task learning with boosted decision trees. We learn several different learning tasks with a joint model, explicitly addressing the specifics of… Expand

Learning to rank with ties

- Computer Science
- SIGIR '08
- 2008

This paper analyzes the properties of ties and develops novel learning frameworks which combine ties and preference data using statistical paired comparison models to improve the performance of learned ranking functions. Expand

Multi-task learning to rank for web search

- Computer Science
- Pattern Recognit. Lett.
- 2012

A boosting framework for learning to rank in the multi-task learning context is proposed to learn non-parametric common structures adaptively from multiple tasks in a stage-wise way to attack the problem of poor quality training data in web search. Expand

Web-Search Ranking with Initialized Gradient Boosted Regression Trees

- Computer Science
- Yahoo! Learning to Rank Challenge
- 2011

This paper investigates Random Forests as a low-cost alternative algorithm to Gradient Boosted Regression Trees (GBRT) (the de facto standard of web-search ranking) and provides an upper bound of the Expected Reciprocal Rank (Chapelle et al., 2009) in terms of classification error. Expand

Generalized Boosting Algorithms for Convex Optimization

- Mathematics, Computer Science
- ICML
- 2011

This work analyzes gradient-based descent algorithms for boosting with respect to any convex objective and introduces a new measure of weak learner performance into this setting which generalizes existing work. Expand

Gradient descent optimization of smoothed information retrieval metrics

- Computer Science
- Information Retrieval
- 2009

This work proposes an algorithm which aims at directly optimizing popular measures such as the Normalized Discounted Cumulative Gain and the Average Precision, to minimize a smooth approximation of these measures with gradient descent. Expand

Learning to rank from Bayesian decision inference

- Computer Science
- CIKM
- 2009

This paper proposes a novel framework that directly optimizes the Bayes risk related to the ranking accuracy in terms of the IR evaluation measures and demonstrates that the framework outperforms several existing methods in most cases. Expand

#### References

SHOWING 1-10 OF 19 REFERENCES

An Efficient Boosting Algorithm for Combining Preferences

- Computer Science
- J. Mach. Learn. Res.
- 2003

This work describes and analyze an efficient algorithm called RankBoost for combining preferences based on the boosting approach to machine learning, and gives theoretical results describing the algorithm's behavior both on the training data, and on new test data not seen during training. Expand

Ranking and Scoring Using Empirical Risk Minimization

- Computer Science
- COLT
- 2005

This work investigates learning methods based on empirical minimization of the natural estimates of the ranking risk of U-statistics and U-processes to give a theoretical framework for ranking algorithms based on boosting and support vector machines. Expand

Greedy function approximation: A gradient boosting machine.

- Mathematics
- 2001

Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions… Expand

Learning to rank using gradient descent

- Mathematics, Computer Science
- ICML
- 2005

RankNet is introduced, an implementation of these ideas using a neural network to model the underlying ranking function, and test results on toy data and on data from a commercial internet search engine are presented. Expand

Optimizing search engines using clickthrough data

- Computer Science
- KDD
- 2002

The goal of this paper is to develop a method that utilizes clickthrough data for training, namely the query-log of the search engine in connection with the log of links the users clicked on in the presented ranking. Expand

Training conditional random fields via gradient tree boosting

- Mathematics, Computer Science
- ICML
- 2004

This paper describes a new method for training CRFs by applying Friedman's (1999) gradient tree boosting method, which scales linearly in the order of the Markov model and in the Order of the feature interactions, rather than exponentially like previous algorithms based on iterative scaling and gradient descent. Expand

Robust Reductions from Ranking to Classification

- 2006

We reduce ranking, as measured by the Area Under the Reciever Operating Characteristic Curve (AUC), to binary classification. The core theorem shows that a binary classification regret of r on the… Expand

Learning to Order Things

- Computer Science, Mathematics
- NIPS
- 1997

An on-line algorithm for learning preference functions that is based on Freund and Schapire's "Hedge" algorithm is considered, and it is shown that the problem of finding the ordering that agrees best with a learned preference function is NP-complete. Expand

Large Margin Methods for Structured and Interdependent Output Variables

- Computer Science
- J. Mach. Learn. Res.
- 2005

This paper proposes to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation and presents a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems. Expand

Robust reductions from ranking to classification

- Mathematics, Computer Science
- Machine Learning
- 2008

The core theorem shows that a binary classification regret of r on the induced binary problem implies an AUC regret of at most 2r, a large improvement over approaches such as ordering according to regressed scores, which have a regret transform of r↦nr where n is the number of elements. Expand