Corpus ID: 7485526

A General Boosting Method and its Application to Learning Ranking Functions for Web Search

@inproceedings{Zheng2007AGB,
  title={A General Boosting Method and its Application to Learning Ranking Functions for Web Search},
  author={Zhaohui Zheng and Hongyuan Zha and Tong Zhang and Olivier Chapelle and Keke Chen and Gordon Sun},
  booktitle={NIPS},
  year={2007}
}
We present a general boosting method extending functional gradient boosting to optimize complex loss functions that are encountered in many machine learning problems. Our approach is based on optimization of quadratic upper bounds of the loss functions which allows us to present a rigorous convergence analysis of the algorithm. More importantly, this general framework enables us to use a standard regression base learner such as single regression tree for fitting any loss function. We illustrate… Expand
Adapting boosting for information retrieval measures
TLDR
This work presents a new ranking algorithm that combines the strengths of two previous methods: boosted tree classification, and LambdaRank, and shows how to find the optimal linear combination for any two rankers, and uses this method to solve the line search problem exactly during boosting. Expand
Ranking, Boosting, and Model Adaptation
We present a new ranking algorithm that combines the strengths of two previous methods: boosted tree classification, and LambdaR ank, which has been shown to be empirically optimal for a widely usedExpand
Ranking function adaptation with boosting trees
TLDR
A new approach called tree-based ranking function adaptation (Trada) is proposed to effectively utilize data sources for training cross-domain ranking functions and is extended to utilize the pairwise preference data from the target domain to further improve the effectiveness of adaptation. Expand
Multi-task learning for boosting with application to web search ranking
In this paper we propose a novel algorithm for multi-task learning with boosted decision trees. We learn several different learning tasks with a joint model, explicitly addressing the specifics ofExpand
Learning to rank with ties
TLDR
This paper analyzes the properties of ties and develops novel learning frameworks which combine ties and preference data using statistical paired comparison models to improve the performance of learned ranking functions. Expand
Multi-task learning to rank for web search
TLDR
A boosting framework for learning to rank in the multi-task learning context is proposed to learn non-parametric common structures adaptively from multiple tasks in a stage-wise way to attack the problem of poor quality training data in web search. Expand
Web-Search Ranking with Initialized Gradient Boosted Regression Trees
TLDR
This paper investigates Random Forests as a low-cost alternative algorithm to Gradient Boosted Regression Trees (GBRT) (the de facto standard of web-search ranking) and provides an upper bound of the Expected Reciprocal Rank (Chapelle et al., 2009) in terms of classification error. Expand
Generalized Boosting Algorithms for Convex Optimization
TLDR
This work analyzes gradient-based descent algorithms for boosting with respect to any convex objective and introduces a new measure of weak learner performance into this setting which generalizes existing work. Expand
Gradient descent optimization of smoothed information retrieval metrics
TLDR
This work proposes an algorithm which aims at directly optimizing popular measures such as the Normalized Discounted Cumulative Gain and the Average Precision, to minimize a smooth approximation of these measures with gradient descent. Expand
Learning to rank from Bayesian decision inference
TLDR
This paper proposes a novel framework that directly optimizes the Bayes risk related to the ranking accuracy in terms of the IR evaluation measures and demonstrates that the framework outperforms several existing methods in most cases. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 19 REFERENCES
An Efficient Boosting Algorithm for Combining Preferences
TLDR
This work describes and analyze an efficient algorithm called RankBoost for combining preferences based on the boosting approach to machine learning, and gives theoretical results describing the algorithm's behavior both on the training data, and on new test data not seen during training. Expand
Ranking and Scoring Using Empirical Risk Minimization
TLDR
This work investigates learning methods based on empirical minimization of the natural estimates of the ranking risk of U-statistics and U-processes to give a theoretical framework for ranking algorithms based on boosting and support vector machines. Expand
Greedy function approximation: A gradient boosting machine.
Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansionsExpand
Learning to rank using gradient descent
TLDR
RankNet is introduced, an implementation of these ideas using a neural network to model the underlying ranking function, and test results on toy data and on data from a commercial internet search engine are presented. Expand
Optimizing search engines using clickthrough data
TLDR
The goal of this paper is to develop a method that utilizes clickthrough data for training, namely the query-log of the search engine in connection with the log of links the users clicked on in the presented ranking. Expand
Training conditional random fields via gradient tree boosting
TLDR
This paper describes a new method for training CRFs by applying Friedman's (1999) gradient tree boosting method, which scales linearly in the order of the Markov model and in the Order of the feature interactions, rather than exponentially like previous algorithms based on iterative scaling and gradient descent. Expand
Robust Reductions from Ranking to Classification
We reduce ranking, as measured by the Area Under the Reciever Operating Characteristic Curve (AUC), to binary classification. The core theorem shows that a binary classification regret of r on theExpand
Learning to Order Things
TLDR
An on-line algorithm for learning preference functions that is based on Freund and Schapire's "Hedge" algorithm is considered, and it is shown that the problem of finding the ordering that agrees best with a learned preference function is NP-complete. Expand
Large Margin Methods for Structured and Interdependent Output Variables
TLDR
This paper proposes to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation and presents a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems. Expand
Robust reductions from ranking to classification
TLDR
The core theorem shows that a binary classification regret of r on the induced binary problem implies an AUC regret of at most 2r, a large improvement over approaches such as ordering according to regressed scores, which have a regret transform of r↦nr where n is the number of elements. Expand
...
1
2
...