• Corpus ID: 12130755

Introducing LETOR 4.0 Datasets

@article{Qin2013IntroducingL4,
  title={Introducing LETOR 4.0 Datasets},
  author={Tao Qin and Tie-Yan Liu},
  journal={ArXiv},
  year={2013},
  volume={abs/1306.2597}
}
LETOR is a package of benchmark data sets for research on LEarning TO Rank, which contains standard features, relevance judgments, data partitioning, evaluation tools, and several baselines. Version 1.0 was released in April 2007. Version 2.0 was released in Dec. 2007. Version 3.0 was released in Dec. 2008. This version, 4.0, was released in July 2009. Very different from previous versions (V3.0 is an update based on V2.0 and V2.0 is an update based on V1.0), LETOR4.0 is a totally new release… 
View PDF on arXiv
230 Citations
Highly Influential Citations
62
Background Citations
69
Methods Citations
116

230 Citations

Feature Selection and Model Comparison on Microsoft Learning-to-Rank Data Sets
TLDR
This report focuses on the core problem of information retrieval: how to learn the relevance between a document and a query given by user, and finds that models of boosting trees, random forest in general achieve the best performance of prediction.
The Istella22 Dataset: Bridging Traditional and Neural Learning to Rank Evaluation
TLDR
Through preliminary experiments on Istella22, it is found that neural re-ranking approaches lag behind LtR models in terms of effectiveness, butLtR models identify the scores from neural models as strong signals, and enables a fair evaluation of traditional learning-to-rank and transfer ranking techniques on the same data.
ES-Rank: evolution strategy learning to rank approach
TLDR
Experimental results from comparing the proposed method to fourteen other approaches from the literature, show that ES-Rank achieves the overall best performance.
WANDS: Dataset for Product Search Relevance Assessment
TLDR
This work proposes a systematic and e-ective way to build a discriminative, reusable, and fair human-labeled dataset, Wayfair Annotation DataSet (WANDS), for e-commerce scenarios and introduces an important cross-referencing step to the annotation process which increases dataset completeness.
UvA-DARE ( Digital Academic Repository ) Query-level Ranker Specialization
TLDR
The Specialized Ranker Model is introduced which assigns queries to di‚erent rankers that become specialized on a subset of the available queries, and a computationally feasible expectation-maximization procedure is derived to infer the model’s parameters.
Listwise Learning to Rank by Exploring Unique Ratings
TLDR
This paper proposes new listwise learning-to-rank models that mitigate the shortcomings of existing ones and proposes a novel and efficient way of refining prediction scores by combining an adapted Vanilla Recurrent Neural Network (RNN) model with pooling given selected documents at previous steps.
Learning to rank, a supervised approach for ranking of documents
TLDR
This thesis investigates state-of-the-art machine learning methods for ranking known as learning to rank to explore if it can be used in enterprise search, which means less data and less document features than web based search.
Sampling Bias Due to Near-Duplicates in Learning to Rank
TLDR
It is demonstrated that duplication causes overfitting and thus less effective models, making a strong case for the benefits of systematic deduplication before training and model evaluation.
ery-level Ranker Specialization
TLDR
The Specialized Ranker Model is introduced which assigns queries to di‚erent rankers that become specialized on a subset of the available queries, and a computationally feasible expectation-maximization procedure is derived to infer the model’s parameters.
Learning a Deep Listwise Context Model for Ranking Refinement
TLDR
This work proposes to use the inherent feature distributions of the top results to learn a Deep Listwise Context Model that helps to fine tune the initial ranked list and can significantly improve the state-of-the-art learning to rank methods on benchmark retrieval corpora.
...
...