# On Sample Complexity Upper and Lower Bounds for Exact Ranking from Noisy Comparisons

@inproceedings{Ren2019OnSC, title={On Sample Complexity Upper and Lower Bounds for Exact Ranking from Noisy Comparisons}, author={Wenbo Ren and Jia Liu and Ness B. Shroff}, booktitle={NeurIPS}, year={2019} }

This paper studies the problem of finding the exact ranking from noisy comparisons. A noisy comparison over a set of $m$ items produces a noisy outcome about the most preferred item, and reveals some information about the ranking. By repeatedly and adaptively choosing items to compare, we want to fully rank the items with a certain confidence, and use as few comparisons as possible. Different from most previous works, in this paper, we have three main novelties: (i) compared to prior works, our… Expand

#### 3 Citations

Batched Coarse Ranking in Multi-Armed Bandits

- Computer Science
- NeurIPS
- 2020

This work proposes algorithms and proves impossibility results which together give almost tight tradeoffs between the total number of arms pulls and the number of policy changes in multi-armed bandits (MAB). Expand

Adaptive Sampling for Heterogeneous Rank Aggregation from Noisy Pairwise Comparisons

- Computer Science
- 2021

This work proposes an elimination-based active sampling strategy, which estimates the ranking of items via noisy pairwise comparisons from users and improves the users’ average accuracy by maintaining an active set of users. Expand

The Sample Complexity of Best-k Items Selection from Pairwise Comparisons

- Computer Science, Mathematics
- ICML
- 2020

This paper studies the sample complexity (aka number of comparisons) bounds for the active best-$k$ items selection from pairwise comparisons and proposes two algorithms based on PAC best items selection algorithms that works for $k=1 and is sample complexity optimal up to a loglog factor. Expand

#### References

SHOWING 1-10 OF 40 REFERENCES

Approximate Ranking from Pairwise Comparisons

- Computer Science, Mathematics
- AISTATS
- 2018

This work analyzes an active ranking algorithm that counts the number of comparisons won, and decides whether to stop or which pair of items to compare next, based on confidence intervals computed from the data collected in previous steps, and shows that this algorithm succeeds in recovering approximate rankings using a number of measurements close to optimal up to logarithmic factors. Expand

PAC Ranking from Pairwise and Listwise Queries: Lower Bounds and Upper Bounds

- Computer Science, Mathematics
- ArXiv
- 2018

This paper derives a lower bound on the sample complexity (aka number of queries), and proposes an algorithm that is sample-complexity-optimal up to an $O(\log(k+l)/\log{k})$ factor and designs ranking algorithms that recover the top-$k$ or total ranking using as few queries as possible. Expand

Simple, Robust and Optimal Ranking from Pairwise Comparisons

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2017

The Copeland counting algorithm is analyzed, and it is shown to be an optimal method up to constant factors, meaning that it achieves the information-theoretic limits for recovering the top k-subset. Expand

A Nearly Instance Optimal Algorithm for Top-k Ranking under the Multinomial Logit Model

- Computer Science, Mathematics
- SODA
- 2018

This work designs a new active ranking algorithm without using any information about the underlying items' preference scores, and establishes a matching lower bound on the sample complexity even when the set of preference scores is given to the algorithm. Expand

Active top-K ranking from noisy comparisons

- Mathematics, Computer Science
- 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
- 2016

It is demonstrated that active ranking can offer significant multiplicative gains in sample complexity over passive ranking and an algorithm that runs linearly in n and which achieves the sample complexity bound is presented. Expand

Active ranking from pairwise comparisons and when parametric assumptions do not help

- Mathematics
- 2016

We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and… Expand

Optimal Sample Complexity of M-wise Data for Top-K Ranking

- Computer Science, Mathematics
- NIPS
- 2017

This work examines an M-wise comparison model that builds on the Plackett-Luce model where for each sample, M items are ranked according to their perceived utilities modeled as noisy observations of their underlying true utilities. Expand

Adaptive Sampling for Coarse Ranking

- Computer Science, Mathematics
- AISTATS
- 2018

A computationally efficient PAC algorithm LUCBRank is proposed for coarse ranking, and an upper bound on its sample complexity is derived and a nearly matching distribution-dependent lower bound is derived. Expand

Spectral Method and Regularized MLE Are Both Optimal for Top-$K$ Ranking

- Mathematics, Computer Science
- Annals of statistics
- 2019

It is demonstrated that under a natural random sampling model, the spectral method alone, or the regularized MLE alone, is minimax optimal in terms of the sample complexity - the number of paired comparisons needed to ensure exact top-K identification, for the fixed dynamic range regime. Expand

Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisons

- Mathematics, Computer Science
- ICML
- 2015

This paper describes the minimax limits on identifiability of top-$K$ ranked items, in the presence of random and non-adaptive sampling, and proposes a nearly linear-time ranking scheme, called Spectral MLE, that allows perfect top-K$ item identification under minimal sample complexity. Expand