Corpus ID: 202539465

On Sample Complexity Upper and Lower Bounds for Exact Ranking from Noisy Comparisons

@inproceedings{Ren2019OnSC,
  title={On Sample Complexity Upper and Lower Bounds for Exact Ranking from Noisy Comparisons},
  author={Wenbo Ren and Jia Liu and Ness B. Shroff},
  booktitle={NeurIPS},
  year={2019}
}
This paper studies the problem of finding the exact ranking from noisy comparisons. A noisy comparison over a set of $m$ items produces a noisy outcome about the most preferred item, and reveals some information about the ranking. By repeatedly and adaptively choosing items to compare, we want to fully rank the items with a certain confidence, and use as few comparisons as possible. Different from most previous works, in this paper, we have three main novelties: (i) compared to prior works, our… Expand

Figures from this paper

Batched Coarse Ranking in Multi-Armed Bandits
TLDR
This work proposes algorithms and proves impossibility results which together give almost tight tradeoffs between the total number of arms pulls and the number of policy changes in multi-armed bandits (MAB). Expand
Adaptive Sampling for Heterogeneous Rank Aggregation from Noisy Pairwise Comparisons
  • Yue Wu, Tao Jin, Hao Lou, Pan Xu, Farzad Farnoud, Quanquan Gu
  • Computer Science
  • 2021
TLDR
This work proposes an elimination-based active sampling strategy, which estimates the ranking of items via noisy pairwise comparisons from users and improves the users’ average accuracy by maintaining an active set of users. Expand
The Sample Complexity of Best-k Items Selection from Pairwise Comparisons
TLDR
This paper studies the sample complexity (aka number of comparisons) bounds for the active best-$k$ items selection from pairwise comparisons and proposes two algorithms based on PAC best items selection algorithms that works for $k=1 and is sample complexity optimal up to a loglog factor. Expand

References

SHOWING 1-10 OF 40 REFERENCES
Approximate Ranking from Pairwise Comparisons
TLDR
This work analyzes an active ranking algorithm that counts the number of comparisons won, and decides whether to stop or which pair of items to compare next, based on confidence intervals computed from the data collected in previous steps, and shows that this algorithm succeeds in recovering approximate rankings using a number of measurements close to optimal up to logarithmic factors. Expand
PAC Ranking from Pairwise and Listwise Queries: Lower Bounds and Upper Bounds
TLDR
This paper derives a lower bound on the sample complexity (aka number of queries), and proposes an algorithm that is sample-complexity-optimal up to an $O(\log(k+l)/\log{k})$ factor and designs ranking algorithms that recover the top-$k$ or total ranking using as few queries as possible. Expand
Simple, Robust and Optimal Ranking from Pairwise Comparisons
TLDR
The Copeland counting algorithm is analyzed, and it is shown to be an optimal method up to constant factors, meaning that it achieves the information-theoretic limits for recovering the top k-subset. Expand
A Nearly Instance Optimal Algorithm for Top-k Ranking under the Multinomial Logit Model
TLDR
This work designs a new active ranking algorithm without using any information about the underlying items' preference scores, and establishes a matching lower bound on the sample complexity even when the set of preference scores is given to the algorithm. Expand
Active top-K ranking from noisy comparisons
  • S. Mohajer, Changho Suh
  • Mathematics, Computer Science
  • 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
  • 2016
TLDR
It is demonstrated that active ranking can offer significant multiplicative gains in sample complexity over passive ranking and an algorithm that runs linearly in n and which achieves the sample complexity bound is presented. Expand
Active ranking from pairwise comparisons and when parametric assumptions do not help
We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, andExpand
Optimal Sample Complexity of M-wise Data for Top-K Ranking
TLDR
This work examines an M-wise comparison model that builds on the Plackett-Luce model where for each sample, M items are ranked according to their perceived utilities modeled as noisy observations of their underlying true utilities. Expand
Adaptive Sampling for Coarse Ranking
TLDR
A computationally efficient PAC algorithm LUCBRank is proposed for coarse ranking, and an upper bound on its sample complexity is derived and a nearly matching distribution-dependent lower bound is derived. Expand
Spectral Method and Regularized MLE Are Both Optimal for Top-$K$ Ranking
TLDR
It is demonstrated that under a natural random sampling model, the spectral method alone, or the regularized MLE alone, is minimax optimal in terms of the sample complexity - the number of paired comparisons needed to ensure exact top-K identification, for the fixed dynamic range regime. Expand
Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisons
TLDR
This paper describes the minimax limits on identifiability of top-$K$ ranked items, in the presence of random and non-adaptive sampling, and proposes a nearly linear-time ranking scheme, called Spectral MLE, that allows perfect top-K$ item identification under minimal sample complexity. Expand
...
1
2
3
4
...