• Corpus ID: 6847667

Finite Sample Prediction and Recovery Bounds for Ordinal Embedding

@article{Jain2016FiniteSP,
  title={Finite Sample Prediction and Recovery Bounds for Ordinal Embedding},
  author={Lalit P. Jain and Kevin G. Jamieson and Robert D. Nowak},
  journal={ArXiv},
  year={2016},
  volume={abs/1606.07081}
}
The goal of ordinal embedding is to represent items as points in a low-dimensional Euclidean space given a set of constraints in the form of distance comparisons like "item $i$ is closer to item $j$ than item $k$". Ordinal constraints like this often come from human judgments. To account for errors and variation in judgments, we consider the noisy situation in which the given constraints are independently corrupted by reversing the correct constraint with some probability. This paper makes… 

Figures from this paper

Insights into Ordinal Embedding Algorithms: A Systematic Evaluation.
TLDR
This paper provides the first comprehensive and systematic empirical evaluation of existing algorithms as well as a new neural network approach for ordinal embedding and finds that simple, relatively unknown, non-convex methods consistently outperform all other algorithms, including elaborate approaches based on neural networks or landmark approaches.
Landmark Ordinal Embedding
TLDR
This paper derives bounds establishing the statistical consistency of LOE under the popular Bradley- Terry-Luce noise model, and shows that LOE is significantly more efficient than conventional ordinal embedding approaches as the number of items grows.
Uncertainty Estimates for Ordinal Embeddings
TLDR
This paper introduces empirical uncertainty estimates for standard embedding algorithms when few noisy triplets are available, using a bootstrap and a Bayesian approach and shows that these estimates are well calibrated and can serve to select embedding parameters or to quantify uncertainty in scientific applications.
Machine learning in a setting of ordinal distance information
TLDR
This talk will talk about a result that states the asymptotic uniqueness of ordinal embeddings, and introduce data-dependent kernel functions that can be evaluated given only ordinal distance information about a data set that provide a generic alternative to the Ordinal embedding approach and avoid some of its drawbacks.
Elliptical Ordinal Embedding
TLDR
This paper proposes to embed each object as a Gaussian distribution, investigating the ability of these embeddings to capture the underlying structure of the data while satisfying the constraints, and exploring properties of the representation.
Learning Ordinal Embedding from Sets
TLDR
This paper proposes a distributional triplet embedding approach in a scalable learning framework and shows that the query complexity of the approach is on par with the single-item approach.
Less but Better: Generalization Enhancement of Ordinal Embedding via Distributional Margin
TLDR
A margin distribution learning paradigm for ordinal embedding, entitled Distributional Margin based Ordinal Embedding (DMOE), is proposed and a concise objective function is formulated which avoids maximizing margin mean and minimizing margin variance directly but exhibits the similar effect.
Ordinal Embedding with a Latent Factor Model
TLDR
This work considers a different model where the embedding is formed within a latent space of factors upon which a user may make judgements in the form of a rank order, where the user gives an answer based on weighting latent factors as opposed to Euclidean distance in the embedded embedding.
Greedy $k$-Center from Noisy Distance Samples
TLDR
Active algorithms are proposed, based on ideas such as UCB and Thompson sampling developed in the closely related Multi-Armed Bandit problem, which adaptively decide which queries to send to the oracle and are able to solve the canonical $k$-center problem within an approximation ratio of two with high probability.
Generalization Error Bound for Hyperbolic Ordinal Embedding
TLDR
A generalization error bound of HOE is provided for the first time, which is at most exponential with respect to the embedding space’s radius, and a comparison between the bounds of Hoe and Euclidean ordinal embedding shows that HOE’'s generalizationerror is reasonable as a cost for its exponential representation ability.
...
...

References

SHOWING 1-10 OF 26 REFERENCES
Low-dimensional embedding using adaptively selected ordinal data
  • Kevin G. Jamieson, R. Nowak
  • Computer Science
    2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
  • 2011
TLDR
An algorithm is proposed that exploits the low-dimensional geometry in order to accurately embed objects based on relatively small number of sequentially selected pairwise comparisons and its performance with experiments is demonstrated.
Local Ordinal Embedding
TLDR
It is proved that in the large sample limit it is enough to know "local ordinal information" in order to perfectly reconstruct a given point configuration, which leads to the Local Ordinal Embedding algorithm, which can also be used for graph drawing.
A Tight Bound of Hard Thresholding
TLDR
A novel stochastic algorithm is presented which performs hard thresholding in each iteration, hence ensuring such parsimonious solutions and proves the {\em global linear convergence} for a number of prevalent statistical models under mild assumptions, even though the problem turns out to be non-convex.
1-Bit Matrix Completion
TLDR
A theory of matrix completion for the extreme case of noisy 1-bit observations is developed and it is shown that the maximum likelihood estimate under a suitable constraint returns an accurate estimate of M when ||M||_{\infty} <= \alpha, and rank(M) <= r.
Individualized rank aggregation using nuclear norm regularization
  • Yu Lu, S. Negahban
  • Computer Science
    2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)
  • 2015
TLDR
A theoretical justification for a nuclear norm regularized optimization procedure is provided, and high-dimensional scaling results that show how the error in estimating user preferences behaves as the number of observations increase are provided.
Uniqueness of Ordinal Embedding
TLDR
The main result of this paper is a formal proof of the conjecture that upon knowledge of all ordinal constraints a large but finite set of points can be approximately reconstructed up to a similarity transformation.
Stochastic triplet embedding
TLDR
A new technique called t-Distributed Stochastic Triplet Embedding (t-STE) is introduced that collapses similar points and repels dissimilar points in the embedding - even when all triplet constraints are satisfied.
Preference Completion: Large-scale Collaborative Ranking from Pairwise Comparisons
TLDR
A large-scale non-convex implementation of AltSVM is developed that trains a factored form of the matrix via alternating minimization, and scales and parallelizes very well to large problem settings.
Sharp Time–Data Tradeoffs for Linear Inverse Problems
TLDR
The results demonstrate that a linear convergence rate is attainable even though the least squares objective is not strongly convex in these settings, and present a unified convergence analysis of the gradient projection algorithm applied to such problems.
Generalized Non-metric Multidimensional Scaling
TLDR
It is argued that this setting is more natural in some experimental settings and proposed algorithm based on convex optimization techniques to solve the non-metric multidimensional scaling problem in which only a set of order relations of the form dij < dkl are provided is provided.
...
...