• Corpus ID: 15966283

Sinkhorn Distances: Lightspeed Computation of Optimal Transport

@inproceedings{Cuturi2013SinkhornDL,
  title={Sinkhorn Distances: Lightspeed Computation of Optimal Transport},
  author={Marco Cuturi},
  booktitle={NIPS},
  year={2013}
}
  • Marco Cuturi
  • Published in NIPS 5 December 2013
  • Computer Science
Optimal transport distances are a fundamental family of distances for probability measures and histograms of features. Despite their appealing theoretical properties, excellent performance in retrieval tasks and intuitive formulation, their computation involves the resolution of a linear program whose cost can quickly become prohibitive whenever the size of the support of these measures or the histograms' dimension exceeds a few hundred. We propose in this work a new family of optimal transport… 
View Paper
machinelearning.wustl.edu
2,293 Citations
Highly Influential Citations
425
Background Citations
940
Methods Citations
1,366
Results Citations
22

Figures from this paper

2,293 Citations

Stochastic algorithms for entropy-regularized optimal transport problems

A family of fast and practical stochastic algorithms for solving the optimal transport problem with an entropic penalization is developed, and the recently developed Greenkhorn algorithm is a limiting case of this family.

Information Geometry for Regularized Optimal Transport and Barycenters of Patterns

A new divergence on the manifold of probability distributions is proposed, building on the entropic regularization of optimal transportation problems, that is able to retain key intuitive aspects of the Wasserstein geometry, such as translation invariance, and admits an intuitive interpretation.

Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration

This paper demonstrates that general optimal transport distances can be approximated in near-linear time by Cuturi's Sinkhorn Distances, and directly suggests a new greedy coordinate descent algorithm, Greenkhorn, with the same theoretical guarantees.

Near-optimal estimation of smooth transport maps with kernel sums-of-squares

This paper proposes the first tractable algorithm for which the statistical L error on the maps nearly matches the existing minimax lower-bounds for smooth map estimation, and leads to an algorithm which has dimension-free polynomial rates in the number of samples, with potentially exponentially dimension-dependent constants.

Regularity as Regularization: Smooth and Strongly Convex Brenier Potentials in Optimal Transport

This work gives algorithms operating on two discrete measures that can recover nearly optimal transport maps with small distortion, or equivalently, nearly optimal Brenier potentials that are strongly convex and smooth.

Chain Rule Optimal Transport

A novel class of distances between statistical multivariate distributions is defined by modeling an optimal transport problem on their marginals with respect to a ground distance defined on their conditionals, and a fast differentiable Sinkhorn-type distance is obtained.

Linear Optimal Transport Embedding: Provable fast Wasserstein distance computation and classification for nonlinear problems

This paper characterize a number of settings in which LOT embeds families of distributions into a space in which they are linearly separable, and proves conditions under which the distance of the LOT embedding between two distributions in arbitrary dimension is nearly isometric to Wasserstein-2 distance between those distributions.

Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance

This work characterize the differential properties of the original Sinkhorn approximation, proving that it enjoys the same smoothness as its regularized version and explicitly provides an efficient algorithm to compute its gradient.

Optimal Transport: Fast Probabilistic Approximation with Exact Solvers

A simple subsampling scheme for fast randomized approximate computation of optimal transport distances based on averaging the exact distances between empirical measures generated from independent samples from the original measures and can be tuned towards higher accuracy or shorter computation times is proposed.

Gaussian-Smoothed Optimal Transport: Metric Structure and Statistical Efficiency

This work proposes a novel Gaussian-smoothed OT (GOT) framework, that achieves the best of both worlds: preserving the 1-Wasserstein metric structure while alleviating the empirical approximation curse of dimensionality.
...

References

SHOWING 1-10 OF 28 REFERENCES

Regularized Discrete Optimal Transport

A generalization of the discrete optimal transport, with applications to color image manipulations, that includes a relaxation of the mass conservation constraint and a regularization term and can be used for color normalization across several images.

The Metric Nearness Problem

This paper formulates and solves the metric nearness problem: Given a set of pairwise dissimilarities, find a “nearest” set of distances that satisfy the properties of a metric—principally the triangle inequality, and suggests various useful extensions and generalizations to metricNearness.

Optimal Transport: Old and New

Couplings and changes of variables.- Three examples of coupling techniques.- The founding fathers of optimal transport.- Qualitative description of optimal transport.- Basic properties.- Cyclical…

Fast and robust Earth Mover's Distances

  • Ofir PeleM. Werman
  • Computer Science
    2009 IEEE 12th International Conference on Computer Vision
  • 2009
A new algorithm is presented for a robust family of Earth Mover's Distances - EMDs with thresholded ground distances so that the number of edges is reduced by an order of magnitude, which makes it possible to compute the EMD on large histograms and databases.

Approximate earth mover’s distance in linear time

It is experimentally show that wavelet EMD is a good approximation to EMD, has similar performance, but requires much less computation, while the comparison is about as fast as for normal Euclidean distance or chi2 statistic.

An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison

  • Haibin LingK. Okada
  • Computer Science
    IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2007
The proposed EMD-L1 significantly simplifies the original linear programming formulation of EMD, and empirically shows that this new algorithm has an average time complexity of O(N2), which significantly improves the best reported supercubic complexity of the original EMD.

Network flows - theory, algorithms and applications

In-depth, self-contained treatments of shortest path, maximum flow, and minimum cost flow problems, including descriptions of polynomial-time algorithms for these core models are presented.

The Sinkhorn-Knopp Algorithm: Convergence and Applications

  • P. Knight
  • Computer Science
    SIAM J. Matrix Anal. Appl.
  • 2008
It is shown that with an appropriate modification, the Sinkhorn-Knopp algorithm is a natural candidate for computing the measure on enormous data sets.

On the Extreme Rays of the Metric Cone

  • D. Avis
  • Mathematics
    Canadian Journal of Mathematics
  • 1980
A classical result in the theory of convex polyhedra is that every bounded polyhedral convex set can be expressed either as the intersection of half-spaces or as a convex combination of extreme…

Planar Earthmover Is Not in L1

It is shown that any $L_1$ embedding of the transportation cost (a.k.a. Earthmover) metric on probability measures supported on the grid incurs distortion $\Omega(\sqrt{\log n}\right)$.