# A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms and Software

@article{Surez2018ATO, title={A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms and Software}, author={Juan-Luis Su{\'a}rez and Salvador Garc{\'i}a and Francisco Herrera}, journal={ArXiv}, year={2018}, volume={abs/1812.05944} }

This paper describes the discipline of distance metric learning, a branch of machine learning that aims to learn distances from the data. Distance metric learning can be useful to improve similarity learning algorithms, and also has applications in dimensionality reduction. We describe the distance metric learning problem and analyze its main mathematical foundations. We discuss some of the most popular distance metric learning techniques used in classification, showing their goals and the…

## Figures and Tables from this paper

## 33 Citations

### Curvilinear Distance Metric Learning

- Computer ScienceNeurIPS
- 2019

A Curvilinear Distance Metric Learning (CDML) method, which adaptively learns the nonlinear geometries of the training data by virtue of Weierstrass theorem, which is equivalently parameterized with a 3-order tensor, and the optimization algorithm is designed to learn the tensor parameter.

### Project and Forget: Solving Large-Scale Metric Constrained Problems

- Computer ScienceArXiv
- 2020

This paper provides an active set algorithm, Project and Forget, that uses Bregman projections, to solve metric constrained problems with many (possibly exponentially) inequality constraints and proves that the algorithm converges to the global optimal solution and that the distance of the current iterate to the optimal solution decays asymptotically at an exponential rate.

### Efficient Machine Learning Methods over Pairwise Space (keynote)

- Computer ScienceCS&P
- 2021

The initial idea of looking for the linear classifier with the maximal margin were transformed into the problem ofLooking for a set of coefficients α = (α1, α2, · · · , αn) related to objects that maximizes an objective function.

### Distance Metric Learned Collaborative Representation Classifier

- Computer ScienceArXiv
- 2019

The proposed method DML-CRC gives state-of-the-art performance on benchmark fine-grained classification datasets CUB Birds, Oxford Flowers and Oxford-IIIT Pets using the VGG-19 deep network and can be used for any similar classification tasks.

### Distance Metric Learned Collaborative Representation Classifier(DML-CRC)

- Computer ScienceIEEE Letters of the Computer Society
- 2020

The proposed method DML-CRC gives state-of-the-art performance on benchmark fine-grained classification datasets CUB Birds, Oxford Flowers and Oxford-IIIT Pets using the VGG-19 deep network and can be used for other similar classification tasks.

### Nonparametric Regression with Comparisons: Escaping the Curse of Dimensionality with Ordinal Information

- Computer ScienceICML
- 2018

This work develops an algorithm called Ranking-Regression (RR) and analyzes its accuracy as a function of size of the labeled and unlabeled datasets and various noise parameters and presents lower bounds, that establish fundamental limits for the task and show that RR is optimal in a variety of settings.

### Reducing Data Complexity using Autoencoders with Class-informed Loss Functions

- Computer ScienceIEEE transactions on pattern analysis and machine intelligence
- 2021

An autoencoder-based approach to complexity reduction is proposed, using class labels in order to inform the loss function about the adequacy of the generated variables, which leads to three different new feature learners, Scorer, Skaler and Slicer.

### MCENN: A variant of extended nearest neighbor method for pattern recognition

- Computer SciencePattern Recognit. Lett.
- 2020

### Supervision and Source Domain Impact on Representation Learning: A Histopathology Case Study

- Computer Science2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)
- 2020

This work explored the performance of a deep neural network and triplet loss in the area of representation learning, investigated the notion of similarity and dissimilarity in pathology whole-slide images and compared different setups from unsupervised and semi-supervised to supervised learning.

### Integrating Language Guidance into Vision-based Deep Metric Learning

- Computer ScienceArXiv
- 2022

Leveraging language embeddings of expert- and pseudo-classnames, this work contextualize and realign visual representation spaces corresponding to meaningful language semantics for better semantic consistency in Deep Metric Learning.

## References

SHOWING 1-10 OF 76 REFERENCES

### Metric Learning: A Survey

- Computer ScienceFound. Trends Mach. Learn.
- 2013

Metric Learning: A Review presents an overview of existing research in this topic, including recent progress on scaling to high-dimensional feature spaces and to data sets with an extremely large number of data points.

### Distance Metric Learning: A Comprehensive Survey

- Computer Science
- 2006

A number of techniques that are central to distance metric learning are discussed, including convex programming, positive semi-definite programming, kernel learning, dimension reduction, K Nearest Neighbor, large margin classification, and graph-based approaches.

### A Survey on Metric Learning for Feature Vectors and Structured Data

- Computer ScienceArXiv
- 2013

A systematic review of the metric learning literature is proposed, highlighting the pros and cons of each approach and presenting a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning.

### Large-scale distance metric learning for k-nearest neighbors regression

- Computer ScienceNeurocomputing
- 2016

### Information-theoretic metric learning

- Computer ScienceICML '07
- 2007

An information-theoretic approach to learning a Mahalanobis distance function that can handle a wide variety of constraints and can optionally incorporate a prior on the distance function and derive regret bounds for the resulting algorithm.

### Metric Learning by Collapsing Classes

- Computer Science, MathematicsNIPS
- 2005

An algorithm for learning a quadratic Gaussian metric (Mahalanobis distance) for use in classification tasks and discusses how the learned metric may be used to obtain a compact low dimensional feature representation of the original input space, allowing more efficient classification with very little reduction in performance.

### Distance Metric Learning with Eigenvalue Optimization

- Computer ScienceJ. Mach. Learn. Res.
- 2012

A novel metric learning approach called DML-eig is introduced which is shown to be equivalent to a well-known eigen value optimization problem called minimizing the maximal eigenvalue of a symmetric matrix.

### Distance metric learning for ordinal classification based on triplet constraints

- Computer ScienceKnowl. Based Syst.
- 2018

### Distance Metric Learning with Application to Clustering with Side-Information

- Computer ScienceNIPS
- 2002

This paper presents an algorithm that, given examples of similar (and, if desired, dissimilar) pairs of points in �”n, learns a distance metric over ℝn that respects these relationships.

### An Overview and Empirical Comparison of Distance Metric Learning Methods

- Computer ScienceIEEE Transactions on Cybernetics
- 2017

An overview of advances in the field of distance metric learning is offered, and well-tested features are utilized to assess the performance of selected methods following the experimental protocol of the state-of-the-art database labeled faces in the wild.