Distance Metric Learning for Large Margin Nearest Neighbor Classification

Abstract

We show how to learn aMahanalobis distance metric for k-nearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. On seven data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification—for example, achieving a test error rate of 1.3% on the MNIST handwritten digits. As in support vector machines (SVMs), the learning problem reduces to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our framework requires no modification or extension for problems in multiway (as opposed to binary) classification.

Extracted Key Phrases

5 Figures and Tables

0200400'05'06'07'08'09'10'11'12'13'14'15'16'17'18
Citations per Year

3,243 Citations

Semantic Scholar estimates that this publication has 3,243 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Weinberger2006DistanceML, title={Distance Metric Learning for Large Margin Nearest Neighbor Classification}, author={Kilian Q. Weinberger and John Blitzer}, year={2006} }