Analysis and extension of spectral methods for nonlinear dimensionality reduction


Many unsupervised algorithms for nonlinear dimensionality reduction, such as locally linear embedding (LLE) and Laplacian eigenmaps, are derived from the spectral decompositions of sparse matrices. While these algorithms aim to preserve certain proximity relations on average, their embeddings are not explicitly designed to preserve local features such as distances or angles. In this paper, we show how to construct a low dimensional embedding that maximally preserves angles between nearby data points. The embedding is derived from the bottom eigenvectors of LLE and/or Laplacian eigenmaps by solving an additional (but small) problem in semidefinite programming, whose size is independent of the number of data points. The solution obtained by semidefinite programming also yields an estimate of the data's intrinsic dimensionality. Experimental results on several data sets demonstrate the merits of our approach.

DOI: 10.1145/1102351.1102450

Extracted Key Phrases


Citations per Year

144 Citations

Semantic Scholar estimates that this publication has 144 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Sha2005AnalysisAE, title={Analysis and extension of spectral methods for nonlinear dimensionality reduction}, author={Fei Sha and Lawrence K. Saul}, booktitle={ICML}, year={2005} }