- Published 2008

Principal Component Analysis (PCA) is a standard tool for dimensional reduction of a set of n observations (samples), each with p variables. In this paper, using a matrix perturbation approach, we study the non-asymptotic relation between the eigenvalues and eigenvectors of PCA computed on a finite sample of size n, to those of the limiting population PCA as n → ∞. As in machine learning, we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample PCA and population PCA under a spiked covariance model. In addition, we also consider the relation between finite sample PCA and the asymptotic results in the joint limit p, n →∞, with p/n = c. We present a matrix perturbation view of the “phase transition phenomenon”, and a simple linear-algebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit. Moreover, our analysis also applies for finite p, n where we show that although there is no sharp phase transition as in the infinite case, either as a function of noise level or as a function of sample size n, the eigenvector of sample PCA may exhibit a sharp ”loss of tracking”, suddenly losing its relation to the (true) eigenvector of the population PCA matrix. This occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise, whose eigenvector points in a random direction.

Citations per Year

Semantic Scholar estimates that this publication has **115** citations based on the available data.

See our **FAQ** for additional information.

Showing 1-10 of 80 extracted citations

Highly Influenced

7 Excerpts

Highly Influenced

5 Excerpts

Highly Influenced

10 Excerpts

Highly Influenced

10 Excerpts

Highly Influenced

13 Excerpts

Highly Influenced

20 Excerpts

Highly Influenced

11 Excerpts

Highly Influenced

5 Excerpts

Highly Influenced

8 Excerpts

Highly Influenced

8 Excerpts

@inproceedings{Nadler2008FiniteSA,
title={Finite Sample Approximation Results for Principal Component Analysis: a Matrix Perturbation Approach},
author={Boaz Nadler},
year={2008}
}