# A Direct Formulation for Sparse PCA Using Semidefinite Programming

@article{dAspremont2007ADF, title={A Direct Formulation for Sparse PCA Using Semidefinite Programming}, author={A. d'Aspremont and L. Ghaoui and Michael I. Jordan and G. Lanckriet}, journal={SIAM Rev.}, year={2007}, volume={49}, pages={434-448} }

Given a covariance matrix, we consider the problem of maximizing the variance explained by a particular linear combination of the input variables while constraining the number of nonzero coefficients in this combination. This problem arises in the decomposition of a covariance matrix into sparse factors or sparse principal component analysis (PCA), and has wide applications ranging from biology to finance. We use a modification of the classical variational representation of the largest… Expand

#### Figures, Tables, and Topics from this paper

#### 620 Citations

A convex integer programming approach for optimal sparse PCA

- Computer Science
- 2018

This paper presents a convex integer programming (IP) framework to solve the SPCA problem to near-optimality, with an emphasis on deriving associated dual bounds and empirically observes that the dual bounds are significantly better than worst-case performance, and are superior to the SDP bounds on some real-life instances. Expand

Using L1-relaxation and integer programming to obtain dual bounds for sparse PCA

- Mathematics
- 2018

Principal component analysis (PCA) is one of the most widely used dimensionality reduction tools in scientific data analysis. The PCA direction, given by the leading eigenvector of a covariance… Expand

Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation

- Computer Science, Mathematics
- IEEE Transactions on Signal Processing
- 2016

Numerical experiments show that the proposed eigenvector extraction algorithm outperforms existing algorithms in terms of support recovery and explained variance, whereas the covariance estimation algorithms improve the sample covariance estimator significantly. Expand

Low-Rank Optimization on the Cone of Positive Semidefinite Matrices

- Mathematics, Computer Science
- SIAM J. Optim.
- 2010

The geometry of that manifold is discussed and a second-order optimization method with guaranteed quadratic convergence is derived, which provides some conditions on the rank of the factorization to ensure equivalence with the original problem. Expand

The Sparse Eigenvalue Problem

- Mathematics
- 2009

In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of the… Expand

DO SEMIDEFINITE RELAXATIONS SOLVE SPARSE PCA UP TO THE INFORMATION LIMIT

- Mathematics
- 2015

Estimating the leading principal components of data, assuming they are sparse, is a central task in modern high-dimensional statistics. Many algorithms were developed for this sparse PCA problem,… Expand

A majorization-minimization approach to the sparse generalized eigenvalue problem

- Mathematics, Computer Science
- Machine Learning
- 2010

The proposed sparse GEV algorithm, which offers a general framework to solve any sparse G EV problem, will give rise to competitive algorithms for a variety of applications where specific instances of GEV problems arise. Expand

A D.C. Programming Approach to the Sparse Generalized Eigenvalue Problem

- Mathematics
- 2009

In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of the… Expand

Fantope Projection and Selection: A near-optimal convex relaxation of sparse PCA

- Mathematics, Computer Science
- NIPS
- 2013

A novel convex relaxation of sparse principal subspace estimation based on the convex hull of rank-d projection matrices (the Fantope) is proposed and implies the near-optimality of DSPCA (d'Aspremont et al. [1]) even when the solution is not rank 1. Expand

Orthogonal sparse eigenvectors: A procrustes problem

- Mathematics, Computer Science
- 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2016

A new method to estimate dominant sparse eigenvectors without trading off their orthogonality is developed using the minorization-maximization (MM) framework where it is shown that the propose method matches or outperforms existing algorithms in terms of recovery probability and explained variance. Expand

#### References

SHOWING 1-10 OF 79 REFERENCES

A Spectral Bundle Method for Semidefinite Programming

- Mathematics, Computer Science
- SIAM J. Optim.
- 2000

This work proposes replacing the traditional polyhedral cutting plane model constructed from subgradient information by a semidefinite model that is tailored to eigenvalue problems, and presents numerical examples demonstrating the efficiency of the approach on combinatorial examples. Expand

Smoothing Technique and its Applications in Semidefinite Optimization

- Mathematics, Computer Science
- Math. Program.
- 2007

This paper develops a simple framework for estimating a Lipschitz constant for the gradient of some symmetric functions of eigenvalues of symmetric matrices and analyzes the efficiency of the special gradient-type schemes on the problems of minimizing the maximal eigenvalue or the spectral radius of the matrix. Expand

Sparse nonnegative solution of underdetermined linear equations by linear programming.

- Mathematics, Medicine
- Proceedings of the National Academy of Sciences of the United States of America
- 2005

It is shown that outward k-neighborliness is equivalent to the statement that, whenever y = Ax has a non negative solution with at most k nonzeros, it is the nonnegative solution to y =Ax having minimal sum. Expand

Interior Point Methods in Semidefinite Programming with Applications to Combinatorial Optimization

- Mathematics, Computer Science
- SIAM J. Optim.
- 1995

It is argued that many known interior point methods for linear programs can be transformed in a mechanical way to algorithms for SDP with proofs of convergence and polynomial time complexity carrying over in a similar fashion. Expand

Non-euclidean restricted memory level method for large-scale convex optimization

- Mathematics, Computer Science
- Math. Program.
- 2005

A new subgradient-type method for minimizing extremely large-scale nonsmooth convex functions over “simple” domains, allowing for flexible handling of accumulated information and tradeoff between the level of utilizing this information and iteration’s complexity. Expand

A rank minimization heuristic with application to minimum order system approximation

- Mathematics
- Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148)
- 2001

We describe a generalization of the trace heuristic that applies to general nonsymmetric, even non-square, matrices, and reduces to the trace heuristic when the matrix is positive semidefinite. The… Expand

Expokit: a software package for computing matrix exponentials

- Mathematics, Computer Science
- TOMS
- 1998

Expokit provides a set of routines aimed at computing matrix exponentials. More precisely, it computes either a small matrix exponential in full, the action of a large sparse matrix exponential on an… Expand

Semideenite Programming

- 2007

In semide nite programming one minimizes a linear function subject to the constraint that an a ne combination of symmetric matrices is positive semide nite. Such a constraint is nonlinear and… Expand

Sparse Principal Components Analysis

- Mathematics
- 2009

Principal components analysis (PCA) is a classical method for the reduction of dimensionality of data in the form of n observations (or cases) of a vector with p variables. For a simple model of… Expand

A Modified Principal Component Technique Based on the LASSO

- Mathematics
- 2003

In many multivariate statistical techniques, a set of linear functions of the original p variables is produced. One of the more difficult aspects of these techniques is the interpretation of the… Expand