A Direct Formulation for Sparse PCA Using Semidefinite Programming

@article{dAspremont2007ADF,
  title={A Direct Formulation for Sparse PCA Using Semidefinite Programming},
  author={A. d'Aspremont and L. Ghaoui and Michael I. Jordan and G. Lanckriet},
  journal={SIAM Rev.},
  year={2007},
  volume={49},
  pages={434-448}
}
Given a covariance matrix, we consider the problem of maximizing the variance explained by a particular linear combination of the input variables while constraining the number of nonzero coefficients in this combination. This problem arises in the decomposition of a covariance matrix into sparse factors or sparse principal component analysis (PCA), and has wide applications ranging from biology to finance. We use a modification of the classical variational representation of the largest… Expand
A convex integer programming approach for optimal sparse PCA
TLDR
This paper presents a convex integer programming (IP) framework to solve the SPCA problem to near-optimality, with an emphasis on deriving associated dual bounds and empirically observes that the dual bounds are significantly better than worst-case performance, and are superior to the SDP bounds on some real-life instances. Expand
Using L1-relaxation and integer programming to obtain dual bounds for sparse PCA
  • Santanu S. Dey, R. Mazumder, Guanyi Wang
  • Mathematics
  • 2018
Principal component analysis (PCA) is one of the most widely used dimensionality reduction tools in scientific data analysis. The PCA direction, given by the leading eigenvector of a covarianceExpand
Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation
TLDR
Numerical experiments show that the proposed eigenvector extraction algorithm outperforms existing algorithms in terms of support recovery and explained variance, whereas the covariance estimation algorithms improve the sample covariance estimator significantly. Expand
Low-Rank Optimization on the Cone of Positive Semidefinite Matrices
TLDR
The geometry of that manifold is discussed and a second-order optimization method with guaranteed quadratic convergence is derived, which provides some conditions on the rank of the factorization to ensure equivalence with the original problem. Expand
The Sparse Eigenvalue Problem
In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of theExpand
DO SEMIDEFINITE RELAXATIONS SOLVE SPARSE PCA UP TO THE INFORMATION LIMIT
Estimating the leading principal components of data, assuming they are sparse, is a central task in modern high-dimensional statistics. Many algorithms were developed for this sparse PCA problem,Expand
A majorization-minimization approach to the sparse generalized eigenvalue problem
TLDR
The proposed sparse GEV algorithm, which offers a general framework to solve any sparse G EV problem, will give rise to competitive algorithms for a variety of applications where specific instances of GEV problems arise. Expand
A D.C. Programming Approach to the Sparse Generalized Eigenvalue Problem
In this paper, we consider the sparse eigenvalue problem wherein the goal is to obtain a sparse solution to the generalized eigenvalue problem. We achieve this by constraining the cardinality of theExpand
Fantope Projection and Selection: A near-optimal convex relaxation of sparse PCA
TLDR
A novel convex relaxation of sparse principal subspace estimation based on the convex hull of rank-d projection matrices (the Fantope) is proposed and implies the near-optimality of DSPCA (d'Aspremont et al. [1]) even when the solution is not rank 1. Expand
Orthogonal sparse eigenvectors: A procrustes problem
TLDR
A new method to estimate dominant sparse eigenvectors without trading off their orthogonality is developed using the minorization-maximization (MM) framework where it is shown that the propose method matches or outperforms existing algorithms in terms of recovery probability and explained variance. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 79 REFERENCES
A Spectral Bundle Method for Semidefinite Programming
TLDR
This work proposes replacing the traditional polyhedral cutting plane model constructed from subgradient information by a semidefinite model that is tailored to eigenvalue problems, and presents numerical examples demonstrating the efficiency of the approach on combinatorial examples. Expand
Smoothing Technique and its Applications in Semidefinite Optimization
  • Y. Nesterov
  • Mathematics, Computer Science
  • Math. Program.
  • 2007
TLDR
This paper develops a simple framework for estimating a Lipschitz constant for the gradient of some symmetric functions of eigenvalues of symmetric matrices and analyzes the efficiency of the special gradient-type schemes on the problems of minimizing the maximal eigenvalue or the spectral radius of the matrix. Expand
Sparse nonnegative solution of underdetermined linear equations by linear programming.
  • D. Donoho, J. Tanner
  • Mathematics, Medicine
  • Proceedings of the National Academy of Sciences of the United States of America
  • 2005
TLDR
It is shown that outward k-neighborliness is equivalent to the statement that, whenever y = Ax has a non negative solution with at most k nonzeros, it is the nonnegative solution to y =Ax having minimal sum. Expand
Interior Point Methods in Semidefinite Programming with Applications to Combinatorial Optimization
  • F. Alizadeh
  • Mathematics, Computer Science
  • SIAM J. Optim.
  • 1995
TLDR
It is argued that many known interior point methods for linear programs can be transformed in a mechanical way to algorithms for SDP with proofs of convergence and polynomial time complexity carrying over in a similar fashion. Expand
Non-euclidean restricted memory level method for large-scale convex optimization
TLDR
A new subgradient-type method for minimizing extremely large-scale nonsmooth convex functions over “simple” domains, allowing for flexible handling of accumulated information and tradeoff between the level of utilizing this information and iteration’s complexity. Expand
A rank minimization heuristic with application to minimum order system approximation
We describe a generalization of the trace heuristic that applies to general nonsymmetric, even non-square, matrices, and reduces to the trace heuristic when the matrix is positive semidefinite. TheExpand
Expokit: a software package for computing matrix exponentials
Expokit provides a set of routines aimed at computing matrix exponentials. More precisely, it computes either a small matrix exponential in full, the action of a large sparse matrix exponential on anExpand
Semideenite Programming
In semide nite programming one minimizes a linear function subject to the constraint that an a ne combination of symmetric matrices is positive semide nite. Such a constraint is nonlinear andExpand
Sparse Principal Components Analysis
Principal components analysis (PCA) is a classical method for the reduction of dimensionality of data in the form of n observations (or cases) of a vector with p variables. For a simple model ofExpand
A Modified Principal Component Technique Based on the LASSO
In many multivariate statistical techniques, a set of linear functions of the original p variables is produced. One of the more difficult aspects of these techniques is the interpretation of theExpand
...
1
2
3
4
5
...