Pathwise Coordinate Optimization for Sparse Learning: Algorithm and Theory
@article{Zhao2014PathwiseCO, title={Pathwise Coordinate Optimization for Sparse Learning: Algorithm and Theory}, author={Tuo Zhao and Han Liu and Tong Zhang}, journal={ArXiv}, year={2014}, volume={abs/1412.7477} }
The pathwise coordinate optimization is one of the most important computational frameworks for high dimensional convex and nonconvex sparse learning problems. It differs from the classical coordinate optimization algorithms in three salient features: {\it warm start initialization}, {\it active set updating}, and {\it strong rule for coordinate preselection}. Such a complex algorithmic structure grants superior empirical performance, but also poses significant challenge to theoretical analysis… CONTINUE READING
Figures, Tables, and Topics from this paper
31 Citations
Statistical Optimization for High Dimensional Machine Learning with Privacy and Sparsity Constraints
- Computer Science
- 2019
- Highly Influenced
- PDF
On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions
- Mathematics, Computer Science
- ArXiv
- 2017
- 2
- PDF
On Faster Convergence of Cyclic Block Coordinate Descent-type Methods for Strongly Convex Minimization
- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2017
- 14
- PDF
On Quadratic Convergence of DC Proximal Newton Algorithm in Nonconvex Sparse Learning
- Computer Science, Mathematics
- NIPS
- 2017
- 6
- PDF
An Interactive Greedy Approach to Group Sparsity in High Dimensions
- Computer Science, Mathematics
- Technometrics
- 2019
- 2
- PDF
Faster Coordinate Descent via Adaptive Importance Sampling
- Computer Science, Mathematics
- AISTATS
- 2017
- 33
- PDF
On Fast Convergence of Proximal Algorithms for SQRT-Lasso Optimization: Don't Worry About its Nonsmooth Loss Function
- Computer Science
- UAI
- 2019
- 2
- PDF
Safe Active Feature Selection for Sparse Learning
- Computer Science, Mathematics
- ArXiv
- 2018
- Highly Influenced
- PDF
References
SHOWING 1-10 OF 76 REFERENCES
OPTIMAL COMPUTATIONAL AND STATISTICAL RATES OF CONVERGENCE FOR SPARSE NONCONVEX LEARNING PROBLEMS.
- Mathematics, Medicine
- Annals of statistics
- 2014
- 134
- PDF
Accelerated Path-Following Iterative Shrinkage Thresholding Algorithm With Application to Semiparametric Graph Estimation
- Mathematics, Medicine
- Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America
- 2016
- 7
- PDF
SparseNet: Coordinate Descent With Nonconvex Penalties
- Mathematics, Medicine
- Journal of the American Statistical Association
- 2011
- 382
- Highly Influential
- PDF
An Improved Convergence Analysis of Cyclic Block Coordinate Descent-type Methods for Strongly Convex Minimization
- Mathematics, Computer Science
- AISTATS
- 2016
- 12
- PDF
Regularized M-estimators with nonconvexity: statistical and algorithmic theory for local optima
- Computer Science, Mathematics
- J. Mach. Learn. Res.
- 2013
- 311
- PDF
Gradient methods for minimizing composite objective function
- Mathematics
- 2007
- 1,222
- Highly Influential
- PDF
Analysis of Multi-stage Convex Relaxation for Sparse Regularization
- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2010
- 389
- PDF
A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems
- Computer Science, Mathematics
- 2011
- 239
- PDF
CALIBRATING NON-CONVEX PENALIZED REGRESSION IN ULTRA-HIGH DIMENSION.
- Mathematics, Medicine
- Annals of statistics
- 2013
- 104
- Highly Influential
- PDF