Corpus ID: 6543419

An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression

@article{Koh2007AnIM,
  title={An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression},
  author={Kwangmoo Koh and Seung-Jean Kim and Stephen P. Boyd},
  journal={J. Mach. Learn. Res.},
  year={2007},
  volume={8},
  pages={1519-1555}
}
Logistic regression with l1 regularization has been proposed as a promising method for feature selection in classification problems. In this paper we describe an efficient interior-point method for solving large-scale l1-regularized logistic regression problems. Small problems with up to a thousand or so features and examples can be solved in seconds on a PC; medium sized problems, with tens of thousands of features and examples, can be solved in tens of seconds (assuming some sparsity in the… Expand
A Fast Hybrid Algorithm for Large-Scale l1-Regularized Logistic Regression
TLDR
A novel hybrid algorithm based on combining two types of optimization iterations: one being very fast and memory friendly while the other being slower but more accurate is proposed, which has global convergence at a geometric rate (a Q-linear rate in optimization terminology). Expand
An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares
TLDR
A specialized interior-point method for solving large-scale -regularized LSPs that uses the preconditioned conjugate gradients algorithm to compute the search direction and can solve large sparse problems, with a million variables and observations, in a few tens of minutes on a PC. Expand
Large-scale sparse logistic regression
TLDR
An adaptive line search scheme which allows to tune the step size adaptively and meanwhile guarantees the optimal convergence rate is proposed, which demonstrates the efficiency of the proposed Lassplore algorithm for large-scale problems. Expand
Linearized Bregman for l 1-regularized Logistic Regression
Sparse logistic regression is an important linear classifier in statistical learning, providing an attractive route for feature selection. A popular approach is based on minimizing anExpand
Large-Scale Sparse Kernel Logistic Regression — with a comparative study on optimization algorithms
Kernel Logistic Regression (KLR) is a powerful probabilistic classification tool, but its training and testing both suffer from severe computational bottlenecks when used with large-scale data.Expand
A Multilevel Acceleration for l 1-regularized Logistic Regression
Logistic Regression is a well studied classification technique. In some cases, its l1-regularized version is used to reduce overfitting to the data and obtain a sparse model. Because this problem isExpand
A coordinate gradient descent method for ℓ1-regularized convex minimization
  • S. Yun, K. Toh
  • Mathematics, Computer Science
  • Comput. Optim. Appl.
  • 2011
TLDR
Comparison with several state-of-the-art algorithms specifically designed for solving large-scale ℓ1-regularized linear least squares or logistic regression problems suggests that an efficiently implemented CGD method may outperform these algorithms despite the fact that theCGD method is not specifically designed just to solve these special classes of problems. Expand
Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems
TLDR
The method extends the scope of existing LASSO algorithms to treat larger data sets, previously out of their reach, and can be extended to general l1 -penalized convex problems and present preliminary results for the Sparse Support Vector Machine and Logistic Regression problems. Expand
Multi-class kernel logistic regression: a fixed-size implementation
TLDR
This research studies a practical iterative algorithm for multi-class kernel logistic regression (KLR) and shows that the optimization problem in each iteration can be solved by a weighted version of least squares support vector machines (LS-SVMs). Expand
A sparse logistic regression framework by difference of convex functions programming
TLDR
A new feature selection method for logistic regression based on a combination of the zero-norm and l2-norm regularization is presented and it is used to directly classify licorice seeds using near-infrared spectroscopy data. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 119 REFERENCES
Efficient L1 Regularized Logistic Regression
TLDR
Theoretical results show that the proposed efficient algorithm for L1 regularized logistic regression is guaranteed to converge to the global optimum, and experiments show that it significantly outperforms standard algorithms for solving convex optimization problems. Expand
The generalized LASSO
  • V. Roth
  • Mathematics, Medicine
  • IEEE Transactions on Neural Networks
  • 2004
TLDR
This paper presents a different class of kernel regressors that effectively overcome the above problems, and presents a highly efficient algorithm with guaranteed global convergence that defies a unique framework for sparse regression models in the very rich class of IRLS models. Expand
High-Dimensional Graphical Model Selection Using ℓ1-Regularized Logistic Regression
TLDR
Under certain mutual incoherence conditions analogous to those imposed in previous work on linear regression, it is proved that consistent neighborhood selection can be obtained as long as the number of observations n grows more quickly than 6d6 log d + 2d5 log p, thereby establishing that logarithmic growth in thenumber of samples n relative to graph size p is sufficient to achieve neighborhood consistency. Expand
Convex optimization techniques for fitting sparse Gaussian graphical models
TLDR
This work considers the problem of fitting a large-scale covariance matrix to multivariate Gaussian data in such a way that the inverse is sparse, thus providing model selection, and presents two new algorithms aimed at solving problems with a thousand nodes. Expand
A comparison of numerical optimizers for logistic regression
Logistic regression is a workhorse of statistics and is closely related to method s used in Machine Learning, including the Perceptron and the Support Vector Machine. This note compares eightExpand
A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs
TLDR
A fast method for solving linear SVMs with L2 loss function that is suited for large scale data mining tasks such as text classification is developed by modifying the finite Newton method of Mangasarian in several ways. Expand
Piecewise linear regularized solution paths
We consider the generic regularized optimization problem β(λ) = argminβ L(y, Xβ) + λJ(β). Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407-499] have shown that for the LASSO-thatExpand
Logistic regression for data mining and high-dimensional classification
TLDR
This thesis will show that logistic regression can outperform modern algorithms like Support Vector Machines on a variety of learning tasks and can compute model parameters for sparse binary datasets with hundreds of thousands of rows and attributes in just a few seconds. Expand
Sparse multinomial logistic regression: fast algorithms and generalization bounds
TLDR
This paper introduces a true multiclass formulation based on multinomial logistic regression and derives fast exact algorithms for learning sparse multiclass classifiers that scale favorably in both the number of training samples and the feature dimensionality, making them applicable even to large data sets in high-dimensional feature spaces. Expand
PATHWISE COORDINATE OPTIMIZATION
We consider “one-at-a-time” coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the L1-penalized regression (lasso) in theExpand
...
1
2
3
4
5
...