# Linear time dynamic programming for the exact path of optimal models selected from a finite set

@article{Hocking2020LinearTD, title={Linear time dynamic programming for the exact path of optimal models selected from a finite set}, author={Toby Hocking and Joseph Vargovich}, journal={ArXiv}, year={2020}, volume={abs/2003.02808} }

Many learning algorithms are formulated in terms of finding model parameters which minimize a data-fitting loss function plus a regularizer. When the regularizer involves the l0 pseudo-norm, the resulting regularization path consists of a finite set of models. The fastest existing algorithm for computing the breakpoints in the regularization path is quadratic in the number of models, so it scales poorly to high dimensional problems. We provide new formal proofs that a dynamic programming… Expand

#### Figures, Tables, and Topics from this paper

#### One Citation

Optimizing ROC Curves with a Sort-Based Surrogate Loss Function for Binary Classification and Changepoint Detection

- Computer Science, Mathematics
- ArXiv
- 2021

This work proposes a convex relaxation of this objective that results in a new surrogate loss function called the AUM, short for Area Under Min(FP, FN), which requires a sort and a sum over the sequence of points on the ROC curve and is efficiently computed and used in a gradient descent learning algorithm. Expand

#### References

SHOWING 1-10 OF 27 REFERENCES

Computationally Efficient Changepoint Detection for a Range of Penalties

- Mathematics
- 2017

ABSTRACT In the multiple changepoint setting, various search methods have been proposed, which involve optimizing either a constrained or penalized cost function over possible numbers and locations… Expand

On optimal multiple changepoint algorithms for large data

- Mathematics, Computer Science
- Stat. Comput.
- 2017

Empirical results show that FPOP is substantially faster than existing dynamic programming methods, and unlike the existing methods its computational efficiency is robust to the number of changepoints in the data. Expand

SparseNet: Coordinate Descent With Nonconvex Penalties

- Mathematics, Medicine
- Journal of the American Statistical Association
- 2011

The properties of penalties suitable for this approach are characterized, their corresponding threshold functions are studied, and a df-standardizing reparametrization is described that assists the pathwise algorithm. Expand

An algorithm for optimal partitioning of data on an interval

- Computer Science, Mathematics
- IEEE Signal Processing Letters
- 2005

This letter describes a simple but powerful algorithm that searches the exponentially large space of partitions of N data points in time O(N/sup 2/), which is guaranteed to find the exact global optimum. Expand

Best Subset Selection via a Modern Optimization Lens

- Mathematics
- 2015

In the last twenty-five years (1990-2014), algorithmic advances in integer optimization combined with hardware improvements have resulted in an astonishing 200 billion factor speedup in solving Mixed… Expand

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

- Mathematics
- 2001

Variable selection is fundamental to high-dimensional statistical modeling, including nonparametric regression. Many approaches in use are stepwise selection procedures, which can be computationally… Expand

SparseStep: Approximating the Counting Norm for Sparse Regularization

- Mathematics
- 2015

The SparseStep algorithm is presented for the estimation of a sparse parameter vector in the linear regression problem. The algorithm works by adding an approximation of the exact counting norm as a… Expand

Fast Bayesian Matching Pursuit: Model Uncertainty and Parameter Estimation for Sparse Linear Models

- Mathematics
- 2009

A low-complexity recursive procedure is presented for model selection and minimum mean squared error (MMSE) estimation in linear regression. Emphasis is given to the case of a sparse parameter vector… Expand

Nearly unbiased variable selection under minimax concave penalty

- Mathematics
- 2010

We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias… Expand

Data-driven Calibration of Penalties for Least-Squares Regression

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2009

A completely data-driven calibration algorithm for these parameters in the least-squares regression framework, without assuming a particular shape for the penalty, based on the concept of minimal penalty, recently introduced by Birge and Massart (2007). Expand