# Iterative Reweighted Least Squares ∗

@inproceedings{Burrus2014IterativeRL, title={Iterative Reweighted Least Squares ∗}, author={C. Sidney Burrus}, year={2014} }

Describes a powerful optimization algorithm which iteratively solves a weighted least squares approximation problem in order to solve an L_p approximation problem. 1 Approximation Methods of approximating one function by another or of approximating measured data by the output of a mathematical or computer model are extraordinarily useful and ubiquitous. In this note, we present a very powerful algorithm most often called Iterative Reweighted Least Squares or (IRLS). Because minimizing the… Expand

#### Figures from this paper

#### 26 Citations

Iterative Reweighted Least Squares * C .

- Mathematics
- 2018

Describes a powerful optimization algorithm which iteratively solves a weighted least squares approximation problem in order to solve an L_p approximation problem. 1 Approximation Methods of… Expand

Iteratively Reweighted Least Squares for 𝓁1-minimization with Global Linear Convergence Rate

- Computer Science, Mathematics
- ArXiv
- 2020

It is proved that IRLS for l1-minimization converges to a sparse solution with a global linear rate, and theory is supported by numerical experiments indicating that the linear rate essentially captures the correct dimension dependence. Expand

Fast, Provably convergent IRLS Algorithm for p-norm Linear Regression

- Computer Science, Mathematics
- NeurIPS
- 2019

Linear regression in $\ell_p$-norm is a canonical optimization problem that arises in several applications, including sparse recovery, semi-supervised learning, and signal processing. Generic convex… Expand

Iteratively reweighted least squares and slime mold dynamics: connection and convergence

- Mathematics
- 2021

We present a connection between two dynamical systems arising in entirely different contexts: the Iteratively Reweighted Least Squares (IRLS) algorithm used in compressed sensing and sparse recovery… Expand

A nonconvex and nonsmooth anisotropic total variation model for image noise and blur removal

- Computer Science
- Multimedia Tools and Applications
- 2019

A nonconvex and nonsmooth anisotropic total variation model is proposed, which can provide a very sparser representation of the derivatives of the function in horizontal and vertical directions and is compared with several state-of-the-art models in denoising and deblurring applications. Expand

Reconstruction of Single Image from Multiple Blurry Measured Images

- Computer Science, Medicine
- IEEE Transactions on Image Processing
- 2018

Numerical studies, including synthetic and real images, demonstrate that the performance of this joint estimation algorithm is superior to the previous state-of-the-art algorithms in terms of both objective and subjective evaluation standards. Expand

Solving Weighted Least Squares (WLS) problems on ARM-based architectures

- Computer Science
- The Journal of Supercomputing
- 2016

This work aims to accelerate the resolution of a WLS problem by reducing the computational cost (relaying on BLAS/LAPACK routines) and the computational precision from double to single and shows that the method that exhibits a high theoretical computational cost overcomes in efficiency other methods with lower theoretical cost in architectures of this type. Expand

RANDOMIZED ALGORITHMS FOR LARGE-SCALE STRONGLY OVER-DETERMINED LINEAR REGRESSION PROBLEMS A DISSERTATION SUBMITTED TO THE INSTITUTE FOR COMPUTATIONAL AND MATHEMATICAL ENGINEERING AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF

- Computer Science
- 2014

This work is most interested in random projection and random sampling algorithms for `2 regression and its robust alternative, `1 regression, with strongly rectangular data and the main result shows that in near input-sparsity time and only a few passes through the data the authors can obtain a good approximate solution, with high probability. Expand

Rain Removal By Image Quasi-Sparsity Priors

- Computer Science
- ArXiv
- 2018

It is shown that the novel rain removal method presented can remove rain from images robustly and outperforms some state-of-the-art rain removal algorithms. Expand

Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments

- Computer Science, Mathematics
- Proceedings of the IEEE
- 2016

Recent work on developing and implementing randomized matrix algorithms in large-scale parallel and distributed environments is reviewed. Expand

#### References

SHOWING 1-10 OF 69 REFERENCES

Iteratively reweighted least squares minimization for sparse recovery

- Mathematics
- 2008

Under certain conditions (known as the restricted isometry property, or RIP) on the mN matrix ˆ (where m<N ), vectors x 2 R N that are sparse (i.e., have most of their entries equal to 0) can be… Expand

A Globally Convergent Method for lp Problems

- Mathematics, Computer Science
- SIAM J. Optim.
- 1993

Numerical experiments indicate that this method is significantly more efficient than the existing iteratively reweighted least-squares method, and it is superlinearly convergent when there is no zero residual at the solution. Expand

Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm

- Mathematics, Computer Science
- IEEE Trans. Signal Process.
- 1997

A view of the algorithm as a novel optimization method which combines desirable characteristics of both classical optimization and learning-based algorithms is provided and Mathematical results on conditions for uniqueness of sparse solutions are also given. Expand

For most large underdetermined systems of linear equations the minimal 1-norm solution is also the sparsest solution

- Mathematics
- 2006

We consider linear equations y = Φx where y is a given vector in ℝn and Φ is a given n × m matrix with n 0 so that for large n and for all Φ's except a negligible fraction, the following property… Expand

A numerical algorithm for complex Chebyshev FIR filter design

- Mathematics
- [Proceedings] 1992 IEEE International Symposium on Circuits and Systems
- 1992

The author presents a multiple exchange algorithm which solves the complex Chebyshev approximation problem by systematically solving a sequence of subproblems. Each subproblem involves optimization… Expand

L1 and L∞ minimization via a variant of Karmarkar's algorithm

- Mathematics, Computer Science
- IEEE Trans. Acoust. Speech Signal Process.
- 1989

Simple iterative algorithms are presented for L/ sub 1/ and L/sub infinity / minimization (regression) based on a variant of Karmarkar's linear programming algorithm based on entirely different theoretical principles to the popular IRLS algorithm. Expand

Iterative reweighted least-squares design of FIR filters

- Mathematics, Computer Science
- IEEE Trans. Signal Process.
- 1994

Develops a new iterative reweighted least squares algorithm for the design of optimal L/sub p/ approximation FIR filters. The algorithm combines a variable p technique with a Newton's method to give… Expand

Approximation theory and methods

- Mathematics
- 1981

Preface 1. The approximation problem and existence of best approximations 2. The uniqueness of best approximations 3. Approximation operators and some approximating functions 4. Polynomial… Expand

Introduction to approximation theory

- Mathematics
- 1966

Introduction: 1 Examples and prospectus 2 Metric spaces 3 Normed linear spaces 4 Inner-product spaces 5 Convexity 6 Existence and unicity of best approximations 7 Convex functions The Tchebycheff… Expand

Globally optimal rational approximation using homotopy continuation methods

- Mathematics, Computer Science
- IEEE Trans. Signal Process.
- 1992

A homotopy function is constructed which guarantees that the globally optimum rational approximation solution may be determined by finding all the solutions of the desired nonlinear problem. Expand