The truncatedSVD as a method for regularization

@article{Hansen1987TheTA,
  title={The truncatedSVD as a method for regularization},
  author={Per Christian Hansen},
  journal={BIT Numerical Mathematics},
  year={1987},
  volume={27},
  pages={534-553}
}
  • P. Hansen
  • Published 1 October 1987
  • Mathematics
  • BIT Numerical Mathematics
The truncated singular value decomposition (SVD) is considered as a method for regularization of ill-posed linear least squares problems. In particular, the truncated SVD solution is compared with the usual regularized solution. Necessary conditions are defined in which the two methods will yield similar results. This investigation suggests the truncated SVD as a favorable alternative to standard-form regularization in cases of ill-conditioned matrices with well-determined numerical rank. 

Truncated Singular Value Decomposition Solutions to Discrete Ill-Posed Problems with Ill-Determined Numerical Rank

TLDR
The purpose of this paper is to show that the success of both truncated SVD and Tikhonov regularization depends on satisfaction of a discrete Picard condition, involving both the matrix and the right-hand side.

A modified truncation singular value decomposition method for solving ill-posed problems

A modified truncated singular value decomposition method for solving ill-posed problems is presented in this paper, in which the solution has a slightly different form. Both theoretical and numerical

Regularization,GSVD and truncatedGSVD

TLDR
This paper analyzes Tikhonov regularization in general form by means of generalized SVD in the same spirit as SVD is used to analyze standard-form regularization and defines a truncated GSVD solution which sheds light on regularization.

An Improved Tikhonov-Regularized Variable Projection Algorithm for Separable Nonlinear Least Squares

TLDR
An improved Tikhonov regularization method is proposed, which neither discards small singular values, nor treats all singular value corrections, and is more effective at reducing the mean square error of the solution and increasing the accuracy of unknowns.

An algorithm for estimating the optimal regularization parameter by the L-curve

In this paper we introduce a new algorithm to estimate the optimal regularization parameter in truncated singular value decomposition (TSVD) regularization methods for the numerical solution of

Model Selection Criteria for a Linear Model to Solve Discrete Ill-Posed Problems on the Basis of Singular Decomposition and Random Projection

Criteria are developed to determine the optimal number of components of a linear model in solving a discrete ill-posed problem by the methods of truncated singular value decomposition and random

Computing Truncated Singular Value Decomposition Least Squares Solutions by Rank Revealing QR-Factorizations

TLDR
An efficient method is presented for computing the TSVD solution via a QR-factorization, without the need for computing a complete SVD.

Modern regularization methods for inverse problems

TLDR
The aim of this paper is to provide a reasonably comprehensive overview of this shift towards modern nonlinear regularization methods, including their analysis, applications and issues for future research.

Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework

TLDR
This paper proposes a multi- objective framework for ill-posed problems, which can handle complex features of problem such as non-convexity, discontinuity and a case study on signal recovery shows the effectiveness of the multi-objective framework.
...

References

SHOWING 1-10 OF 35 REFERENCES

Algorithms for the regularization of ill-conditioned least squares problems

TLDR
Two regularization methods for ill-conditioned least squares problems are studied from the point of view of numerical efficiency and it is shown that if they are transformed into a certain standard form, very efficient algorithms can be used for their solution.

Perturbation bounds in connection with singular value decomposition

LetA be anm ×n-matrix which is slightly perturbed. In this paper we will derive an estimate of how much the invariant subspaces ofAHA andAAH will then be affected. These bounds have the sin ϑ theorem

On the almost rank deficient case of the least squares problem

This paper studies properties of the solutions to overdetermined systems of linear equations whose matrices are almost rank deficient. Let such a system be approximated by the system of rankr which

On the Numerical Solution of Ill-Conditioned Linear Systems with Applications to Ill-Posed Problems

We consider the solution of ill-conditioned linear systems using the singular value decomposition, and show how this can improve the accuracy of the computed solution for certain kinds of right-hand

Rank degeneracy and least squares problems

This paper is concerned with least squares problems when the least squares matrix A is near a matrix that is not of full rank. A definition of numerical rank is given. It is shown that under certain

A Numerical Method for Solving Fredholm Integral Equations of the First Kind Using Singular Values

The integral equation in question is approximated by simple numerical quadrature formulas plus collocation.Each row of the resulting matrix equation for the unknown function values is weighted by the

A Practical Examination of Some Numerical Methods for Linear Discrete Ill-Posed Problems

Four well-known methods for the numerical solution of linear discrete ill-posed problems are investigated from a common point of view: namely, the type of algebraic expansion generated for the solu...

The generalized eigenstructure problem in linear system theory

TLDR
The numerical aspects of a certain class of such algorithms-dealing with what the author calls generalized eigenstructure problems-are discussed and some new and/or modified algorithms are presented.

The Numerical Solution of a Non-Characteristic Cauchy Probelm for a Parabolic Equation

We study the problem of solving numerically a parabolic partial differential equation in one space dimension, where boundary values are given on one boundary only. For the analysis and numerical