A Nonmonotone Matrix-Free Algorithm for Nonlinear Equality-Constrained Least-Squares Problems

  title={A Nonmonotone Matrix-Free Algorithm for Nonlinear Equality-Constrained Least-Squares Problems},
  author={El Houcine Bergou and Youssef Diouane and Vyacheslav Kungurtsev and Cl{\'e}ment W. Royer},
  journal={SIAM J. Sci. Comput.},
Least squares form one of the most prominent classes of optimization problems, with numerous applications in scientific computing and data fitting. When such formulations aim at modeling complex systems, the optimization process must account for nonlinear dynamics by incorporating constraints. In addition, these systems often incorporate a large number of variables, which increases the difficulty of the problem, and motivates the need for efficient algorithms amenable to large-scale… 

Figures and Tables from this paper



A globally convergent Levenberg–Marquardt method for equality-constrained optimization

This work develops a special globalization of the Levenberg–Marquardt method when it is applied to the Lagrange optimality system, based on linesearch for a smooth exact penalty function of the optimization problem, which in particular involves the objectivefunction of the problem.

Constrained Levenberg-Marquardt Method with Global Complexity Bound

This paper proposes a new rule for updating the damping parameter that is based on the perspective of majorization-minimization method and is the first LM method that has an iteration complexity bound for constrained problems.

A regularization method for constrained nonlinear least squares

The proposed regularization method for nonlinear least-squares problems with equality constraints is similar to applying an SQP method with an exact merit function on a related problem, and the implementation compares favorably to IPOPT in IEEE double precision.


The standard method for solving least squares problems which lead to non-linear normal equations depends upon a reduction of the residuals to linear form by first order Taylor approximations taken

Levenberg-Marquardt Methods Based on Probabilistic Gradient Models and Inexact Subproblem Solution, with Application to Data Assimilation

This paper considers the extension of the classical Levenberg--Marquardt algorithm to the scenarios where the linearized least squares subproblems are solved inexactly and/or the gradient model is noisy and accurate only within a certain probability.

Adaptive Algorithm for Constrained Least-Squares Problems

The results indicate that, for least-squares problems, the approach taken here is a viable alternative to standard general optimization methods such as the Byrd–Omojokun trust-region method and the Powell damped BFGS line search method.

MINRES-QLP: A Krylov Subspace Method for Indefinite or Singular Symmetric Systems

This work derives preconditioned MINRES-QLP, new stopping rules, and better estimates of the solution and residual norms, the matrix norm, and the condition number.

Convergence and Complexity Analysis of a Levenberg–Marquardt Algorithm for Inverse Problems

A novel Levenberg-Marquardt method is introduced that matches, simultaneously, the state of the art in all of these convergence properties with a single seamless algorithm.

A unified local convergence analysis of inexact constrained Levenberg–Marquardt methods

It is shown that the best results known for the unconstrained case also hold for the constrained Levenberg–Marquardt method, and the influence of the regularization parameter on the level of inexactness and the convergence rate is described.

Deterministic non periodic flow

  • J. Atmos. Sci,
  • 1963