@article{Na2021AnAS,
author={Sen Na and Mihai Anitescu and Mladen Kolar},
journal={ArXiv},
year={2021},
volume={abs/2102.05320}
}
• Published 10 February 2021
• Computer Science
• ArXiv
We consider solving nonlinear optimization problems with stochastic objective and deterministic equality constraints. We assume for the objective that its evaluation, gradient, and Hessian are inaccessible, while one can compute their stochastic estimates by, for example, subsampling. We propose a stochastic algorithm based on sequential quadratic programming (SQP) that uses a differentiable exact augmented Lagrangian as the merit function. To motivate our algorithm design, we first revisit and…
• Mathematics
• 2022
We apply a stochastic sequential quadratic programming (StoSQP) algorithm to solve constrained nonlinear optimization problems, where the objective is stochastic and the constraints are in equality
• Computer Science
Mathematical Programming
• 2023
An active-set stochastic sequential quadratic programming (StoSQP) algorithm that utilizes a differentiable exact augmented Lagrangian as the merit function that allows nonlinear inequality constraints without requiring the strict complementary condition.
• Computer Science, Mathematics
Mathematical Programming
• 2022
There exists a universal weighted averaging scheme that transitions to local convergence at an optimal stage, and still exhibits a superlinear convergence rate nearly (up to a logarithmic factor) matching that of uniform Hessian averaging.
• Mathematics
• 2023
In this paper, a robust sequential quadratic programming method of [1] for constrained optimization is generalized to problem with stochastic objective function, deterministic equality and inequality
• Computer Science, Mathematics
• 2021
An algorithm that allows inexact subproblem solutions to be employed, which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize is proposed.
• Computer Science
• 2021
The proposed sequential quadratic optimization algorithm both allows the use of stochastic objective gradient estimates and possesses convergence guarantees even in the setting in which the constraint Jacobians may be rank deficient.
• Computer Science
• 2021
We propose a fast temporal decomposition procedure for solving long-horizon nonlinear dynamic programs. The core of the procedure is sequential quadratic programming (SQP), with a differentiable
• Computer Science
• 2022
A practical adaptive inexact stochastic sequential quadratic programming (PAIS-SQP) method is described and criteria for controlling the sample size and the accuracy in the solutions of the SQP subproblems based on the variance estimates obtained as the optimization progresses is proposed.
• Computer Science
• 2023
A sequential quadratic optimization algorithm for minimizing an objective function defined by an expectation subject to nonlinear inequality and equality constraints is proposed, analyzed, and tested and is proved to possess convergence guarantees in expectation.
• Computer Science, Mathematics
• 2022
The global almost sure convergence guarantee for TR-StoSQP is established, and its empirical performance on both a subset of problems in the CUTEst test set and constrained logistic regression problems using data from the LIBSVM collection is illustrated.

References

SHOWING 1-10 OF 141 REFERENCES

• Computer Science
SIAM J. Optim.
• 2021
Under reasonable assumptions, convergence (resp.,~convergence in expectation) from remote starting points is proved for the proposed deterministic (resp,~stochastic) algorithm.
• Computer Science, Mathematics
SIAM J. Optim.
• 2017
The proposed framework extends the classic quasi-Newton methods working in deterministic settings to stochastic settings, and it is proved its almost sure convergence to stationary points.
• Computer Science
SIAM J. Optim.
• 2002
An SQP algorithm that uses a smooth augmented Lagrangian merit function and makes explicit provision for infeasibility in the original problem and the QP subproblems is discussed and a reduced-Hessian semidefinite QP solver (SQOPT) is discussed.
• Computer Science, Mathematics
SIAM J. Optim.
• 2009
It is intended to demonstrate that a properly modified SA approach can be competitive and even significantly outperform the SAA method for a certain class of convex stochastic problems.
• Mathematics, Computer Science
SIAM J. Optim.
• 2018
It is shown that the inner product test improves upon the well known norm test, and can be used as a basis for an algorithm that is globally convergent on nonconvex functions and enjoys a global linear rate of convergence on strongly convex functions.
• Computer Science
NeurIPS
• 2019
This work proposes to use line-search techniques to automatically set the step-size when training models that can interpolate the data, and proves that SGD with a stochastic variant of the classic Armijo line- search attains the deterministic convergence rates for both convex and strongly-convex functions.
• Computer Science, Mathematics
NeurIPS
• 2018
The proposed algorithm efficiently escapes saddle points and finds approximate local minima for general smooth, nonconvex functions in only $\mathcal{\tilde{O}}(\epsilon^{-3.5})$ stochastic gradient and stochastically Hessian-vector product evaluations.
• Computer Science, Mathematics
SIAM J. Optim.
• 2014
An approach for nonlinear programming based on the direct minimization of an exact differentiable penalty function using trust-region Newton techniques that provides desirable features required for scalability and presents features that are desirable for parametric optimization problems that must be solved in a latency-limited environment.
In this thesis first and second order algorithms are proposed for solving equality constrained finite dimensional minimization problems and optimal control problems with terminal equality constraints using the exact penalty function approach.
• Computer Science, Mathematics
AISTATS
• 2019
This paper theoretically analyzes in the convex and non-convex settings a generalized version of the AdaGrad stepsizes, and shows sufficient conditions for these stepsizes to achieve almost sure asymptotic convergence of the gradients to zero, proving the first guarantee for generalized AdaGrad Stepsizes in the non- Convex setting.