• Corpus ID: 14336572

Gradient methods for minimizing composite objective function

@article{Nesterov2007GradientMF,
  title={Gradient methods for minimizing composite objective function},
  author={Yurii Nesterov},
  journal={Research Papers in Economics},
  year={2007}
}
  • Y. Nesterov
  • Published 1 September 2007
  • Computer Science, Mathematics
  • Research Papers in Economics
In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two convex terms: one is smooth and given by a black-box oracle, and another is general but simple and its structure is known. Despite to the bad properties of the sum, such problems, both in convex and nonconvex cases, can be solved with eciency typical for the good part of the objective. For convex problems of the above structure, we consider primal and dual variants… 
Gradient methods for minimizing composite functions
  • Y. Nesterov
  • Mathematics, Computer Science
    Math. Program.
  • 2013
In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two terms: one is smooth and given by a black-box oracle, and another is
An adaptive accelerated proximal gradient method and its homotopy continuation for sparse optimization
TLDR
An accelerated proximal gradient method is presented for problems where the smooth part of the objective function is also strongly convex, and this method incorporates an efficient line-search procedure, and achieves the optimal iteration complexity for such composite optimization problems.
A family of subgradient-based methods for convex optimization problems in a unifying framework
TLDR
A new family of subgradient- and gradient-based methods which converges with optimal complexity for convex optimization problems whose feasible region is simple enough, and which can partially become special cases of other universal methods, too.
Fast Multiple-Splitting Algorithms for Convex Optimization
TLDR
It is proved that the number of iterations needed by the first class of algorithms to obtain an $\epsilon$-optimal solution is $O((K-1)L/\ep silon)$, where $L$ is an upper bound on all of the Lipschitz constants.
Conditional gradient type methods for composite nonlinear and stochastic optimization
TLDR
This paper presents a unified analysis for the CGT method in the sense that it achieves the best known rate of convergence when the weakly smooth term is nonconvex and possesses (nearly) optimal complexity if it turns out to be convex.
Optimal subgradient methods: computational properties for large-scale linear inverse problems
TLDR
The computational properties of the optimal subgradient algorithm (OSGA) for applications of linear inverse problems involving high-dimensional data and several Nesterov-type optimal methods adapted to solve nonsmooth problems by simply passing a subgradient instead of the gradient are studied.
MOCCA: Mirrored Convex/Concave Optimization for Nonconvex Composite Functions
TLDR
The MOCCA (mirrored convex/concave) algorithm is proposed, a primal/dual optimization approach that takes a local convex approximation to each term at every iteration, and offers theoretical guarantees for convergence when the overall problem is approximately convex.
An optimal method for stochastic composite optimization
TLDR
The accelerated stochastic approximation (AC-SA) algorithm based on Nesterov’s optimal method for smooth CP is introduced, and it is shown that the AC-SA algorithm can achieve the aforementioned lower bound on the rate of convergence for SCO.
Efficient random coordinate descent algorithms for large-scale structured nonconvex optimization
In this paper we analyze several new methods for solving nonconvex optimization problems with the objective function consisting of a sum of two terms: one is nonconvex and smooth, and another is
PPA-Like Contraction Methods for Convex Optimization: A Framework Using Variational Inequality Approach
  • B. He
  • Computer Science
  • 2015
TLDR
This paper gives a review on the recently developed customized PPA in H-norm (H is a positive definite matrix), and uses the variational inequality approach to construct the contraction-type methods for convex optimization with different linear constraints.
...
...

References

SHOWING 1-10 OF 29 REFERENCES
Accelerating the cubic regularization of Newton’s method on convex problems
TLDR
An accelerated version of the cubic regularization of Newton’s method that converges for the same problem class with order, keeping the complexity of each iteration unchanged and arguing that for the second-order schemes, the class of non-degenerate problems is different from the standard class.
Rounding of convex sets and efficient gradient methods for linear programming problems
  • Y. Nesterov
  • Mathematics, Computer Science
    Optim. Methods Softw.
  • 2008
TLDR
It is proved that the upper complexity bound for both schemes is O((√(n ln m)/δ)ln n) iterations of a gradient-type method, where n and m are the sizes of the corresponding linear programming problems.
Introductory Lectures on Convex Optimization - A Basic Course
TLDR
It was in the middle of the 1980s, when the seminal paper by Kar markar opened a new epoch in nonlinear optimization, and it became more and more common that the new methods were provided with a complexity analysis, which was considered a better justification of their efficiency than computational experiments.
Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems
TLDR
This paper proposes gradient projection algorithms for the bound-constrained quadratic programming (BCQP) formulation of these problems and test variants of this approach that select the line search parameters in different ways, including techniques based on the Barzilai-Borwein method.
Just relax: convex programming methods for identifying sparse signals in noise
  • J. Tropp
  • Computer Science
    IEEE Transactions on Information Theory
  • 2006
TLDR
A method called convex relaxation, which attempts to recover the ideal sparse signal by solving a convex program, which can be completed in polynomial time with standard scientific software.
Smooth minimization of non-smooth functions
TLDR
A new approach for constructing efficient schemes for non-smooth convex optimization is proposed, based on a special smoothing technique, which can be applied to functions with explicit max-structure, and can be considered as an alternative to black-box minimization.
Iterative solution of nonlinear equations in several variables
TLDR
Convergence of Minimization Methods An Annotated List of Basic Reference Books Bibliography Author Index Subject Index.
Regression Shrinkage and Selection via the Lasso
TLDR
A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Atomic Decomposition by Basis Pursuit
TLDR
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
Deconvolution with the l 1 norm
Given a wavelet w and a noisy trace t = s * w + n, an approximation s of the spike train s can be obtained using the l 1 norm. This extraction has the advantage of preserving isolated spikes in s. On
...
...