# Hybrid Conditional Gradient - Smoothing Algorithms with Applications to Sparse and Low Rank Regularization

@article{Argyriou2014HybridCG, title={Hybrid Conditional Gradient - Smoothing Algorithms with Applications to Sparse and Low Rank Regularization}, author={Andreas Argyriou and Marco Signoretto and Johan A. K. Suykens}, journal={ArXiv}, year={2014}, volume={abs/1404.3591} }

We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cases with multiple nonsmooth terms, in which standard conditional gradient methods may be difficult to apply. The HCGS algorithm borrows techniques from smoothing proximal methods…

## 18 Citations

Model Based Conditional Gradient Method with Armijo-like Line Search

- 2019

The Conditional Gradient Method is generalized to a class of non-smooth non-convex optimization problems with many applications in machine learning. The proposed algorithm iterates by minimizing…

Model Function Based Conditional Gradient Method with Armijo-like Line Search

- Computer Science, MathematicsICML
- 2019

The proposed algorithm iterates by minimizing so-called model functions over the constraint set and is shown to perform favorably on a sparse non-linear robust regression problem and the flexibility of the proposed framework in several matrix factorization formulations is discussed.

Fast column generation for atomic norm regularization

- Computer ScienceAISTATS 2017
- 2017

The fully corrective Frank-Wolfe (FCFW) algorithm can be made particularly efficient for difficult problems in this family by solving the simplicial or conical subproblems produced by FCFW using a special instance of a classical active set algorithm for quadratic programming.

Solving Separable Nonsmooth Problems Using Frank-Wolfe with Uniform Affine Approximations

- Mathematics, Computer ScienceIJCAI
- 2018

This paper proposes a modified FW algorithm amenable to nonsmooth functions, subject to a separability assumption, by optimizing for approximation quality over all affine functions, given a neighborhood of interest.

New results on subgradient methods for strongly convex optimization problems with a unified analysis

- Mathematics, Computer ScienceComput. Optim. Appl.
- 2016

A unifying framework for subgradient methods which yields two kinds of methods, namely, the proximal gradient method (PGM) and the conditional gradient method(CGM), unifying several existing methods is proposed.

Decomposition Techniques for Bilinear Saddle Point Problems and Variational Inequalities with Affine Monotone Operators

- Mathematics, Computer ScienceJ. Optim. Theory Appl.
- 2017

The goal of this paper was to develop alternative decomposition techniques based on LMO for bilinear saddle point problems and for variational inequalities with affine monotone operators.

Adaptive smoothing algorithms for nonsmooth composite convex minimization

- Mathematics, Computer ScienceComput. Optim. Appl.
- 2017

This work develops a new adaptive smoothing algorithm that has the O1ε-worst-case iteration-complexity while preserves the same complexity-per-iteration as in Nesterov’s method and allows one to automatically update the smoothness parameter at each iteration.

Low-Rank Spectral Optimization via Gauge Duality

- Computer Science, MathematicsSIAM J. Sci. Comput.
- 2016

An algorithm is described that is based on solving a certain constrained eigenvalue optimization problem that corresponds to the gauge dual which, unlike the more typical Lagrange dual, has an especially simple constraint.

A Deterministic Nonsmooth Frank Wolfe Algorithm with Coreset Guarantees

- Mathematics, Computer ScienceINFORMS Journal on Optimization
- 2019

A new Frank-Wolfe (FW) type algorithm that is applicable to minimization problems with a nonsmooth convex objective is presented and it is shown that the scheme yields so-called coreset results for various Machine Learning problems including 1-median, Balanced Development, Sparse PCA, Graph Cuts, and the SVM.

Faster Projection-free Online Learning

- Computer Science, MathematicsCOLT
- 2020

This paper gives an efficient projection-free algorithm that guarantees regret for general online convex optimization with smooth cost functions and one linear optimization computation per iteration.

## References

SHOWING 1-10 OF 61 REFERENCES

Mirror Prox algorithm for multi-term composite minimization and semi-separable problems

- Mathematics, Computer ScienceComput. Optim. Appl.
- 2015

In the paper, we develop a composite version of Mirror Prox algorithm for solving convex–concave saddle point problems and monotone variational inequalities of special structure, allowing to cover…

Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization

- Mathematics, Computer ScienceICML
- 2013

A new general framework for convex optimization over matrix factorizations, where every Frank-Wolfe iteration will consist of a low-rank update, is presented, and the broad application areas of this approach are discussed.

PRISMA: PRoximal Iterative SMoothing Algorithm

- Mathematics, Computer ScienceArXiv
- 2012

A time variant smoothing strategy is used that allows us to obtain a guarantee that does not depend on knowing in advance the total number of iterations nor a bound on the domain.

Mirror Prox Algorithm for Multi-Term Composite Minimization and Alternating Directions

- Computer Science, MathematicsArXiv
- 2013

In the paper, we develop a composite version of Mirror Prox algorithm for solving convex-concave saddle point problems and monotone variational inequalities of special structure, allowing to cover…

Convex Optimization without Projection Steps

- Computer Science, MathematicsArXiv
- 2011

This framework generalizes the sparse greedy algorithm of Frank & Wolfe and its primal-dual analysis by Clarkson 2010 (and the low-rank SDP approach by Hazan 2008) to arbitrary convex domains and gives a convergence proof guaranteeing {\epsilon}-small duality gap after O(1/{\ep silon}) iterations.

A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

- Mathematics, Computer ScienceSIAM J. Imaging Sci.
- 2009

A new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically.

Duality Between Subgradient and Conditional Gradient Methods

- Computer Science, MathematicsSIAM J. Optim.
- 2015

This paper shows the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method through convex duality, and implies that for certain problems, such as for supervised machine learning problems with non-smooth losses or problems regularized by non-Smooth regularizers, the primal subgradient method and the dual conditionalgradient method are formally equivalent.

Greedy Algorithms for Structurally Constrained High Dimensional Problems

- Computer Science, MathematicsNIPS
- 2011

This framework not only unifies existing greedy algorithms by recovering them as special cases but also yields novel ones that solve convex optimization problems that arise when dealing with structurally constrained high-dimensional problems.

The Complexity of Large-scale Convex Programming under a Linear Optimization Oracle

- Mathematics
- 2013

This paper considers a general class of iterative optimization algorithms, referred to as linear-optimization-based convex programming (LCP) methods, for solving large-scale convex programming (CP)…

Smoothing and First Order Methods: A Unified Framework

- Computer Science, MathematicsSIAM J. Optim.
- 2012

It is proved that independently of the structure of the convex nonsmooth function involved, and of the given fast first order iterative scheme, it is always possible to improve the complexity rate and reach an O(\varepsilon^{-1}) efficiency estimate by solving an adequately smoothed approximation counterpart.