Stochastic Forward–Backward Splitting for Monotone Inclusions

@article{Rosasco2016StochasticFS,
  title={Stochastic Forward–Backward Splitting for Monotone Inclusions},
  author={Lorenzo Rosasco and Silvia Villa and Bang C{\^o}ng Vu},
  journal={Journal of Optimization Theory and Applications},
  year={2016},
  volume={169},
  pages={388-406}
}
We propose and analyze the convergence of a novel stochastic algorithm for monotone inclusions that are sum of a maximal monotone operator and a single-valued cocoercive operator. The algorithm we propose is a natural stochastic extension of the classical forward–backward method. We provide a non-asymptotic error analysis in expectation for the strongly monotone case, as well as almost sure convergence under weaker assumptions. For minimization problems, we recover rates matching those obtained… 
View on Springer
dspace.mit.edu
35 Citations
Highly Influential Citations
7
Background Citations
20
Methods Citations
10

35 Citations

A Tseng type stochastic forward-backward algorithm for monotone inclusions

In this paper, we propose a stochastic version of the classical Tseng’s forward-backwardforward method with inertial term for solving monotone inclusions given by the sum of a maximal monotone…

Convergence analysis of the stochastic reflected forward-backward splitting algorithm

We propose and analyze the convergence of a novel stochastic algorithm for solving monotone inclusions that are the sum of a maximal monotone operator and a monotone, Lipschitzian operator. The…

Stochastic Forward Douglas-Rachford Splitting for Monotone Inclusions

We propose a stochastic Forward-Douglas-Rachford Splitting framework for finding a zero point of the sum of three maximally monotone operators in real separable Hilbert space, where one of the…

A First-Order Stochastic Primal-Dual Algorithm with Correction Step

ABSTRACT In this article, we investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive…

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

TLDR
A restricted gap function based on the Fitzpatrick function is defined, which proves that the expected gap of an averaged sequence diminishes at a sublinear rate of 1/k and allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability.

Forward-backward-forward methods with variance reduction for stochastic variational inequalities

TLDR
To the best of the knowledge, this is the first stochastic look-ahead algorithm achieving this by using only a single projection at each iteration, and leads to almost sure convergence to an optimal solution.

Forward–Backward algorithms for stochastic Nash equilibrium seeking in restricted strongly and strictly monotone games

We study stochastic Nash equilibrium problems with expected valued cost functions whose pseudogradient satisfies restricted monotonicity properties which hold only with respect to the solution. We…

Variance-Reduced Splitting Schemes for Monotone Stochastic Generalized Equations

TLDR
Variancereduced stochastic modified forward-backward splitting schemes rely on weaker moment requirements on noise and allow for weakening unbiasedness requirements on oracles in strongly monotone regimes, and show that the variance-reduced schemes outperform stoChastic approximation schemes and sample-average approximation approaches.

Forward–Backward algorithms for stochastic Nash equilibrium seeking in restricted cocoercive and strictly monotone games

We study stochastic Nash equilibrium problems with expected valued cost functions whose pseudogradient satisfies restricted monotonicity properties which hold only with respect to the solution. We…

Inexact Operator Splitting Method for Monotone Inclusion Problems

The Douglas–Peaceman–Rachford–Varga operator splitting methods are a class of efficient methods for finding a zero of the sum of two maximal monotone operators in a real Hilbert space; however, they…

References

SHOWING 1-10 OF 68 REFERENCES

Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators

We investigate the asymptotic behavior of a stochastic version of the forward-backward splitting algorithm for finding a zero of the sum of a maximally monotone set-valued operator and a cocoercive…

A stochastic inertial forward–backward splitting algorithm for multivariate monotone inclusions

TLDR
Almost sure convergence in real Hilbert spaces of the sequence of iterates to an optimal solution is established and two new classes of stochastic inertial primal–dual splitting methods for solving structured systems of composite monotone inclusions are introduced.

Solving monotone inclusions via compositions of nonexpansive averaged operators

A unified fixed point theoretic framework is proposed to investigate the asymptotic behavior of algorithms for finding solutions to monotone inclusion problems. The basic iterative scheme under…

Solving variational inequalities with Stochastic Mirror-Prox algorithm

TLDR
A novel Stochastic Mirror-Prox algorithm is developed for solving s.v.i. variational inequalities with monotone operators and it is shown that with the convenient stepsize strategy it attains the optimal rates of convergence with respect to the problem parameters.

Accelerated and Inexact Forward-Backward Algorithms

We propose a convergence analysis of accelerated forward-backward splitting methods for composite function minimization, when the proximity operator is not available in closed form, and can only be…

Almost sure convergence of stochastic gradient processes with matrix step sizes

Regularized Iterative Stochastic Approximation Methods for Stochastic Variational Inequality Problems

TLDR
This work introduces two classes of stochastic approximation methods, each of which requires exactly one projection step at every iteration, and provides convergence analysis for each of them.

Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework

TLDR
This paper investigates the AC-SA algorithms for solving strongly convex stochastic composite optimization problems in more detail by establishing the large-deviation results associated with the convergence rates and introducing an efficient validation procedure to check the accuracy of the generated solutions.

Variable metric quasi-Fejér monotonicity

On the maximal monotonicity of subdifferential mappings.

The subdifferential of a lower semicontinuous proper convex function on a Banach space is a maximal monotone operator, as well as a maximal cyclically monotone operator. This result was announced by…
...