Stochastic Forward–Backward Splitting for Monotone Inclusions

@article{Rosasco2016StochasticFS,
  title={Stochastic Forward–Backward Splitting for Monotone Inclusions},
  author={Lorenzo Rosasco and Silvia Villa and Bang C{\^o}ng Vu},
  journal={Journal of Optimization Theory and Applications},
  year={2016},
  volume={169},
  pages={388-406}
}
We propose and analyze the convergence of a novel stochastic algorithm for monotone inclusions that are sum of a maximal monotone operator and a single-valued cocoercive operator. The algorithm we propose is a natural stochastic extension of the classical forward–backward method. We provide a non-asymptotic error analysis in expectation for the strongly monotone case, as well as almost sure convergence under weaker assumptions. For minimization problems, we recover rates matching those obtained… 
View on Springer
dspace.mit.edu
34 Citations
Highly Influential Citations
7
Background Citations
20
Methods Citations
9

34 Citations

A Tseng type stochastic forward-backward algorithm for monotone inclusions
In this paper, we propose a stochastic version of the classical Tseng’s forward-backwardforward method with inertial term for solving monotone inclusions given by the sum of a maximal monotone…
Convergence analysis of the stochastic reflected forward-backward splitting algorithm
We propose and analyze the convergence of a novel stochastic algorithm for solving monotone inclusions that are the sum of a maximal monotone operator and a monotone, Lipschitzian operator. The…
Stochastic Forward Douglas-Rachford Splitting for Monotone Inclusions
We propose a stochastic Forward-Douglas-Rachford Splitting framework for finding a zero point of the sum of three maximally monotone operators in real separable Hilbert space, where one of the…
A First-Order Stochastic Primal-Dual Algorithm with Correction Step
ABSTRACT In this article, we investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive…
Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces
TLDR
A restricted gap function based on the Fitzpatrick function is defined, which proves that the expected gap of an averaged sequence diminishes at a sublinear rate of 1/k and allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability.
Forward-backward-forward methods with variance reduction for stochastic variational inequalities
TLDR
To the best of the knowledge, this is the first stochastic look-ahead algorithm achieving this by using only a single projection at each iteration, and leads to almost sure convergence to an optimal solution.
Forward–Backward algorithms for stochastic Nash equilibrium seeking in restricted strongly and strictly monotone games
We study stochastic Nash equilibrium problems with expected valued cost functions whose pseudogradient satisfies restricted monotonicity properties which hold only with respect to the solution. We…
Variance-Reduced Splitting Schemes for Monotone Stochastic Generalized Equations
TLDR
Variancereduced stochastic modified forward-backward splitting schemes rely on weaker moment requirements on noise and allow for weakening unbiasedness requirements on oracles in strongly monotone regimes, and show that the variance-reduced schemes outperform stoChastic approximation schemes and sample-average approximation approaches.
Forward–Backward algorithms for stochastic Nash equilibrium seeking in restricted cocoercive and strictly monotone games
We study stochastic Nash equilibrium problems with expected valued cost functions whose pseudogradient satisfies restricted monotonicity properties which hold only with respect to the solution. We…
Inexact Operator Splitting Method for Monotone Inclusion Problems
The Douglas–Peaceman–Rachford–Varga operator splitting methods are a class of efficient methods for finding a zero of the sum of two maximal monotone operators in a real Hilbert space; however, they…
...
...

References

SHOWING 1-10 OF 68 REFERENCES
Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators
We investigate the asymptotic behavior of a stochastic version of the forward-backward splitting algorithm for finding a zero of the sum of a maximally monotone set-valued operator and a cocoercive…
A stochastic inertial forward–backward splitting algorithm for multivariate monotone inclusions
TLDR
Almost sure convergence in real Hilbert spaces of the sequence of iterates to an optimal solution is established and two new classes of stochastic inertial primal–dual splitting methods for solving structured systems of composite monotone inclusions are introduced.
Solving monotone inclusions via compositions of nonexpansive averaged operators
A unified fixed point theoretic framework is proposed to investigate the asymptotic behavior of algorithms for finding solutions to monotone inclusion problems. The basic iterative scheme under…
Solving variational inequalities with Stochastic Mirror-Prox algorithm
TLDR
A novel Stochastic Mirror-Prox algorithm is developed for solving s.v.i. variational inequalities with monotone operators and it is shown that with the convenient stepsize strategy it attains the optimal rates of convergence with respect to the problem parameters.
Accelerated and Inexact Forward-Backward Algorithms
We propose a convergence analysis of accelerated forward-backward splitting methods for composite function minimization, when the proximity operator is not available in closed form, and can only be…
Almost sure convergence of stochastic gradient processes with matrix step sizes
Regularized Iterative Stochastic Approximation Methods for Stochastic Variational Inequality Problems
TLDR
This work introduces two classes of stochastic approximation methods, each of which requires exactly one projection step at every iteration, and provides convergence analysis for each of them.
Variable metric quasi-Fejér monotonicity
On the maximal monotonicity of subdifferential mappings.
The subdifferential of a lower semicontinuous proper convex function on a Banach space is a maximal monotone operator, as well as a maximal cyclically monotone operator. This result was announced by…
Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping II: mean-square and linear convergence
TLDR
Results on the mean-square and linear convergence of the iterates of the block-coordinate fixed point algorithms are established and applications to monotone operator splitting and proximal optimization algorithms are presented.
...
...