Corpus ID: 44117287

Stochastic Zeroth-order Optimization via Variance Reduction method

@article{Liu2018StochasticZO,
  title={Stochastic Zeroth-order Optimization via Variance Reduction method},
  author={L. Liu and Minhao Cheng and C. Hsieh and D. Tao},
  journal={ArXiv},
  year={2018},
  volume={abs/1805.11811}
}
  • L. Liu, Minhao Cheng, +1 author D. Tao
  • Published 2018
  • Computer Science, Mathematics
  • ArXiv
  • Derivative-free optimization has become an important technique used in machine learning for optimizing black-box models. To conduct updates without explicitly computing gradient, most current approaches iteratively sample a random search direction from Gaussian distribution and compute the estimated gradient along that direction. However, due to the variance in the search direction, the convergence rates and query complexities of existing methods suffer from a factor of $d$, where $d$ is the… CONTINUE READING
    11 Citations
    Nonconvex Zeroth-Order Stochastic ADMM Methods with Lower Function Query Complexity
    • 5
    • PDF
    Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization
    • 7
    • Highly Influenced
    • PDF
    ZEROTH-ORDER STOCHASTIC PROJECTED GRADIENT DESCENT FOR NONCONVEX OPTIMIZATION
    • 4
    • PDF
    Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization
    • 17
    • Highly Influenced
    • PDF
    Learning to Learn by Zeroth-Order Oracle
    • 3
    • PDF
    VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning
    • 30
    • PDF
    ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
    • 13
    • PDF
    An Accelerated DFO Algorithm for Finite-sum Convex Functions
    • 1
    • PDF
    Stochastically Controlled Stochastic Gradient for the Convex and Non-convex Composition problem
    • 4
    • PDF

    References

    SHOWING 1-10 OF 31 REFERENCES
    Zeroth-order Asynchronous Doubly Stochastic Algorithm with Variance Reduction
    • 12
    • PDF
    Random Gradient-Free Minimization of Convex Functions
    • 375
    • Highly Influential
    • PDF
    Finite Sample Convergence Rates of Zero-Order Stochastic Optimization Methods
    • 16
    • PDF
    Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations
    • 187
    • PDF
    Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
    • 592
    • Highly Influential
    • PDF
    Stochastic Zeroth-order Optimization in High Dimensions
    • 37
    • PDF
    Non-convex Finite-Sum Optimization Via SCSG Methods
    • 135
    • Highly Influential
    • PDF
    Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
    • 1,576
    • Highly Influential
    • PDF
    Stochastic Variance Reduction for Nonconvex Optimization
    • 349
    • Highly Influential
    • PDF