Small Batch or Large Batch?: Gaussian Walk with Rebound Can Teach

@inproceedings{Yin2017SmallBO,
  title={Small Batch or Large Batch?: Gaussian Walk with Rebound Can Teach},
  author={Peifeng Yin and Ping Luo and Taiga Nakamura},
  booktitle={KDD},
  year={2017}
}
Efficiency of large-scale learning is a hot topic in both academic and industry. The stochastic gradient descent (SGD) algorithm, and its extension mini-batch SGD, allow the model to be updated without scanning the whole data set. However, the use of approximate gradient leads to the uncertainty issue, slowing down the decreasing of objective function… CONTINUE READING