Recurrent neural network training with preconditioned stochastic gradient descent

Abstract

Recurrent neural networks (RNN), especially the ones requiring extremely long term memories, are difficult to training. Hence, they provide an ideal testbed for benchmarking the performance of optimization algorithms. This paper reports test results of a recently proposed preconditioned stochastic gradient descent (PSGD) algorithm on RNN training. We find… (More)

Topics

3 Figures and Tables

Slides referencing similar topics