Learning Longer-term Dependencies in RNNs with Auxiliary Losses

Abstract

Despite recent advances in training recurrent neural networks (RNNs), capturing long-term dependencies in sequences remains a fundamental challenge. Most approaches use backpropagation through time (BPTT), which is difficult to scale to very long sequences. This paper proposes a simple method that improves the ability to capture long term dependencies in… (More)

Topics

11 Figures and Tables

Slides referencing similar topics