Corpus ID: 3140413

Neural Speed Reading via Skim-RNN

@article{Seo2018NeuralSR,
  title={Neural Speed Reading via Skim-RNN},
  author={Minjoon Seo and Sewon Min and Ali Farhadi and Hannaneh Hajishirzi},
  journal={ArXiv},
  year={2018},
  volume={abs/1711.02085}
}
  • Minjoon Seo, Sewon Min, +1 author Hannaneh Hajishirzi
  • Published 2018
  • Computer Science
  • ArXiv
  • Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens. Skim-RNN gives computational advantage over an RNN that always updates the entire hidden state. Skim-RNN uses the same input and output interfaces as a standard RNN and can be easily used instead of RNNs in existing models. In our experiments, we show that Skim-RNN can achieve… CONTINUE READING
    Neural Speed Reading with Structural-Jump-LSTM
    • 10
    • Highly Influenced
    • PDF
    Reading selectively via Binary Input Gated Recurrent Unit
    • 2
    • PDF
    Skipping RNN State Updates without Retraining the Original Model
    • 1
    • Highly Influenced
    Speed Reading: Learning to Read ForBackward via Shuttle
    • 7
    • PDF
    Learning to Remember More with Less Memorization
    • 12
    • Highly Influenced
    • PDF
    Learning Longer-term Dependencies in RNNs with Auxiliary Losses
    • 70
    • PDF
    Long Short-Term Memory with Dynamic Skip Connections
    • 6
    • PDF
    Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
    • 2
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 34 REFERENCES
    Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
    • 106
    • PDF
    Variable Computation in Recurrent Neural Networks
    • 31
    • Highly Influential
    • PDF
    Learning to Skim Text
    • 73
    • Highly Influential
    • PDF
    Modeling Human Reading with Neural Attention
    • 28
    • PDF
    Strongly-Typed Recurrent Neural Networks
    • 42
    • PDF
    Learning Longer Memory in Recurrent Neural Networks
    • 184
    • PDF
    Segmental Recurrent Neural Networks
    • 92
    • PDF
    Coarse-to-Fine Question Answering for Long Documents
    • 96
    • PDF
    XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
    • 1,366
    • PDF