A Lightweight Recurrent Network for Sequence Modeling

@inproceedings{Zhang2019ALR,
  title={A Lightweight Recurrent Network for Sequence Modeling},
  author={Biao Zhang and Rico Sennrich},
  booktitle={ACL},
  year={2019}
}
Recurrent networks have achieved great success on various sequential tasks with the assistance of complex recurrent units, but suffer from severe computational inefficiency due to weak parallelization. One direction to alleviate this issue is to shift heavy computations outside the recurrence. In this paper, we propose a lightweight recurrent network, or LRN. LRN uses input and forget gates to handle long-range dependencies as well as gradient vanishing and explosion, with all parameter related… CONTINUE READING
4
Twitter Mentions

References

Publications referenced by this paper.
SHOWING 1-10 OF 42 REFERENCES

Deep contextualized word representations

VIEW 5 EXCERPTS
HIGHLY INFLUENTIAL

Neural Machine Translation with Deep Attention.

  • IEEE transactions on pattern analysis and machine intelligence
  • 2018
VIEW 5 EXCERPTS
HIGHLY INFLUENTIAL