RNN Architecture Learning with Sparse Regularization

@article{Dodge2019RNNAL,
  title={RNN Architecture Learning with Sparse Regularization},
  author={Jesse Dodge and Roy Schwartz and Hao Peng and Noah A. Smith},
  journal={ArXiv},
  year={2019},
  volume={abs/1909.03011}
}
Neural models for NLP typically use large numbers of parameters to reach state-of-the-art performance, which can lead to excessive memory usage and increased runtime. We present a structure learning method for learning sparse, parameter-efficient NLP models. Our method applies group lasso to rational RNNs (Peng et al., 2018), a family of models that is closely connected to weighted finite-state automata (WFSAs). We take advantage of rational RNNs' natural grouping of the weights, so the group… CONTINUE READING

References

Publications referenced by this paper.
SHOWING 1-10 OF 36 REFERENCES