Corpus ID: 211069143

Soft Threshold Weight Reparameterization for Learnable Sparsity

@article{Kusupati2020SoftTW,
  title={Soft Threshold Weight Reparameterization for Learnable Sparsity},
  author={Aditya Kusupati and V. Ramanujan and Raghav Somani and Mitchell Wortsman and Prateek Jain and S. Kakade and A. Farhadi},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.03231}
}

• Aditya Kusupati, V. Ramanujan, +4 authors A. Farhadi
• Published 2020
• Computer Science, Mathematics
• ArXiv

Sparsity in Deep Neural Networks (DNNs) is studied extensively with the focus of maximizing prediction accuracy given an overall parameter budget. Existing methods rely on uniform or heuristic non-uniform sparsity budgets which have sub-optimal layer-wise parameter allocation resulting in a) lower prediction accuracy or b) higher inference cost (FLOPs). This work proposes Soft Threshold Reparameterization (STR), a novel use of the soft-threshold operator on DNN weights. STR smoothly induces… Expand

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed