• DOI:10.18653/v1/2020.emnlp-main.499
Corpus ID: 222290473

Adversarial Self-Supervised Data-Free Distillation for Text Classification

@inproceedings{Ma2020AdversarialSD,
  title={Adversarial Self-Supervised Data-Free Distillation for Text Classification},
  author={Xinyin Ma and Yongliang Shen and Gongfan Fang and Chen Chen and Chenghao Jia and Weiming Lu},
  booktitle={EMNLP},
  year={2020}
}

• Xinyin Ma, Yongliang Shen, +3 authors Weiming Lu
• Published in EMNLP 2020
• Computer Science

Large pre-trained transformer-based language models have achieved impressive results on a wide range of NLP tasks. In the past few years, Knowledge Distillation(KD) has become a popular paradigm to compress a computationally expensive model to a resource-efficient lightweight model. However, most KD algorithms, especially in NLP, rely on the accessibility of the original training dataset, which may be unavailable due to privacy issues. To tackle this problem, we propose a novel two-stage data… Expand

View on ACL

arxiv.org

Save to Library

Create Alert

Cite

Launch Research Feed