• DOI:10.18653/v1/2020.acl-main.593
Corpus ID: 215785895

The Right Tool for the Job: Matching Model and Instance Complexities

@inproceedings{Schwartz2020TheRT,
  title={The Right Tool for the Job: Matching Model and Instance Complexities},
  author={Roy Schwartz and Gabriel Stanovsky and Swabha Swayamdipta and Jesse Dodge and Noah A. Smith},
  booktitle={ACL},
  year={2020}
}

• Roy Schwartz, Gabriel Stanovsky, +2 authors Noah A. Smith
• Published in ACL 2020
• Computer Science

As NLP models become larger, executing a trained model requires significant computational resources incurring monetary and environmental costs. To better respect a given inference budget, we propose a modification to contextual representation fine-tuning which, during inference, allows for an early (and fast) "exit" from neural network calculations for simple instances, and late (and accurate) exit for hard instances. To achieve this, we add classifiers to different layers of BERT and use their… Expand