Corpus ID: 231979430

Calibrate Before Use: Improving Few-Shot Performance of Language Models

@article{Zhao2021CalibrateBU,
  title={Calibrate Before Use: Improving Few-Shot Performance of Language Models},
  author={Tony Zhao and Eric Wallace and Shi Feng and D. Klein and S. Singh},
  journal={ArXiv},
  year={2021},
  volume={abs/2102.09690}
}
GPT-3 can perform numerous tasks when provided a natural language prompt that contains a few training examples. We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art. We demonstrate that this instability arises from the bias of language models towards predicting certain answers, e.g., those that are placed near the end of the… Expand
6 Citations
AdaPrompt: Adaptive Prompt-based Finetuning for Relation Extraction
  • Xiang Chen, Xin Xie, +6 authors Huajun Chen
  • Computer Science
  • ArXiv
  • 2021
  • PDF
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
  • Highly Influenced
  • PDF
Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
  • Highly Influenced
  • PDF
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
  • Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, Woomyeong Park
  • Computer Science
  • 2021
  • Highly Influenced
  • PDF
GPT Understands, Too
  • Xiao Liu, Yanan Zheng, +4 authors Jie Tang
  • Computer Science
  • ArXiv
  • 2021
  • 4
  • PDF
Natural Instructions: Benchmarking Generalization to New Tasks from Natural Language Instructions
  • PDF

References

SHOWING 1-10 OF 36 REFERENCES
Making Pre-trained Language Models Better Few-shot Learners
  • 13
  • Highly Influential
  • PDF
Language Models are Few-Shot Learners
  • 900
  • PDF
Language Models are Unsupervised Multitask Learners
  • 2,720
  • Highly Influential
  • PDF
Improving Language Understanding by Generative Pre-Training
  • 1,938
  • PDF
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
  • 29
  • PDF
Learning and Evaluating General Linguistic Intelligence
  • 89
  • PDF
A Deep Reinforced Model for Abstractive Summarization
  • 778
  • PDF
...
1
2
3
4
...