Corpus ID: 233296494

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

@article{Lu2021FantasticallyOP,
  title={Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity},
  author={Yao Lu and Max Bartolo and A. Moore and S. Riedel and Pontus Stenetorp},
  journal={ArXiv},
  year={2021},
  volume={abs/2104.08786}
}
When primed with only a handful of training samples, very large pretrained language models such as GPT-3, have shown competitive results when compared to fully-supervised fine-tuned large pretrained language models. We demonstrate that the order in which the samples are provided can be the difference between near state-of-the-art and random guess performance: Essentially some permutations are “fantastic” and some not. We analyse this phenomenon in detail, establishing that: it is present across… Expand

Figures and Tables from this paper

FLEX: Unifying Evaluation for Few-Shot NLP
True Few-Shot Learning with Language Models
What's in a Measurement? Using GPT-3 on SemEval 2021 Task 8 - MeasEval

References

SHOWING 1-10 OF 28 REFERENCES
Language Models are Unsupervised Multitask Learners
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
How Context Affects Language Models' Factual Predictions
A Deep Reinforced Model for Abstractive Summarization
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Language Models as Knowledge Bases?
Deep contextualized word representations
...
1
2
3
...