• Computer Science
  • Published in ArXiv 2019

How Can We Know What Language Models Know?

@article{Jiang2019HowCW,
  title={How Can We Know What Language Models Know?},
  author={Zhengbao Jiang and Frank F. Xu and Jun Araki and Graham Neubig},
  journal={ArXiv},
  year={2019},
  volume={abs/1911.12543}
}
Recent work has presented intriguing results examining the knowledge contained in language models (LM) by having the LM fill in the blanks of prompts such as “Obama is a by profession”. These prompts are usually manually created, and quite possibly sub-optimal; another prompt such as “Obama worked as a ” may result in more accurately predicting the correct profession. Because of this, given an inappropriate prompt, we might fail to retrieve facts that the LM does know, and thus any given prompt… CONTINUE READING

Citations

Publications citing this paper.

References

Publications referenced by this paper.