Corpus ID: 231740560

Measuring and Improving Consistency in Pretrained Language Models

@article{Elazar2021MeasuringAI,
  title={Measuring and Improving Consistency in Pretrained Language Models},
  author={Yanai Elazar and Nora Kassner and Shauli Ravfogel and Abhilasha Ravichander and E. Hovy and Hinrich Schutze and Y. Goldberg},
  journal={ArXiv},
  year={2021},
  volume={abs/2102.01017}
}

Yanai Elazar, Nora Kassner, +4 authors Y. Goldberg

Published 2021

Computer Science

ArXiv

Consistency of a model — that is, the invariance of its behavior under meaning-preserving alternations in its input — is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge? To this end, we create PARAREL , a high-quality resource of cloze-style query English paraphrases. It contains a total of 328 paraphrases for thirty-eight relations. Using PARAREL , we show that… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed