• DOI:10.48550/arXiv.2204.12632
• Corpus ID: 248406097

Testing the Ability of Language Models to Interpret Figurative Language

@inproceedings{Liu2022TestingTA,
  title={Testing the Ability of Language Models to Interpret Figurative Language},
  author={Emmy Liu and Chenxuan Cui and Kenneth Zheng and Graham Neubig},
  booktitle={North American Chapter of the Association for Computational Linguistics},
  year={2022}
}

• Emmy Liu, Chenxuan Cui, +1 author Graham Neubig
• Published in
  North American Chapter of the…
  26 April 2022
• Linguistics

Figurative and metaphorical language are commonplace in discourse, and figurative expressions play an important role in communication and cognition. However, figurative language has been a relatively under-studied area in NLP, and it remains an open question to what extent modern language models can interpret nonliteral phrases. To address this question, we introduce Fig-QA, a Winograd-style nonliteral language understanding task consisting of correctly interpreting paired figurative phrases with divergent meanings. We evaluate the performance of several state-of-the-art language models on this task, and find that although language models achieve performance significantly over chance, they still fall short of human performance, particularly in zero- or few-shot settings. This suggests that further work is needed to improve the nonliteral reasoning capabilities of language models.