Testing the Ability of Language Models to Interpret Figurative Language

  title={Testing the Ability of Language Models to Interpret Figurative Language},
  author={Emmy Liu and Chenxuan Cui and Kenneth Zheng and Graham Neubig},
  booktitle={North American Chapter of the Association for Computational Linguistics},
Figurative and metaphorical language are commonplace in discourse, and figurative expressions play an important role in communication and cognition. However, figurative language has been a relatively under-studied area in NLP, and it remains an open question to what extent modern language models can interpret nonliteral phrases. To address this question, we introduce Fig-QA, a Winograd-style nonliteral language understanding task consisting of correctly interpreting paired figurative phrases… 

FLUTE: Figurative Language Understanding and Textual Explanations

FLUTE is released, a dataset of 8,000 figurative NLI instances with explanations, spanning three cat-egories: Sarcasm, Simile, and Metaphor, and it is shown how uti-lizing GPT-3 in conjunction with human experts can aid in scaling up the creation of datasets even for such complex linguistic phe-nomena as flgurative language.

MiQA: A Benchmark for Inference on Metaphorical Questions

We propose a benchmark to assess the capability of large language models to reason with conventional metaphors. Our benchmark combines the previously isolated topics of metaphor detection and

FigurativeQA: A Test Benchmark for Figurativeness Comprehension for Question Answering

It is demonstrated that a state-of-the-art RoBERTa QA model has considerably lower performance in question answering when the contexts are figurative rather than literal, indicating a gap in current models.

FLUTE: Figurative Language Understanding through Textual Explanations

FLUTE is released, a dataset of 9,000 figurative NLI instances with explanations, spanning four categories: Sarcasm, Simile, Metaphor, and Idioms, which shows how utilizing GPT-3 in conjunction with human annotators (novices and experts) can aid in scaling up the creation of datasets even for such complex linguistic phenomena as figurative language.

Leveraging a New Spanish Corpus for Multilingual and Cross-lingual Metaphor Detection

This work presents the first corpus annotated with naturally occurring metaphors in Spanish large enough to develop systems to perform metaphor detection, and presents the, to the best of the authors' knowledge, first cross-lingual experiments on supervised metaphor detection.

A Survey in Automatic Irony Processing: Linguistic, Cognitive, and Multi-X Perspectives

This paper will provide a comprehensive overview of computational irony, insights from linguisic theory and cognitive science, as well as its interactions with downstream NLP tasks and newly proposed multi-X irony processing perspectives.

Detecting Euphemisms with Literal Descriptions and Visual Imagery

This paper describes a two-stage system for the Euphemism Detection shared task hosted by the 3rd Workshop on Figurative Language Processing in conjunction with EMNLP 2022, and integrates visual supervision into the system using visual imageries.

Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5

It is shown that simple sequential fine-tuning of text-to-text models is an extraordinarily powerful method of achieving cross-task knowledge transfer while simultaneously predicting multiple interdependent targets.

Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Review

The importance of NLG being guided by knowledge, in order to convey human-like reasoning through language generation, is explored, and ten goals for intelligent NLG systems to pursue are proposed.

MAPS-KB: A Million-scale Probabilistic Simile Knowledge Base

This work constructs MAPS-KB, a million-scale probabilistic simile knowledge base, covering 4.3 million triplets over 0.4 million terms from 70 GB corpora, and proposes a novel framework for large-scale simileknowledge base construction, as well as two Probabilistic metrics which enable an improved under- standing of simile phenomena in natural language.



It’s not Rocket Science: Interpreting Figurative Language in Narratives

This paper studies the interpretation of two non-compositional figurative languages (idioms and similes), and proposes knowledge-enhanced models, adopting human strategies for interpreting figurative language types: inferring meaning from the context and relying on the constituent words’ literal meanings.

Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages

The authors' extensive experiments suggest that contextual representations in PLMs do encode metaphorical knowledge, and mostly in their middle layers, and the knowledge is transferable between languages and datasets, especially when the annotation is consistent across training and testing sets.

Investigating Robustness of Dialog Models to Popular Figurative Language Constructs

This work analyzes the performance of existing dialog models in situations where the input dialog context exhibits use of figurative language, and proposes lightweight solutions to help existing models become more robust to figurativelanguage.

Metaphor Detection with Cross-Lingual Model Transfer

We show that it is possible to reliably discriminate whether a syntactic construction is meant literally or metaphorically using lexical semantic features of the words that participate in the

Figurative Language in Recognizing Textual Entailment

A collection of recognizing textual entailment (RTE) datasets focused on figurative language is introduced, indicating that state-of-the-art models trained on popular RTE datasets might not sufficiently capture figurativelanguage.

A howling success or a working sea? Testing what BERT knows about metaphors

Metaphor is a widespread linguistic and cognitive phenomenon that is ruled by mechanisms which have received attention in the literature. Transformer Language Models such as BERT have brought

Word Embedding and WordNet Based Metaphor Identification and Interpretation

An unsupervised learning method that identifies and interprets metaphors at word-level without any preprocessing, outperforming strong baselines in the metaphor identification task and extends to interpret the identified metaphors, paraphrasing them into their literal counterparts, so they can be better translated by machines.

Experience Grounds Language

It is posited that the present success of representation learning approaches trained on large text corpora can be deeply enriched from the parallel tradition of research on the contextual and social nature of language.

An LSTM-CRF Based Approach to Token-Level Metaphor Detection

This paper proposes a method for detection of metaphors at the token level using a hybrid model of Bidirectional-LSTM and CRF, which used fewer features, as compared to the previous state-of-the-art sequential model.

Leveraging Syntactic Constructions for Metaphor Identification

This work undertake a corpus-based analysis of predicate-argument constructions and their metaphoric properties, and attempt to effectively represent syntactic constructions as features for metaphor processing, both in identifying source and target domains and in distinguishing metaphoric words from non-metaphoric.