Do Trajectories Encode Verb Meaning?

@inproceedings{Ebert2022DoTE,
  title={Do Trajectories Encode Verb Meaning?},
  author={Dylan Ebert and Chen Sun and Elizabeth-Jane Pavlick},
  booktitle={NAACL},
  year={2022}
}
Distributional models learn representations of words from text, but are criticized for their lack of grounding, or the linking of text to the non-linguistic world. Grounded language models have had success in learning to connect concrete categories like nouns and adjectives to the world via images and videos, but can struggle to isolate the meaning of the verbs themselves from the context in which they typically occur. In this paper, we investigate the extent to which trajectories (i.e. the… 

Figures and Tables from this paper

References

SHOWING 1-10 OF 38 REFERENCES

Grounded Models of Semantic Representation

TLDR
Experimental results show that a closer correspondence to human data can be obtained by uncovering latent information shared among the textual and perceptual modalities rather than arriving at semantic knowledge by concatenating the two.

Experience Grounds Language

TLDR
It is posited that the present success of representation learning approaches trained on large text corpora can be deeply enriched from the parallel tradition of research on the contextual and social nature of language.

A Visuospatial Dataset for Naturalistic Verb Learning

TLDR
A new dataset for training and evaluating grounded language models, designed to emulate the quality of language data to which a pre-verbal child is likely to have access, and uses the collected data to compare several distributional semantics models for verb learning.

Combining Language and Vision with a Multimodal Skip-gram Model

TLDR
Since they propagate visual information to all words, the MMSKIP-GRAM models discover intriguing visual properties of abstract words, paving the way to realistic implementations of embodied theories of meaning.

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

TLDR
It is argued that a system trained only on form has a priori no way to learn meaning, and a clear understanding of the distinction between form and meaning will help guide the field towards better science around natural language understanding.

Lexical Conceptual Structure of Literal and Metaphorical Spatial Language: A Case Study of “Push”

TLDR
It is demonstrated that literal and metaphorical extensions exhibit patterns predicted and represented by the Lexical Conceptual Structure model, and that spatial predicates naturally extend into other fields (e.g., circumstantial or temporal).

Generating Simulations of Motion Events from Verbal Descriptions

TLDR
A computational model for motion events in natural language that maps from linguistic expressions, through a dynamic event interpretation, into three-dimensional temporal simulations in a model that acts as a conceptual “debugger” for the semantics of different motion verbs.

Using Event Representations to Generate Robot Semantics

TLDR
A semantic framework based on events can generate a general representational framework for human-robot communication and an implementation of the framework involving communication with an iCub will be described.

Learning Visually Grounded Sentence Representations

TLDR
In this work, grounded sentence representations are investigated, where a sentence encoder is trained to predict the image features of a given caption and use the resultant features as sentence representations.

Spatial and Linguistic Aspects of Visual Imagery in Sentence Comprehension

TLDR
The results demonstrate that either subject nouns or main verbs can trigger visual imagery, but only when used in literal sentences about real space-metaphorical language does not yield significant effects-which implies that it is the comprehension of the sentence as a whole and not simply lexical associations that yields imagery effects.