• Corpus ID: 8712237

Distributional Semantics in Technicolor

@inproceedings{Bruni2012DistributionalSI,
  title={Distributional Semantics in Technicolor},
  author={Elia Bruni and Gemma Boleda and Marco Baroni and Nam Khanh Tran},
  booktitle={ACL},
  year={2012}
}
Our research aims at building computational models of word meaning that are perceptually grounded. [] Key Result Moreover, we show that visual and textual information are tapping on different aspects of meaning, and indeed combining them in multimodal models often improves performance.

Figures and Tables from this paper

Multimodal Distributional Semantics
TLDR
This work proposes a flexible architecture to integrate text- and image-based distributional information, and shows in a set of empirical tests that the integrated model is superior to the purely text-based approach, and it provides somewhat complementary semantic information with respect to the latter.
Edinburgh Research Explorer Visually Grounded Meaning Representations
TLDR
A new model which uses stacked autoencoders to learn higher-level representations from textual and visual input is introduced which yields a better fit to behavioral data compared to baselines and related models which either rely on a single modality or do not make use of attribute-based input.
Explorer Visually Grounded Meaning Representations
TLDR
A new model which uses stacked autoencoders to learn higher-level representations from textual and visual input is introduced which yields a better fit to behavioral data compared to baselines and related models which either rely on a single modality or do not make use of attribute-based input.
Visually Grounded Meaning Representations
TLDR
A new model which uses stacked autoencoders to learn higher-level representations from textual and visual input is introduced which yields a better fit to behavioral data compared to baselines and related models which either rely on a single modality or do not make use of attribute-based input.
Explorer Models of Semantic Representation with Visual Attributes
TLDR
This work creates a new large-scale taxonomy of visual attributes covering more than 500 concepts and their corresponding 688K images and shows that these bimodal models give a better fit to human word association data compared to amodal model and word representations based on handcrafted norming data.
Learning visually grounded meaning representations
TLDR
An approach is presented which draws inspiration from the successful application of attribute classifiers in image classification, and represents images and the concepts depicted by them by automatically predicted visual attributes, and can act as a substitute for human-produced attributes without any critical information.
Models of Semantic Representation with Visual Attributes
TLDR
This work creates a new large-scale taxonomy of visual attributes covering more than 500 concepts and their corresponding 688K images and shows that these bimodal models give a better fit to human word association data compared to amodal model and word representations based on handcrafted norming data.
Learning Multi-Modal Word Representation Grounded in Visual Context
TLDR
This work explores various choices for what can serve as a visual context and presents an end-to-end method to integrate visual context elements in a multimodal skip-gram model and provides experiments and extensive analysis of the obtained results.
TOWARDS UNIFYING GROUNDED AND DISTRIBUTIONAL SEMANTICS USING THE WORDS-AS-CLASSIFIERS MODEL OF LEXICAL SEMANTICS
TLDR
This thesis evaluates wac2vec, a model that brings together grounded and distributional semantics to work towards leveraging the relative strengths of both, and uses empirical analysis to explore whether wac1vec adds semantic information to traditional embeddings.
Simple Learning and Compositional Application of Perceptually Grounded Word Meanings for Incremental Reference Resolution
TLDR
This work presents an account of word and phrase meaning that is perceptually grounded, trainable, compositional, and ‘dialogueplausible’ in that it computes meanings word-by-word.
...
...

References

SHOWING 1-10 OF 40 REFERENCES
Visual Information in Semantic Representation
TLDR
Experimental results show that a closer correspondence to human data can be obtained by taking the visual modality into account and a model of multimodal meaning representation which is based on the linguistic and visual context is developed.
Using Visual Information to Predict Lexical Preference
TLDR
It is shown that for certain verbs, using visual information can significantly improve performance over a baseline, and for the successful cases, visual information is useful even in the presence of cooccurrence information derived from webscale text.
Concepts and properties in word spaces
TLDR
This article presents a qualitative analysis of the properties generated by humans in two different settings, as well as those produced, for the same concepts, by two computational models, in terms of property types, finding that differences and similarities among models cut across the human/computational distinction.
Symbol Interdependency in Symbolic and Embodied Cognition
TLDR
An overview of the symbolic and embodied cognition accounts is given and it is shown how meaning induction attributed to a specific statistical process or to activation of embodied representations should be attributed to language itself.
Distributional Memory: A General Framework for Corpus-Based Semantics
TLDR
The Distributional Memory approach is shown to be tenable despite the constraints imposed by its multi-purpose nature, and performs competitively against task-specific algorithms recently reported in the literature for the same tasks, and against several state-of-the-art methods.
A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge.
TLDR
A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena.
From Frequency to Meaning: Vector Space Models of Semantics
TLDR
The goal in this survey is to show the breadth of applications of VSMs for semantics, to provide a new perspective on VSMs, and to provide pointers into the literature for those who are less familiar with the field.
Producing high-dimensional semantic spaces from lexical co-occurrence
TLDR
A procedure that processes a corpus of text and produces numeric vectors containing information about its meanings for each word, which provide the basis for a representational model of semantic memory, hyperspace analogue to language (HAL).
Learning to Connect Language and Perception
TLDR
It is argued that the time is now ripe to make a concerted effort to address this important, integrative AI problem of connecting language and perception.
Every Picture Tells a Story: Generating Sentences from Images
TLDR
A system that can compute a score linking an image to a sentence, which can be used to attach a descriptive sentence to a given image, or to obtain images that illustrate a given sentence.
...
...