Corpus ID: 235390553

Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responses

  title={Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responses},
  author={Richard Antonello and Javier Turek and Vy A. Vo and Alexander G. Huth},
How related are the representations learned by neural language models, translation models, and language tagging tasks? We answer this question by adapting an encoder-decoder transfer learning method from computer vision to investigate the structure among 100 different feature spaces extracted from hidden representations of various networks trained on language tasks. This method reveals a low-dimensional structure where language models and translation models smoothly interpolate between word… Expand

Figures and Tables from this paper


Incorporating Context into Language Encoding Models for fMRI
The models built here show a significant improvement in encoding performance relative to state-of-the-art embeddings in nearly every brain area and suggest that LSTM language models learn high-level representations that are related to representations in the human brain. Expand
Language processing in brains and deep neural networks: computational convergence and its limits
It is shown that image, word and contextualized word embeddings separate the hierarchical levels of language processing in the brain and how perceptual, lexical and compositional representations precisely unfold within each cortical region and contribute to uncovering the governing principles of languageprocessing in brains and algorithms. Expand
Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)
It is hypothesize that altering BERT to better align with brain recordings would enable it to also better understand language, and closes the loop to allow the interaction between NLP and cognitive neuroscience to be a true cross-pollination. Expand
Interpretable multi-timescale models for predicting fMRI responses to continuous natural speech
This work constructs interpretable multi-timescale representations by forcing individual units in an LSTM LM to integrate information over specific temporal scales, which provides a finer-grained map of timescale information in the human language pathway. Expand
Neural Taskonomy: Inferring the Similarity of Task-Derived Representations from Brain Activity
These computationally-driven results—arising out of state-of-the-art computer vision methods—begin to reveal the task-specific architecture of the human visual system. Expand
Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses
This approach is the first to simultaneously track diverse reading subprocesses during complex story processing and predict the detailed neural representation of diverse story features, ranging from visual word properties to the mention of different story characters and different actions they perform. Expand
Natural speech reveals the semantic maps that tile human cerebral cortex
This study systematically map semantic selectivity across the cortex using voxel-wise modelling of functional MRI data collected while subjects listened to hours of narrative stories, and uses a novel generative model to create a detailed semantic atlas. Expand
Lack of selectivity for syntax relative to word meanings throughout the language network
To understand what you are reading now, your mind retrieves the meanings of words and constructions from a linguistic knowledge store (lexico-semantic processing) and identifies the relationshipsExpand
Task representations in neural networks trained to perform many cognitive tasks
It is found that after training, recurrent units can develop into clusters that are functionally specialized for different cognitive processes, and a simple yet effective measure is introduced to quantify relationships between single-unit neural representations of tasks. Expand
GloVe: Global Vectors for Word Representation
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure. Expand