• Corpus ID: 246430154

Compositionality as Lexical Symmetry

@article{Akyrek2022CompositionalityAL,
  title={Compositionality as Lexical Symmetry},
  author={Ekin Aky{\"u}rek and Jacob Andreas},
  journal={ArXiv},
  year={2022},
  volume={abs/2201.12926}
}
Standard deep network models lack the induc001 tive biases needed to generalize composition002 ally from small datasets in tasks like semantic 003 parsing, translation, and question answering. 004 A large body of work in natural language pro005 cessing seeks to overcome this limitation with 006 new model architectures that enforce a compo007 sitional process of sentence interpretation. In 008 this paper, we present a domain-general and 009 model-agnostic framework for compositional 010 modeling… 

Figures and Tables from this paper

References

SHOWING 1-10 OF 67 REFERENCES
Improving Compositional Generalization with Latent Structure and Data Augmentation
TLDR
This work presents a more powerful data recombination method using a model called Compositional Structure Learner (CSL), a generative model with a quasi-synchronous context-free grammar backbone, which results in a model even stronger than a T5-CSL ensemble on two real world compositional generalization tasks.
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?
TLDR
NQG-T5 is proposed, a hybrid model that combines a high-precision grammar-based approach with a pre-trained sequence-to-sequence model that outperforms existing approaches across several compositional generalization challenges on non-synthetic data, while also being competitive with the state of theart on standard evaluations.
Permutation Equivariant Models for Compositional Generalization in Language
TLDR
This paper hypothesizes that language compositionality is a form of group-equivariance, and proposes a set of tools for constructing equivariant sequence-to-sequence models that are able to achieve the type compositional generalization required in human language understanding.
Lexicon Learning for Few Shot Sequence Modeling
TLDR
This work augments neural decoders with a lexical translation mechanism that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules, and shows that it improves systematic generalization on a diverse set of sequence modeling tasks drawn from cognitive science, formal semantics, and machine translation.
Good-Enough Compositional Data Augmentation
TLDR
A simple data augmentation protocol aimed at providing a compositional inductive bias in conditional and unconditional sequence models that reduces error rate by as much as 87% on diagnostic tasks from the SCAN dataset and 16% on a semantic parsing task.
Making a Point: Pointer-Generator Transformers for Disjoint Vocabularies
TLDR
This work proposes a model that does not beat the best performing baseline, and demonstrates that it provides complementary information to it: an oracle that combines the best outputs of the two models improves over the strongest baseline by 7.7 WER on average in the low-resource setting.
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
TLDR
A Sentiment Treebank that includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality, and introduces the Recursive Neural Tensor Network.
Data Recombination for Neural Semantic Parsing
TLDR
Data recombination improves the accuracy of the RNN model on three semantic parsing datasets, leading to new state-of-the-art performance on the standard GeoQuery dataset for models with comparable supervision.
SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation
TLDR
An extremely simple data augmentation strategy for NMT: randomly replacing words in both the source sentence and the target sentence with other random words from their corresponding vocabularies is proposed.
Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks
TLDR
The only factor that consistently contributed a hierarchical bias across tasks was the use of a tree-structured model rather than a model with sequential recurrence, suggesting that human-like syntactic generalization requires architectural syntactic structure.
...
...