Skip to search formSkip to main contentSkip to account menu

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

The Cross-lingual TRansfer Evaluation of Multilingual Encoders XTREME benchmark is introduced, a multi-task benchmark for evaluating the cross-lingually generalization capabilities of multilingual representations across 40 languages and 9 tasks.
View PDF on arXiv (opens in a new tab)

Are Sixteen Heads Really Better than One?

It is made the surprising observation that even if models have been trained using multiple heads, in practice, a large percentage of attention heads can be removed at test time without significantly impacting performance.
View PDF on arXiv (opens in a new tab)

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

TaBERT is a pretrained LM that jointly learns representations for NL sentences and (semi-)structured tables that achieves new best results on the challenging weakly-supervised semantic parsing benchmark WikiTableQuestions, while performing competitively on the text-to-SQL dataset Spider.
View on ACL (opens in a new tab)

Lagging Inference Networks and Posterior Collapse in Variational Autoencoders

This paper investigates posterior collapse from the perspective of training dynamics and proposes an extremely simple modification to VAE training to reduce inference lag: depending on the model's current mutual information between latent variable and observation, the inference network is optimized before performing each model update.
View PDF on arXiv (opens in a new tab)

Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow

A novel method to mine high-quality aligned data from SO using two sets of features: hand-crafted features considering the structure of the extracted snippets, and correspondence features obtained by training a probabilistic model to capture the correlation between NL and code using neural networks.
View on ACM (opens in a new tab)

Weight Poisoning Attacks on Pretrained Models

It is shown that it is possible to construct “weight poisoning” attacks where pre-trained weights are injected with vulnerabilities that expose “backdoors” after fine-tuning, enabling the attacker to manipulate the model prediction simply by injecting an arbitrary keyword.
View on ACL (opens in a new tab)

Competence-based Curriculum Learning for Neural Machine Translation

A curriculum learning framework for NMT that reduces training time, reduces the need for specialized heuristics or large batch sizes, and results in overall better performance, which can help improve the training time and the performance of both recurrent neural network models and Transformers.
View on ACL (opens in a new tab)

Stress Test Evaluation for Natural Language Inference

This work proposes an evaluation methodology consisting of automatically constructed “stress tests” that allow us to examine whether systems have the ability to make real inferential decisions, and reveals strengths and weaknesses of these models with respect to challenging linguistic phenomena.
View on ACL (opens in a new tab)

Controllable Invariance through Adversarial Feature Learning

This paper shows that the proposed framework induces an invariant representation, and leads to better generalization evidenced by the improved performance on three benchmark tasks.
View PDF on arXiv (opens in a new tab)

Stack-Pointer Networks for Dependency Parsing

A novel architecture for dependency parsing: stack-pointer networks (StackPtr), which first reads and encodes the whole sentence, then builds the dependency tree top-down in a depth-first fashion, yielding an efficient decoding algorithm with O(n^2) time complexity.
View on ACL (opens in a new tab)
...
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)