• Publications
  • Influence
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems
TLDR
We propose a method called ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying prioritized historical exemplars, together with an adaptive regularization technique based on ElasticWeight Consolidation. Expand
  • 3
  • 1
  • PDF
Embedding Learning Through Multilingual Concept Induction
TLDR
We present a new method for estimating vector space representations of words: embedding learning by concept induction. Expand
  • 10
  • PDF
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
TLDR
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning. Expand
  • 7
  • PDF
A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction
TLDR
We present a new method for sentiment lexicon induction that is designed to be applicable to the entire range of typological diversity of the world’s languages. Expand
  • 5
  • PDF
Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable
TLDR
We propose Co+Co, a simple and scalable multilingual embedding learner that combines context-based and concept-based learning. Expand
  • 2
  • PDF
Quantifying the Contextualization of Word Representations with Semantic Class Probing
TLDR
Pretrained language models have achieved a new state of the art on many NLP tasks, but there are still many open questions about how and why they work so well. Expand
  • 2
  • PDF
A Closer Look at Few-Shot Crosslingual Transfer: Variance, Benchmarks and Baselines
TLDR
We present a focused study of few-shot crosslingual transfer, a recently proposed NLP scenario: a pretrained multilingual encoder is first finetuned on many annotations in a high resource language (typically English), and then finetuning on a few annotations (the “few shots”) in a target language. Expand
  • 2
  • PDF