• Publications
  • Influence
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?
TLDR
It is observed that intermediate tasks requiring high-level inference and reasoning abilities tend to work best and that target task performance is strongly correlated with higher-level abilities such as coreference resolution, but it is failed to observe more granular correlations between probing and target taskperformance. Expand
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?
TLDR
It is observed that intermediate tasks requiring high-level inference and reasoning abilities tend to work best and that target task performance is strongly correlated with higher-level abilities such as coreference resolution, but it is failed to observe more granular correlations between probing and target taskperformance. Expand
Consistency of a Recurrent Language Model with Respect to Incomplete Decoding
TLDR
It is proved that commonly used incomplete decoding algorithms - greedy search, beam search, top-k sampling, and nucleus sampling - are inconsistent, despite the fact that recurrent language models are trained to produce sequences of finite length. Expand
ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation
TLDR
This work views their non-autoregressive translation system as an inference network trained to minimize the autoregressive teacher energy, which achieves state-of-the-art non-Autoregressive results on the IWSLT 2014 DE-EN and WMT 2016 RO-EN datasets, approaching the performance of autore progressive models. Expand
Towards Actual (Not Operational) Textual Style Transfer Auto-Evaluation
TLDR
There are advances on developing methods that do not require parallel corpora, but issues remain with automatic evaluation metrics, and current works agree on the following three evaluation aspects. Expand
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
TLDR
This work considers the problem of automatically generating textual paraphrases with modified attributes or properties, focusing on the setting without parallel data, and proposes additional metrics based on semantic preservation and fluency as well as a way to combine them into a single overall score. Expand
Learning Criteria and Evaluation Metrics for Textual Transfer between Non-Parallel Corpora
TLDR
The problem of automatically generating textual paraphrases with modified attributes or stylistic properties, focusing on the setting without parallel data, is considered, and the metric of post-transfer classification accuracy is shown to be insufficient, and additional metrics based on semantic content preservation and fluency are proposed. Expand
The Daunting Task of Real-World Textual Style Transfer Auto-Evaluation
TLDR
Given that current tasks do not represent real use cases of style transfer, current auto-evaluation approach is flawed, and this discussion aims to bring researchers to think about the future of styletransfer and style transfer evaluation research. Expand
Improving Joint Training of Inference Networks and Structured Prediction Energy Networks
TLDR
This paper designs a compound objective to jointly train both cost-augmented and test-time inference networks along with the energy function, and proposes joint parameterizations for the inference networks that encourage them to capture complementary functionality during learning. Expand
Text Generation by Learning from Demonstrations
TLDR
It is found that GOLD outperforms the baselines according to automatic and human evaluation on summarization, question generation, and machine translation, including attaining state-of-the-art results for CNN/DailyMail summarization. Expand
...
1
2
...