Share This Author
RoBERTa: A Robustly Optimized BERT Pretraining Approach
It is found that BERT was significantly undertrained, and can match or exceed the performance of every model published after it, and the best model achieves state-of-the-art results on GLUE, RACE and SQuAD.
Dense Passage Retrieval for Open-Domain Question Answering
This work shows that retrieval can be practically implemented using dense representations alone, where embeddings are learned from a small number of questions and passages by a simple dual-encoder framework.
Reasoning With Neural Tensor Networks for Knowledge Base Completion
An expressive neural tensor network suitable for reasoning over relationships between two entities given a subset of the knowledge base is introduced and performance can be improved when entities are represented as an average of their constituting word vectors.
Reading Wikipedia to Answer Open-Domain Questions
This approach combines a search component based on bigram hashing and TF-IDF matching with a multi-layer recurrent neural network model trained to detect answers in Wikipedia paragraphs, indicating that both modules are highly competitive with respect to existing counterparts.
SpanBERT: Improving Pre-training by Representing and Predicting Spans
- Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, Omer Levy
- Computer ScienceTACL
- 24 July 2019
The approach extends BERT by masking contiguous random spans, rather than random tokens, and training the span boundary representations to predict the entire content of the masked span, without relying on the individual token representations within it.
of the Association for Computational Linguistics:
SimCSE: Simple Contrastive Learning of Sentence Embeddings
SimCSE is presented, a simple contrastive learning framework that greatly advances the state-of-the-art sentence embeddings and regularizes pre-trainedembeddings’ anisotropic space to be more uniform, and it better aligns positive pairs when supervised signals are available.
Observed versus latent features for knowledge base and text inference
- Kristina Toutanova, Danqi Chen
- Computer ScienceProceedings of the 3rd Workshop on Continuous…
- 30 July 2015
It is shown that the observed features model is most effective at capturing the information present for entity pairs with textual relations, and a combination of the two combines the strengths of both model types.
A Fast and Accurate Dependency Parser using Neural Networks
This work proposes a novel way of learning a neural network classifier for use in a greedy, transition-based dependency parser that can work very fast, while achieving an about 2% improvement in unlabeled and labeled attachment scores on both English and Chinese datasets.
Position-aware Attention and Supervised Data Improve Slot Filling
- Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, Christopher D. Manning
- Computer ScienceEMNLP
- 1 September 2017
An effective new model is proposed, which combines an LSTM sequence model with a form of entity position-aware attention that is better suited to relation extraction that builds TACRED, a large supervised relation extraction dataset obtained via crowdsourcing and targeted towards TAC KBP relations.