• Publications
  • Influence
SciBERT: A Pretrained Language Model for Scientific Text
TLDR
We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. Expand
  • 384
  • 85
  • PDF
Longformer: The Long-Document Transformer
TLDR
Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length, making it easy to process documents of thousands of tokens or longer. Expand
  • 198
  • 48
  • PDF
SciBERT: Pretrained Contextualized Embeddings for Scientific Text
TLDR
We release SciBERT, a pretrained contextualized embedding model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. Expand
  • 143
  • 40
  • PDF
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
TLDR
We propose the first model for abstractive summarization of single, longer-form documents (e.g., research papers). Expand
  • 147
  • 28
  • PDF
CEDR: Contextualized Embeddings for Document Ranking
TLDR
We propose a joint approach that incorporates BERT's classification vector into existing neural models and show that it outperforms state-of-the-art ad-hoc ranking baselines. Expand
  • 115
  • 16
  • PDF
Depression and Self-Harm Risk Assessment in Online Forums
TLDR
We propose methods for identifying posts in support communities that may indicate a risk of self-harm, and demonstrate that our approach outperforms strong baselines on this general forum dataset. Expand
  • 87
  • 13
  • PDF
Scientific Article Summarization Using Citation-Context and Article's Discourse Structure
TLDR
We propose a summarization approach for scientific articles which takes advantage of citation-context and the document discourse model to overcome the problem of inconsistency between the citation summary and the article’s content. Expand
  • 58
  • 7
  • PDF
Structural Scaffolds for Citation Intent Classification in Scientific Publications
TLDR
We propose structural scaffolds, a multitask model to incorporate structural information of scientific papers into citations for effective classification of citation intents. Expand
  • 37
  • 6
  • PDF
Pretrained Language Models for Sequential Sentence Classification
TLDR
We show that pretrained language models, BERT (Devlin et al., 2018) in particular, can be used for this task to capture contextual dependencies without the need for hierarchical encoding. Expand
  • 26
  • 5
  • PDF
Fact or Fiction: Verifying Scientific Claims
TLDR
We introduce scientific claim verification, a new task to select abstracts from the research literature containing evidence that supports or refutes a given scientific claim, and to identify rationales justifying each decision. Expand
  • 24
  • 5
  • PDF
...
1
2
3
4
5
...