• Publications
  • Influence
SciBERT: A Pretrained Language Model for Scientific Text
Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et. al., 2018) toExpand
  • 279
  • 59
  • PDF
SciBERT: Pretrained Contextualized Embeddings for Scientific Text
Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained contextualized embedding model based on BERT (Devlin et al.,Expand
  • 124
  • 36
  • PDF
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
Language models pretrained on text from a wide variety of sources form the foundation of today's NLP. In light of the success of these broad-coverage models, we investigate whether it is stillExpand
  • 111
  • 22
  • PDF
Longformer: The Long-Document Transformer
Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce theExpand
  • 102
  • 22
  • PDF
Longformer: The Long-Document Transformer
Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce theExpand
  • 95
  • 21
  • PDF
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing
Despite recent advances in natural language processing, many statistical models for processing text perform extremely poorly under domain shift. Processing biomedical and clinical text is aExpand
  • 101
  • 18
  • PDF
Construction of the Literature Graph in Semantic Scholar
We describe a deployed scalable system for organizing published scientific literature into a heterogeneous graph to facilitate algorithmic manipulation and discovery. The resulting literature graphExpand
  • 99
  • 10
  • PDF
Probabilistic Soft Logic for Semantic Textual Similarity
Probabilistic Soft Logic (PSL) is a recently developed framework for probabilistic logic. We use PSL to combine logical and distributional representations of natural-language meaning, whereExpand
  • 53
  • 8
  • PDF
Representing Meaning with a Combination of Logical and Distributional Models
NLP tasks differ in the semantic information they require, and at this time no single semantic representation fulfills all requirements. Logic-based representations characterize sentence structure,Expand
  • 48
  • 8
  • PDF
A new routing metric and protocol for multipath routing in cognitive networks
Routing in cognitive networks is a challenging problem due to the primary users' (PU) activities and mobility. Multipath routing is a general solution to improve reliability of connections. ExistingExpand
  • 39
  • 5