Skip to search formSkip to main contentSkip to account menu

SciBERT: A Pretrained Language Model for Scientific Text

TLDR
SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks and demonstrates statistically significant improvements over BERT.
View on ACL (opens in a new tab)

Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

TLDR
It is consistently found that multi-phase adaptive pretraining offers large gains in task performance, and it is shown that adapting to a task corpus augmented using simple data selection strategies is an effective alternative, especially when resources for domain-adaptive pretraining might be unavailable.
View on ACL (opens in a new tab)

CORD-19: The COVID-19 Open Research Dataset

TLDR
The mechanics of dataset construction are described, highlighting challenges and key design decisions, an overview of how CORD-19 has been used, and several shared tasks built around the dataset are described.
View on ACL (opens in a new tab)

SciBERT: Pretrained Contextualized Embeddings for Scientific Text

TLDR
SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks and demonstrates statistically significant improvements over BERT.

S2ORC: The Semantic Scholar Open Research Corpus

TLDR
In S2ORC, a large corpus of 81.1M English-language academic papers spanning many academic disciplines is introduced, which is expected to facilitate research and development of tools and tasks for text mining over academic text.
View on ACL (opens in a new tab)

Construction of the Literature Graph in Semantic Scholar

TLDR
This paper reduces literature graph construction into familiar NLP tasks, point out research challenges due to differences from standard formulations of these tasks, and report empirical results for each task.
View on ACL (opens in a new tab)

TREC-COVID: Constructing a Pandemic Information Retrieval Test Collection

TREC-COVID is a community evaluation designed to build a test collection that captures the information needs of biomedical researchers using the scientific literature during a pandemic. One of the…
View PDF on arXiv (opens in a new tab)

Fact or Fiction: Verifying Scientific Claims

We introduce scientific claim verification, a new task to select abstracts from the research literature containing evidence that supports or refutes a given scientific claim, and to identify…
View on ACL (opens in a new tab)

TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19

TLDR
TREC-COVID differs from traditional IR shared task evaluations with special considerations for the expected users, IR modality considerations, topic development, participant requirements, assessment process, relevance criteria, evaluation metrics, iteration process, projected timeline, and the implications of data use as a post-task test collection.
View on PubMed (opens in a new tab)

TLDR: Extreme Summarization of Scientific Documents

TLDR
This work introduces SCITLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers, and proposes CATTS, a simple yet effective learning strategy for generatingTLDRs that exploits titles as an auxiliary training signal.
View on ACL (opens in a new tab)
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)