SPECTER: Document-level Representation Learning using Citation-informed Transformers

@article{Cohan2020SPECTERDR,
  title={SPECTER: Document-level Representation Learning using Citation-informed Transformers},
  author={Arman Cohan and Sergey Feldman and Iz Beltagy and Doug Downey and Daniel S. Weld},
  journal={ArXiv},
  year={2020},
  volume={abs/2004.07180}
}
Representation learning is a critical ingredient for natural language processing systems. Recent Transformer language models like BERT learn powerful textual representations, but these models are targeted towards token- and sentence-level training objectives and do not leverage information on inter-document relatedness, which limits their document-level representation power. For applications on scientific documents, such as classification and recommendation, accurate embeddings of documents are… Expand
16 Citations
CSFCube - A Test Collection of Computer Science Research Articles for Faceted Query by Example
  • 1
  • Highly Influenced
  • PDF
MS2: Multi-Document Summarization of Medical Studies
  • Highly Influenced
  • PDF
Aspect-based Document Similarity for Research Papers
  • 1
  • Highly Influenced
  • PDF
Document Embedding using piped ELM-GAN Model
  • Arefeh Yavary, H. Sajedi
  • Computer Science
  • 2021 15th International Conference on Ubiquitous Information Management and Communication (IMCOM)
  • 2021
Powering COVID-19 community Q&A with Curated Side Information
  • PDF
WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach
  • 1
  • PDF
...
1
2
...

References

SHOWING 1-10 OF 59 REFERENCES
A Comprehensive Survey on Graph Neural Networks
  • 1,312
  • Highly Influential
  • PDF
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  • 16,788
  • Highly Influential
  • PDF
A Simple but Tough-to-Beat Baseline for Sentence Embeddings
  • 785
  • Highly Influential
An Overview of Microsoft Academic Service (MAS) and Applications
  • 430
  • Highly Influential
  • PDF
Improving Textual Network Embedding with Global Attention via Optimal Transport
  • 10
  • Highly Influential
  • PDF
Improving Textual Network Learning with Variational Homophilic Embeddings
  • 8
  • Highly Influential
  • PDF
Simplifying Graph Convolutional Networks
  • 429
  • Highly Influential
  • PDF
Attention is All you Need
  • 17,927
  • Highly Influential
  • PDF
Inductive Representation Learning on Large Graphs
  • 3,010
  • Highly Influential
  • PDF
SciBERT: A Pretrained Language Model for Scientific Text
  • 401
  • PDF
...
1
2
3
4
5
...