Language Models as Knowledge Bases?
@inproceedings{Petroni2019LanguageMA, title={Language Models as Knowledge Bases?}, author={F. Petroni and Tim Rockt{\"a}schel and Patrick Lewis and A. Bakhtin and Yuxiang Wu and Alexander H. Miller and S. Riedel}, booktitle={EMNLP}, year={2019} }
Recent progress in pretraining language models on large textual corpora led to a surge of improvements for downstream NLP tasks. Whilst learning linguistic knowledge, these models may also be storing relational knowledge present in the training data, and may be able to answer queries structured as “fill-in-the-blank” cloze statements. Language models have many advantages over structured knowledge bases: they require no schema engineering, allow practitioners to query about an open class of… Expand
Supplemental Code
Paper Mentions
253 Citations
Relational world knowledge representation in contextual language models: A review
- Computer Science
- 2021
- PDF
KnowlyBERT - Hybrid Query Answering over Language Models and Knowledge Graphs
- Computer Science
- SEMWEB
- 2020
- 2
- PDF
Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries
- Computer Science
- ArXiv
- 2020
- 4
- Highly Influenced
- PDF
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
- Computer Science
- ICLR
- 2020
- 44
- PDF
References
SHOWING 1-10 OF 43 REFERENCES
Improving Language Understanding by Generative Pre-Training
- Computer Science
- 2018
- 1,929
- Highly Influential
- PDF
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
- Computer Science
- BlackboxNLP@EMNLP
- 2018
- 1,265
- PDF
T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples
- Computer Science
- LREC
- 2018
- 43
- Highly Influential
- PDF
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Computer Science
- NAACL-HLT
- 2019
- 15,760
- Highly Influential
- PDF
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
- Computer Science
- NAACL-HLT
- 2019
- 182
- PDF
A Survey of Reinforcement Learning Informed by Natural Language
- Computer Science, Mathematics
- IJCAI
- 2019
- 52
- PDF
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
- Computer Science
- ACL
- 2019
- 253
- PDF
Dissecting Contextual Word Embeddings: Architecture and Representation
- Computer Science
- EMNLP
- 2018
- 187
- Highly Influential
- PDF