COMET: Commonsense Transformers for Automatic Knowledge Graph Construction

@article{Bosselut2019COMETCT,
  title={COMET: Commonsense Transformers for Automatic Knowledge Graph Construction},
  author={Antoine Bosselut and Hannah Rashkin and Maarten Sap and Chaitanya Malaviya and Asli Celikyilmaz and Yejin Choi},
  journal={ArXiv},
  year={2019},
  volume={abs/1906.05317}
}
We present the first comprehensive study on automatic knowledge base construction for two prevalent commonsense knowledge graphs: ATOMIC (Sap et al., 2019) and ConceptNet (Speer et al., 2017. [] Key Result Empirical results demonstrate that COMET is able to generate novel knowledge that humans rate as high quality, with up to 77.5% (ATOMIC) and 91.7% (ConceptNet) precision at top 1, which approaches human performance for these resources.
Commonsense Generative Model for Chinese Automatic Knowledge Graph Construction
  • Xiaowen Shi, J. Yang, Liang He
  • Computer Science
    2022 IEEE 2nd International Conference on Electronic Technology, Communication and Information (ICETCI)
  • 2022
TLDR
A commonsense generative model with a novel attention mechanism is proposed and whether pre-trained language models can effectively learn and generate novel knowledge is discussed, showing that the model could generate correct commonsense knowledge with high scores.
Dynamic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
TLDR
Empirical results on the SOCIALIQA and STORYCOMMONSENSE datasets in a zero-shot setting demonstrate that using commonsense knowledge models to dynamically construct and reason over knowledge graphs achieves performance boosts over pre-trained language models and usingknowledge models to directly evaluate answers.
Exploiting Structural and Semantic Context for Commonsense Knowledge Base Completion
TLDR
This paper investigates two key ideas: (1) learning from local graph structure, using graph convolutional networks and automatic graph densification and (2) transfer learning from pre-trained language models to knowledge graphs for enhanced contextual representation of knowledge.
COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs
TLDR
It is proposed that manually constructed CSKGs will never achieve the coverage necessary to be applicable in all situations encountered by NLP agents, and a new evaluation framework for testing the utility of KGs based on how effectively implicit knowledge representations can be learned from them is proposed.
On Symbolic and Neural Commonsense Knowledge Graphs
TLDR
It is proposed that manually constructed CSKGs will never achieve the coverage necessary to be applicable in all situations encountered by NLP agents, and a new evaluation framework for testing the utility of KGs based on how effectively implicit knowledge representations can be learned from them is proposed.
Commonsense Knowledge Base Completion with Structural and Semantic Context
TLDR
This paper investigates two key ideas: (1) learning from local graph structure, using graph convolutional networks and automatic graph densification and (2) transfer learning from pre-trained language models to knowledge graphs for enhanced contextual representation of knowledge.
Alleviating the Knowledge-Language Inconsistency: A Study for Deep Commonsense Knowledge
TLDR
It is shown that deep commonsense knowledge occupies a significant part of Commonsense knowledge, while the conventional methods based on pre-trained language models fail to capture it effectively, and a novel method is proposed to mine the deep commonsens knowledge from raw text that is exactly language expression, alleviating the reliance of conventional methods on the triple representation form.
Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
TLDR
This paper presents a novel approach that generates contextually-relevant symbolic knowledge structures on demand using generative neural commonsense knowledge models and achieves significant performance boosts over pretrained language models and vanilla knowledge models, all while providing interpretable reasoning paths for its predictions.
Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering
TLDR
This paper augments a general commonsense QA framework with a knowledgeable path generator by extrapolating over existing paths in a KG with a state-of-the-art language model, which learns to connect a pair of entities in text with a dynamic, and potentially novel, multi-hop relational path.
DISCOS: Bridging the Gap between Discourse Knowledge and Commonsense Knowledge
TLDR
Experiments demonstrate that the proposed commonsense knowledge acquisition framework DISCOS can successfully convert discourse knowledge about eventualities from ASER, a large-scale discourse knowledge graph, into if-then Commonsense knowledge defined in ATOMIC without any additional annotation effort.
...
...

References

SHOWING 1-10 OF 38 REFERENCES
Commonsense Knowledge Base Completion
TLDR
This work develops neural network models for scoring tuples on arbitrary phrases and evaluates them by their ability to distinguish true held-out tuples from false ones and finds strong performance from a bilinear model using a simple additive architecture to model phrases.
Commonsense Knowledge Base Completion and Generation
TLDR
Experimental results show that the joint learning method improved completion accuracy and the generation model created reasonable knowledge, which could also be used to augment data and improve the accuracy of completion.
ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning
TLDR
Experimental results demonstrate that multitask models that incorporate the hierarchical structure of if-then relation types lead to more accurate inference compared to models trained in isolation, as measured by both automatic and human evaluation.
Knowledge vault: a web-scale approach to probabilistic knowledge fusion
TLDR
The Knowledge Vault is a Web-scale probabilistic knowledge base that combines extractions from Web content (obtained via analysis of text, tabular data, page structure, and human annotations) with prior knowledge derived from existing knowledge repositories that computes calibrated probabilities of fact correctness.
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
TLDR
A new version of the linked open data resource ConceptNet is presented that is particularly well suited to be used with modern NLP techniques such as word embeddings, with state-of-the-art results on intrinsic evaluations of word relatedness that translate into improvements on applications of word vectors, including solving SAT-style analogies.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Yago: a core of semantic knowledge
TLDR
YAGO builds on entities and relations and currently contains more than 1 million entities and 5 million facts, which includes the Is-A hierarchy as well as non-taxonomic relations between entities (such as HASONEPRIZE).
Scalable knowledge harvesting with high precision and high recall
TLDR
A new notion of ngram-itemsets for richer patterns is proposed, and MaxSat-based constraint reasoning is used on both the quality of patterns and the validity of fact candidates, to use in a scalable system for high-quality knowledge harvesting.
Zero-Shot Relation Extraction via Reading Comprehension
TLDR
It is shown that relation extraction can be reduced to answering simple reading comprehension questions, by associating one or more natural-language questions with each relation slot, and that zero-shot generalization to unseen relation types is possible, at lower accuracy levels.
Improving Language Understanding by Generative Pre-Training
TLDR
The general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, improving upon the state of the art in 9 out of the 12 tasks studied.
...
...