Evaluating the Factual Consistency of Abstractive Text Summarization
@article{Kryscinski2020EvaluatingTF, title={Evaluating the Factual Consistency of Abstractive Text Summarization}, author={Wojciech Kryscinski and B. McCann and Caiming Xiong and R. Socher}, journal={ArXiv}, year={2020}, volume={abs/1910.12840} }
Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents. We propose a weakly-supervised, model-based approach for verifying factual consistency and identifying conflicts between source documents and a generated summary. Training data is generated by applying a series of rule-based transformations to the sentences of source documents. The factual consistency model is then trained jointly for three tasks: 1… CONTINUE READING
Supplemental Code
Github Repo
Via Papers with Code
Code for the "Evaluating the Factual Consistency of Abstractive Text Summarization" paper
Figures, Tables, and Topics from this paper
46 Citations
Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation
- Computer Science
- ArXiv
- 2020
- 1
- Highly Influenced
- PDF
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
- Computer Science
- ACL
- 2020
- 35
- Highly Influenced
- PDF
Factual Error Correction for Abstractive Summarization Models
- Computer Science
- EMNLP
- 2020
- 2
- Highly Influenced
- PDF
Boosting Factual Correctness of Abstractive Summarization with Knowledge Graph
- Computer Science
- ArXiv
- 2020
- 10
- Highly Influenced
- PDF
Entity-level Factual Consistency of Abstractive Text Summarization
- Computer Science
- ArXiv
- 2021
- Highly Influenced
- PDF
On Faithfulness and Factuality in Abstractive Summarization
- Computer Science
- ACL
- 2020
- 28
- Highly Influenced
- PDF
References
SHOWING 1-10 OF 58 REFERENCES
Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization
- Computer Science
- COLING
- 2018
- 36
- PDF
FEVER: a large-scale dataset for Fact Extraction and VERification
- Computer Science
- NAACL-HLT
- 2018
- 274
- PDF
BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization
- Computer Science
- ACL
- 2019
- 28
- PDF
Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference
- Computer Science
- ACL
- 2019
- 37
- Highly Influential
- PDF