Evaluating the Factual Consistency of Abstractive Text Summarization

@article{Kryscinski2020EvaluatingTF,
  title={Evaluating the Factual Consistency of Abstractive Text Summarization},
  author={Wojciech Kryscinski and B. McCann and Caiming Xiong and R. Socher},
  journal={ArXiv},
  year={2020},
  volume={abs/1910.12840}
}
  • Wojciech Kryscinski, B. McCann, +1 author R. Socher
  • Published 2020
  • Computer Science
  • ArXiv
  • Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents. We propose a weakly-supervised, model-based approach for verifying factual consistency and identifying conflicts between source documents and a generated summary. Training data is generated by applying a series of rule-based transformations to the sentences of source documents. The factual consistency model is then trained jointly for three tasks: 1… CONTINUE READING
    46 Citations
    Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation
    • 1
    • Highly Influenced
    • PDF
    Go Figure! A Meta Evaluation of Factuality in Summarization
    • 2
    • PDF
    Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
    • 35
    • Highly Influenced
    • PDF
    Boosting Factual Correctness of Abstractive Summarization
    • 4
    • Highly Influenced
    Factual Error Correction for Abstractive Summarization Models
    • 2
    • Highly Influenced
    • PDF
    Boosting Factual Correctness of Abstractive Summarization with Knowledge Graph
    • 10
    • Highly Influenced
    • PDF
    Entity-level Factual Consistency of Abstractive Text Summarization
    • Highly Influenced
    • PDF
    On Faithfulness and Factuality in Abstractive Summarization
    • 28
    • Highly Influenced
    • PDF
    Compressive Summarization with Plausibility and Salience Modeling
    • PDF

    References

    SHOWING 1-10 OF 58 REFERENCES
    Faithful to the Original: Fact Aware Neural Abstractive Summarization
    • 100
    • PDF
    On Faithfulness and Factuality in Abstractive Summarization
    • 28
    • PDF
    Improving Abstraction in Text Summarization
    • 61
    • PDF
    FEVER: a large-scale dataset for Fact Extraction and VERification
    • 274
    • PDF
    Assessing The Factual Accuracy of Generated Text
    • 30
    • PDF
    BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization
    • 28
    • PDF
    Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference
    • 37
    • Highly Influential
    • PDF
    Bottom-Up Abstractive Summarization
    • 276
    • PDF
    Text Summarization with Pretrained Encoders
    • 247
    • PDF