An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems

@inproceedings{Chen2020AnES,
  title={An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems},
  author={Yiran Chen and Pengfei Liu and Ming Zhong and Zi-Yi Dou and Danqing Wang and Xipeng Qiu and X. Huang},
  booktitle={FINDINGS},
  year={2020}
}
Neural network-based models augmented with unsupervised pre-trained knowledge have achieved impressive performance on text summarization. However, most existing evaluation methods are limited to an in-domain setting, where summarizers are trained and evaluated on the same dataset. We argue that this approach can narrow our understanding of the generalization ability for different summarization systems. In this paper, we perform an in-depth analysis of characteristics of different datasets and… Expand
3 Citations
QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
  • 2
  • PDF
Retrieving Complex Tables with Multi-Granular Graph Representation Learning
  • PDF
Summarization, Simplification, and Generation: The Case of Patents
  • PDF

References

SHOWING 1-10 OF 41 REFERENCES
Exploring Domain Shift in Extractive Text Summarization
  • 9
  • PDF
Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
  • 15
  • PDF
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks
  • 44
  • Highly Influential
  • PDF
Searching for Effective Neural Extractive Summarization: What Works and What's Next
  • 40
  • PDF
A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization
  • 9
  • PDF
BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization
  • 32
  • Highly Influential
  • PDF
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension
  • 75
  • PDF
Content Selection in Deep Learning Models of Summarization
  • 70
  • PDF
Text Summarization with Pretrained Encoders
  • 294
  • Highly Influential
  • PDF
...
1
2
3
4
5
...