Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics

@inproceedings{Lin2003AutomaticEO,
  title={Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics},
  author={Chin-Yew Lin and Eduard H. Hovy},
  booktitle={HLT-NAACL},
  year={2003}
}
Following the recent adoption by the machine translation community of automatic evaluation using the BLEU/NIST scoring process, we conduct an in-depth study of a similar idea for evaluating summaries. The results show that automatic evaluation using unigram cooccurrences between summary pairs correlates surprising well with human evaluations, based on various statistical metrics; while direct application of the BLEU evaluation procedure does not always give good results. 

6 Figures & Tables

Topics

Statistics

050100150'04'06'08'10'12'14'16'18
Citations per Year

1,460 Citations

Semantic Scholar estimates that this publication has 1,460 citations based on the available data.

See our FAQ for additional information.