Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics

  title={Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics},
  author={Chin-Yew Lin and Eduard H. Hovy},
Following the recent adoption by the machine translation community of automatic evaluation using the BLEU/NIST scoring process, we conduct an in-depth study of a similar idea for evaluating summaries. The results show that automatic evaluation using unigram cooccurrences between summary pairs correlates surprising well with human evaluations, based on various statistical metrics; while direct application of the BLEU evaluation procedure does not always give good results. 

6 Figures & Tables



Citations per Year

1,460 Citations

Semantic Scholar estimates that this publication has 1,460 citations based on the available data.

See our FAQ for additional information.