Evaluating Word Order Recursively over Permutation-Forests

@inproceedings{Stanojevic2014EvaluatingWO,
  title={Evaluating Word Order Recursively over Permutation-Forests},
  author={Milos Stanojevic and Khalil Sima'an},
  booktitle={SSST@EMNLP},
  year={2014}
}
Automatically evaluating word order of MT system output at the sentence-level is challenging. At the sentence-level, ngram counts are rather sparse which makes it difficult to measure word order quality effectively using lexicalized units. Recent approaches abstract away from lexicalization by assigning a score to the permutation representing how word positions in system output move around relative to a reference translation. Metrics over permutations exist (e.g., Kendal tau or Spearman Rho… CONTINUE READING

Similar Papers

Loading similar papers…