LAYERED: Metric for Machine Translation Evaluation

@inproceedings{Gautam2014LAYEREDMF,
  title={LAYERED: Metric for Machine Translation Evaluation},
  author={Shubham Gautam and Pushpak Bhattacharyya},
  booktitle={WMT@ACL},
  year={2014}
}
This paper describes the LAYERED metric which is used for the shared WMT’14 metrics task. Various metrics exist for MT evaluation: BLEU (Papineni, 2002), METEOR (Alon Lavie, 2007), TER (Snover, 2006) etc., but are found inadequate in quite a few language settings like, for example, in case of free word order languages. In this paper, we propose an MT evaluation scheme that is based on the NLP layers: lexical, syntactic and semantic. We contend that higher layer metrics are after all needed… CONTINUE READING