Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings

@article{Ghazarian2019BetterAE,
  title={Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings},
  author={Sarik Ghazarian and Johnny Tian-Zheng Wei and Aram Galstyan and Nanyun Peng},
  journal={ArXiv},
  year={2019},
  volume={abs/1904.10635}
}
  • Sarik Ghazarian, Johnny Tian-Zheng Wei, +1 author Nanyun Peng
  • Published in ArXiv 2019
  • Computer Science
  • Despite advances in open-domain dialogue systems, automatic evaluation of such systems is still a challenging problem. Traditional reference-based metrics such as BLEU are ineffective because there could be many valid responses for a given context that share no common words with reference responses. A recent work proposed Referenced metric and Unreferenced metric Blended Evaluation Routine (RUBER) to combine a learning-based metric, which predicts relatedness between a generated response and a… CONTINUE READING

    Create an AI-powered research feed to stay up to date with new papers like this posted to ArXiv

    Figures, Tables, and Topics from this paper.

    Explore Further: Topics Discussed in This Paper

    Citations

    Publications citing this paper.
    SHOWING 1-3 OF 3 CITATIONS

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 26 REFERENCES

    Deep contextualized word representations

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Improving Language Understanding by Generative Pre-Training

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Dialogue Generation With GAN

    VIEW 2 EXCERPTS

    DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset

    VIEW 1 EXCERPT