Using cross-evaluation to evaluate interactive QA systems

@article{Sun2011UsingCT,
  title={Using cross-evaluation to evaluate interactive QA systems},
  author={Ying Sun and Paul B. Kantor and Emile L. Morse},
  journal={JASIST},
  year={2011},
  volume={62},
  pages={1653-1665}
}
In this article, we report on an experiment to assess the possibility of rigorous evaluation of interactive questionanswering (QA) systems using the cross-evaluation method. This method takes into account the effects of tasks and context, and of the users of the systems. Statistical techniques are used to remove these effects, isolating the effect of the system itself. The results show that this approach yields meaningful measurements of the impact of systems on user task performance, using a… CONTINUE READING

From This Paper

Topics from this paper.
1 Citations
12 References
Similar Papers

Citations

Publications citing this paper.

References

Publications referenced by this paper.
Showing 1-10 of 12 references

A hidden world, growing beyond control

  • D. Priest, W. M. Arkin
  • 2010
1 Excerpt

A model for realistic evaluation of an end - to - end question answering system

  • N. Wacholder, D. Kelly, +4 authors T. Strzalkowski
  • Journal of the American Society for Information…
  • 2007
1 Excerpt

Annual Meeting of American Society for Information Science and Technology (pp. 383–392)

  • N. Wacholder, D. Kelly, +4 authors T. Strzalkowski
  • 2004

Similar Papers

Loading similar papers…