• Computer Science
  • Published in NTCIR 2004

New Performance Metrics Based on Multigrade Relevance: Their Application to Question Answering

@inproceedings{Sakai2004NewPM,
  title={New Performance Metrics Based on Multigrade Relevance: Their Application to Question Answering},
  author={Tetsuya Sakai},
  booktitle={NTCIR},
  year={2004}
}
This paper proposes two new InformationRetrieval performance metrics based on multigrade relevance, called Q-measure and R-measure, which are akin to Cumulative Gain and Average Weighted Precision but are arguably more reliable. We then show how Qmeasure can be appliedto Question Answering involving ranked lists of exact answers, and discuss its advantages over Reciprocal Rank through an experiment using the QAC1 test collection. The appendices of this paper contain theorem proofs concerning Q… CONTINUE READING

Figures, Tables, and Topics from this paper.

Citations

Publications citing this paper.
SHOWING 1-10 OF 42 CITATIONS

Semantic Information Management for Pervasive Computing

VIEW 5 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

On the reliability of factoid question answering evaluation

  • ACM Trans. Asian Lang. Inf. Process.
  • 2007
VIEW 11 EXCERPTS
CITES METHODS

eXtended cumulated gain measures for the evaluation of content-oriented XML retrieval

  • ACM Trans. Inf. Syst.
  • 2006
VIEW 4 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

References

Publications referenced by this paper.
SHOWING 1-10 OF 19 REFERENCES

Information Retrieval System Evaluation using Multi-Grade Relevance Judgments - Discussion on Averageable Single-Numbered Measures (in Japanese)

N. Kando, K. Kuriyama, M. Yoshioka
  • IPSJ SIG Notes,
  • 2001
VIEW 7 EXCERPTS
HIGHLY INFLUENTIAL