• Publications
  • Influence
The TREC-8 Question Answering Track Report
TLDR
An overtemperature and overcurrent resistor fuse has a relatively low electrical resistance below a selected melting temperature and has an irreversible abrupt increase of electrical resistance above the selected meltingTemperature range caused by serious overload or overheating conditions. Expand
Query expansion using lexical-semantic relations
TLDR
Examination of the utility of lexical query expansion in the large, diverse TREC collection shows this query expansion technique makes little difference in retrieval effectiveness if the original queries are relatively complete descriptions of the information being sought even when the concepts to be expanded are selected by hand. Expand
Retrieval evaluation with incomplete information
TLDR
It is shown that current evaluation measures are not robust to substantially incomplete relevance judgments, and a new measure is introduced that is both highly correlated with existing measures when complete judgments are available and more robust to incomplete judgment sets. Expand
DRAFT Overview of the TREC 2003 Question Answering Track
TLDR
The TREC 2003 question answering track contained two tasks, the passages task and the main task; the evaluation metric was the number of snippets that contained a correct answer; the reliability of the evaluation used for these tasks was examined. Expand
Variations in relevance judgments and the measurement of retrieval effectiveness
TLDR
Very high correlations were found among the rankings of systems produced using diAerent relevance judgment sets, indicating that the comparative evaluation of retrieval performance is stable despite substantial diAerences in relevance judgments, and thus reaArm the use of the TREC collections as laboratory tools. Expand
The TREC-8 Question Answering Track Evaluation
TLDR
The TREC-8 Question Answering track was the first large-scale evaluation of systems that return answers, as opposed to lists of documents, in response to a question, and the examination uncovered no serious flaws in the methodology, supporting its continued use for question answering evaluation. Expand
The TREC Spoken Document Retrieval Track: A Success Story
TLDR
The SDR Track can be declared a success in that it has provided objective, demonstrable proof that this technology can be successfully applied to realistic audio collections using a combination of existing technologies and that it can be objectively evaluated. Expand
Variations in relevance judgments and the measurement of retrieval effectiveness
  • E. Voorhees
  • Computer Science
  • Inf. Process. Manag.
  • 1 September 2000
TLDR
Very high correlations were found among the rankings of systems produced using different relevance judgment sets, indicating that the comparative evaluation of retrieval performance is stable despite substantial differences in relevance judgments, and thus reaffirm the use of the TREC collections as laboratory tools. Expand
Overview of the Eighth Text REtrieval Conference (TREC-8)
TLDR
The eighth Text REtrieval Conference TREC was held at the National Institute of Standards and Tech nology NIST on November and outlined the goals of the series of workshops designed to foster research in text retrieval. Expand
Overview of the TREC 2002 Question Answering Track
TLDR
This paper provides an overview of the TREC 2002 QA track, which defined how answer strings were judged, and established that different assessors have different ideas as to what constitutes a correct answer even for the limited type of questions used in the track. Expand
...
1
2
3
4
5
...