Technical Brief: Agreement, the F-Measure, and Reliability in Information Retrieval

@article{Hripcsak2005TechnicalBA,
  title={Technical Brief: Agreement, the F-Measure, and Reliability in Information Retrieval},
  author={G. Hripcsak and A. Rothschild},
  journal={Journal of the American Medical Informatics Association : JAMIA},
  year={2005},
  volume={12 3},
  pages={
          296-8
        }
}
  • G. Hripcsak, A. Rothschild
  • Published 2005
  • Computer Science, Medicine
  • Journal of the American Medical Informatics Association : JAMIA
Information retrieval studies that involve searching the Internet or marking phrases usually lack a well-defined number of negative cases. [...] Key Result Positive specific agreement-or the equivalent F-measure-may be an appropriate way to quantify interrater reliability and therefore to assess the reliability of a gold standard in these studies.Expand
575 Citations
Testing and Best Practices
Gold Standards in Enterprise Knowledge Projects
  • PDF
Reliability and type of consumer health documents on the World Wide Web: an annotation study
  • M. Martin
  • Computer Science, Medicine
  • J. Biomed. Semant.
  • 2011
  • 5
  • PDF
A scaleable automated quality assurance technique for semantic representations and proposition banks
  • 1
  • PDF
Domain-independent automatic keyphrase indexing with small training sets
  • 79
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 13 REFERENCES
Measuring agreement in medical informatics reliability studies
  • 201
Reference Standards, Judges, and Comparison Subjects
  • 74
  • PDF
Inter-annotator Agreement for a German Newspaper Corpus
  • 110
  • PDF
Measuring agreement between two judges on the presence or absence of a trait.
  • 469
Information Retrieval
  • A. Dekhtyar
  • Medicine, Computer Science
  • Lecture Notes in Computer Science
  • 1968
  • 3,241
  • PDF
Statistical methods for rates and proportions
  • 10,780
Evaluation Methods in Medical Informatics
  • 501
Available in:
  • 9,073
  • PDF
cited 2005March 23].Available from: http://ourworld. compuserve.com/homepages/jsuebersax/agree.htm
  • 2005
...
1
2
...