Overview of QA4MRE Main Task at CLEF 2013

Abstract

This paper describes the Question Answering for Machine Reading (QA4MRE) Main Task at the 2013 Cross Language Evaluation Forum. In the main task, systems answered multiple-choice questions on documents concerned with four different topics. There were also two pilot tasks, Machine Reading on Biomedical Texts about Alzheimer's disease, and Japanese Entrance Exams. This paper describes the preparation of the data sets, the definition of the background collections, the metric used for the evaluation of the systems’ submissions, and the results. We introduced two novelties this year: auxiliary questions to evaluate systems level of inference, and a portion of questions where none of the options were correct. Nineteen groups participated in the task submitting a total of 77 runs in five languages.

Extracted Key Phrases

19 Figures and Tables

Cite this paper

@inproceedings{Sutcliffe2013OverviewOQ, title={Overview of QA4MRE Main Task at CLEF 2013}, author={Richard F. E. Sutcliffe and Anselmo Pe{\~n}as and Eduard H. Hovy and Pamela Forner and {\'A}lvaro Rodrigo and Corina Forascu and Yassine Benajiba and Petya Osenova}, booktitle={CLEF}, year={2013} }