• Computer Science
  • Published in ArXiv 2019

What Does My QA Model Know? Devising Controlled Probes using Expert Knowledge

@article{Richardson2019WhatDM,
  title={What Does My QA Model Know? Devising Controlled Probes using Expert Knowledge},
  author={Kyle Richardson and Ashish Sabharwal},
  journal={ArXiv},
  year={2019},
  volume={abs/1912.13337}
}
Open-domain question answering (QA) is known to involve several underlying knowledge and reasoning challenges, but are models actually learning such knowledge when trained on benchmark tasks? To investigate this, we introduce several new challenge tasks that probe whether state-of-theart QA models have general knowledge about word definitions and general taxonomic reasoning, both of which are fundamental to more complex forms of reasoning and are widespread in benchmark datasets. As an… CONTINUE READING

References

Publications referenced by this paper.
SHOWING 1-10 OF 54 REFERENCES

Enhanced LSTM for Natural Language Inference

  • ACL
  • 2016
VIEW 9 EXCERPTS
HIGHLY INFLUENTIAL

Language Models as Knowledge Bases?

  • EMNLP/IJCNLP
  • 2019
VIEW 4 EXCERPTS
HIGHLY INFLUENTIAL