A Simple Method for Commonsense Reasoning

@article{Trinh2018ASM,
  title={A Simple Method for Commonsense Reasoning},
  author={Trieu H. Trinh and Quoc V. Le},
  journal={ArXiv},
  year={2018},
  volume={abs/1806.02847}
}
Commonsense reasoning is a long-standing challenge for deep learning. For example, it is difficult to use neural networks to tackle the Winograd Schema dataset~\cite{levesque2011winograd}. In this paper, we present a simple method for commonsense reasoning with neural networks, using unsupervised learning. Key to our method is the use of language models, trained on a massive amount of unlabled data, to score multiple choice questions posed by commonsense reasoning tests. On both Pronoun… CONTINUE READING

Figures, Tables, Results, and Topics from this paper.

Key Quantitative Results

  • On a Pronoun Disambiguation dataset, PDP-60, our method achieves 70.0% accuracy, which is better than the state-of-art accuracy of 66.7%. On a Winograd Schema dataset, WSC-273, our method achieves 63.7% accuracy, 11% above that of the current state-of-art result (52.8%)2 A unique feature of Winograd Schema questions is the presence of a special word that decides the correct reference choice.
  • Remarkably on the later benchmark, we are able to achieve 63.7% accuracy, comparing to 52.8% accuracy of the previous state-of-the-art, who utilizes supervised learning and expensively annotated knowledge bases.

Citations

Publications citing this paper.
SHOWING 1-10 OF 21 CITATIONS

On the Evaluation of Common-Sense Reasoning in Natural Language Understanding

VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Language Models are Unsupervised Multitask Learners

VIEW 3 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

References

Publications referenced by this paper.
SHOWING 1-10 OF 37 REFERENCES

The Winograd Schema Challenge

  • AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning
  • 2011
VIEW 8 EXCERPTS
HIGHLY INFLUENTIAL

Long Short-Term Memory

  • Neural Computation
  • 1997
VIEW 8 EXCERPTS
HIGHLY INFLUENTIAL

Attention Is All You Need

VIEW 2 EXCERPTS
HIGHLY INFLUENTIAL

State-of-the-Art Speech Recognition with Sequence-to-Sequence Models

  • 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2017
VIEW 1 EXCERPT

Similar Papers

Loading similar papers…