• Publications
  • Influence
Probing Natural Language Inference Models through Semantic Fragments
TLDR
We propose the use of semantic fragments---systematically generated datasets that each target a different semantic phenomenon---for probing, and efficiently improving, such capabilities of linguistic models. Expand
  • 28
  • 3
  • PDF
Transformers as Soft Reasoners over Language
TLDR
This paper investigates a modern approach to this problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. Expand
  • 19
  • 3
  • PDF
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project
TLDR
This paper reports unprecedented success on the Grade 8 New York Regents Science Exam, where for the first time a system scores more than 90% on the exam's multiple choice (NDMC) questions. Expand
  • 16
  • 2
  • PDF
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge
TLDR
We introduce a methodology for automatically building probe datasets from expert knowledge sources, allowing for systematic control for artifacts that may arise during this process. Expand
  • 12
  • 2
  • PDF
CLUE: A Chinese Language Understanding Evaluation Benchmark
TLDR
We introduce CLUE, a Chinese Language Understanding Evaluation benchmark that contains eight different natural language understanding tasks, including single-sentence classification, sentence pair classification, and machine reading comprehension. Expand
  • 8
  • 1
  • PDF
MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity
TLDR
We present a new logic-based inference engine for natural language inference (NLI) called MonaLog, which is based on natural logic and the monotonicity calculus. Expand
  • 7
  • PDF
Function Assistant: A Tool for NL Querying of APIs
TLDR
In this paper, we describe Function Assistant, a lightweight Python-based toolkit for querying and exploring source code repositories using natural language. Expand
  • 8
  • PDF
Polyglot Semantic Parsing in APIs
TLDR
We develop a novel graph-based decoding framework that achieves state-of-the-art performance on the above datasets, and apply this method to two other benchmark SP tasks. Expand
  • 7
  • PDF
Accessing Structured Health Information through English Queries and Automatic Deduction
TLDR
We describe ongoing research in using natural English text queries and automated deduction to obtain answers based on multiple structured data sources in a specific subject domain. Expand
  • 7
  • PDF