SemEval-2019 Task 10: Math Question Answering

@inproceedings{Hopkins2019SemEval2019T1,
  title={SemEval-2019 Task 10: Math Question Answering},
  author={Mark Hopkins and Ronan Le Bras and Cristian Petrescu-Prahova and Gabriel Stanovsky and Hannaneh Hajishirzi and Rik Koncel-Kedziorski},
  booktitle={International Workshop on Semantic Evaluation},
  year={2019}
}
We report on the SemEval 2019 task on math question answering. [] Key Method For a significant subset of these questions, we also provided SMT-LIB logical form annotations and an interpreter that could solve these logical forms. Systems were evaluated based on the percentage of correctly answered questions. The top system correctly answered 45% of the test questions, a considerable improvement over the 17% random guessing baseline.
View on ACL
aclanthology.org
22 Citations
Highly Influential Citations
2
Background Citations
12
Methods Citations
7

Figures and Tables from this paper

22 Citations

ProblemSolver at SemEval-2019 Task 10: Sequence-to-Sequence Learning and Expression Trees

A dual-pronged approach was used, building a Sequence-to-Sequence Neural Network pre-trained with augmented data that could answer all categories of questions and a Tree system, which can only answer a certain type of questions.

Finding Old Answers to New Math Questions: The ARQMath Lab at CLEF 2020

The ARQMath Lab at CLEF 2020 considers the problem of finding answers to new mathematical questions among posted answers on a community question answering site (Math Stack Exchange), and creates a standard test collection for researchers to use for benchmarking.

Answer Retrieval for Questions on Math

The ARQMath Lab at CLEF considers finding answers to new mathematical questions among posted answers on a community question answering site (Math Stack Exchange). Queries are question posts held out…

Plans for the ARQMath Lab at CLEF 2020

The ARQMath Lab at CLEF 2020 considers the problem of finding answers to new mathematical questions among posted answers on a community question answering site (Math Stack Exchange), and creates a standard test collection for researchers to use for benchmarking.

Overview of ARQMath 2020 (Updated Working Notes Version): CLEF Lab on Answer Retrieval for Questions on Math

The ARQMath Lab at CLEF considers finding answers to new mathematical questions among posted answers on a community question answering site (Math Stack Exchange), which includes a formula retrieval sub-task.

AiFu at SemEval-2019 Task 10: A Symbolic and Sub-symbolic Integrated System for SAT Math Question Answering

This paper is to describe how AiFu works technically and to report and analyze some essential experimental results.

Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)

This paper provides an overview of the third and final year of the Answer Retrieval for Questions on Math (ARQMath-3) lab, run as part of CLEF 2022. ARQMath has aimed to introduce test collections…

Mining Mathematical Documents for Question Answering via Unsupervised Formula Labeling

This paper presents data mining methods and benchmark results to employ Mathematical Entity Linking and Unsupervised Formula Labeling for semantic formula search and mathematical question answering (MathQA) on the arXiv preprint repository, Wikipedia, and Wikidata.

Overview of ARQMath-2 (2021): Second CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)

An overview of the second year of the Answer Retrieval for Questions on Math (ARQMath-2) lab is provided, suggesting that some combination of experience with the task design and the training data available from ARQMath1 was beneficial, with greater improvements in ARQ Math-2 relative to baselines for both Task 1 and Task 2 than for ARZMath-1 relative to those same baselines.

NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions

The Natural Language for Optimization (NL4Opt) Competition was created to investigate methods of extracting the meaning and formulation of an optimization problem based on its text description by allowing non-experts to interface with them using natural language.

References

SHOWING 1-10 OF 21 REFERENCES

ProblemSolver at SemEval-2019 Task 10: Sequence-to-Sequence Learning and Expression Trees

A dual-pronged approach was used, building a Sequence-to-Sequence Neural Network pre-trained with augmented data that could answer all categories of questions and a Tree system, which can only answer a certain type of questions.

Beyond Sentential Semantic Parsing: Tackling the Math SAT with a Cascade of Tree Transducers

We present an approach for answering questions that span multiple sentences and exhibit sophisticated cross-sentence anaphoric phenomena, evaluating on a rich source of such questions – the math…

Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems

Experimental results show that indirect supervision of program learning via answer rationales is a promising strategy for inducing arithmetic programs.

AiFu at SemEval-2019 Task 10: A Symbolic and Sub-symbolic Integrated System for SAT Math Question Answering

This paper is to describe how AiFu works technically and to report and analyze some essential experimental results.

Solving Probability Problems in Natural Language

A two-step endto-end fully automated approach for solving probability word problems that is able to automatically provide answers to exercises about probability formulated in natural language is developed.

Learning to Automatically Solve Algebra Word Problems

An approach for automatically learning to solve algebra word problems by reasons across sentence boundaries to construct and solve a system of linear equations, while simultaneously recovering an alignment of the variables and numbers to the problem text.

The Most Uncreative Examinee: A First Step toward Wide Coverage Natural Language Math Problem Solving

Evaluation on entrance exam mock tests revealed that an optimistic estimate of the system’s performance already matches human averages on a few test sets, and a prototype system that accepts as its input a linguistically annotated problem text was developed.

MAWPS: A Math Word Problem Repository

MAWPS allows for the automatic construction of datasets with particular characteristics, providing tools for tuning the lexical and template overlap of a dataset as well as for filtering ungrammatical problems from web-sourced corpora.

Learning to Solve Arithmetic Word Problems with Verb Categorization

The paper analyzes the arithmetic-word problems “genre”, identifying seven categories of verbs used in such problems, and reports the first learning results on this task without reliance on predefined templates and makes the data publicly available.

Parsing Algebraic Word Problems into Equations

This paper formalizes the problem of solving multi-sentence algebraic word problems as that of generating and scoring equation trees. We use integer linear programming to generate equation trees and…