Multi-hop Inference for Sentence-level TextGraphs: How Challenging is Meaningfully Combining Information for Science Question Answering?

@article{Jansen2018MultihopIF,
  title={Multi-hop Inference for Sentence-level TextGraphs: How Challenging is Meaningfully Combining Information for Science Question Answering?},
  author={Peter Alexander Jansen},
  journal={ArXiv},
  year={2018},
  volume={abs/1805.11267}
}
Question Answering for complex questions is often modelled as a graph construction or traversal task, where a solver must build or traverse a graph of facts that answer and explain a given question. This “multi-hop” inference has been shown to be extremely challenging, with few models able to aggregate more than two facts before being overwhelmed by “semantic drift”, or the tendency for long chains of facts to quickly drift off topic. This is a major barrier to current inference models, as even… 

Figures and Tables from this paper

Extracting Common Inference Patterns from Semi-Structured Explanations
TLDR
This work presents a prototype tool for identifying common inference patterns from corpora of semi-structured explanations, and uses it to successfully extract 67 inference patternsfrom a “matter” subset of standardized elementary science exam questions that span scientific and world knowledge.
WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference
TLDR
This work presents the second iteration of the WorldTree project, a corpus of 5,114 standardized science exam questions paired with large detailed multi-fact explanations that combine core scientific knowledge and world knowledge, and uses this explanation corpus to author a set of 344 high-level science domain inference patterns similar to semantic frames supporting multi-hop inference.
TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration
TLDR
The Shared Task on Multi-Hop Inference for Explanation Regeneration tasks participants with regenerating detailed gold explanations for standardized elementary science exam questions by selecting facts from a knowledge base of semi-structured tables.
QASC: A Dataset for Question Answering via Sentence Composition
TLDR
This work presents a multi-hop reasoning dataset, Question Answering via Sentence Composition (QASC), that requires retrieving facts from a large corpus and composing them to answer a multiple-choice question, and provides annotation for supporting facts as well as their composition.
Explainable Inference Over Grounding-Abstract Chains for Science Questions
TLDR
This paper frames question answering as a natural language abductive reasoning problem, constructing plausible explanations for each candidate answer and then selecting the candidate with the best explanation as the final answer by employing a linear programming formalism.
PGL at TextGraphs 2020 Shared Task: Explanation Regeneration using Language and Graph Learning Methods
TLDR
This paper describes the system designed by the Baidu PGL Team which achieved the first place in the TextGraphs 2020 Shared Task, which uses a pre-trained language model to recall the top-K relevant explanations for each question and develops an architecture consisting both powerful pre- trained transformers and GNNs to tackle the multi-hop inference problem.
A Survey on Multi-hop Question Answering and Generation
TLDR
A general and formal definition of MHQA task is provided, the existing attempts to this highly interesting, yet quite challenging task are summarized, and the best methods to createMHQA datasets are outlined.
If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering
TLDR
This work proposes a simple approach that retrieves and reranks set of evidence facts jointly and shows that jointly retrieving candidate evidence leads to substantially higher evidence retrieval performance when fed to the same supervised reranker.
ExplanationLP: Abductive Reasoning for Explainable Science Question Answering
TLDR
A novel approach for answering and explaining multiple-choice science questions by reasoning on grounding and abstract inference chains that elicits explanations by constructing a weighted graph of relevant facts for each candidate answer and extracting the facts that satisfy certain structural and semantic constraints.
Understanding Dataset Design Choices for Multi-hop Reasoning
TLDR
This paper investigates two recently proposed datasets, WikiHop and HotpotQA, and explores sentence-factored models for these tasks; by design, these models cannot do multi-hop reasoning, but they are still able to solve a large number of examples in both datasets.
...
1
2
3
...

References

SHOWING 1-10 OF 13 REFERENCES
WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-Hop Inference
TLDR
A corpus of explanations for standardized science exams, a recent challenge task for question answering, is presented and an explanation-centered tablestore is provided, a collection of semi-structured tables that contain the knowledge to construct these elementary science explanations.
Higher-order Lexical Semantic Models for Non-factoid Answer Reranking
TLDR
This work introduces a higher-order formalism that allows all these lexical semantic models to chain direct evidence to construct indirect associations between question and answer texts, by casting the task as the traversal of graphs that encode direct term associations.
Tables as Semi-structured Knowledge for Question Answering
TLDR
This paper first uses the structure of tables to guide the construction of a dataset of over 9000 multiple-choice questions with rich alignment annotations, then uses this annotated data to train a semi-structured feature-driven model for question answering that uses tables as a knowledge base.
Question Answering via Integer Programming over Semi-Structured Knowledge
TLDR
This work proposes a structured inference system for this task, formulated as an Integer Linear Program (ILP), that answers natural language questions using a semi-structured knowledge base derived from text, including questions requiring multi-step inference and a combination of multiple facts.
Controlling Information Aggregation for Complex Question Answering
TLDR
Empirical evaluation on an elementary science exam benchmark shows that the proposed methods enables effective aggregation even over larger graphs and demonstrates the complementary value of information aggregation for answering complex questions.
Constructing Datasets for Multi-hop Reading Comprehension Across Documents
TLDR
A novel task to encourage the development of models for text understanding across multiple documents and to investigate the limits of existing methods, in which a model learns to seek and combine evidence — effectively performing multihop, alias multi-step, inference.
Answering Complex Questions Using Open Information Extraction
TLDR
This work develops a new inference model for Open IE that can work effectively with multiple short facts, noise, and the relational structure of tuples, and significantly outperforms a state-of-the-art structured solver on complex questions of varying difficulty.
Structural relationships for large-scale learning of answer re-ranking
TLDR
Novel supervised approaches that exploit structural relationships between a question and their candidate answer passages to learn a re-ranking model are defined, which shows that, despite its simplicity, shallow syntactic trees allow for learning complex relational structures, which exhibits a steep learning curve with the increase in the training size.
What’s in an Explanation? Characterizing Knowledge and Inference Requirements for Elementary Science Exams
TLDR
This work develops an explanation-based analysis of knowledge and inference requirements, which supports a fine-grained characterization of the challenges, and compares a retrieval and an inference solver on 212 questions.
Framing QA as Building and Ranking Intersentence Answer Justifications
TLDR
A question answering approach for standardized science exams that both identifies correct answers and produces compelling human-readable justifications for why those answers are correct is proposed, and it is shown that information aggregation is key to addressing the information need in complex questions.
...
1
2
...