Evaluating Semantic Parsing against a Simple Web-based Question Answering Model

@article{Talmor2017EvaluatingSP,
  title={Evaluating Semantic Parsing against a Simple Web-based Question Answering Model},
  author={Alon Talmor and Mor Geva and Jonathan Berant},
  journal={ArXiv},
  year={2017},
  volume={abs/1707.04412}
}
Semantic parsing shines at analyzing complex natural language that involves composition and computation over multiple pieces of evidence. However, datasets for semantic parsing contain many factoid questions that can be answered from a single web document. In this paper, we propose to evaluate semantic parsing-based question answering models by comparing them to a question answering baseline that queries the web and extracts the answer only from web snippets, without access to the target… 
View on ACL
arxiv.org
11 Citations
Background Citations
3
Methods Citations
3

Figures and Tables from this paper

11 Citations

Citation Type

Citation Type

The Web as a Knowledge-Base for Answering Complex Questions
TLDR
This paper proposes to decompose complex questions into a sequence of simple questions, and compute the final answer from the sequence of answers, and empirically demonstrates that question decomposition improves performance from 20.8 precision@1 to 27.5 precision @1 on this new dataset.
A Survey of Question Answering over Knowledge Base
TLDR
A survey of KBQA approaches is given by classifying them in two categories, and current mainstream techniques in KBZA are introduced, and similarities and differences among them are discussed.
BEAT: Considering question types for bug question answering via templates
A Linguistic Approach to Short Query and Answer Systems
TLDR
The main contribution of this approach is the generalization of the system, which can answer questions related to multiple domains, and the results obtained in the classification of question types and identification of nominal complements indicate promising results.
Point-of-Interest Oriented Question Answering with Joint Inference of Semantic Matching and Distance Correlation
TLDR
A novel deep learning framework integrated with joint inference to capture both tag semantic and geographic correlation between question and POIs and a special cross attention question embedding neural network structure to obtain question- to-POI and POI-to-question information is presented.
Knowledge Graph and Semantic Computing: Knowledge Computing and Language Understanding: 4th China Conference, CCKS 2019, Hangzhou, China, August 24–27, 2019, Revised Selected Papers
TLDR
This paper proposes a cross-lingual entity linking framework with the help of more adaptive representations, and applies trainable adjusting matrices to fine-tune the semantic representations built from multilingual embeddings.
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension
TLDR
It is shown that training on a source RC dataset and transferring to a target dataset substantially improves performance, even in the presence of powerful contextual representations from BERT (Devlin et al., 2019).
A Survey of Document Grounded Dialogue Systems (DGDS)
TLDR
The classification, architecture, datasets, models, and future development trends of the DGDS are analyzed, hoping to help researchers in this field.
A Survey of Document Grounded Dialogue Systems (DGDS)
TLDR
This paper presents a meta-modelling system that automates the very labor-intensive and therefore time-heavy and expensive and therefore expensive and expensive process of social computing and information Retrieval.
Smart question answering system using vectorization approach and statistical scoring method
...
1
2
...

References

SHOWING 1-10 OF 30 REFERENCES
Semantic Parsing on Freebase from Question-Answer Pairs
TLDR
This paper trains a semantic parser that scales up to Freebase and outperforms their state-of-the-art parser on the dataset of Cai and Yates (2013), despite not having annotated logical forms.
Compositional Semantic Parsing on Semi-Structured Tables
TLDR
This paper proposes a logical-form driven parsing algorithm guided by strong typing constraints and shows that it obtains significant improvements over natural baselines and is made publicly available.
Large-scale Semantic Parsing without Question-Answer Pairs
TLDR
This paper introduces a novel semantic parsing approach to query Freebase in natural language without requiring manual annotations or question-answer pairs and converts sentences to semantic graphs using CCG and subsequently grounds them to Freebase guided by denotations as a form of weak supervision.
Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base
TLDR
This work proposes a novel semantic parsing framework for question answering using a knowledge base that leverages the knowledge base in an early stage to prune the search space and thus simplifies the semantic matching problem.
Freebase QA: Information Extraction or Semantic Parsing?
TLDR
It is concluded that the differences between these technologies, both in task performance, and in how they get there, is not significant, and suggests that the semantic parsing community should target answering more compositional open-domain questions that are beyond the reach of more direct information extraction methods.
Learning Dependency-Based Compositional Semantics
TLDR
A new semantic formalism, dependency-based compositional semantics (DCS) is developed and a log-linear distribution over DCS logical forms is defined and it is shown that the system obtains comparable accuracies to even state-of-the-art systems that do require annotated logical forms.
Scaling Semantic Parsers with On-the-Fly Ontology Matching
TLDR
A new semantic parsing approach that learns to resolve ontological mismatches, which is learned from question-answer pairs, uses a probabilistic CCG to build linguistically motivated logicalform meaning representations, and includes an ontology matching model that adapts the output logical forms for each target ontology.
Constraint-Based Question Answering with Knowledge Graph
TLDR
A novel systematic KBQA approach to solve multi-constraint questions is proposed, which not only obtains comparable results on the two existing benchmark data-sets, but also achieves significant improvements on the ComplexQuestions.
Scaling question answering to the web
TLDR
Mulder is introduced, which is believed to be the first general-purpose, fully-automated question-answering system available on the web, and its architecture is described, which relies on multiple search-engine queries, natural-language parsing, and a novel voting procedure to yield reliable answers coupled with high recall.
SQuAD: 100,000+ Questions for Machine Comprehension of Text
TLDR
A strong logistic regression model is built, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%).
...
1
2
3
...