Answering Comparative Questions: Better than Ten-Blue-Links?

@article{Schildwchter2019AnsweringCQ,
  title={Answering Comparative Questions: Better than Ten-Blue-Links?},
  author={Matthias Schildw{\"a}chter and Alexander Bondarenko and Julian Zenker and Matthias Hagen and Chris Biemann and Alexander Panchenko},
  journal={Proceedings of the 2019 Conference on Human Information Interaction and Retrieval},
  year={2019}
}
We present CAM (comparative argumentative machine), a novel open-domain IR system to argumentatively compare objects with respect to information extracted from the Common Crawl. In a user study, the participants obtained 15% more accurate answers using CAM compared to a "traditional" keyword-based search and were 20% faster in finding the answer to comparative questions. 

Figures from this paper

Answering Comparative Questions with Web-based Arguments

CAM (comparative argumentative machine), an open-domain information retrieval system to argumentatively compare options using information extracted from the Common Crawl, is proposed, which was 15% more accurate and 20% faster at answering comparative questions of the type “How does X compare to Y with respect to Z?” than when using standard keyword-based search.

Question Answering for Comparative Questions with GPT-2

This work tries to leverage language models like GPT-2 by generating substitute answer articles conditioned on the given query, which are then re-ranked by their textual similarity to the generated substitute answerarticles.

Answering Comparative Questions with Arguments

The DFG-funded project “ACQuA: Answering Comparative Questions with Arguments” is described, which describes the achieved results so far, and ongoing activities like the first shared task on argument retrieval.

An Open-Domain Web Search Engine for Answering Comparative Questions

We present an open-domain web search engine that can help answer comparative questions like ”Is X better than Y for Z?” by providing argumentative documents. Building such a system requires multiple

Towards Understanding and Answering ComparativeQuestions

This paper develops an approach that detects the stances of potential result nuggets and develops approaches to extract the mentioned comparison objects and aspects from existing datasets that are labelled as comparative or not.

A User Study on Clarifying Comparative Questions

In an experiment that reflects a conversational search interface with a clarification component, 70% of the study participants find clarifications useful to retrieve relevant results for questions with unclear comparison aspects.

Towards Understanding and Answering Comparative Questions

This paper develops an approach that detects the stances of potential result nuggets and develops approaches to extract the mentioned comparison objects and aspects from existing datasets that are classified as comparative or not.

Comparative Web Search Questions

A precision-oriented classifier is developed by combining carefully hand-crafted lexico-syntactic rules with feature-based and neural approaches and an important insight is that reliable direct answers to comparative questions require more than the facts from a search engine's knowledge graph.

Categorization of Comparative Sentences for Argument Mining

This work presents the first publicly available open-domain dataset for CAM, which is the automatic extraction of direct comparisons from text and paves the way for domain-independent comparison extraction from web-scale corpora for the use in result ranking and presentation for comparative queries.

Retrieving Comparative Arguments using Deep Language Models

This paper uses a model trained on a question-answering dataset MSMARCO, with the proposed weights and weights pre-trained by the team Katana, and customize ColBERT for the comparative retrieval domain by fine-tuning the model on the data from the previous years’ Touché competitions.

References

SHOWING 1-10 OF 19 REFERENCES

ArgumenText: Searching for Arguments in Heterogeneous Sources

This paper presents an argument retrieval system capable of retrieving sentential arguments for any given controversial topic, and finds that its system covers 89% of arguments found in expert-curated lists of arguments from an online debate portal, and also identifies additional valid arguments.

Understanding and Detecting Supporting Arguments of Diverse Types

Experimental results show that LambdaMART (Burges, 2010) ranker that uses features informed by argument types yields better performance than the same ranker trained without type information.

Categorization of Comparative Sentences for Argument Mining

This work presents the first publicly available open-domain dataset for CAM, which is the automatic extraction of direct comparisons from text and paves the way for domain-independent comparison extraction from web-scale corpora for the use in result ranking and presentation for comparative queries.

Building an Argument Search Engine for the Web

An argument search framework for studying how people query for arguments, how to mine arguments from the web, or how to rank them is developed and a prototype search engine is built that relies on an initial, freely accessible index of nearly 300k arguments crawled from reliable web resources.

What is the Essence of a Claim? Cross-Domain Claim Identification

While the divergent conceptualization of claims in different datasets is indeed harmful to cross-domain classification, it is shown that there are shared properties on the lexical level as well as system configurations that can help to overcome these gaps.

CWS: a comparative web search system

A system called CWS is developed to effectively facilitate Web users' comparison needs and developed a novel interface which supports two types of view modes: a pair-view which displays the result in the page level, and a cluster- view which organizes the comparative pages into the themes and displays the extracted phrases to facilitate users' compare.

Building a Web-Scale Dependency-Parsed Corpus from CommonCrawl

An index of all sentences and their linguistic meta-data enabling quick search across the corpus is built, demonstrating the utility of this corpus on the verb similarity task by showing that a distributional model trained on the corpus yields better results than models trained on smaller corpora, like Wikipedia.

Social Media Argumentation Mining: The Quest for Deliberateness in Raucousness

The motivation for social media argumentation mining, as well as the tasks and challenges involved, are discussed.

Mining Comparative Sentences and Relations

This paper proposes two novel techniques based on two new types of sequential rules to perform the tasks of comparative sentence mining and results show that these techniques are very promising.

What works and what does not: Classifier and feature analysis for argument mining

Although the performance of classifiers varies depending on the feature combinations and corpora used for training and testing, Random Forest seems to be among the best performing classifiers.