Easy Questions First? A Case Study on Curriculum Learning for Question Answering

@inproceedings{Sachan2016EasyQF,
  title={Easy Questions First? A Case Study on Curriculum Learning for Question Answering},
  author={Mrinmaya Sachan and Eric P. Xing},
  booktitle={ACL},
  year={2016}
}
Cognitive science researchers have emphasized the importance of ordering a complex task into a sequence of easy to hard problems. [] Key Method We introduce a number of heuristics that improve upon selfpaced learning. Then, we argue that incorporating easy, yet, a diverse set of samples can further improve learning. We compare these curriculum learning proposals in the context of four non-convex models for QA and show that they lead to real improvements in each of them.

Figures and Tables from this paper

Curriculum Learning: A Survey
TLDR
This survey shows how limits have been tackled in the literature, and presents curriculum learning instantiations for various tasks in machine learning, and constructs a multi-perspective clustering algorithm, linking the discovered clusters with the taxonomy.
Training Curricula for Open Domain Answer Re-Ranking
TLDR
This work proposes several heuristics to estimate the difficulty of a given training sample and shows that this approach leads to superior performance of two leading neural ranking architectures, namely BERT and ConvKNRM, using both pointwise and pairwise losses.
Curriculum Learning Strategies for IR
TLDR
The task of conversation response ranking: ranking responses given the conversation history is resorted to, which finds that by just intelligently sorting the training data, by performing curriculum learning, can improve the retrieval effectiveness by up to 2%.
Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
TLDR
The task of conversation response ranking: ranking responses given the conversation history is resorted to, which finds that by just intelligently sorting the training data (i.e., by performing curriculum learning) it can improve the retrieval effectiveness by up to 2%.
A Competence-aware Curriculum for Visual Concepts Learning via Question Answering
TLDR
This work designs a neural-symbolic concept learner for learning the visual concepts and a multi-dimensional Item Response Theory (mIRT) model for guiding the learning process with an adaptive curriculum.
Learning by Asking Questions
TLDR
This work presents a model that performs LBA on the CLEVR dataset, and shows that it automatically discovers an easy-to-hard curriculum when learning interactively from an oracle and asks questions that generalize to state-of-the-art VQA models and to novel test time distributions.
Curriculum Learning for Natural Answer Generation
TLDR
A curriculum learning based framework for natural answer generation (CL-NAG), which is able to take full advantage of the valuable learning data from a noisy and uneven-quality corpus, and outperforms the state-of-the-arts.
Curriculum Learning for Natural Language Understanding
TLDR
By reviewing the trainset in a crossed way, this work is able to distinguish easy examples from difficult ones, and arrange a curriculum for language models, and obtains significant and universal performance improvements on a wide range of NLU tasks.
Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives 1
TLDR
A curriculum learning (CL) based Pointer-Generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty, and a new Introspective Alignment Layer (IAL), which reasons over decomposed alignments using block-based self-attention.
Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives
TLDR
A curriculum learning (CL) based Pointer-Generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty, and a new Introspective Alignment Layer (IAL), which reasons over decomposed alignments using block-based self-attention.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 48 REFERENCES
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
TLDR
This work argues for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering, and classify these tasks into skill sets so that researchers can identify (and then rectify) the failings of their systems.
Curriculum learning
TLDR
It is hypothesized that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
My Computer Is an Honor Student - but How Intelligent Is It? Standardized Tests as a Measure of AI
TLDR
It is argued that machine performance on standardized tests should be a key component of any new measure of AI, because attaining a high level of performance requires solving significant AI problems involving language understanding and world modeling - critical skills for any machine that lays claim to intelligence.
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
TLDR
MCTest is presented, a freely available set of stories and associated questions intended for research on the machine comprehension of text that requires machines to answer multiple-choice reading comprehension questions about fictional stories, directly tackling the high-level goal of open-domain machine comprehension.
Learning Answer-Entailing Structures for Machine Comprehension
TLDR
A unified max-margin framework is presented that learns to find hidden structures that explain the relation between the question, correct answer, and text, and is extended to incorporate multi-task learning on the different subtasks that are required to perform machine comprehension.
Self-Paced Curriculum Learning
TLDR
The missing link between CL and SPL is discovered, and a unified framework named self-paced curriculum leaning (SPCL) is proposed, formulated as a concise optimization problem that takes into account both prior knowledge known before training and the learning progress during training.
Self-Paced Learning for Latent Variable Models
TLDR
A novel, iterative self-paced learning algorithm where each iteration simultaneously selects easy samples and learns a new parameter vector that outperforms the state of the art method for learning a latent structural SVM on four applications.
Self-Paced Learning with Diversity
TLDR
This work proposes an approach called self-paced learning with diversity (SPLD) which formalizes the preference for both easy and diverse samples into a general regularization term, independent of the learning objective, and thus can be easily generalized into various learning tasks.
Active Learning Literature Survey
TLDR
This report provides a general introduction to active learning and a survey of the literature, including a discussion of the scenarios in which queries can be formulated, and an overview of the query strategy frameworks proposed in the literature to date.
Question Answering Using Enhanced Lexical Semantic Models
TLDR
This work focuses on improving the performance using models of lexical semantic resources and shows that these systems can be consistently and significantly improved with rich lexical semantics information, regardless of the choice of learning algorithms.
...
1
2
3
4
5
...