Corpus ID: 127983438

Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension

@article{Sun2019ProbingPK,
  title={Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension},
  author={Kai Sun and Dian Yu and Dong Yu and Claire Cardie},
  journal={ArXiv},
  year={2019},
  volume={abs/1904.09679}
}
With an ultimate goal of narrowing the gap between human and machine readers in text comprehension, we present the first collection of Challenging Chinese machine reading Comprehension datasets (C^3) collected from language and professional certification exams, which contains 13,924 documents and their associated 23,990 multiple-choice questions. [...] Key Method We further explore how to leverage linguistic knowledge including a lexicon of idioms and proverbs, graphs of general world knowledge (e.g…Expand
A Sentence Cloze Dataset for Chinese Machine Reading Comprehension
TLDR
This paper proposes a new task called Sentence Cloze-style Machine Reading Comprehension (SC-MRC), which aims to fill the right candidate sentence into the passage that has several blanks, and builds a Chinese dataset called CMRC 2019 to evaluate the difficulty of the task. Expand
Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models
TLDR
Experimental results demonstrate that pre-training models using the proposed approach followed by fine-tuning achieve significant improvements over previous state-of-the-art models on two commonsense-related benchmarks, including CommonsenseQA and Winograd Schema Challenge. Expand
Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach
TLDR
This work proposes a simple yet effective method to teach pretrained models with commonsense reasoning by leveraging the structured knowledge in ConceptNet, the largest commonsense knowledge base (KB). Expand
BURT: BERT-inspired Universal Representation from Learning Meaningful Segment
TLDR
A universal representation model, BURT (BERT-inspired Universal Representation from learning meaningful segmenT), to encode different levels of linguistic unit into the same vector space, which significantly outperforms existing information retrieval methods and yields universal representations that can be directly applied to retrieval-based question-answering and natural language. Expand
ERNIE-DOC: The Retrospective Long-Document Modeling Transformer
TLDR
Two well-designed techniques, namely the retrospective feed mechanism and the enhanced recurrence mechanism enable ERNIE-DOC with much longer effective context length to capture the contextual information of a whole document. Expand
Cross-lingual Transferring of Pre-trained Contextualized Language Models
TLDR
This work proposes a novel cross-lingual model transferring framework for PrLMs: TRELM, and showcases an embedding aligning that adversarially adapts a PrLM's non-contextualized embedding space and the TRILayer structure to learn a text transformation network across languages, which addresses the vocabulary difference between languages. Expand
AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization
TLDR
This paper proposes a novel pre-trained language model, referred to as AMBERT (A Multi-grained BERT), on the basis of both fine- grained and coarse-grains tokenizations, which outperforms the existing best performing models in almost all cases. Expand
ICLR 2021 English English PrLM Chinese ChinesePrLM transfer transfer mPrLM mPrLM English Chinese TreLM
  • 2020
Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons: other languagesExpand
Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
TLDR
This technical report releases a lightweight model called Mengzi, which stands for a family of discriminative, generative, domain-specific, and multimodal pre-trained model variants, capable of a wide range of language and vision tasks, and has achieved new state-of-the-art results on the widely-used CLUE benchmark. Expand
Deep Neural Networks with Short Circuits for Improved Gradient Learning
TLDR
The proposed short circuit is a unidirectional connection that single back propagates the sensitive from the deep layer to the shallows, which can plug into the backbone deep neural networks without introducing external training parameters. Expand
...
1
2
...

References

SHOWING 1-10 OF 66 REFERENCES
Improving Machine Reading Comprehension with General Reading Strategies
TLDR
Three general strategies aimed to improve non-extractive machine reading comprehension (MRC) are proposed and the effectiveness of these proposed strategies and the versatility and general applicability of fine-tuned models that incorporate these strategies are demonstrated. Expand
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
TLDR
MCTest is presented, a freely available set of stories and associated questions intended for research on the machine comprehension of text that requires machines to answer multiple-choice reading comprehension questions about fictional stories, directly tackling the high-level goal of open-domain machine comprehension. Expand
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
TLDR
DuReader is a new large-scale, open-domain Chinese ma- chine reading comprehension (MRC) dataset, designed to address real-world MRC, and provides rich annotations for more question types that leaves plenty of room for the research community to make improvements. Expand
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
TLDR
The dataset is the first to study multi-sentence inference at scale, with an open-ended set of question types that requires reasoning skills, and finds human solvers to achieve an F1-score of 88.1%. Expand
A Span-Extraction Dataset for Chinese Machine Reading Comprehension
TLDR
This paper introduces a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities in this area and hosted the Second Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC 2018). Expand
RACE: Large-scale ReAding Comprehension Dataset From Examinations
TLDR
The proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension, and there is a significant gap between the performance of the state-of-the-art models and the ceiling human performance. Expand
Broad Context Language Modeling as Reading Comprehension
TLDR
This work views LAMBADA as a reading comprehension problem and applies comprehension models based on neural networks, finding that neural network readers perform well in cases that involve selecting a name from the context based on dialogue or discourse cues but struggle when coreference resolution or external knowledge is needed. Expand
The NarrativeQA Reading Comprehension Challenge
TLDR
A new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts are presented, designed so that successfully answering their questions requires understanding the underlying narrative rather than relying on shallow pattern matching or salience. Expand
Constructing Datasets for Multi-hop Reading Comprehension Across Documents
TLDR
A novel task to encourage the development of models for text understanding across multiple documents and to investigate the limits of existing methods, in which a model learns to seek and combine evidence — effectively performing multihop, alias multi-step, inference. Expand
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
TLDR
This new dataset is aimed to overcome a number of well-known weaknesses of previous publicly available datasets for the same task of reading comprehension and question answering, and is the most comprehensive real-world dataset of its kind in both quantity and quality. Expand
...
1
2
3
4
5
...