• DOI:10.48550/arXiv.2208.08750
• Corpus ID: 251643623

Exploring and Exploiting Multi-Granularity Representations for Machine Reading Comprehension

@article{Chen2022ExploringAE,
  title={Exploring and Exploiting Multi-Granularity Representations for Machine Reading Comprehension},
  author={Nuo Chen and Chenyu You},
  journal={ArXiv},
  year={2022},
  volume={abs/2208.08750}
}

• Nuo Chen, Chenyu You
• Published 18 August 2022
• Computer Science
• ArXiv

Recently, the attention-enhanced multi-layer encoder, such as Transformer, has been extensively studied in Machine Reading Comprehension (MRC). To predict the answer, it is common practice to employ a predictor to draw information only from the final encoder layer which generates the coarse-grained representations of the source sequences, i.e., passage and question. The analysis shows that the representation of source sequence becomes more coarse-grained from fine-grained as the encoding layer increases. It is generally believed that with the growing number of layers in deep neural networks, the encoding process will gather relevant information for each location increasingly, resulting in more coarse-grained representations, which adds the likeli-hood of similarity to other locations (referring to homogeneity). Such phenomenon will mislead the model to make wrong judgement and degrade the performance. In this paper, we argue that it would be better if the predictor could exploit representations of different granularity from the encoder, providing different views of the source sequences, such that the expressive power of the model could be fully utilized. To this end, we propose a novel approach called Adaptive Bidirectional Attention-Capsule Network (ABA-Net), which adaptively exploits the source representations of different levels to the predictor. Furthermore, due to the better representations are at the core for boosting MRC performance, the capsule network and self-attention module are carefully designed as the building blocks of our encoders, which provides the capability to explore the local and global representations, respectively. Experimental results on three benchmark datasets, i.e., SQuAD 1.0, SQuAD 2.0 and COQA, demonstrate the effectiveness of our approach. In particular, we set the new state-of-the-art performance on the SQuAD 1.0 dataset. source representations of different levels for question answering by dynamically cap-turing the connection between all layers of representations of each word. Furthermore, to explore better representations, we incorporate the capsule network and self-attention module as the building blocks of encoders. The experimental results show that our model achieves competitive performance compared to the state-of-the-art on three public benchmark datasets. The systematic analysis demonstrates the effectiveness of each component in our model and shows that our model can learn effective representations for MRC and is capable of understanding rich context and answering complex questions. Future work can