Adversarial Augmentation Policy Search for Domain and Cross-Lingual Generalization in Reading Comprehension

  title={Adversarial Augmentation Policy Search for Domain and Cross-Lingual Generalization in Reading Comprehension},
  author={Adyasha Maharana and Mohit Bansal},
Reading comprehension models often overfit to nuances of training datasets and fail at adversarial evaluation. Training with adversarially augmented dataset improves robustness against those adversarial attacks but hurts generalization of the models. In this work, we present several effective adversaries and automated data augmentation policy search methods with the goal of making reading comprehension models more robust to adversarial evaluation, but also improving generalization to the source… 
Unseen Target Stance Detection with Adversarial Domain Generalization
  • Zhen Wang
  • Computer Science
    2020 International Joint Conference on Neural Networks (IJCNN)
  • 2020
This study investigates the domain difference between targets and thus incorporate attention-based conditional encoding with adversarial domain generalization to perform unseen target stance detection and achieves new state-of-the-art performance on the SemEval-2016 dataset.
Are Multilingual BERT models robust? A Case Study on Adversarial Attacks for Multilingual Question Answering
This paper is the first to explore and successfully attack a multilingual QA (MLQA) system pre-trained on multilingual BERT using several attack strategies for the adversarial statement reducing performance by as much as 85%.
A survey of methods for revealing and overcoming weaknesses of data-driven Natural Language Understanding
This structured survey provides an overview of the evolving research area by categorising reported weaknesses in models and datasets and the methods proposed to reveal and alleviate those weaknesses for the English language.
Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection
This paper proposes to automatically identify and reduce spurious correlations using attribution methods with dynamic refinement of the list of terms that need to be regularized during training and improves the cross-corpora performance over previous work independently and in combination with pre-defined dictionaries.
Using calibrator to improve robustness in Machine Reading Comprehension
A method to improve the robustness by using a calibrator as the post-hoc reranker, which is implemented based on XGBoost model, which can achieve performance improvement by more than 10% and also make improvement on the original and generalization datasets.
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
An empirical survey of recent progress on data augmentation for NLP in the limited labeled data setting is provided, summarizing the landscape of methods and carrying out experiments on 11 datasets covering topics/news classification, inference tasks, paraphrasing tasks, and single-sentence tasks.
ContraQA: Question Answering under Contradicting Contexts
A misinformation-aware QA system is built as a counter-measure that integrates question answering and misinformation detection in a joint fashion to defend against the threat of misinformation.
Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020
Unsupervised Few-Bits Semantic Hashing with Implicit Topics Modeling and Grid Tagging Scheme for End-to-End Fine-grained Opinion Extraction.


Robust Machine Comprehension Models via Adversarial Training
A novel alternative adversary-generation algorithm is proposed, AddSentDiverse, that significantly increases the variance within the adversarial training data by providing effective examples that punish the model for making certain superficial assumptions, and can make a state-of-the-art model significantly more robust.
Improving Machine Reading Comprehension via Adversarial Training
Virtual adversarial training (VAT) is applied on machine reading comprehension (MRC) and the possibility of improving the RC models with semi-supervised learning is explored and it is proved that examples from a different task are also beneficial.
Adversarial Domain Adaptation for Machine Reading Comprehension
An Adversarial Domain Adaptation framework for Machine Reading Comprehension (MRC), where pseudo questions are first generated for unlabeled passages in the target domain, and then a domain classifier is incorporated into an MRC model to predict which domain a given passage-question pair comes from.
Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models
We present two categories of model-agnostic adversarial strategies that reveal the weaknesses of several generative, task-oriented dialogue models: Should-Not-Change strategies that evaluate
Generating Natural Adversarial Examples
This paper proposes a framework to generate natural and legible adversarial examples that lie on the data manifold, by searching in semantic space of dense and continuous data representation, utilizing the recent advances in generative adversarial networks.
Generating Natural Language Adversarial Examples
A black-box population-based optimization algorithm is used to generate semantically and syntactically similar adversarial examples that fool well-trained sentiment analysis and textual entailment models with success rates of 97% and 70%, respectively.
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency
A new word replacement order determined by both the wordsaliency and the classification probability is introduced, and a greedy algorithm called probability weighted word saliency (PWWS) is proposed for text adversarial attack.
Adversarial Examples for Evaluating Reading Comprehension Systems
This work proposes an adversarial evaluation scheme for the Stanford Question Answering Dataset that tests whether systems can answer questions about paragraphs that contain adversarially inserted sentences without changing the correct answer or misleading humans.
Explaining and Harnessing Adversarial Examples
It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.
Domain-agnostic Question-Answering with Adversarial Training
An adversarial training framework for domain generalization in Question Answering (QA) task is utilized, where the two models constantly compete, so that QA model can learn domain-invariant features.