On Sensitivity of Deep Learning Based Text Classification Algorithms to Practical Input Perturbations

  title={On Sensitivity of Deep Learning Based Text Classification Algorithms to Practical Input Perturbations},
  author={Aamir Miyajiwala and Arnav Ladkat and Samiksha Jagadale and Raviraj Joshi},
Text classification is a fundamental Natural Language Processing task that has a wide variety of applications, where deep learning approaches have produced state-of-the-art results. While these models have been heavily criticized for their black-box nature, their robustness to slight perturbations in input text has been a matter of concern. In this work, we carry out a data-focused study evaluating the impact of systematic practical perturbations on the performance of the deep learning based… 

TabText: a Systematic Approach to Aggregate Knowledge Across Tabular Data Structures

tabText, a methodology that leverages the unstructured data format of language to encode efficiently and accurately tabular data from different table structures and time periods is presented.



ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models

It is shown that simple models based on CNN or LSTM as well as complex models like BERT are questionable in terms of their syntactic and semantic understanding.

Deep Learning for Hindi Text Classification: A Comparison

Translated versions of English data-sets are used to evaluate models based on CNN, LSTM and Attention for text classification of Hindi text and serve as a tutorial for popular text classification techniques.

Experimental Evaluation of Deep Learning models for Marathi Text Classification

It is shown that basic single layer models based on CNN and LSTM coupled with FastText embeddings perform on par with the BERT based models on the available datasets and is hoped this paper aids focused research and experiments in the area of Marathi NLP.

A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification

A sensitivity analysis of one-layer CNNs is conducted to explore the effect of architecture components on model performance; the aim is to distinguish between important and comparatively inconsequential design decisions for sentence classification.

Evaluating the Robustness of Neural Language Models to Input Perturbations

This study designs and implements various types of character-level and word-level perturbation methods to simulate realistic scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained.

Assessing Robustness of Text Classification through Maximal Safe Radius Computation

This paper focuses on robustness of text classification against word substitutions, aiming to provide guarantees that the model prediction does not change if a word is replaced with a plausible alternative, such as a synonym.

Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

TextFooler is presented, a simple but strong baseline to generate adversarial text that outperforms previous attacks by success rate and perturbation rate, and is utility-preserving and efficient, which generates adversarialtext with computational complexity linear to the text length.

Black-Box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers

A novel algorithm is presented, DeepWordBug, to effectively generate small text perturbations in a black-box setting that forces a deep-learning classifier to misclassify a text input.

A C-LSTM Neural Network for Text Classification

C-LSTM is a novel and unified model for sentence representation and text classification that outperforms both CNN and LSTM and can achieve excellent performance on these tasks.

The influence of preprocessing on text classification using a bag-of-words representation

An extensive and systematic set of TC experiments were performed to explore the impact of all possible combinations of five/six basic preprocessing methods on four benchmark text corpora using three ML methods and training and test sets and the general conclusion is that it is always advisable to perform an extensive and systematically variety of pre processing methods combined with TC experiments to improve TC accuracy.