• Corpus ID: 227054413

An Experimental Study of Semantic Continuity for Deep Learning Models

  title={An Experimental Study of Semantic Continuity for Deep Learning Models},
  author={Shangxi Wu and Jitao Sang and Xian Zhao and Lizhang Chen},
Deep learning models suffer from the problem of semantic discontinuity: small perturbations in the input space tend to cause semantic-level interference to the model output. We argue that the semantic discontinuity results from these inappropriate training targets and contributes to notorious issues such as adversarial robustness, interpretability, etc. We first conduct data analysis to provide evidence of semantic discontinuity in existing deep learning models, and then design a simple… 


Learning Perceptually-Aligned Representations via Adversarial Robustness
This work identifies the pervasive brittleness of deep networks' learned representations as a fundamental barrier to attaining human-aligned models, and re-cast robust optimization as a tool for enforcing human priors on the features learned by deep neural networks.
Adversarial Robustness as a Prior for Learned Representations
This work shows that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks, and indicates adversarial robustness as a promising avenue for improving learned representations.
Does Interpretability of Neural Networks Imply Adversarial Robustness?
Two factors are identified---namely the suppression of the gradient's magnitude and the selective use of features guided by high-quality interpretations---which explain model behaviors under various regularization and target interpretation settings.
Adversarially-Trained Deep Nets Transfer Better
It is demonstrated that adversarially- trained models transfer better across new domains than naturally-trained models, even though it's known that these models do not generalize as well as naturally-training models on the source domain.
Explaining and Harnessing Adversarial Examples
It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.
Improving the Robustness of Deep Neural Networks via Stability Training
This paper presents a general stability training method to stabilize deep networks against small input distortions that result from various types of common image processing, such as compression, rescaling, and cropping.
A survey on Image Data Augmentation for Deep Learning
This survey will present existing methods for Data Augmentation, promising developments, and meta-level decisions for implementing DataAugmentation, a data-space solution to the problem of limited data.
Gradients of Counterfactuals
This work proposes to examine interior gradients, which are gradients of counterfactual inputs constructed by scaling down the original input, and applies this method to the GoogleNet architecture, as well as a ligand-based virtual screening network with categorical features and an LSTM based language model for the Penn Treebank dataset.
Data augmentation for improving deep learning in image classification problem
This paper has compared and analyzed multiple methods of data augmentation in the task of image classification, starting from classical image transformations like rotating, cropping, zooming, histogram based methods and finishing at Style Transfer and Generative Adversarial Networks, along with the representative examples.
Interpreting Adversarially Trained Convolutional Neural Networks
Surprisingly, it is found that adversarial training alleviates the texture bias of standard CNNs when trained on object recognition tasks, and helps CNNs learn a more shape-biased representation.