Dense Associative Memory Is Robust to Adversarial Inputs

@article{Krotov2018DenseAM,
  title={Dense Associative Memory Is Robust to Adversarial Inputs},
  author={Dmitry Krotov and John J. Hopfield},
  journal={Neural Computation},
  year={2018},
  volume={30},
  pages={3151-3167}
}
Deep neural networks (DNNs) trained in a supervised way suffer from two known problems. First, the minima of the objective function used in learning correspond to data points (also known as rubbish examples or fooling images) that lack semantic similarity with the training data. Second, a clean input can be changed by a small, and often imperceptible for human vision, perturbation so that the resulting deformed input is misclassified by the network. These findings emphasize the differences… 
A DVERSARIAL S PHERES
TLDR
A simple synthetic dataset of classifying between two concentric high dimensional spheres is studied and it is proved that any model which misclassifies a small constant fraction of a sphere will be vulnerable to adversarial perturbations of sizeO(1/ √ d).
Adversarial Spheres
TLDR
A fundamental tradeoff between the amount of test error and the average distance to nearest error is shown, which proves that any model which misclassifies a small constant fraction of a sphere will be vulnerable to adversarial perturbations of size O(1/\sqrt{d})$.
The Relationship Between High-Dimensional Geometry and Adversarial Examples
TLDR
A fundamental bound relating the classification error rate to the average distance to the nearest misclassification, which is independent of the model is shown, which will point a way forward to explore how the geometry of complex real-world data sets leads to adversarial examples.
Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
TLDR
This work takes inspiration from a popular framework in neuroscience: “predictive coding”, and provides an open-sourced PyTorch-based package called Predify, which can be used to implement and investigate the impacts of the predictive coding dynamics in any convolutional neural network.
Improving adversarial robustness of deep neural networks by using semantic information
Adversarial Examples on Object Recognition
TLDR
The hypotheses behind their existence, the methods used to construct or protect against them, and the capacity to transfer adversarial examples between different machine learning models are introduced.
Adversarial Examples on Object Recognition: A Comprehensive Survey
TLDR
The hypotheses behind their existence, the methods used to construct or protect against them, and the capacity to transfer adversarial examples between different machine learning models are introduced to provide a comprehensive and self-contained survey of this growing field of research.
Threat of Adversarial Attacks on Deep Learning in Computer Vision: Survey II
TLDR
A literature review of the contributions made by the computer vision community in adversarial attacks on deep learning until the advent of year 2018, which focuses on the advances in this area since 2018.
On the Adversarial Robustness of Vision Transformers
TLDR
This work provides the first and comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations and finds that ViTs possess better adversarial robustness when compared with convolutional neural networks (CNNs).
DNDNet: Reconfiguring CNN for Adversarial Robustness
TLDR
A novel "defense layer" in a network which aims to block the generation of adversarial noise and prevents an adversarial attack in black-box and gray-box settings is presented.
...
...

References

SHOWING 1-10 OF 18 REFERENCES
Towards Deep Neural Network Architectures Robust to Adversarial Examples
TLDR
Deep Contractive Network is proposed, a model with a new end-to-end training procedure that includes a smoothness penalty inspired by the contractive autoencoder (CAE) to increase the network robustness to adversarial examples, without a significant performance penalty.
Intriguing properties of neural networks
TLDR
It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
TLDR
This work takes convolutional neural networks trained to perform well on either the ImageNet or MNIST datasets and finds images with evolutionary algorithms or gradient ascent that DNNs label with high confidence as belonging to each dataset class, and produces fooling images, which are then used to raise questions about the generality of DNN computer vision.
Explaining and Harnessing Adversarial Examples
TLDR
It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.
Distributional Smoothing with Virtual Adversarial Training
TLDR
When the LDS based regularization was applied to supervised and semi-supervised learning for the MNIST dataset, it outperformed all the training methods other than the current state of the art method, which is based on a highly advanced generative model.
Improving Back-Propagation by Adding an Adversarial Gradient
TLDR
First experimental results on MNIST show that the "adversarial back-propagation" method increases the resistance to adversarial examples and boosts the classification performance, and results on CIFAR-10 indicate that the method has a regularizing effect similar to dropout in fully connected networks.
Adversarial examples in the physical world
TLDR
It is found that a large fraction of adversarial examples are classified incorrectly even when perceived through the camera, which shows that even in physical world scenarios, machine learning systems are vulnerable to adversarialExamples.
Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples
TLDR
This work introduces the first practical demonstration that cross-model transfer phenomenon enables attackers to control a remotely hosted DNN with no access to the model, its parameters, or its training data, and introduces the attack strategy of fitting a substitute model to the input-output pairs in this manner, then crafting adversarial examples based on this auxiliary model.
Dense Associative Memory for Pattern Recognition
TLDR
The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions - the higher rectified polynomials which until now have not been used in deep learning.
Random Feature Nullification for Adversary Resistant Deep Architecture
Deep neural networks (DNN) have been proven to be quite effective in many applications such as image recognition and using software to process security or traffic camera footage, for example to
...
...