• Corpus ID: 247997004

Exploring Robust Architectures for Deep Artificial Neural Networks

  title={Exploring Robust Architectures for Deep Artificial Neural Networks},
  author={Asim Waqas and Ghulam Rasool and Hamza Farooq and Nidhal Carla Bouaynaya and Rowan University and University of Minnesota},
The architectures of deep artificial neural networks (DANNs) are rou-tinely studied to improve their predictive performance. However, the relationship between the architecture of a DANN and its robustness to noise and adversarial attacks is less explored. We investigate how the robustness of DANNs relates to their underlying graph architectures or struc-tures. This study: (1) starts by exploring the design space of architectures of DANNs using graph-theoretic robustness measures; (2) transforms… 

Figures from this paper



Towards Evaluating the Robustness of Neural Networks

It is demonstrated that defensive distillation does not significantly increase the robustness of neural networks, and three new attack algorithms are introduced that are successful on both distilled and undistilled neural networks with 100% probability are introduced.

Towards Deep Learning Models Resistant to Adversarial Attacks

This work studies the adversarial robustness of neural networks through the lens of robust optimization, and suggests the notion of security against a first-order adversary as a natural and broad security guarantee.

Explaining and Harnessing Adversarial Examples

It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.

Intriguing properties of neural networks

It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.

PremiUm-CNN: Propagating Uncertainty Towards Robust Convolutional Neural Networks

The theoretical and algorithmic foundations of uncertainty or belief propagation are established by developing new deep learning models named PremiUm-CNNs (Propagating Uncertainty in Convolutional Neural Networks), which introduce a tensor normal distribution as a prior over convolutional kernels and estimate the variational posterior by maximizing the evidence lower bound.

Exploring Randomly Wired Neural Networks for Image Recognition

The results suggest that new efforts focusing on designing better network generators may lead to new breakthroughs by exploring less constrained search spaces with more room for novel design.

Deep Residual Learning for Image Recognition

This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.

Learning Multiple Layers of Features from Tiny Images

It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network.

ImageNet classification with deep convolutional neural networks

A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.

On Random Weights and Unsupervised Feature Learning

The answer is that certain convolutional pooling architectures can be inherently frequency selective and translation invariant, even with random weights, and the viability of extremely fast architecture search is demonstrated by using random weights to evaluate candidate architectures, thereby sidestepping the time-consuming learning process.