• Corpus ID: 239050288

Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs

  title={Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs},
  author={Avinash Baidya and Joel Dapello and James J. DiCarlo and Tiago Marques},
While some convolutional neural networks (CNNs) have surpassed human visual abilities in object classification, they often struggle to recognize objects in images corrupted with different types of common noise patterns, highlighting a major limitation of this family of models. Recently, it has been shown that simulating a primary visual cortex (V1) at the front of CNNs leads to small improvements in robustness to these image perturbations. In this study, we start with the observation that… 

Figures and Tables from this paper


Simulating a Primary Visual Cortex at the Front of CNNs Improves Robustness to Image Perturbations
While current CNN architectures are arguably brain-inspired, the results presented here demonstrate that more precisely mimicking just one stage of the primate visual system leads to new gains in ImageNet-level computer vision applications.
Learning From Brains How to Regularize Machines
This work denoised the notoriously variable neural activity using strong predictive models trained on this large corpus of responses from the mouse visual system, and used the neural representation similarity to regularize CNNs trained on image classification by penalizing intermediate representations that deviated from neural ones.
Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network
Detailed experiments to validate that carefully assembling these techniques and applying them to basic CNN models can improve the accuracy and robustness of the models while minimizing the loss of throughput showed that the improvement to backbone network performance boosted transfer learning performance significantly.
Generalisation in humans and deep neural networks
The robustness of humans and current convolutional deep neural networks on object recognition under twelve different types of image degradations is compared and it is shown that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on.
Towards robust vision by multi-task learning on monkey visual cortex
This work jointly trained a deep network to perform image classification and to predict neural activity in macaque primary visual cortex in response to the same natural stimuli, and found that co-training on monkey V1 data indeed leads to increased robustness despite the absence of those distortions during training.
Effective training of convolutional neural networks for age estimation based on knowledge distillation
An effective training procedure of CNNs for age estimation based on knowledge distillation is proposed, able to allow smaller and simpler “student” models to be trained to match the predictions of a larger “teacher” model.
Very Deep Convolutional Networks for Large-Scale Image Recognition
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
This work introduces Patch Gaussian, a simple augmentation scheme that adds noise to randomly selected patches in an input image that leads to reduced sensitivity to high frequency noise(similar to Gaussian) while retaining the ability to take advantage of relevant high frequency information in the image.
An Effective Anti-Aliasing Approach for Residual Networks
This work shows that it can mitigate Frequency aliasing by placing non-trainable blur filters and using smooth activation functions at key locations, particularly where networks lack the capacity to learn them, and lead to substantial improvements in out-of-distribution generalization on both image classification under natural corruptions on ImageNet-C and few-shot learning on Meta-Dataset.