Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domain

  title={Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domain},
  author={Guangyao Chen and Peixi Peng and Li Ma and Jia Li and Lin Du and Yonghong Tian},
  journal={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},
Recently, the generalization behavior of Convolutional Neural Networks (CNN) is gradually transparent through explanation techniques with the frequency components decomposition. However, the importance of the phase spectrum of the image for a robust vision system is still ignored. In this paper, we notice that the CNN tends to converge at the local optimum which is closely related to the high-frequency components of the training images, while the amplitude spectrum is easily disturbed such as… 

PADDLES: Phase-Amplitude Spectrum Disentangled Early Stopping for Learning with Noisy Labels

It is observed that PS, which captures more semantic information, can increase the robustness of DNNs to label noise, more so than AS can, and the proposed Phase-AmplituDe DisentangLed Early Stopping method outperforms other early stopping methods and obtains state-of-the-art performance.

Improving Robustness to Out-of-Distribution Data by Frequency-based Augmentation

A frequency-based data augmentation technique that replaces the frequency components with other images of the same class to improve the robustness against out-of-distribution data.

Adversarial amplitude swap towards robust image classifiers

Results showed that adversarial amplitude images can serve as a better data augmentation method to achieve general robustness against both common corruptions and adversarial perturbations even in an adversarial training setup, and contributed to the understanding and the training of truly robust classifiers.

Uncertainty Modeling for Out-of-Distribution Generalization

This paper hypothesizes that the feature statistic, after considering the potential uncertainties, follows a multivariate Gaussian distribution, so that each feature statistic is no longer a deterministic value, but a probabilistic point with diverse distribution possibilities.

Deeper Insights into ViTs Robustness towards Common Corruptions

This paper investigates how CNN-like architectural designs and CNN-based data augmentation strategies impact on ViTs’ robustness towards common corruptions through an extensive and rigorous benchmarking, and introduces a novel conditional method enabling input-varied augmentations from two angles.

Analyzing Adversarial Robustness of Vision Transformers against Spatial and Spectral Attacks

It is argued that the image quality should be simultaneously considered in evaluating adversarial robustness of Transformers, and that the superiority of one architecture to another in terms of robustness can change depending on the attack strength expressed by the quality of the attacked images.

PRIME: A Few Primitives Can Boost Robustness to Common Corruptions

This work proposes PRIME, a general data augmentation scheme that relies on simple yet rich families of max-entropy image transformations that outperforms the prior art in terms of corruption robustness, while its simplicity and plug-and-play nature enable combination with other methods to further boost their robustness.

Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification

The spectral analysis of different datasets shows that in contrast to common visual recognition, dermoscopic skin lesion classification, by nature, is reliant on complex feature combinations beyond shape-bias, and shifting away from the prevalent desire of shape-biasing models can even improve skin lesions classifiers in some cases.

Detecting Generated Images by Real Images

This work observed that the noise pattern of real images exhibits similar characteristics in the frequency domain, while the generated images are far different, and can perform image authentication by checking whether an image follows the patterns of authentic images.

Generalized Out-of-Distribution Detection: A Survey

A framework is presented called generalized OOD detection, which encompasses the aforementioned problems, i.e .



High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks

This work first notices CNN's ability in capturing the high-frequency components of images, which are almost imperceptible to a human, and leads to multiple hypotheses that are related to the generalization behaviors of CNN, including a potential explanation for adversarial examples.

Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain

A novel Spatial-Phase Shallow Learning (SPSL) method is presented, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery to improve the transferability and theoretically analyze the validity of utilizing the phase spectrum.

Improved Regularization of Convolutional Neural Networks with Cutout

This paper shows that the simple regularization technique of randomly masking out square regions of input during training, which is called cutout, can be used to improve the robustness and overall performance of convolutional neural networks.

Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks

The proposed ODIN method, based on the observation that using temperature scaling and adding small perturbations to the input can separate the softmax score distributions between in- and out-of-distribution images, allowing for more effective detection, consistently outperforms the baseline approach by a large margin.

Finding the Secret of Image Saliency in the Frequency Domain

It is found that the secret of visual saliency may mainly hide in the phases of intermediate frequencies, and a novel approach to design the saliency detector under the assistance of prior knowledge obtained through both unsupervised and supervised learning processes is proposed.

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

Patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches, and CutMix consistently outperforms state-of-the-art augmentation strategies on CIFAR and ImageNet classification tasks, as well as on ImageNet weakly-supervised localization task.

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

This work proposes a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit and derives a robust initialization method that particularly considers the rectifier nonlinearities.

Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data

This work bases its work on a popular method ODIN, proposing two strategies for freeing it from the needs of tuning with OoD data, while improving its OoD detection performance, and proposing to decompose confidence scoring as well as a modified input pre-processing method.

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

A reparameterization of the weight vectors in a neural network that decouples the length of those weight vectors from their direction is presented, improving the conditioning of the optimization problem and speeding up convergence of stochastic gradient descent.

Deep Residual Learning for Image Recognition

This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.