Learning Data Augmentation with Online Bilevel Optimization for Image Classification

@article{Mounsaveng2021LearningDA,
  title={Learning Data Augmentation with Online Bilevel Optimization for Image Classification},
  author={Saypraseuth Mounsaveng and Issam H. Laradji and Ismail Ben Ayed and David V{\'a}zquez and Marco Pedersoli},
  journal={2021 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  year={2021},
  pages={1690-1699}
}
Data augmentation is a key practice in machine learning for improving generalization performance. However, finding the best data augmentation hyperparameters requires domain knowledge or a computationally demanding search. We address this issue by proposing an efficient approach to automatically train a network that learns an effective distribution of transformations to improve its generalization. Using bilevel optimization, we directly optimize the data augmentation parameters using a… 

Figures and Tables from this paper

Direct Differentiable Augmentation Search
TLDR
This paper proposes an efficient differentiable search algorithm called Direct Differentiable Augmentation Search (DDAS), which exploits meta-learning with one-step gradient update and continuous relaxation to the expected training loss for efficient search.
GABO: Graph Augmentations with Bi-level Optimization
TLDR
This framework combines a GIN layer augmentation generator with a bias transformation and outperforms the same classifier augmented using the state-of-the-art FLAG augmentation.
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond
TLDR
A best-response-based single-level reformulation is constructed and a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies are established, covering aspects ranging from fundamental automatic differentiation schemes to various accelerations, simplifications, extensions and their convergence and complexity properties.
A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images
TLDR
A consistency-based (CB) loss function that encourages the output predictions to be consistent with spatial transformations of the input images, and yields significant improvement over conventional point-level loss functions and almost matches the performance of models trained with full supervision with much less human effort.
A realistic fish-habitat dataset to evaluate algorithms for underwater visual analysis
TLDR
This work presents DeepFish as a benchmark suite with a large-scale dataset to train and test methods for several computer vision tasks, and collects point-level and segmentation labels to have a more comprehensive fish analysis benchmark.
An overview of mixing augmentation methods and augmentation strategies
TLDR
This survey focuses on two DA research streams: image mixing and automated selection of augmentation strategies and mainly covers the methods published in the materials of top-tier conferences and in leading journals in the years 2017-2021.
Meta Approach to Data Augmentation Optimization
TLDR
This paper proposes to optimize image recognition models and data augmentation policies simultaneously to improve the performance using gradient descent, and achieves efficient and scalable training by approximating the gradient of policies by implicit gradient with Neumann series approximation.
On Automatic Data Augmentation for 3D Point Cloud Classification
TLDR
This work proposes to automatically learn a data augmentation strategy using bilevel optimization, which achieves competitive performance on both tasks and provides further insight into the augmentor’s ability to learn the validation set distribution.
SBO-RNN: Reformulating Recurrent Neural Networks via Stochastic Bilevel Optimization
TLDR
It is proved that under mild conditions there is no vanishing or exploding gradient in training SBO-RNN, an RNN where the feedforward and backpropagation solve the lower and upper-level optimization for learning hidden states and their hyperparameters, respectively.
Detecting aerial objects: drones, birds, and helicopters
TLDR
An object detection dataset consisting of videos that depict helicopter exercises recorded in an unconstrained, maritime environment is presented and special consideration was taken to emphasize small instances of helicopters relative to the field-of-view, providing a more even ratio of small, medium, and large-sized object appearances for training more robust detectors in this specific domain.
...
...

References

SHOWING 1-10 OF 55 REFERENCES
RandAugment: Practical data augmentation with no separate search
TLDR
RandAugment can be used uniformly across different tasks and datasets and works out of the box, matching or surpassing all previous learned augmentation approaches on CIFAR-10, CIFar-100, SVHN, and ImageNet.
AutoAugment: Learning Augmentation Strategies From Data
TLDR
This paper describes a simple procedure called AutoAugment to automatically search for improved data augmentation policies, which achieves state-of-the-art accuracy on CIFAR-10, CIFar-100, SVHN, and ImageNet (without additional data).
Faster AutoAugment: Learning Augmentation Strategies using Backpropagation
TLDR
This paper proposes a differentiable policy search pipeline for data augmentation, which achieves significantly faster searching than prior work without a performance drop and introduces approximate gradients for several transformation operations with discrete parameters.
The Effectiveness of Data Augmentation in Image Classification using Deep Learning
TLDR
A method to allow a neural net to learn augmentations that best improve the classifier, which is called neural augmentation is proposed, and the successes and shortcomings of this method are discussed.
Adversarial Learning of General Transformations for Data Augmentation
TLDR
This work learns data augmentation directly from the training data by learning to transform images with an encoder-decoder architecture combined with a spatial transformer network.
Fast AutoAugment
TLDR
This paper proposes an algorithm called Fast AutoAugment that finds effective augmentation policies via a more efficient search strategy based on density matching that speeds up the search time by orders of magnitude while achieves comparable performances on image recognition tasks with various models and datasets.
Online Hyper-Parameter Learning for Auto-Augmentation Strategy
TLDR
This paper proposes Online Hyper-parameter Learning for Auto-Augmentation (OHL-Auto-Aug), an economical solution that learns the augmentation policy distribution along with network training that eliminates the need of re-training and dramatically reduces the cost of the overall search process.
A Bayesian Data Augmentation Approach for Learning Deep Models
TLDR
A novel Bayesian formulation to data augmentation is provided, where new annotated training points are treated as missing variables and generated based on the distribution learned from the training set, and this approach produces better classification results than similar GAN models.
Smart Augmentation Learning an Optimal Data Augmentation Strategy
TLDR
Smart augmentation works, by creating a network that learns how to generate augmented data during the training process of a target network in a way that reduces that networks loss, and allows to learn augmentations that minimize the error of that network.
Learning to Compose Domain-Specific Transformations for Data Augmentation
TLDR
The proposed method can make use of arbitrary, non-deterministic transformation functions, is robust to misspecified user input, and is trained on unlabeled data, which can be used to perform data augmentation for any end discriminative model.
...
...