• DOI:10.1109/TPAMI.2019.2895781
Corpus ID: 47014389

Training Faster by Separating Modes of Variation in Batch-Normalized Models

@article{Kalayeh2020TrainingFB,
  title={Training Faster by Separating Modes of Variation in Batch-Normalized Models},
  author={M. Kalayeh and M. Shah},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2020},
  volume={42},
  pages={1483-1500}
}

M. Kalayeh, M. Shah

Published 2020

Computer Science, Mathematics, Medicine

IEEE Transactions on Pattern Analysis and Machine Intelligence

Batch Normalization (BN) is essential to effectively train state-of-the-art deep Convolutional Neural Networks (CNN). It normalizes the layer outputs during training using the statistics of each mini-batch. BN accelerates training procedure by allowing to safely utilize large learning rates and alleviates the need for careful initialization of the parameters. In this work, we study BN from the viewpoint of Fisher kernels that arise from generative probability models. We show that assuming… CONTINUE READING

View on IEEE

arxiv.org

Save to Library

Create Alert

Cite

Launch Research Feed