Corpus ID: 46373531

Reducing Duplicate Filters in Deep Neural Networks

@inproceedings{RoyChowdhury2018ReducingDF,
  title={Reducing Duplicate Filters in Deep Neural Networks},
  author={Aruni RoyChowdhury and Prakhar Sharma and E. Learned-Miller},
  year={2018}
}
  • Aruni RoyChowdhury, Prakhar Sharma, E. Learned-Miller
  • Published 2018
  • This paper investigates the presence of duplicate neurons or filters in neural networks. This phenomenon is prevalent in networks and increases with the number of filters in a layer. We observe the emergence of duplicate filters over training iterations, study the factors that affect their concentration and compare existing network reducing operations. We validate our findings using convolutional and fully-connected networks on the CIFAR-10 dataset. 
    14 Citations

    Figures and Tables from this paper

    Methods for Pruning Deep Neural Networks
    • PDF
    Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks
    • 1
    • PDF
    Variable batch size across layers for efficient prediction on CNNs
    Compression of Deep Neural Networks by Combining Pruning and Low Rank Decomposition
    • 1
    Understanding Convolutional Neural Network Training with Information Theory
    • 21
    International Journal of Recent Technology and Engineering (IJRTE)
    • PDF
    Deep Multi-view Features from Raw Audio for Acoustic Scene Classification
    • PDF
    Fast and Accurate Person Re-Identification with RMNet
    • 4
    • PDF

    References

    SHOWING 1-10 OF 22 REFERENCES
    Pruning Filters for Efficient ConvNets
    • 1,346
    • Highly Influential
    • PDF
    Visualizing and Understanding Convolutional Networks
    • 8,978
    • PDF
    Reducing Overfitting in Deep Networks by Decorrelating Representations
    • 212
    • PDF
    Symmetry-invariant optimization in deep networks
    • 15
    • PDF
    Pruning Convolutional Neural Networks for Resource Efficient Inference
    • 615
    • Highly Influential
    Learning Multiple Layers of Features from Tiny Images
    • 10,072
    • PDF
    Convergent Learning: Do different neural networks learn the same representations?
    • 139
    • PDF