Corpus ID: 232135069

Clusterability in Neural Networks

@article{Filan2021ClusterabilityIN,
  title={Clusterability in Neural Networks},
  author={Daniel Filan and Stephen Casper and Shlomi Hod and C. Wild and Andrew Critch and Stuart J. Russell},
  journal={ArXiv},
  year={2021},
  volume={abs/2103.03386}
}
The learned weights of a neural network have often been considered devoid of scrutable internal structure. In this paper, however, we look for structure in the form of clusterability: how well a network can be divided into groups of neurons with strong internal connectivity but weak external connectivity. We find that a trained neural network is typically more clusterable than randomly initialized networks, and often clusterable relative to random networks with the same distribution of weights… Expand
1 Citations
Extreme sparsity gives rise to functional specialization

References

SHOWING 1-10 OF 46 REFERENCES
Understanding Community Structure in Layered Neural Networks
Modular representation of layered neural networks
Graph Structure of Neural Networks
Interpreting Layered Neural Networks via Hierarchical Modular Representation
A review of modularization techniques in artificial neural networks
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Modular Networks: Learning to Decompose Neural Computation
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
...
1
2
3
4
5
...