Spectral estimation for detecting low-dimensional structure in networks using arbitrary null models

  title={Spectral estimation for detecting low-dimensional structure in networks using arbitrary null models},
  author={M. Humphries and J. A. Caballero and M. Evans and S. Maggi and Abhinav Singh},
  journal={PLoS ONE},
Discovering low-dimensional structure in real-world networks requires a suitable null model that defines the absence of meaningful structure. Here we introduce a spectral approach for detecting a network’s low-dimensional structure, and the nodes that participate in it, using any null model. We use generative models to estimate the expected eigenvalue distribution under a specified null model, and then detect where the data network’s eigenspectra exceed the estimated bounds. On synthetic… Expand

Figures and Tables from this paper

Strong and weak principles of neural dimension reduction
It is suggested that the brain has both principles of dimension reduction at play, and it is shown how either the weak or strong principles appear to be true based on innocuous looking decisions about how to use dimension reduction on data. Expand


Significance-based community detection in weighted networks
This paper proposes a community extraction algorithm for weighted networks which incorporates iterative hypothesis testing under the null, and proves a central limit theorem for edge-weight sums and asymptotic consistency of the algorithm under a weighted stochastic block model. Expand
Spectral redemption in clustering sparse networks
A way of encoding sparse data using a “nonbacktracking” matrix, and it is shown that the corresponding spectral algorithm performs optimally for some popular generative models, including the stochastic block model. Expand
Estimation of Global Network Statistics from Incomplete Data
The treatment allows us to find support for Dunbar's hypothesis in detecting an upper threshold for the number of active social contacts that individuals maintain over the course of one week, and it is demonstrated that the scaling methods can provide very good estimates of true network statistics while acknowledging limits. Expand
Learning Latent Block Structure in Weighted Networks
This model learns from both the presence and weight of edges, allowing it to discover structure that would otherwise be hidden when weights are discarded or thresholded, and a Bayesian variational algorithm is described for efficiently approximating this model's posterior distribution over latent block structures. Expand
Evaluating Overfit and Underfit in Models of Network Community Structure
A broad investigation of over and underfitting across 16 state-of-the-art community detection algorithms applied to a novel benchmark corpus of 572 structurally diverse real-world networks finds that algorithms vary widely in the number and composition of communities they find, given the same input. Expand
Community detection for correlation matrices
This work introduces, via a consistent redefinition of null models based on random matrix theory, the appropriate correlation-based counterparts of the most popular community detection techniques, and can filter out both unit-specific noise and system-wide dependencies, and the resulting communities are internally correlated and mutually anti-correlated. Expand
Spectral community detection in sparse networks
This work makes use of a relaxation method to derive a spectral community detection algorithm that works well even in the sparse regime where other methods break down, Interestingly, however, the matrix at the heart of the method is not exactly the non-backtracking matrix, but a variant of it with a somewhat different definition. Expand
Scalable detection of statistically significant communities and hierarchies, using message passing for modularity
  • Pan Zhang, C. Moore
  • Mathematics, Computer Science
  • Proceedings of the National Academy of Sciences
  • 2014
By applying the proposed algorithm recursively, subdividing communities until no statistically significant subcommunities can be found, it is shown that the algorithm can detect hierarchical structure in real-world networks more efficiently than previous methods. Expand
Finding community structure in networks using the eigenvectors of matrices.
  • M. Newman
  • Mathematics, Medicine
  • Physical review. E, Statistical, nonlinear, and soft matter physics
  • 2006
A modularity matrix plays a role in community detection similar to that played by the graph Laplacian in graph partitioning calculations, and a spectral measure of bipartite structure in networks and a centrality measure that identifies vertices that occupy central positions within the communities to which they belong are proposed. Expand
Configuration model for correlation matrices preserving the node strength.
The present algorithm generates random networks that preserve the expectation of total connectivity of each node to other nodes, akin to configuration models for conventional networks, derived from the maximum entropy principle. Expand