Finite size analysis of the detectability limit of the stochastic block model

@article{Young2017FiniteSA,
  title={Finite size analysis of the detectability limit of the stochastic block model},
  author={Jean-Gabriel Young and Patrick Desrosiers and Laurent H{\'e}bert-Dufresne and Edward Laurence and Louis J. Dub{\'e}},
  journal={Physical review. E},
  year={2017},
  volume={95 6-1},
  pages={
          062304
        }
}
It has been shown in recent years that the stochastic block model is sometimes undetectable in the sparse limit, i.e., that no algorithm can identify a partition correlated with the partition used to generate an instance, if the instance is sparse enough and infinitely large. In this contribution, we treat the finite case explicitly, using arguments drawn from information theory and statistics. We give a necessary condition for finite-size detectability in the general SBM. We then distinguish… 

Figures from this paper

Algorithmic detectability threshold of the stochastic blockmodel

TLDR
This study considers the expectation-maximization (EM) algorithm with belief propagation (BP) and derives its algorithmic detectability threshold and is not restricted to the community structure but includes general modular structures.

Community detection and stochastic block models: recent developments

  • E. Abbe
  • Computer Science
    J. Mach. Learn. Res.
  • 2017
TLDR
The recent developments that establish the fundamental limits for community detection in the stochastic block model are surveyed, both with respect to information-theoretic and computational thresholds, and for various recovery requirements such as exact, partial and weak recovery.

Community Detection and Stochastic Block Models

  • E. Abbe
  • Computer Science
    Found. Trends Commun. Inf. Theory
  • 2018
TLDR
The recent developments that establish the fundamental limits for community detection in the stochastic block model are surveyed, both with respect to information-theoretic and computational thresholds, and for various recovery requirements such as exact, partial and weak recovery.

Decoding communities in networks.

TLDR
Two main applications of standard coding-theoretical methods to community detection are illustrated and the Shannon's noisy-channel coding theorem can be invoked to establish a lower bound, here named as decodability bound, for the maximum amount of noise tolerable by an ideal decoder to achieve perfect detection of communities.

missSBM: An R Package for Handling Missing Values in the Stochastic Block Model

The Stochastic Block Model (SBM) is a popular probabilistic model for random graph. It is commonly used to perform clustering on network data by aggregating nodes that share similar connectivity

Duality between predictability and reconstructability in complex systems

TLDR
It is shown that predictability and reconstructability, even though closely connected by the mutual information, can behave differently, even in a dual manner, and it is proved how such duality universally emerges when changing the number of steps in the process.

Universality of the stochastic block model

TLDR
It is shown that the most popular algorithms designed to solve MPE problems can in fact be understood as special cases of the maximum likelihood formulation of the stochastic block model, or one of its direct generalizations.

Threefold way to the dimension reduction of dynamics on networks: An application to synchronization

TLDR
DART generalizes recent approaches for dimension reduction by allowing the treatment of complex-valued dynamical variables, heterogeneities in the intrinsic properties of the nodes as well as modular networks with strongly interacting communities.

Réduire la dimension des systèmes complexes : un regard sur l'émergence de la synchronisation

Complex systems are characterized by the emergence of macroscopic phenomena that cannot be explained by the properties of their basic constituents. Synchronization is one of these phenomena in which

Inférence et réseaux complexes

TLDR
A thorough analysis of a well-known generative model, introduced 40 years ago to identify patterns and regularities in the structure of real networks, and a random model for these objects, and the associated efficient sampling algorithm are obtained.

References

SHOWING 1-10 OF 90 REFERENCES

Community detection and the stochastic block model : recent developments

TLDR
The recent developments that establish the fundamental limits for community detection in the stochastic block model are surveyed, both with respect to statistical and computational tradeoffs, and for various recovery requirements such as exact, partial and weak recovery.

De la détection de la structure communautaire des réseaux complexes

TLDR
This thesis presents the basis of a unified theory of community detection, which builds upon recent advances of the spectral theory of complex networks and proves that a large class of detection algorithm is equivalent to an optimization problem that can be solved approximately though the spectral decomposition of a very general cost matrix.

IEEE TRANSACTIONS ON INFORMATION THEORY

Since at most T of these equations can come from (7), at least (m T) of them come from (8’). Thus, for the distribution (PI’, . . . , p,‘), at most T of the probabilities are nonzero. The proof of

Elements of Information Theory

TLDR
The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment.

Convex Optimization

TLDR
A comprehensive introduction to the subject of convex optimization shows in detail how such problems can be solved numerically with great efficiency.

Journal of Classification

TLDR
This book offers papers in the field of classification, numerical taxonomy, multidimensional scaling and other ordination techniques, clustering, tree structures and other network models, and extends to statistics, psychology, biology, information retrieval, anthropology, archeology and more.

and a at

The xishacorene natural products are structurally unique apolar diterpenoids that feature a bicyclo[3.3.1] framework. These secondary metabolites likely arise from the well-studied, structurally

“A and B”:

Direct fabrication of large micropatterned single crystals. p1205 21 Feb 2003. (news): Academy plucks best biophysicists from a sea of mediocrity. p994 14 Feb 2003.

Phys. Rev. E

  • Phys. Rev. E
  • 2016

Phys

  • Rev. E 84, 066106
  • 2011
...