Unsupervised Belief Representation Learning with Information-Theoretic Variational Graph Auto-Encoders

  title={Unsupervised Belief Representation Learning with Information-Theoretic Variational Graph Auto-Encoders},
  author={Jinning Li and Huajie Shao and Dachun Sun and Ruijie Wang and Yuchen Yan and Jinyang Li and Shengzhong Liu and Hanghang Tong and Tarek F. Abdelzaher},
  journal={Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval},
  • Jinning LiHuajie Shao T. Abdelzaher
  • Published 1 October 2021
  • Computer Science
  • Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
This paper develops a novel unsupervised algorithm for belief representation learning in polarized networks that (i) uncovers the latent dimensions of the underlying belief space and (ii) jointly embeds users and content items (that they interact with) into that space in a manner that facilitates a number of downstream tasks, such as stance detection, stance prediction, and ideology mapping. Inspired by total correlation in information theory, we propose the Information-Theoretic Variational… 

Figures and Tables from this paper

Polarization identification on multiple timescale using representation learning on temporal graphs in Eulerian description

A case is made for approaching polarization problems with a Eulerian description, the concurrent description in fluid mechanics, and the temporal evolution of nodes embeddings is represented with a deformation of velocity vector fields.

NTULM: Enriching Social Media Text Representations with Non-Textual Units

This work constructs an NTU-centric social heterogeneous network to co-embed NTUs and principally integrates these NTU embeddings into a large pretrained language model by fine-tuning with these additional units, leading to the generation of holistic general purpose social media content embedding.



Interpretable Variational Graph Autoencoder with Noninformative Prior

This work exploits the noninformative prior as the prior distribution of latent variables, which enables the posterior distribution parameters to be almost learned from the sample data, thereby improving the interpretability of the model.

Auto-Encoding Total Correlation Explanation

An information-theoretic approach to characterizing disentanglement and dependence in representation learning using multivariate mutual information, also called total correlation, is proposed and it is found that this lower bound is equivalent to the one in variational autoencoders (VAE) under certain conditions.

Isolating Sources of Disentanglement in VAEs

A decomposition of the variational lower bound is shown that can be used to explain the success of the β-VAE in learning disentangled representations, and a new information-theoretic disentanglement metric is proposed, which is classifier-free and generalizable to arbitrarily-distributed and non-scalar latent variables.

DeepWalk: online learning of social representations

DeepWalk is an online learning algorithm which builds useful incremental results, and is trivially parallelizable, which make it suitable for a broad class of real world applications such as network classification, and anomaly detection.

InfoVAE: Balancing Learning and Inference in Variational Autoencoders

It is shown that the proposed Info-VAE model can significantly improve the quality of the variational posterior and can make effective use of the latent features regardless of the flexibility of the decoding distribution.

Disentangling Overlapping Beliefs by Structured Matrix Factorization

A new class of Non-negative Matrix Factorization algorithms that allow identification of both agreement and disagreement points when beliefs of different communities partially overlap, and shows that social beliefs overlap even in polarized scenarios.

Modeling Relational Data with Graph Convolutional Networks

It is shown that factorization models for link prediction such as DistMult can be significantly improved through the use of an R-GCN encoder model to accumulate evidence over multiple inference steps in the graph, demonstrating a large improvement of 29.8% on FB15k-237 over a decoder-only baseline.

Computational Modeling of Hierarchically Polarized Groups by Structured Matrix Factorization

An (enhanced) unsupervised Non-negative Matrix Factorization (NMF) algorithm is described for computational modeling of hierarchically polarized groups and is enhanced with a language model, and with a proof of orthogonality of factorized components.

Fake News Stance Detection Using Deep Learning Architecture (CNN-LSTM)

A hybrid Neural Network architecture, that combines the capabilities of CNN and LSTM, is used with two different dimensionality reduction approaches, Principle Component Analysis (PCA) and Chi-Square, to determine the relative stance of a news article towards its headline.

ControlVAE: Controllable Variational Autoencoder

A new non-linear PI controller is designed, a variant of the proportional-integral-derivative (PID) control, to automatically tune the hyperparameter (weight) added in the VAE objective using the output KL-divergence as feedback during model training.