Dropout Training of Matrix Factorization and Autoencoder for Link Prediction in Sparse Graphs

@article{Zhai2015DropoutTO,
  title={Dropout Training of Matrix Factorization and Autoencoder for Link Prediction in Sparse Graphs},
  author={Shuangfei Zhai and Zhongfei Zhang},
  journal={ArXiv},
  year={2015},
  volume={abs/1512.04483}
}
Matrix factorization (MF) and Autoencoder (AE) are among the most successful approaches of unsupervised learning. While MF based models have been extensively exploited in the graph modeling and link prediction literature, the AE family has not gained much attention. In this paper we investigate both MF and AE's application to the link prediction problem in sparse graphs. We show the connection between AE and MF from the perspective of multiview learning, and further propose MF+AE: a model… 

Figures and Tables from this paper

Dropout non-negative matrix factorization

Experimental results on multiple datasets confirm that the dropout NMF methods can not only improve NMF but also further improve existing representative matrix factorization models.

Dropout non-negative matrix factorization

Experimental results on multiple datasets confirm that the dropout NMF methods can not only improve NMF but also further improve existing representative matrix factorization models.

Dropout as a Low-Rank Regularizer for Matrix Factorization

A theoretical analysis of dropout for matrix factorization (MF), where Bernoulli random variables are used to drop columns of the factors, concludes that dropout can be used as a low-rank regularizer with data dependent singular-value thresholding.

A fusion probability matrix factorization framework for link prediction

An Analysis of Dropout for Matrix Factorization

This work demonstrates the equivalence between dropout and a fully deterministic model for matrix factorization in which the factors are regularized by the sum of the product of the norms of the columns.

Survey on graph embeddings and their applications to machine learning problems on graphs

This survey aims to describe the core concepts of graph embeddings and provide several taxonomies for their description, and presents an in-depth analysis of models based on network types, and overviews a wide range of applications to machine learning problems on graphs.

Survey on graph embeddings and their applications to machine learning problems on graphs

This survey covers a new rapidly growing family of automated graph feature engineering techniques, presents an in-depth analysis of models based on network types, and overviews a wide range of applications to machine learning problems on graphs.

Learning network representations

A description of the state-of-the-art of network representation learning as well as a detailed account of the connections with other fields of study such as continuous word embeddings and deep learning architectures are provided.

node2vec: Scalable Feature Learning for Networks

In node2vec, an algorithmic framework for learning continuous feature representations for nodes in networks, a flexible notion of a node's network neighborhood is defined and a biased random walk procedure is designed, which efficiently explores diverse neighborhoods.

References

SHOWING 1-10 OF 30 REFERENCES

Link Prediction via Matrix Factorization

The model learns latent features from the topological structure of a (possibly directed) graph, and is shown to make better predictions than popular unsupervised scores, and may be combined with optional explicit features for nodes or edges, which yields better performance.

Feature Noising for Log-Linear Structured Prediction

This work reinterpreted noising as an explicit regularizer, and approximate it with a second-order formula that can be used during training without actually generating fake data, and shows how to apply this method to structured prediction using multinomial logistic regression and linear-chain CRFs.

A Deep Learning Approach to Link Prediction in Dynamic Networks

A novel deep learning framework, i.e., Conditional Temporal Restricted Boltzmann Machine (ctRBM), which predicts links based on individual transition variance as well as influence introduced by local neighbors is proposed, which outperforms existing algorithms in link inference on dynamic networks.

New perspectives and methods in link prediction

This paper examines important factors for link prediction in networks and provides a general, high-performance framework for the prediction task and presents an effective flow-based predicting algorithm, formal bounds on imbalance in sparse network link prediction, and employ an evaluation method appropriate for the observed imbalance.

Relational learning via collective matrix factorization

This model generalizes several existing matrix factorization methods, and therefore yields new large-scale optimization algorithms for these problems, which can handle any pairwise relational schema and a wide variety of error models.

Probabilistic Matrix Factorization

The Probabilistic Matrix Factorization (PMF) model is presented, which scales linearly with the number of observations and performs well on the large, sparse, and very imbalanced Netflix dataset and is extended to include an adaptive prior on the model parameters.

Marginalized Denoising Auto-encoders for Nonlinear Representations

The marginalized Denoising Auto-encoder (mDAE) is presented, which (approximately) marginalizes out the corruption during training and is able to match or outperform the DAE with much fewer training epochs.

Link prediction using supervised learning

This research identifies a set of features that are key to the superior performance under the supervised learning setup, and shows that a small subset of features always plays a significant role in the link prediction job.

DeepWalk: online learning of social representations

DeepWalk is an online learning algorithm which builds useful incremental results, and is trivially parallelizable, which make it suitable for a broad class of real world applications such as network classification, and anomaly detection.

A Marginalized Denoising Method for Link Prediction in Relational Data

The basic idea is that the multiplication of matrices can be explicitly written in a summation of individual terms, e.g. (AXX )ij = ∑ kl AikXklXjl if k 6= j, otherwise it will be qAijXjlXjl when k = j.