• Corpus ID: 235458300

Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

  title={Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA},
  author={Hermanni H{\"a}lv{\"a} and Sylvain Le Corff and Luc Leh'ericy and Jonathan So and Yongjie Zhu and Elisabeth Gassiat and Aapo Hyv{\"a}rinen},
We introduce a new general identifiable framework for principled disentanglement referred to as Structured Nonlinear Independent Component Analysis (SNICA). Our contribution is to extend the identifiability theory of deep generative models for a very broad class of structured models. While previous works have shown identifiability for specific classes of time-series models, our theorems extend this to more general temporal structures as well as to models with more complex structures such as… 

Figures from this paper

Identifiable Variational Autoencoders via Sparse Decoding
The Sparse VAE is identifiable: given data drawn from the model, there exists a uniquely optimal set of factors, and it is found that it recovers meaningful latent factors and has smaller heldout reconstruction error than related methods.


Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series
This work combines nonlinear ICA with a Hidden Markov Model, resulting in a model where a latent state acts in place of the observed segment index, and proves identifiability of the proposed model for a general mixing nonlinearity, such as a neural network.
Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning
This work provides a comprehensive proof of the identifiability of the model as well as the consistency of the estimation method, and proposes to learn nonlinear ICA by discriminating between true augmented data, or data in which the auxiliary variable has been randomized.
Variational Autoencoders and Nonlinear ICA: A Unifying Framework
This work shows that for a broad family of deep latent-variable models, identification of the true joint distribution over observed and latent variables is actually possible up to very simple transformations, thus achieving a principled and powerful form of disentanglement.
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
This work proposes a new intuitive principle of unsupervised deep learning from time series which uses the nonstationary structure of the data, and shows how TCL can be related to a nonlinear ICA model, when ICA is redefined to include temporal nonstationarities.
Nonlinear ICA of Temporally Dependent Stationary Sources
It is proved that the method estimates the sources for general smooth mixing nonlinearities, assuming the sources have sufficiently strong temporal dependencies, and these dependencies are in a certain way different from dependencies found in Gaussian processes.
beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
Learning an interpretable factorised representation of the independent data generative factors of the world without supervision is an important precursor for the development of artificial
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
This paper theoretically shows that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data, and trains more than 12000 models covering most prominent methods and evaluation metrics on seven different data sets.
Causal Discovery with General Non-Linear Relationships using Non-Linear ICA
It is shown rigorously that in the case of bivariate causal discovery, such non-linear ICA can be used to infer the causal direction via a series of independence tests, and an alternative measure of causal direction based on asymptotic approximations to the likelihood ratio is proposed.
Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE
The ability to record activities from hundreds of neurons simultaneously in the brain has placed an increasing demand for developing appropriate statistical techniques to analyze such data. Recently,
Auto-Encoding Variational Bayes
A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.