Learnable latent embeddings for joint behavioral and neural analysis

  title={Learnable latent embeddings for joint behavioral and neural analysis},
  author={Steffen Schneider and Jin Hwa Lee and Mackenzie W. Mathis},
Mapping behavioral actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioral data increases, there is growing interest in modeling neural dynamics during adaptive behaviors to probe neural representations. In particular, neural latent embeddings can reveal underlying correlates of behavior, yet, we lack non-linear techniques that can explicitly and flexibly leverage joint behavior and neural data. Here, we fill this gap with a novel… 

Figures from this paper

Understanding Neural Coding on Latent Manifolds by Sharing Features and Dividing Ensembles

This work proposes feature sharing across neural tuning curves, which significantly improves performance and leads to better-behaved optimization and a solution to the problem of ensemble detection, whereby different groups of neurons, i.e., ensembles, can be modulated by different latent manifolds.

A model-free approach to link neural activity to behavioral tasks

“Model-free identification of neural encoding (MINE)” is developed using convolutional neural networks (CNN) to relate aspects of tasks to neural activity and identifies a new class of neurons that integrate thermosensory and behavioral information which eluded us previously when using traditional clustering and regression-based approaches.

Neural manifold analysis of brain circuit dynamics in health and disease

A number of linear and non-linear approaches to neural manifold learning are reviewed, including principal component analysis (PCA), multi-dimensional scaling (MDS), Isomap, locally linear embedding (LLE), Laplacian eigenmaps (LEM), t-SNE, and uniform manifold approximation and projection (UMAP), and are outlined under a common mathematical nomenclature.

Distinguishing discrete and continuous behavioral variability using warped autoregressive HMMs

A class of warped ARHMMs (WARHMM) is introduced in which the warping variable affects the dynamics of each syllable either linearly or nonlinearly, and achieves similar performance to the standard ARHMM while using fewer behavioral syllables.

Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE

Though the identifiability is appealing, it is shown that iVAEs could have local minimum solution where observations and the approximated ICs are independent given covariates.

Automatically annotated motion tracking identifies a distinct social behavioral profile following chronic social defeat stress

Evidence is provided that the DeepOF supervised and unsupervised pipelines detect a distinct stress-induced social behavioral pattern, which was particularly observed at the beginning of a novel social encounter.



Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE

The ability to record activities from hundreds of neurons simultaneously in the brain has placed an increasing demand for developing appropriate statistical techniques to analyze such data. Recently,

Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity

This work introduces a benchmark suite for latent variable modeling of neural population activity, and identifies unsupervised evaluation as a common framework for evaluating models across datasets, and applies several baselines that demonstrate benchmark diversity.

Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification

Modeling data in two monkeys performing three-dimensional reach and grasp tasks revealed that the behaviorally relevant dynamics are significantly lower-dimensional than otherwise implied, and PSID discovered distinct rotational dynamics that were more predictive of behavior.

Inferring single-trial neural population dynamics using sequential auto-encoders

LFADS, a deep learning method for analyzing neural population activity, can extract neural dynamics from single-trial recordings, stitch separate datasets into a single model, and infer perturbations, for example, from behavioral choices to these dynamics.

Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding

Evidence that objects in segmented natural movies undergo transitions that are typically small in magnitude with occasional large jumps, which is characteristic of a temporally sparse distribution is provided and SlowVAE, a model for unsupervised representation learning that uses a sparse prior on temporally adjacent observations to disentangle generative factors without any assumptions on the number of changing factors is presented.

Large-scale neural recordings call for new insights to link brain and behavior

emerging tools and technologies being used to probe large-scale brain activity and new approaches to characterize behavior in the context of such measurements, and insights obtained from large- scale neural recordings in diverse model systems are highlighted.

Parallel inference of hierarchical latent dynamics in two-photon calcium imaging of neuronal populations

The system VaLPACa (Variational Ladders for Parallel Autoencoding of Calcium imaging data) solves the problem of disentangling deeper- and shallower-level dynamics by incorporating a ladder architecture that can infer a hierarchy of dynamical systems by incorporating sequential variational autoencoders.

The intrinsic attractor manifold and population dynamics of a canonical cognitive circuit across waking and sleep

This work shows that a mammalian brain circuit of thousands of neurons constructs a navigational compass with only a one-dimensional ring of stable activity states, and develops a method to decode unknown variables by finding shapes in neural data.

Cortical population activity within a preserved neural manifold underlies multiple motor behaviors

It is reported that dominant population activity patterns, the neural modes, are largely preserved across various tasks, with many displaying consistent temporal dynamics and reliably mapping onto muscle activity.