Learnable latent embeddings for joint behavioral and neural analysis

  title={Learnable latent embeddings for joint behavioral and neural analysis},
  author={Steffen Schneider and Jin Hwa Lee and M. Mathis},
Mapping behavioral actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioral data increases, there is growing interest in modeling neural dynamics during adaptive behaviors to probe neural representations. In particular, neural latent embeddings can reveal underlying correlates of behavior, yet, we lack non-linear techniques that can explicitly and flexibly leverage joint behavior and neural data. Here, we fill this gap with a novel… 

Figures from this paper

A model-free approach to link neural activity to behavioral tasks

“Model-free identification of neural encoding (MINE)” is developed using convolutional neural networks (CNN) to relate aspects of tasks to neural activity and identifies a new class of neurons that integrate thermosensory and behavioral information which eluded us previously when using traditional clustering and regression-based approaches.

Distinguishing discrete and continuous behavioral variability using warped autoregressive HMMs

A class of warped ARHMMs (WARHMM) is introduced in which the warping variable affects the dynamics of each syllable either linearly or nonlinearly, and achieves similar performance to the standard ARHMM while using fewer behavioral syllables.

Automatically annotated motion tracking identifies a distinct social behavioral profile following chronic social defeat stress

Evidence is provided that the DeepOF supervised and unsupervised pipelines detect a distinct stress-induced social behavioral pattern, which was particularly observed at the beginning of a novel social encounter.



Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE

The ability to record activities from hundreds of neurons simultaneously in the brain has placed an increasing demand for developing appropriate statistical techniques to analyze such data. Recently,

Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity

This work introduces a benchmark suite for latent variable modeling of neural population activity, and identifies unsupervised evaluation as a common framework for evaluating models across datasets, and applies several baselines that demonstrate benchmark diversity.

Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification

Modeling data in two monkeys performing three-dimensional reach and grasp tasks revealed that the behaviorally relevant dynamics are significantly lower-dimensional than otherwise implied, and PSID discovered distinct rotational dynamics that were more predictive of behavior.

Inferring single-trial neural population dynamics using sequential auto-encoders

LFADS, a deep learning method for analyzing neural population activity, can extract neural dynamics from single-trial recordings, stitch separate datasets into a single model, and infer perturbations, for example, from behavioral choices to these dynamics.

Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding

Evidence that objects in segmented natural movies undergo transitions that are typically small in magnitude with occasional large jumps, which is characteristic of a temporally sparse distribution is provided and SlowVAE, a model for unsupervised representation learning that uses a sparse prior on temporally adjacent observations to disentangle generative factors without any assumptions on the number of changing factors is presented.

Large-scale neural recordings call for new insights to link brain and behavior

emerging tools and technologies being used to probe large-scale brain activity and new approaches to characterize behavior in the context of such measurements, and insights obtained from large- scale neural recordings in diverse model systems are highlighted.

Parallel inference of hierarchical latent dynamics in two-photon calcium imaging of neuronal populations

The system VaLPACa (Variational Ladders for Parallel Autoencoding of Calcium imaging data) solves the problem of disentangling deeper- and shallower-level dynamics by incorporating a ladder architecture that can infer a hierarchy of dynamical systems by incorporating sequential variational autoencoders.

Demixed principal component analysis of neural population data

A new dimensionality reduction technique, demixed principal component analysis (dPCA), that decomposes population activity into a few components and exposes the dependence of the neural representation on task parameters such as stimuli, decisions, or rewards is demonstrated.

Cortical population activity within a preserved neural manifold underlies multiple motor behaviors

It is reported that dominant population activity patterns, the neural modes, are largely preserved across various tasks, with many displaying consistent temporal dynamics and reliably mapping onto muscle activity.