• Corpus ID: 231846902

Few-shot time series segmentation using prototype-defined infinite hidden Markov models

  title={Few-shot time series segmentation using prototype-defined infinite hidden Markov models},
  author={Yazan Qarout and Yordan P. Raykov and Max A. Little},
We propose a robust framework for interpretable, few-shot analysis of non-stationary sequential data based on flexible graphical models to express the structured distribution of sequential events, using prototype radial basis function (RBF) neural network emissions. A motivational link is demonstrated between prototypical neural network architectures for few-shot learning and the proposed RBF network infinite hidden Markov model (RBF-iHMM). We show that RBF networks can be efficiently specified… 

Figures and Tables from this paper



Stationary Activations for Uncertainty Calibration in Deep Learning

A new family of non-linear neural network activation functions that mimic the properties induced by the widely-used Matern family of kernels in Gaussian process (GP) models are introduced and it is demonstrated that the local stationarity property together with limited mean-square differentiability shows both good performance and uncertainty calibration in Bayesian deep learning tasks.

Learning higher-order sequential structure with cloned HMMs

The experiments show that CHMMs can beat n-grams, sequence memoizers, and LSTMs on character-level language modeling tasks and can be a viable alternative to these methods in some tasks that require variable order sequence modeling and the handling of uncertainty.

Neural Autoregressive Distribution Estimation

We present Neural Autoregressive Distribution Estimation (NADE) models, which are neural network architectures applied to the problem of unsupervised distribution and density estimation. They

Understanding deep learning requires rethinking generalization

These experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data, and confirm that simple depth two neural networks already have perfect finite sample expressivity.

Composing graphical models with neural networks for structured representations and fast inference

A general modeling and inference framework that composes probabilistic graphical models with deep learning methods and combines their respective strengths is proposed, giving a scalable algorithm that leverages stochastic variational inference, natural gradients, graphical model message passing, and the reparameterization trick.

NICE: Non-linear Independent Components Estimation

We propose a deep learning framework for modeling complex high-dimensional densities called Non-linear Independent Component Estimation (NICE). It is based on the idea that a good representation is

Nonlinear time series modelling with the radial basis function-based state-dependent autoregressive model

It is shown that the RBF-AR model can not only reconstruct the dynamics of given nonlinear time series effectively, but also give much better fitting to complextime series than the approach of directly RBF neural network modelling.

Spectral Representations for Convolutional Neural Networks

This work proposes spectral pooling, which performs dimensionality reduction by truncating the representation in the frequency domain, and demonstrates the effectiveness of complex-coefficient spectral parameterization of convolutional filters.

The sequence memoizer

The sequence memoizer is a new hierarchical Bayesian model for discrete sequence data that captures long range dependencies and power-law characteristics, while remaining computationally attractive.

Prototypical Networks for Few-shot Learning

This work proposes Prototypical Networks for few-shot classification, and provides an analysis showing that some simple design decisions can yield substantial improvements over recent approaches involving complicated architectural choices and meta-learning.