• Corpus ID: 22207570

A Connection between Feed-Forward Neural Networks and Probabilistic Graphical Models

  title={A Connection between Feed-Forward Neural Networks and Probabilistic Graphical Models},
  author={Dmitrij Schlesinger},
Two of the most popular modelling paradigms in computer vision are feed-forward neural networks (FFNs) and probabilistic graphical models (GMs). Various connections between the two have been studied in recent works, such as e.g. expressing mean-field based inference in a GM as an FFN. This paper establishes a new connection between FFNs and GMs. Our key observation is that any FFN implements a certain approximation of a corresponding Bayesian network (BN). We characterize various benefits of… 

Figures from this paper



Learning Stochastic Feedforward Neural Networks

A stochastic feedforward network with hidden layers composed of both deterministic and stochastics variables is proposed that achieves superior performance on synthetic and facial expressions datasets compared to conditional Restricted Boltzmann Machines and Mixture Density Networks.

Conditional Random Fields Meet Deep Neural Networks for Semantic Segmentation

The literature on combining the modelling power of CRFs with the representation-learning ability of DNNs is reviewed, ranging from early work that combines these two techniques as independent stages of a common pipeline to recent approaches that embed inference of probabilistic models directly in the neural network itself.

Learning Multiple Layers of Features from Tiny Images

It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network.

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

This paper considers fully connected CRF models defined on the complete set of pixels in an image and proposes a highly efficient approximate inference algorithm in which the pairwise edge potentials are defined by a linear combination of Gaussian kernels.

Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

This work considers a small-scale version of {\em conditional computation}, where sparse stochastic units form a distributed representation of gaters that can turn off in combinatorially many ways large chunks of the computation performed in the rest of the neural network.

Dropout: a simple way to prevent neural networks from overfitting

It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.

Deeply Learning the Messages in Message Passing Inference

A new, efficient deep structured model learning scheme, in which deep Convolutional Neural Networks can be used to directly estimate the messages in message passing inference for structured prediction with Conditional Random Fields (CRFs).

Gradient-based learning applied to document recognition

This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.

A Learning Algorithm for Boltzmann Machines

Joint Training of Generic CNN-CRF Models with Stochastic Optimization

A new CNN-CRF end-to-end learning framework, which is based on joint stochastic optimization with respect to both Convolutional Neural Network and Conditional Random Field parameters, is proposed and empirically evaluates the method on the task of semantic labeling of body parts in depth images and shows that it compares favorably to competing techniques.