• Corpus ID: 63747657

Mixture density networks for distribution and uncertainty estimation

@inproceedings{Guillaumes2017MixtureDN,
  title={Mixture density networks for distribution and uncertainty estimation},
  author={Axel Brando Guillaumes},
  year={2017}
}
The deep learning techniques have made neural networks the leading option for solving some computational problems and it has been shown the production of the state-of-the-art results in many fields like computer vision, automatic speech recognition, natural language processing, and audio recognition. In fact, we may be tempted to make use of neural networks directly, as we know them nowadays, in order to make predictions and solve many problems, but if the decision that has to be taken… 
Quantifying Multimodality in World Models
TLDR
The correct modelling & detection of uncertain future states lays the foundation for handling critical situations in a safe way, which is a prerequisite for deploying RL systems in real-world environments.
Learning to Predict Diverse Human Motions from a Single Image via Mixture Density Networks
TLDR
A novel approach to predict future human motions from a much weaker condition, i.e., a single image, with mixture density networks (MDN) modeling, and introduces an energybased prior over learnable parameters of MDN to maintain motion coherence, as well as improve the prediction accuracy.
Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling
TLDR
The proposed uncertainty-aware learning from demonstration method outperforms other compared methods in terms of safety using a complex real-world driving dataset and is suitable for real-time robotics applications.
Improved Deep Mixture Density Network for Regional Wind Power Probabilistic Forecasting
TLDR
An Improved Deep Mixture Density Network is proposed for short-term WPPF of multiple wind farms and the entire region and a laconic and accurate probabilistic expression of predicted power at each time step is produced by the proposed model.
Domain Adaptation for Autoencoder-Based End-to-End Communication Over Wireless Channels
TLDR
This paper proposes a method for adapting the autoencoder without modifying the encoder and decoder neural networks, and adapting only the MDN model of the channel, and effectively present to the decoder samples close to the source distribution.
Generative Adversarial Model-Guided Deep Active Learning for Voltage Dip Labelling
TLDR
A novel deep active learning method, guided by a generative adversarial network (GAN), where the generator is formed by modelling data with a Gaussian mixture model and provides the estimated probability distribution function (pdf) where the query criterion of the deep activeLearning method is built upon.
Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network
  • Chen Li, Gim Hee Lee
  • Computer Science, Mathematics
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
The experiments show that the 3D poses estimated by the approach from an input of 2D joints are consistent in 2D reprojections, which supports the argument that multiple solutions exist for the 2D-to-3D inverse problem.
Likelihood Approximation Networks (LANs) for Fast Inference of Simulation Models in Cognitive Neuroscience
TLDR
This work proposes neural networks that learn approximate likelihoods for arbitrary generative models, allowing fast posterior sampling with only a one-off cost for model simulations that is amortized for future inference.
Likelihood approximation networks (LANs) for fast inference of simulation models in cognitive neuroscience
TLDR
This work proposes neural networks that learn approximate likelihoods for arbitrary generative models, allowing fast posterior sampling with only a one-off cost for model simulations that is amortized for future inference.
CGMVAE: Coupling GMM Prior and GMM Estimator for Unsupervised Clustering and Disentanglement
TLDR
This paper proposes a novel method on top of one recent architecture with a novel explanation of Gaussian mixture model (GMM) membership, accompanied by a GMM loss to enhance the clustering.
...
1
2
3
...

References

SHOWING 1-10 OF 33 REFERENCES
Intriguing properties of neural networks
TLDR
It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.
Mixture density networks
TLDR
This paper introduces a new class of network models obtained by combining a conventional neural network with a mixture density model, called a Mixture Density Network, which can in principle represent arbitrary conditional probability distributions in the same way that aventional neural network can represent arbitrary functions.
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
TLDR
This work proposes an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
TLDR
Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
Deep Learning
TLDR
Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
TLDR
The "exponential linear unit" (ELU) which speeds up learning in deep neural networks and leads to higher classification accuracies and significantly better generalization performance than ReLUs and LReLUs on networks with more than 5 layers.
Simplifying Neural Networks by Soft Weight-Sharing
TLDR
A more complicated penalty term is proposed in which the distribution of weight values is modeled as a mixture of multiple gaussians, which allows the parameters of the mixture model to adapt at the same time as the network learns.
Improving Back-Propagation by Adding an Adversarial Gradient
TLDR
First experimental results on MNIST show that the "adversarial back-propagation" method increases the resistance to adversarial examples and boosts the classification performance, and results on CIFAR-10 indicate that the method has a regularizing effect similar to dropout in fully connected networks.
Learning long-term dependencies with gradient descent is difficult
TLDR
This work shows why gradient based learning algorithms face an increasingly difficult problem as the duration of the dependencies to be captured increases, and exposes a trade-off between efficient learning by gradient descent and latching on information for long periods.
Explaining and Harnessing Adversarial Examples
TLDR
It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.
...
1
2
3
4
...