# Mixture density networks for distribution and uncertainty estimation

@inproceedings{Guillaumes2017MixtureDN, title={Mixture density networks for distribution and uncertainty estimation}, author={Axel Brando Guillaumes}, year={2017} }

The deep learning techniques have made neural networks the leading
option for solving some computational problems and it has been
shown the production of the state-of-the-art results in many fields like
computer vision, automatic speech recognition, natural language processing,
and audio recognition. In fact, we may be tempted to make
use of neural networks directly, as we know them nowadays, in order
to make predictions and solve many problems, but if the decision that
has to be taken…

## Figures from this paper

figure 1.1 figure 1.2 figure 1.3 figure 1.4 figure 1.5 figure 1.6 figure 1.7 figure 1.8 figure 1.9 figure 2.1 figure 2.2 figure 2.3 figure 2.4 figure 2.5 figure 3.1 figure 3.2 figure 3.3 figure 3.4 figure 3.5 figure 3.6 figure 4.1 figure 4.2 figure 5.1 figure 5.2 figure 5.3 figure 5.4 figure 5.5 figure 5.6 figure 6.1 figure 6.10 figure 6.11 figure 6.12 figure 6.13 figure 6.14 figure 6.15 figure 6.16 figure 6.2 figure 6.3 figure 6.4 figure 6.5 figure 6.6 figure 6.7 figure 6.8 figure 6.9

## 23 Citations

Quantifying Multimodality in World Models

- Computer ScienceICAART
- 2022

The correct modelling & detection of uncertain future states lays the foundation for handling critical situations in a safe way, which is a prerequisite for deploying RL systems in real-world environments.

Learning to Predict Diverse Human Motions from a Single Image via Mixture Density Networks

- Computer ScienceArXiv
- 2021

A novel approach to predict future human motions from a much weaker condition, i.e., a single image, with mixture density networks (MDN) modeling, and introduces an energybased prior over learnable parameters of MDN to maintain motion coherence, as well as improve the prediction accuracy.

Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling

- Computer Science2018 IEEE International Conference on Robotics and Automation (ICRA)
- 2018

The proposed uncertainty-aware learning from demonstration method outperforms other compared methods in terms of safety using a complex real-world driving dataset and is suitable for real-time robotics applications.

Improved Deep Mixture Density Network for Regional Wind Power Probabilistic Forecasting

- Computer ScienceIEEE Transactions on Power Systems
- 2020

An Improved Deep Mixture Density Network is proposed for short-term WPPF of multiple wind farms and the entire region and a laconic and accurate probabilistic expression of predicted power at each time step is produced by the proposed model.

Domain Adaptation for Autoencoder-Based End-to-End Communication Over Wireless Channels

- Computer ScienceArXiv
- 2021

This paper proposes a method for adapting the autoencoder without modifying the encoder and decoder neural networks, and adapting only the MDN model of the channel, and effectively present to the decoder samples close to the source distribution.

Generative Adversarial Model-Guided Deep Active Learning for Voltage Dip Labelling

- Computer Science2019 IEEE Milan PowerTech
- 2019

A novel deep active learning method, guided by a generative adversarial network (GAN), where the generator is formed by modelling data with a Gaussian mixture model and provides the estimated probability distribution function (pdf) where the query criterion of the deep activeLearning method is built upon.

Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network

- Computer Science, Mathematics2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2019

The experiments show that the 3D poses estimated by the approach from an input of 2D joints are consistent in 2D reprojections, which supports the argument that multiple solutions exist for the 2D-to-3D inverse problem.

Likelihood Approximation Networks (LANs) for Fast Inference of Simulation Models in Cognitive Neuroscience

- Computer Science, BiologybioRxiv
- 2020

This work proposes neural networks that learn approximate likelihoods for arbitrary generative models, allowing fast posterior sampling with only a one-off cost for model simulations that is amortized for future inference.

Likelihood approximation networks (LANs) for fast inference of simulation models in cognitive neuroscience

- Computer Science, BiologyeLife
- 2021

This work proposes neural networks that learn approximate likelihoods for arbitrary generative models, allowing fast posterior sampling with only a one-off cost for model simulations that is amortized for future inference.

CGMVAE: Coupling GMM Prior and GMM Estimator for Unsupervised Clustering and Disentanglement

- Computer ScienceIEEE Access
- 2021

This paper proposes a novel method on top of one recent architecture with a novel explanation of Gaussian mixture model (GMM) membership, accompanied by a GMM loss to enhance the clustering.

## References

SHOWING 1-10 OF 33 REFERENCES

Intriguing properties of neural networks

- Computer ScienceICLR
- 2014

It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.

Mixture density networks

- Computer Science
- 1994

This paper introduces a new class of network models obtained by combining a conventional neural network with a mixture density model, called a Mixture Density Network, which can in principle represent arbitrary conditional probability distributions in the same way that aventional neural network can represent arbitrary functions.

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

- Computer ScienceNIPS
- 2017

This work proposes an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates.

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

- Computer ScienceICML
- 2015

Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.

Deep Learning

- Computer ScienceNature
- 2015

Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

- Computer ScienceICLR
- 2016

The "exponential linear unit" (ELU) which speeds up learning in deep neural networks and leads to higher classification accuracies and significantly better generalization performance than ReLUs and LReLUs on networks with more than 5 layers.

Simplifying Neural Networks by Soft Weight-Sharing

- Computer ScienceNeural Computation
- 1992

A more complicated penalty term is proposed in which the distribution of weight values is modeled as a mixture of multiple gaussians, which allows the parameters of the mixture model to adapt at the same time as the network learns.

Improving Back-Propagation by Adding an Adversarial Gradient

- Computer ScienceArXiv
- 2015

First experimental results on MNIST show that the "adversarial back-propagation" method increases the resistance to adversarial examples and boosts the classification performance, and results on CIFAR-10 indicate that the method has a regularizing effect similar to dropout in fully connected networks.

Learning long-term dependencies with gradient descent is difficult

- Computer ScienceIEEE Trans. Neural Networks
- 1994

This work shows why gradient based learning algorithms face an increasingly difficult problem as the duration of the dependencies to be captured increases, and exposes a trade-off between efficient learning by gradient descent and latching on information for long periods.

Explaining and Harnessing Adversarial Examples

- Computer ScienceICLR
- 2015

It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.