#### Filter Results:

#### Publication Year

1999

2017

#### Publication Type

#### Co-author

#### Publication Venue

#### Key Phrases

Learn More

We combine supervised learning with unsupervised learning in deep neural networks. The proposed model is trained to simultaneously minimize the sum of supervised and unsupervised cost functions by backpropagation, avoiding the need for layer-wise pre-training. Our work builds on top of the Ladder network proposed by Valpola [1] which we extend by combining… (More)

- Harri Valpola, Juha Karhunen
- Neural Computation
- 2002

A Bayesian ensemble learning method is introduced for unsupervised extraction of dynamic processes from noisy data. The data are assumed to be generated by an unknown nonlinear mapping from unknown factors. The dynamics of the factors are modeled using a nonlinear state-space model. The nonlinear mappings in the model are represented using multilayer… (More)

- Jaakko Särelä, Harri Valpola
- Journal of Machine Learning Research
- 2005

A new algorithmic framework called denoising source separation (DSS) is introduced. The main benefit of this framework is that it allows for easy development of new source separation algorithms which are optimised for specific problems. In this framework, source separation algorithms are constucted around denoising procedures. The resulting algorithms can… (More)

- Alexander Ilin, Harri Valpola
- Neural Processing Letters
- 2005

We show that the choice of posterior approximation of sources affects the solution found in Bayesian varia-tional learning of linear independent component analysis models. Assuming the sources to be independent a posteriori favours a solution which has an orthogonal mixing matrix. A linear dynamic model which uses second-order statistics is considered but… (More)

- Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, Tapani Raiko
- ArXiv
- 2015

SUMMARY Blind separation of sources from their linear mixtures is a well understood problem. However, if the mixtures are nonlinear, this problem becomes generally very difficult. This is because both the nonlinear mapping and the underlying sources must be learned from the data in a blind manner, and the problem is highly ill-posed without a suitable… (More)

Many existing independent component analysis algorithms include a preprocessing stage where the inputs are sphered. This amounts to normalising the data such that all correlations between the variables are removed. In this work, I show that sphering allows very weak contextual modulation to steer the development of meaningful features. Context-biased… (More)

- Tapani Raiko, Harri Valpola, Yann LeCun
- AISTATS
- 2012

We transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average , and use separate shortcut connections to model the linear dependencies instead. This transformation aims at separating the problems of learning the linear and nonlin-ear parts of the whole input-output mapping, which has many… (More)

- Harri Valpola, Markus Harva, Juha Karhunen
- Signal Processing
- 2004

In many models, variances are assumed to be constant although this assumption is known to be unrealistic. Joint modelling of means and variances can lead to infinite probability densities which makes it a difficult problem for many learning algorithms. We show that a Bayesian variational technique which is sensitive to probability mass instead of density is… (More)

We introduce building blocks from which a large variety of latent variable models can be built. The blocks include continuous and discrete variables, summation, addition, non-linearity and switching. Ensemble learning provides a cost function which can be used for updating the variables as well as optimising the model structure. The blocks are designed to… (More)