• Corpus ID: 51640191

Neural Networks Regularization Through Representation Learning

  title={Neural Networks Regularization Through Representation Learning},
  author={Soufiane Belharbi},
Les modeles de reseaux de neurones et en particulier les modeles profonds sont aujourd'hui l'un des modeles a l'etat de l'art en apprentissage automatique et ses applications. Les reseaux de neurones profonds recents possedent de nombreuses couches cachees ce qui augmente significativement le nombre total de parametres. L'apprentissage de ce genre de modeles necessite donc un grand nombre d'exemples etiquetes, qui ne sont pas toujours disponibles en pratique. Le sur-apprentissage est un des… 
1 Citations



Sequence to Sequence Learning with Neural Networks

This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.

Learning Recurrent Neural Networks with Hessian-Free Optimization

This work solves the long-outstanding problem of how to effectively train recurrent neural networks on complex and difficult sequence modeling problems which may contain long-term data dependencies and offers a new interpretation of the generalized Gauss-Newton matrix of Schraudolph which is used within the HF approach of Martens.

Knowledge transfer via multiple model local structure mapping

A locally weighted ensemble framework to combine multiple models for transfer learning, where the weights are dynamically assigned according to a model's predictive power on each test example, is proposed.

Unsupervised Pretraining for Sequence to Sequence Learning

This work presents a general unsupervised learning method to improve the accuracy of sequence to sequence (seq2seq) models by pretraining the weights of the encoder and decoder with the pretrained weights of two language models and then fine-tuned with labeled data.

Training Products of Experts by Minimizing Contrastive Divergence

A product of experts (PoE) is an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary because it is hard even to approximate the derivatives of the renormalization term in the combination rule.

Mastering the game of Go with deep neural networks and tree search

Using this search algorithm, the program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0.5, the first time that a computer program has defeated a human professional player in the full-sized game of Go.

Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies

D3EGF(FIH)J KMLONPEGQSRPETN UCV.WYX(Z R.[ V R6\M[ X N@]_^O\`JaNcb V RcQ W d EGKeL(^(QgfhKeLOE?i)^(QSj ETNPfPQkRl[ V R)m"[ X ^(KeLOEG^ npo qarpo m"[ X ^(KeLOEG^tsAu EGNPb V ^ v wyx

Human-level control through deep reinforcement learning

This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.

Regularization methods for neural networks and related models

A comprehensive overview of existing regularization techniques for neural networks is provided and their theoretical explanation is provided to make them applicable to a wide range of problems.

Multi-Task Learning for Classification with Dirichlet Process Priors

Experimental results on two real life MTL problems indicate that the proposed algorithms automatically identify subgroups of related tasks whose training data appear to be drawn from similar distributions are more accurate than simpler approaches such as single-task learning, pooling of data across all tasks, and simplified approximations to DP.