Learn More
Product units provide a method of automatically learning the higher-order input combinations required for eecient learning in neural networks. However, we show that problems are encountered when using backpropagation to train networks containing these units. This paper examines these problems, and proposes some atypical heuristics to improve learning. Using(More)
We present an algorithm for the training of feedforward and recurrent neural networks. It detects internal representation conflicts and uses these conflicts in a constructive manner to add new neu-rons to the network. The advantages are twofold: (1) starting with a small network neurons are only allocated when required; (2) by detecting and resolving(More)
| We investigate the learning of de-terministic nite-state automata (DFA's) with recurrent networks with a single input neu-ron, where each input symbol is represented as a temporal pattern and strings as sequences of temporal patterns. We empirically demonstrate that obvious temporal encodings can make learning very diicult or even impossible. Based on(More)
  • 1