# Learning task-dependent distributed representations by backpropagation through structure

@article{Goller1996LearningTD, title={Learning task-dependent distributed representations by backpropagation through structure}, author={Christoph Goller and Andreas K{\"u}chler}, journal={Proceedings of International Conference on Neural Networks (ICNN'96)}, year={1996}, volume={1}, pages={347-352 vol.1} }

While neural networks are very successfully applied to the processing of fixed-length vectors and variable-length sequences, the current state of the art does not allow the efficient processing of structured objects of arbitrary shape (like logical terms, trees or graphs. [] Key Method The major difference of our approach compared to others is that the structure-representations are exclusively tuned for the intended inference task.

## 597 Citations

### Inductive Learning in Symbolic Domains Using Structure-Driven Recurrent Neural Networks

- Computer ScienceKI
- 1996

A connectionist architecture together with a novel supervised learning scheme which is capable of solving inductive inference tasks on complex symbolic structures of arbitrary size and first results from experiments with inductive learning tasks consisting in the classification of logical terms are given.

### Comparing Structures Using a Hopfield-Style Neural Network

- Computer ScienceApplied Intelligence
- 2004

The former approaches of structural matching and constraint relaxation by spreading activation in neural networks and the method of solving optimization tasks using Hopfield-style nets are combined.

### Learning Efficiently with Neural Networks: A Theoretical Comparison between Structured and Flat Representations

- Computer ScienceECAI
- 2000

The message of this paper is that, whenever structured representations are available, they should be preferred to "flat" (array based) representations because they are likely to simplify learning in terms of time complexity.

### A general framework for adaptive processing of data structures

- Computer ScienceIEEE Trans. Neural Networks
- 1998

The framework described in this paper is an attempt to unify adaptive models like artificial neural nets and belief nets for the problem of processing structured information, where relations between data variables are expressed by directed acyclic graphs, where both numerical and categorical values coexist.

### From Hopfield nets to recursive networks to graph machines: Numerical machine learning for structured data

- Computer ScienceTheor. Comput. Sci.
- 2005

### Neural Representation Learning in Linguistic Structured Prediction

- Computer Science
- 2016

This thesis argues for the importance of modeling discrete structure in language, even when learning continuous representations, and proposes dynamic recurrent acyclic graphical neural networks (DRAGNN), a modular neural architecture that generalizes the encoder/decoder concept to include explicit linguistic structures.

### Selective Training: A Strategy for Fast Backpropagation on Sentence Embeddings

- Computer SciencePAKDD
- 2019

This work presents a method to reduce training time substantially by selecting training instances that provide relevant information for training, based on the similarity of the learned representations over input instances, thus allowing for learning a non-trivial weighting scheme from multi-dimensional representations.

### Recursive Neural Networks Applied to Discourse Representation Theory

- Computer ScienceICANN
- 2002

A novel technique is introduced, combining Discourse Representation Theory (DRT) with Recursive Neural Networks (RNN) in order to yield a neural model capable to discover properties and relationships among constituents of a knowledge-base expressed by natural language sentences.

### Neural Networks for Processing Data Structures

- Computer ScienceSummer School on Neural Networks
- 1997

It is not difficult to figure out how to extract tree automata from a neural network for structures, and this would allow the above scheme to work on the other side around, with a neural module which is driven by a symbolic subsystem.

## References

SHOWING 1-10 OF 14 REFERENCES

### Learning Recursive Distributed Representations for Holistic Computation

- Computer Science
- 1991

Two possible forms of holistic inference, transformational inference and confluent inference, are identified and compared and a dual-ported RAAM architecture based on Pollack's Recursive Auto-Associative Memory is implemented and demonstrated in the domain of Natural Language translation.

### Distributed representations for terms in hybrid reasoning systems

- Computer Science
- 1997

The intended applications of the approach are hybrid (symbolic/connectionist) systems, where the connectionist part has to solve logic-oriented inductive learning tasks similar to the term-classiication problems used in the experiments.

### A Connectionist Parser with Recursive Sentence Structure and Lexical Disambiguation

- Computer ScienceAAAI
- 1992

XERIC networks, presented here, are distributed representation connectionist parsers which can analyze and represent syntactically varied sentences, including ones with recursive phrase structure constructs.

### Backpropagation Through Time: What It Does and How to Do It

- Computer Science, MathematicsProc. IEEE
- 1990

This paper first reviews basic backpropagation, a simple method which is now being widely used in areas like pattern recognition and fault diagnosis, and describes further extensions of this method, to deal with systems other than neural networks, systems involving simultaneous equations or true recurrent networks, and other practical issues which arise with this method.

### Exploring the Symbolic/Subsymbolic Continuum: A case study of RAAM

- Computer Science
- 1992

It is di cult to clearly de ne the symbolic and subsymbolic paradigms; each is usually described by its tendencies rather than any one de nitive property. Symbolic processing is generally…

### Encoding Labeled Graphs by Labeling RAAM

- Computer ScienceNIPS
- 1993

The Labeling RAAM (LRAAM), an extension to the RAAM by Pollack, can encode labeled graphs with cycles by representing pointers explicitly by transforming the encoder network of the LRAAM into an analog Hopfield network with hidden units.

### Structure Sensitivity in Connectionist Models

- Biology
- 1993

Annotation: Published in The Proceedings of the 1993 Connectionist Models Summer School, (Eds) Mozer et al., Lawrence Erlbaum, 1993.