• Corpus ID: 13434324

Autoencoder Trees

@article{Irsoy2015AutoencoderT,
  title={Autoencoder Trees},
  author={Ozan Irsoy and Ethem Alpaydin},
  journal={ArXiv},
  year={2015},
  volume={abs/1409.7461}
}
We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. The encoder tree takes the input and generates a lower dimensional representation in the leaves and the decoder tree takes this and reconstructs the original input. Exploiting the… 
View PDF on arXiv
15 Citations
Background Citations
4
Methods Citations
5

Figures from this paper

15 Citations

Citation Type

Citation Type

Unsupervised feature extraction with autoencoder trees
Convolutional Soft Decision Trees
TLDR
This work shows that dropout and dropconnect on input units, previously proposed for deep multi-layer neural networks, can also be used with soft decision trees for regularization, and proposes a convolutional extension of the soft decision tree with local feature detectors in successive layers that are trained together with the other parameters of thesoft decision tree.
GrCAN: Gradient Boost Convolutional Autoencoder with Neural Decision Forest
TLDR
A gradient boost module is developed and embedded into the proposed convolutional autoencoder with neural decision forest to improve the performance and the extensive experiments demonstrate that the proposed model achieves good efficiency and prediction performance compared with a series of baseline methods.
D input : X 1 D input : X Input Processing : Convolutional Autoencoders Max Pooling Unpooling Deconvolutional Layer Fully Connected Layer Hidden representation : H Convolutional Layer Neural Decision Forest Neural Decision Forest Base Learner Gradient Boost
TLDR
This work develops a gradient boost module and embeds it into the proposed convolutional autoencoder with neural decision forest to improve the performance and design a structure to learn the parameters of the neural decision Forest and gradient boost modules at contiguous steps.
Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree
TLDR
The proposed pooling operations provide a boost in invariance properties relative to conventional pooling and set the state of the art on several widely adopted benchmark datasets; they are also easy to implement, and can be applied within various deep neural network architectures.
BranchConnect: Image Categorization with Learned Branch Connections
  • Karim Ahmed, L. Torresani
  • Computer Science
    2018 IEEE Winter Conference on Applications of Computer Vision (WACV)
  • 2018
TLDR
An architecture for image categorization that enables the end-to-end learning of separate visual features for the different classes to distinguish is introduced and it is suggested that this scheme acts as a form of beneficial regularization improving generalization performance.
Truncating Wide Networks Using Binary Tree Architectures
TLDR
In this paper, a binary tree architecture to truncate architecture of wide networks by reducing the width of the networks is proposed in order to increase the expressive capacity of networks with a less increase on parameter size.
Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree
TLDR
The proposed pooling operations are easy to implement and can be applied within various deep neural network architectures and provide a boost in invariance properties relative to conventional pooling.
Transition Matrix Representation of Trees with Transposed Convolutions
TLDR
Tart (Transition Matrix Representation with Transposed Convolutions), the novel generalized tree representation for optimal structural search, represents a tree model with a series of transposed convolutions that boost the speed of inference by avoiding the creation of transition matrices.
Unsupervised Learning of Discourse Structures using a Tree Autoencoder
TLDR
A new strategy to generate tree structures in a task-agnostic, unsupervised fashion by extending a latent tree induction framework with an auto-encoding objective is proposed, inferring general tree structures of natural text in multiple domains, showing promising results on a diverse set of tasks.
...
1
2
...

References

SHOWING 1-9 OF 9 REFERENCES
Learning Deep Architectures for AI
TLDR
The motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer modelssuch as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks are discussed.
Budding Trees
TLDR
This work proposes a new decision tree model, named the budding tree, where a node can be both a leaf and an internal decision node, and shows that the tree and its parameters can be trained using gradient-descent.
Soft decision trees
TLDR
A novel decision tree architecture with soft decisions at the internal nodes where both children are chosen with probabilities given by a sigmoid gating function is discussed and the proposed model has significantly higher accuracy using fewer nodes.
Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
TLDR
This work describes and analyze an apparatus for adaptively modifying the proximal function, which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal functions that can be chosen in hindsight.
Hierarchical mixtures of experts and the EM algorithm
  • M. I. Jordan, R. Jacobs
  • Computer Science
    Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan)
  • 1993
TLDR
An expectation-maximization (EM) algorithm for adjusting the parameters of the tree-structured architecture for supervised learning is presented and an online learning algorithm in which the parameters are updated incrementally is developed.
The mnist database of handwritten digits
TLDR
An improved articulated bar flail having shearing edges for efficiently shredding materials and an improved shredder cylinder with a plurality of these flails circumferentially spaced and pivotally attached to the periphery of a rotatable shaft are disclosed.
Learning deep architectures for AI Foundations and trends R in Machine Learning
  • Learning deep architectures for AI Foundations and trends R in Machine Learning
  • 2009
Learning deep architectures for AI. Foundations and trends Learning internal representations from gray-scale images: An example of extensional programming
  • Machine Learning Ninth annual conference of the cognitive science society
  • 1987
Learning internal representations from gray-scale images: An example of extensional programming
  • Ninth annual conference of the cognitive science society, pp. 462–473, 1987.
  • 1987