• Corpus ID: 59413788

Hyperspherical Prototype Networks

@inproceedings{Mettes2019HypersphericalPN,
  title={Hyperspherical Prototype Networks},
  author={Pascal Mettes and Elise van der Pol and Cees G. M. Snoek},
  booktitle={NeurIPS},
  year={2019}
}
This paper introduces hyperspherical prototype networks, which unify regression and classification by prototypes on hyperspherical output spaces. [] Key Method Furthermore, hyperspherical prototype networks generalize to regression, by optimizing outputs as an interpolation between two prototypes on the hypersphere. Since both tasks are now defined by the same loss function, they can be jointly optimized for multi-task problems. Experimental evaluation shows the benefits of hyperspherical prototype networks…

Figures and Tables from this paper

Improvising the Learning of Neural Networks on Hyperspherical Manifold
TLDR
Experiments have demonstrated that applying stereographic projection on existing state-of-the-art angular margin objective functions improved performance for standard image classification data sets (CIFAR-10,100) and ran the authors' experiments on malaria-thin blood smear images, resulting in effective outcomes.
Hyperbolic Busemann Learning with Ideal Prototypes
TLDR
The main idea behind the approach is to position prototypes on the ideal boundary of the Poincaré ball, which does not require prior label knowledge, and to be able to compute proximities to ideal prototypes, the penalised Busemann loss is introduced.
Distance-based Hyperspherical Classification for Multi-source Open-Set Domain Adaptation
TLDR
This work tackles multi-source Open-Set domain adaptation by introducing HyMOS: a straightforward model that exploits the power of contrastive learning and the properties of its hyperspherical feature space to correctly predict known labels on the target, while rejecting samples belonging to any unknown class.
Leveraging Class Hierarchies with Metric-Guided Prototype Learning
TLDR
This work proposes to integrate this metric in a new and versatile classification layer in order to model the disparity of errors, and relies on jointly learning a feature-extracting network and a set of class representations, or prototypes, which incorporate the error metric into their relative arrangement in the embedding space.
LEVERAGING CLASS HIERARCHIES
  • Computer Science
  • 2020
TLDR
This work proposes to integrate this metric in the supervision of a prototypical network in order to model the hierarchical class structure, and shows that this simultaneous training allows for consistent improvement of the severity of the network’s errors with regard to the class hierarchy when compared to traditional methods and other prototypebased strategies.
SEN: A Novel Feature Normalization Dissimilarity Measure for Prototypical Few-Shot Learning Networks
TLDR
The theoretical analysis shows that the proposed dissimilarity measure, denoted the Squared root of the Euclidean distance and the Norm distance (SEN), forces embedding points to be attracted to its correct prototype, while being repelled from all other prototypes, keeping the norm of all points the same.
Rethinking Semantic Segmentation: A Prototype View
TLDR
This study un-covers several limitations of such parametric segmentation regime, and proposes a nonparametric alternative based on non-learnable prototypes, which yields compelling results over several datasets, and performs well in the large-vocabulary situation.
FoldHSphere: deep hyperspherical embeddings for protein fold recognition
TLDR
The proposed FoldHSpherePro ensemble method is efficient in learning discriminative and fold-representative embeddings for the protein domains and effective at identifying the protein fold class by pairwise comparison, even when amino acid sequence similarities are low.
Regularizing Neural Networks via Minimizing Hyperspherical Energy (with Appendix)
TLDR
The compressive minimum hyperspherical energy (CoMHE) is proposed as a more effective regularization for neural networks that consistently outperforms existing regularization methods, and can be easily applied to different neural networks.
Dissecting Supervised Constrastive Learning
TLDR
This work proves, under mild assumptions, that both losses attain their minimum once the representations of each class collapse to the vertices of a regular simplex, inscribed in a hypersphere.
...
...

References

SHOWING 1-10 OF 50 REFERENCES
Deep Hyperspherical Learning
TLDR
Deep hyperspherical convolution networks that are distinct from conventional inner product based convolutional networks are introduced, and it is shown that SphereNet can effectively encode discriminative representation and alleviate training difficulty, leading to easier optimization, faster convergence and comparable (even better) classification accuracy over Convolutional counterparts.
Prototypical Networks for Few-shot Learning
TLDR
This work proposes Prototypical Networks for few-shot classification, and provides an analysis showing that some simple design decisions can yield substantial improvements over recent approaches involving complicated architectural choices and meta-learning.
Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost
TLDR
Two distance-based classifiers, the k-nearest neighbor (k-NN) and nearest class mean (NCM) classifiers are considered, and a new metric learning approach is introduced for the latter, and an extension of the NCM classifier is introduced to allow for richer class representations.
Regressive Virtual Metric Learning
TLDR
A theoretical analysis showing the consistency of the approach and establishing some links with other classical metric learning methods and an efficient solution to the difficult problem of selecting virtual points based in part on recent works in optimal transport are proposed.
Infinite Mixture Prototypes for Few-Shot Learning
TLDR
It is shown the importance of adaptive capacity for capturing complex data distributions such as alphabets, with 25% absolute accuracy improvements over prototypical networks, while still maintaining or improving accuracy on the standard Omniglot and mini-ImageNet benchmarks.
NormFace: L2 Hypersphere Embedding for Face Verification
TLDR
This work identifies and study four issues related to normalization through mathematical analysis, which yields understanding and helps with parameter settings, and proposes two strategies for training using normalized features.
Robust Classification with Convolutional Prototype Learning
TLDR
This paper argues that the lack of robustness for CNN is caused by the softmax layer, which is a totally discriminative model and based on the assumption of closed world (i.e., with a fixed number of categories), and proposes a novel learning framework called convolutional prototype learning (CPL).
Prototype adjustment for zero shot classification
SphereFace: Deep Hypersphere Embedding for Face Recognition
TLDR
This paper proposes the angular softmax (A-Softmax) loss that enables convolutional neural networks (CNNs) to learn angularly discriminative features in deep face recognition (FR) problem under open-set protocol.
Fix your classifier: the marginal value of training the last weight layer
TLDR
This work argues that this classifier can be fixed, up to a global scale constant, with little or no loss of accuracy for most tasks, allowing memory and computational benefits.
...
...