• Corpus ID: 236134149

Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning

  title={Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning},
  author={Timo Milbich and Karsten Roth and Samarth Sinha and Ludwig Schmidt and Marzyeh Ghassemi and Bj{\"o}rn Ommer},
Deep Metric Learning (DML) aims to find representations suitable for zero-shot transfer to a priori unknown test distributions. However, common evaluation protocols only test a single, fixed data split in which train and test classes are assigned randomly. More realistic evaluations should consider a broad spectrum of distribution shifts with potentially varying degree and difficulty. In this work, we systematically construct train-test splits of increasing difficulty and present the ooDML… 

Figures and Tables from this paper

Guided Deep Metric Learning

This paper proposes a novel approach to DML that is capable of a better manifold generalization and representation to up to 40% improvement, using guidelines suggested by Musgrave et al. to perform a more fair and realistic comparison.

Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning

This paper is the first to evaluate state-of-the-art DML methods trained on imbalanced data, and to show the negative impact these representations have on minority subgroup performance when used for downstream tasks.

A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning

This work introduces non-isotropic probabilistic proxy-based DML, which derives non- isotropic von Mises-Fisher distributions for class proxies to better represent complex class-specific variances and measures the proxy-to-image distance between these models.

Non-isotropy Regularization for Proxy-based Deep Metric Learning

Extensive experiments highlight consistent generalization benefits of NIR while achieving competitive and state-of-the-art performance on the standard benchmarks CUB200-2011, Cars196 and Stanford Online Products.

Integrating Language Guidance into Vision-based Deep Metric Learning

Leveraging language embeddings of expert- and pseudo-classnames, this work contextualize and realign visual representation spaces corresponding to meaningful language semantics for better semantic consistency in DML.

CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

The novel “Contrastive Leave One Out Boost” (CLOOB), which uses modern Hopfield networks for covariance enrichment together with the InfoLOOB objective to mitigate this saturation effect of the InfoNCE objective.

TeST: Test-time Self-Training under Distribution Shift

This paper proposes Test-Time Self-Training (TeST): a technique that takes as input a model trained on some source data and a novel data distribution at test time, and learns invariant and robust representations using a student-teacher framework and finds that models adapted with TeST significantly improve over baseline test-time adaptation algorithms.



Revisiting Training Strategies and Generalization Performance in Deep Metric Learning

A simple, yet effective, training regularization is proposed to reliably boost the performance of ranking-based DML models on various standard benchmark datasets.

Unbiased Evaluation of Deep Metric Learning Algorithms

An unbiased comparison of the most popular DML baseline methods under same conditions and more importantly, not obfuscating any hyper parameter tuning or adjustment needed to favor a particular method is performed.

Deep Variational Metric Learning

This paper proposes a deep variational metric learning (DVML) framework to explicitly model the intra-class variance and disentangle the intra -class invariance, namely, the class centers, and can simultaneously generate discriminative samples to improve robustness.

Uniform Priors for Data-Efficient Transfer.

It is shown that features that are most transferable have high uniformity in the embedding space and a uniformity regularization scheme is proposed that encourages better transfer and feature reuse.

Sharing Matters for Generalization in Deep Metric Learning

Experiments show that, independent of the underlying network architecture and the specific ranking loss, the approach significantly improves performance in deep metric learning, leading to new the state-of-the-art results on various standard benchmark datasets.

Rethinking Zero-Shot Video Classification: End-to-End Training for Realistic Applications

This work proposes the first end-to-end algorithm for ZSL in video classification, which uses a trainable 3D CNN to learn the visual features and outperforms the state-of-the-art by a wide margin.

Out-of-Distribution Generalization via Risk Extrapolation (REx)

This work introduces the principle of Risk Extrapolation (REx), and shows conceptually how this principle enables extrapolation, and demonstrates the effectiveness and scalability of instantiations of REx on various OoD generalization tasks.

Learning Intra-Batch Connections for Deep Metric Learning

This work proposes an approach based on message passing networks that takes into account all the relations in a mini-batch of samples into account, and refine embedding vectors by exchanging messages among all samples in a given batch allowing the training process to be aware of the overall structure.

Improved Deep Metric Learning with Multi-class N-pair Loss Objective

This paper proposes a new metric learning objective called multi-class N-pair loss, which generalizes triplet loss by allowing joint comparison among more than one negative examples and reduces the computational burden of evaluating deep embedding vectors via an efficient batch construction strategy using only N pairs of examples.

Meta-Learning With Differentiable Convex Optimization

The objective is to learn feature embeddings that generalize well under a linear classification rule for novel categories and this work exploits two properties of linear classifiers: implicit differentiation of the optimality conditions of the convex problem and the dual formulation of the optimization problem.