• Corpus ID: 232045968

Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks

@inproceedings{Sharma2021SketchingCF,
  title={Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks},
  author={Apoorva Sharma and Navid Azizan and Marco Pavone},
  booktitle={UAI},
  year={2021}
}
In order to safely deploy Deep Neural Networks (DNNs) within the perception pipelines of real-time decision making systems, there is a need for safeguards that can detect out-of-training-distribution (OoD) inputs both efficiently and ac-curately. Building on recent work leveraging the local curvature of DNNs to reason about epistemic uncertainty, we propose Sketching Curvature for OoD Detection (SCOD) , an architecture-agnostic framework for equipping any trained DNN with a task-relevant… 

Figures and Tables from this paper

Uncertainty in Contrastive Learning: On the Predictability of Downstream Performance
TLDR
This work explores if the downstream performance on a given datapoint is predictable, directly from its pre-trained embedding, and shows that this goal can be achieved by directly estimating the distribution of the training data in the embedding space and accounting for the local consistency of the representations.
Task-Driven Out-of-Distribution Detection with Statistical Guarantees for Robot Learning
: Our goal is to perform out-of-distribution (OOD) detection , i.e., to detect when a robot is operating in environments that are drawn from a different distribution than the environments used to
Task-Driven Detection of Distribution Shifts with Statistical Guarantees for Robot Learning
TLDR
The resulting approach provides guaranteed confidence bounds on OOD detection including bounds on both the false positive and false negative rates of the detector and is task-driven and sensitive only to changes that impact the robot’s performance.
One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares
TLDR
This paper develops Orthogonal Recursive Fitting ( ORFit), an algorithm for one-pass learning which seeks to perfectly fit every new datapoint while changing the parameters in a direction that causes the least change to the predictions on previousdatapoints.
Task-Driven Data Augmentation for Vision-Based Robotic Control
TLDR
It is shown that re-training vision models on these adversarial datasets improves control performance on OoD test scenarios by up to 28.2% compared to standard task-agnostic data augmentation.
Kronecker-Factored Optimal Curvature
TLDR
It is proved that this formulation is equivalent to the best rank-one approximation problem, where the well-known power iteration method is guaranteed to converge to an optimal rank- one solution resulting in the novel algorithm: the Kronecker-Factored Optimal Curvature (K-FOC).
Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes
TLDR
It is argued that the practical and principled combination of DNNs with sparse Gaussian Processes can pave the way towards reliable and fast robot learning systems with uncertainty awareness.
A DNN with Confidence Measure as an MPC Copycat
  • Computer Science
  • 2021
TLDR
The results achieved so far have shown that the implemented MPC optimizer can reliably solve the specified problem, that the neural network can learn using information from the solver as training and validation datasets, and that the added tools can estimate a measure of confidence for each prediction so that the original solver can be called upon to solve the problem.
A Framework and Benchmark for Deep Batch Active Learning for Regression
TLDR
An open-source benchmark with 15 large tabular data sets is introduced, which is used to compare different BMDAL methods and shows that a combination of the novel components yields new state-of-the-art results in terms of RMSE and is computationally efficient.
Laplace Redux - Effortless Bayesian Deep Learning
TLDR
This work reviews the range of variants of the Laplace approximation, an easy-to-use software library for PyTorch offering user-friendly access to all major versions of the LA, and demonstrates that the LA is competitive with more popular alternatives in terms of performance, while excelling in Terms of computational cost.
...
...

References

SHOWING 1-10 OF 41 REFERENCES
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
TLDR
This work proposes to amplify human effort through a partially automated labeling scheme, leveraging deep learning with humans in the loop, and constructs a new image dataset, LSUN, which contains around one million labeled images for each of 10 scene categories and 20 object categories.
A Scalable Laplace Approximation for Neural Networks
TLDR
This work uses recent insights from second-order optimisation for neural networks to construct a Kronecker factored Laplace approximation to the posterior over the weights of a trained network, enabling practitioners to estimate the uncertainty of models currently used in production without having to retrain them.
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
TLDR
This work proposes an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates.
Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation
TLDR
It is proved that, for a classification problem with a large number of classes, if an input has a high confidence classification score, the inclusion of the Hessian term has small impacts in the final solution, and the empirical results indicate that considering group features can improve deep learning interpretation significantly.
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
TLDR
A new theoretical framework is developed casting dropout training in deep neural networks (NNs) as approximate Bayesian inference in deep Gaussian processes, which mitigates the problem of representing uncertainty in deep learning without sacrificing either computational complexity or test accuracy.
Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
TLDR
The proposed ODIN method, based on the observation that using temperature scaling and adding small perturbations to the input can separate the softmax score distributions between in- and out-of-distribution images, allowing for more effective detection, consistently outperforms the baseline approach by a large margin.
Orthogonal Gradient Descent for Continual Learning
TLDR
The Orthogonal Gradient Descent (OGD) method is presented, which accomplishes this goal by projecting the gradients from new tasks onto a subspace in which the neural network output on previous task does not change and the projected gradient is still in a useful direction for learning the new task.
Learning Multiple Layers of Features from Tiny Images
TLDR
It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network.
Detecting Extrapolation with Local Ensembles
TLDR
This work focuses on underdetermination as a key component of extrapolation: it aims to detect when many possible predictions are consistent with the training data and model class, and uses local second-order information to approximate the variance of predictions across an ensemble of models from the same class.
Convolutional Networks with Dense Connectivity
TLDR
The Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion, and has several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially improve parameter efficiency.
...
...