• Corpus ID: 235669893

Certifiable Machine Unlearning for Linear Models

  title={Certifiable Machine Unlearning for Linear Models},
  author={Ananth Mahadevan and Michael Mathioudakis},
ABSTRACT Machine unlearning is the task of updating machine learning (ML) models after a subset of the training data they were trained on is deleted. Methods for the task are desired to combine effectiveness and efficiency, i.e., they should effectively ‘unlearn’ deleted data, but in a way that does not require excessive computational effort (e.g., a full retraining) for a small amount of deletions. Such a combination is typically achieved by tolerating some amount of approximation in the… 
Zero-Shot Machine Unlearning
This paper introduces the novel problem of zero-shot machine unlearning that caters for the extreme but practical scenario where zero original data samples are available for use, and proposes two novel solutions based on error minimizing-maximizing noise and gated knowledge transfer.
Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher
This work proposes a novel machine unlearning method by exploring the utility of competent and incompetent teachers in a student-teacher framework to induce forgetfulness, and introduces a zero retrain forgetting (ZRF) metric to evaluate the un learning method.
Learning to Refit for Convex Learning Problems
This work focuses on convex learning problems and proposes a general framework to learn to estimate optimized model parameters for different training sets using neural networks and rigorously characterize the expressive power of neural networks to approximate the optimizer of convex problems.
The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining
This paper starts with a formal definition of the unlearning problem in FL and proposes a rapid retraining approach to fully erase data samples from a trained FL model, which allows data holders to jointly conduct the un learning process efficiently while keeping their training data locally.
Fast Yet Effective Machine Unlearning
A novel machine unlearning framework with error-maximizing noise generation and impair-repair based weight manipulation that offers an efficient solution to the above questions and works well in both small and large-scale vision tasks.


Certifiable Machine Unlearning for Linear Models [Experiment, Analysis & Benchmark Papers]
An experimental study of the three stateof-the-art approximate unlearning methods for linear models and the trade-offs between efficiency, effectiveness and certifiability offered by each method is presented.
Descent-to-Delete: Gradient-Based Methods for Machine Unlearning
This work gives the first data deletion algorithms that are able to handle an arbitrarily long sequence of adversarial updates while promising both per-deletion run-time and steady-state error that do not grow with the length of the update sequence.
“Amnesia” – Towards Machine Learning Models That Can Forget User Data Very Fast
This work formulates the problem of “decrementally” updating trained ML models to “forget” the data of a user, and presents efficient decremental update procedures for three popular ML algorithms.
DeltaGrad: Rapid retraining of machine learning models
This work proposes the DeltaGrad algorithm for rapid retraining machine learning models based on information cached during the training phase, and shows that it compares favorably to the state of the art.
Towards Making Systems Forget with Machine Unlearning
This paper presents a general, efficient unlearning approach by transforming learning algorithms used by a system into a summation form, and applies to all stages of machine learning, including feature selection and modeling.
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
An upper-bound on the amount of information remaining in the weights is introduced, which can be estimated efficiently even for deep neural networks, and is a generalized and weaker form of Differential Privacy.
DART: Data Addition and Removal Trees
This paper introduces DART, a variant of random forests that supports adding and removing training data with minimal retraining, and finds that DART is orders of magnitude faster than retraining from scratch while sacrificing very little in terms of predictive performance.
Adam: A Method for Stochastic Optimization
This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
A Parallel Mixture of SVMs for Very Large Scale Problems
This article proposes a new mixture of SVMs that can be easily implemented in parallel and where each SVM is trained on a small subset of the whole data set.
Multiple Incremental Decremental Learning of Support Vector Machines
An extension of incremental decremental algorithm which efficiently works for simultaneous update of multiple data points for online SVM learning in which the authors need to remove old data points and add new data points in a short time.