Proof-of-Learning: Definitions and Practice

  title={Proof-of-Learning: Definitions and Practice},
  author={Hengrui Jia and Mohammad Yaghini and Christopher A. Choquette-Choo and Natalie Dullerud and Anvith Thudi and Varun Chandrasekaran and Nicolas Papernot},
  journal={2021 IEEE Symposium on Security and Privacy (SP)},
Training machine learning (ML) models typically involves expensive iterative optimization. Once the model’s final parameters are released, there is currently no mechanism for the entity which trained the model to prove that these parameters were indeed the result of this optimization procedure. Such a mechanism would support security of ML applications in several ways. For instance, it would simplify ownership resolution when multiple parties contest ownership of a specific model. It would also… 

On the Fundamental Limits of Formally (Dis)Proving Robustness in Proof-of-Learning

It is shown that, until the aforementioned open problems are addressed, relying more heavily on cryptography is likely needed to formulate a new class of PoL protocols with formal robustness guarantees, and that establishing precedence robustly also reduces to an open problem in learning theory.

An Introduction to Machine Unlearning

Seven state-of-the-art machine unlearning algorithms are summarized and compared, core concepts of core concepts used in the “machine unlearning” literature are consolidated, approaches for evaluating algorithms are reconciled, and issues related to applying machine un learning in practice are discussed.

Dataset Inference for Self-Supervised Models

This work introduces a new dataset inference defense, which uses the private training set of the victim encoder model to attribute its ownership in the event of stealing, and demonstrates that dataset inference is a promising direction for defending self-supervised models against model stealing.

A Survey of Machine Unlearning

This survey paper seeks to provide a thorough investigation of machine unlearning in its definitions, scenarios, mechanisms, and applications as a categorical collection of state-of-the-art research for ML researchers as well as those seeking to innovate privacy technologies.


This paper proposes to compute distance between black-box models by comparing their Local Interpretable Model-Agnostic Explanations (LIME), and empirically shows that the method, which is called Zest, helps in several tasks that require measurements of model similarity: verifying machine unlearning, and detecting many forms of model reuse.

Dataset Obfuscation: Its Applications to and Impacts on Edge Machine Learning

Comprehensive experiments are conducted to investigate how the dataset obfuscation can affect the resultant model weights and indicate broad application prospects for training outsourcing in edge computing and guarding against attacks in Federated Learning among edge devices.

On the Difficulty of Defending Self-Supervised Learning against Model Extraction

This work constructs several novel attacks against model stealing attacks against self-Supervised Learning and shows that existing defenses against model extraction are inadequate and not easily retro-fitted to the specificities of SSL.

Trustworthy Graph Neural Networks: Aspects, Methods and Trends

A comprehensive roadmap to build trustworthy GNNs from the view of the various computing technologies involved is proposed, including robustness, explainability, privacy, fairness, accountability, and environmental well-being.

Unownability of AI : Why Legal Ownership of Artificial Intelligence is Hard

It is concluded that it is difficult if not impossible to establish ownership claims over AI models beyond a reasonable doubt.

Fingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbations

A novel and practical mecha-nism to enable the service provider to verify whether a suspect model is stolen from the victim model via model extraction attacks and train an encoder via contrastive learning that takes fingerprints as inputs, outputs a similarity score.



Learning Multiple Layers of Features from Tiny Images

It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network.

Pricing via Processing or Combatting Junk Mail

We present a computational technique for combatting junk mail in particular and controlling access to a shared resource in general. The main idea is to require a user to compute a moderately hard,

Stealing Machine Learning Models via Prediction APIs

Simple, efficient attacks are shown that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees against the online services of BigML and Amazon Machine Learning.

Dataset Inference: Ownership Resolution in Machine Learning

This paper introduces dataset inference, the process of identifying whether a suspected model copy has private knowledge from the original model’s dataset, as a defense against model stealing, and develops an approach for dataset inference that combines statistical testing with the ability to estimate the distance of multiple data points to the decision boundary.

A Survey on Model Watermarking Neural Networks

This document at hand provides the first extensive literature review on ML model watermarking schemes and attacks against them, which offers a taxonomy of existing approaches and systemizes general knowledge around them, and assembles the security requirements to water marking approaches.

vCNN: Verifiable Convolutional Neural Network

The proposed vCNNs framework is the first practical pairing-based zk-SNARK scheme for CNNs, and it significantly reduces space and time complexities to generate a proof with providing perfect zero-knowledge and computational knowledge soundness.

Bitcoin : A Peer-to-Peer Electronic Cash System

Entangled Watermarks as a Defense against Model Extraction

Entangled Watermarking Embeddings (EWE) is introduced, which encourages the model to learn common features for classifying data that is sampled from the task distribution, but also data that encodes watermarks, which forces an adversary attempting to remove watermarks that are entangled with legitimate data to sacrifice performance on legitimate data.

Piracy Resistant Watermarks for Deep Neural Networks.

This work proposes null embedding, a new way to build piracy-resistant watermarks into DNNs that can only take place at a model's initial training, and empirically shows that the proposed watermarks achieve piracy resistance and other watermark properties, over a wide range of tasks and models.

On the Intrinsic Privacy of Stochastic Gradient Descent

This work takes the first step towards analysing the intrinsic privacy properties of stochastic gradient descent (SGD), and proposes a method to augment the intrinsic noise of SGD to achieve the desired $\epsilon$.