TamperNN: Efficient Tampering Detection of Deployed Neural Nets

@article{Merrer2019TamperNNET,
  title={TamperNN: Efficient Tampering Detection of Deployed Neural Nets},
  author={Erwan Le Merrer and Gilles Tr{\'e}dan},
  journal={2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)},
  year={2019},
  pages={424-434}
}
Neural networks are powering the deployment of embedded devices and Internet of Things. Applications range from personal assistants to critical ones such as self-driving cars. It has been shown recently that models obtained from neural nets can be trojaned; an attacker can then trigger an arbitrary model behavior facing crafted inputs. This has a critical impact on the security and reliability of those deployed devices. We introduce novel algorithms to detect the tampering with deployed models… Expand
Adversarial frontier stitching for remote neural network watermarking
TLDR
This paper formally introduces the problem and proposes a novel zero-bit watermarking algorithm that makes use of adversarial model examples, and allows subsequent extraction of the watermark using only few queries. Expand

References

SHOWING 1-10 OF 36 REFERENCES
Trojaning Attack on Neural Networks
TLDR
A trojaning attack on neuron networks that can be successfully triggered without affecting its test accuracy for normal input data, and it only takes a small amount of time to attack a complex neuron network model. Expand
Practical Black-Box Attacks against Machine Learning
TLDR
This work introduces the first practical demonstration of an attacker controlling a remotely hosted DNN with no such knowledge, and finds that this black-box attack strategy is capable of evading defense strategies previously found to make adversarial example crafting harder. Expand
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
TLDR
The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN, and analytically investigates the generalizability and robustness properties granted by the use of defensive Distillation when training DNNs. Expand
Stealing Machine Learning Models via Prediction APIs
TLDR
Simple, efficient attacks are shown that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees against the online services of BigML and Amazon Machine Learning. Expand
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
TLDR
This tutorial introduces the fundamentals of adversarial machine learning to the security community, and presents novel techniques that have been recently proposed to assess performance of pattern classifiers and deep learning algorithms under attack, evaluate their vulnerabilities, and implement defense strategies that make learning algorithms more robust to attacks. Expand
Towards the Science of Security and Privacy in Machine Learning
TLDR
It is shown that there are (possibly unavoidable) tensions between model complexity, accuracy, and resilience that must be calibrated for the environments in which they will be used, and formally explores the opposing relationship between model accuracy and resilience to adversarial manipulation. Expand
Have You Stolen My Model? Evasion Attacks Against Deep Neural Network Watermarking Techniques
TLDR
Verifying the robustness and reliability of state-of- the-art deep neural network watermarking schemes shows that, a malicious adversary, even in scenarios where the watermark is difficult to remove, can still evade the verification by the legitimate owners, thus avoiding the detection of model theft. Expand
Adversarial frontier stitching for remote neural network watermarking
TLDR
This paper formally introduces the problem and proposes a novel zero-bit watermarking algorithm that makes use of adversarial model examples, and allows subsequent extraction of the watermark using only few queries. Expand
Adversarial examples in the physical world
TLDR
It is found that a large fraction of adversarial examples are classified incorrectly even when perceived through the camera, which shows that even in physical world scenarios, machine learning systems are vulnerable to adversarialExamples. Expand
DeepXplore: Automated Whitebox Testing of Deep Learning Systems
TLDR
DeepXplore efficiently finds thousands of incorrect corner case behaviors in state-of-the-art DL models with thousands of neurons trained on five popular datasets including ImageNet and Udacity self-driving challenge data. Expand
...
1
2
3
4
...