Enabling Homomorphically Encrypted Inference for Large DNN Models

  title={Enabling Homomorphically Encrypted Inference for Large DNN Models},
  author={Guillermo Lloret-Talavera and Marc Jord{\`a} and Harald Servat and Fabian Boemer and Chetan Chauhan and Shigeki Tomishima and Nilesh N. Shah and Antonio J. Pe{\~n}a},
The proliferation of machine learning services in the last few years has raised data privacy concerns. Homomorphic encryption (HE) enables inference using encrypted data but it incurs 100x-10,000x memory and runtime overheads. Secure deep neural network (DNN) inference using HE is currently limited by computing and memory resources, with frameworks requiring hundreds of gigabytes of DRAM to evaluate small models. To overcome these limitations, in this paper we explore the feasibility of… Expand

Figures and Tables from this paper

CryptInfer: Enabling Encrypted Inference on Skin Lesion Images for Melanoma Detection
Results demonstrate that privacy-preserving machine learning as a service (MLaaS) based on encrypted data is indeed practically feasible, and a modified LeNet-like architecture is designed and implemented to achieve the end goal of enabling encrypted inference on melanoma dataset. Expand
Fighting COVID-19 in the Dark: Methodology for Improved Inference Using Homomorphically Encrypted DNN
This work proposes a structured methodology to replace ReLU with a quadratic polynomial activation, and uses a pre-trained model that trains another HE-friendly model, using techniques such as ’trainable activation’ functions and knowledge distillation to address the accuracy degradation issue. Expand
Lasagna: Accelerating Secure Deep Learning Inference in SGX-enabled Edge Cloud
  • Yuepeng Li, Deze Zeng, +4 authors Minyi Guo
  • Computer Science
  • SoCC
  • 2021
Lasagna, an SGX oriented DNN inference performance acceleration framework without compromising the task security, which consists of a local task scheduler and a global task balancer to optimize the system performance by exploring the layered-structure of DNN models. Expand


nGraph-HE2: A High-Throughput Framework for Neural Network Inference on Encrypted Data
The proposed nGraph-HE2 framework leverages the CKKS scheme, whose support for real numbers is friendly to data science, and a client-aided model using a two-party approach to compute activation functions to enable privacy-preserving inference on standard, pre-trained models using their native activation functions and number fields. Expand
Low Latency Privacy Preserving Inference
This work applies the method of transfer learning to provide private inference services using deep networks with latency of ∼0.16 seconds and presents more than 10× improvement in latency and enable inference on wider networks compared to prior attempts with the same level of security. Expand
nGraph-HE: a graph compiler for deep learning on homomorphically encrypted data
Homomorphic encryption (HE)---the ability to perform computation on encrypted data---is an attractive remedy to increasing concerns about data privacy in deep learning (DL). However, building DLExpand
CHET: an optimizing compiler for fully-homomorphic neural-network inferencing
CHET is a domain-specific optimizing compiler designed to make the task of programming FHE applications easier, and generates homomorphic circuits that outperform expert-tuned circuits and makes it easy to switch across different encryption schemes. Expand
Gazelle: A Low Latency Framework for Secure Neural Network Inference
Gazelle is designed, a scalable and low-latency system for secure neural network inference, using an intricate combination of homomorphic encryption and traditional two-party computation techniques (such as garbled circuits). Expand
Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference
This work develops a pruning and quantization approach that leverages sparse representations in the underlying cryptosystem to accelerate inference and derives an optimal approximation for popular activation functions that achieves maximally-sparse encodings and minimizes approximation error. Expand
XONN: XNOR-based Oblivious Deep Neural Network Inference
XONN is first to perform oblivious inference on Fitnet architectures with up to 21 layers, suggesting a new level of scalability compared with state-of-the-art, and evaluates XONN on four datasets to perform privacy-preserving medical diagnosis. Expand
SEALion: a Framework for Neural Network Inference on Encrypted Data
We present SEALion: an extensible framework for privacy-preserving machine learning with homomorphic encryption. It allows one to learn deep neural networks that can be seamlessly utilized forExpand
MP2ML: a mixed-protocol machine learning framework for private inference
MP2ML is a machine learning framework which integrates nGraph-HE and the secure two-party computation framework ABY to execute DL inference while maintaining the privacy of both the input data and model weights and is compatible with popular DL frameworks such as TensorFlow. Expand
CryptoDL: Deep Neural Networks over Encrypted Data
New techniques to adopt deep neural networks within the practical limitation of current homomorphic encryption schemes are developed and show that CryptoDL provides efficient, accurate and scalable privacy-preserving predictions. Expand