This Face Does Not Exist... But It Might Be Yours! Identity Leakage in Generative Models

@article{Tinsley2021ThisFD,
  title={This Face Does Not Exist... But It Might Be Yours! Identity Leakage in Generative Models},
  author={Patrick J. Tinsley and Adam Czajka and Patrick J. Flynn},
  journal={2021 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  year={2021},
  pages={1319-1327}
}
Generative adversarial networks (GANs) are able to generate high resolution photo-realistic images of objects that "do not exist." These synthetic images are rather difficult to detect as fake. However, the manner in which these generative models are trained hints at a potential for information leakage from the supplied training data, especially in the context of synthetic faces. This paper presents experiments suggesting that identity information in face images can flow from the training… 

Figures and Tables from this paper

Fooling a Face Recognition System with a Marker-Free Label-Consistent Backdoor Attack
TLDR
This paper generates a label-consistent poisoned dataset, where the poisoned images matches their labels and are di cult to spot by a quick visual inspection, and shows that the network finetuned on the poisoned dataset is successfully fooled, identifying one of the author as a specific target identity.
PrintsGAN: Synthetic Fingerprint Generator
TLDR
PrintsGAN is proposed, a synthetic fingerprint generator capable of generating unique fingerprints along with multiple impressions for a given fingerprint, and the utility of the PrintsGAN generated dataset is shown by training a deep network to extract a fixed-length embedding from a fingerprint.
When do GANs replicate? On the choice of dataset size
TLDR
It is shown that dataset size and its complexity play an important role in GANs replication and perceptual quality of the generated images and a practical tool for one-shot estimation on minimal dataset size is found which can be used to guide datasets construction and selection.
High-Resolution Image Synthesis with Latent Diffusion Models
TLDR
These latent diffusion models achieve new state-of-the-art scores for image inpainting and class-conditional image synthesis and highly competitive performance on various tasks, includ-ing text-to-image synthesis, unconditional image generation and super-resolution, while significantly reducing computational requirements compared to pixel-based DMs.
Biometrics: Trust, but Verify
TLDR
Insightful insights are provided into how the biometric community can address core biometric recognition systems design issues to better instill trust, fairness, and security for all.
GAUDI: A Neural Architect for Immersive 3D Scene Generation
TLDR
GAUDI obtains state-of-the-art performance in the unconditional generative setting across multiple datasets and allows for conditional generation of 3D scenes given conditioning variables like sparse image observations or text that describes the scene.
Analyzing the Impact of Shape & Context on the Face Recognition Performance of Deep Networks
—In this article, we analyze how changing the un- derlying 3D shape of the base identity in face images can distort their overall appearance, especially from the perspective of deep face recognition.

References

SHOWING 1-10 OF 32 REFERENCES
GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection
TLDR
The results obtained in the empirical evaluation show that additional efforts are required to develop robust facial manipulation detection systems against unseen conditions and spoof techniques, such as the one proposed in this study.
Differentially Private Data Generative Models
TLDR
It is demonstrated that both DP-AuGM and DP-VaeGM can be easily integrated with real-world machine learning applications, such as machine learning as a service and federated learning, which are otherwise threatened by the membership inference attack and the GAN-based attack, respectively.
A Style-Based Generator Architecture for Generative Adversarial Networks
TLDR
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation.
VGGFace2: A Dataset for Recognising Faces across Pose and Age
TLDR
A new large-scale face dataset named VGGFace2 is introduced, which contains 3.31 million images of 9131 subjects, with an average of 362.6 images for each subject, and the automated and manual filtering stages to ensure a high accuracy for the images of each identity are described.
GANobfuscator: Mitigating Information Leakage Under GAN via Differential Privacy
TLDR
GANobfuscator, a differentially private GAN, which can achieve differential privacy under GANs by adding carefully designed noise to gradients during the learning procedure, is proposed and theoretically proves that GANob obfuscator can provide strict privacy guarantee with differential privacy.
Generative Adversarial Nets
We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a
SphereFace: Deep Hypersphere Embedding for Face Recognition
TLDR
This paper proposes the angular softmax (A-Softmax) loss that enables convolutional neural networks (CNNs) to learn angularly discriminative features in deep face recognition (FR) problem under open-set protocol.
Analyzing and Improving the Image Quality of StyleGAN
TLDR
This work redesigns the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images, and thereby redefines the state of the art in unconditional image modeling.
Learning Face Representation from Scratch
TLDR
A semi-automatical way to collect face images from Internet is proposed and a large scale dataset containing about 10,000 subjects and 500,000 images, called CASIAWebFace is built, based on which a 11-layer CNN is used to learn discriminative representation and obtain state-of-theart accuracy on LFW and YTF.
FaceNet: A unified embedding for face recognition and clustering
TLDR
A system that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure offace similarity, and achieves state-of-the-art face recognition performance using only 128-bytes perface.
...
...