How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals

  title={How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals},
  author={Umur Aybars Ciftci and Ilke Demir and Lijun Yin},
  journal={2020 IEEE International Joint Conference on Biometrics (IJCB)},
Fake portrait video generation techniques have been posing a new threat to the society with photorealistic deep fakes for political propaganda, celebrity imitation, forged evidences, and other identity related manipulations. Following these generation techniques, some detection approaches have also been proved useful due to their high classification accuracy. Nevertheless, almost no effort was spent to track down the source of deep fakes. We propose an approach not only to separate deep fakes… 

Figures and Tables from this paper

Review of Deepfake Detection Techniques

Methods of deepfake detection are evaluated and how they can be combined or modified to get more accurate results are discussed.

How Do Deepfakes Move? Motion Magnification for Deepfake Source Detection

This work contrasts the movement in deepfakes and authentic videos by motion magnification towards building a generalized deepfake source detector, and exploits the difference between real motion and the amplifier GANs to detect whether a video is fake and its source generator if so.

Deepfake Detection using Biological Features: A Survey

The history of the Deepfake is described and biological features such as eyebrow recognition, eye blinking detection, eye movement detection, ear and mouth detection, and heartbeat detection are clearly described and a scope in this domain is proposed.

FakeTransformer: Exposing Face Forgery From Spatial-Temporal Representation Modeled By Facial Pixel Variations

This work attempts to find inconsistent patterns that can distinguish between real videos and synthetic videos from the variations of facial pixels, which are highly related to physiological information.

Using Deep Learning to Detecting Deepfakes

This study looks at various deepfake detection models that use deep learning algorithms to combat this looming threat of deepfakes and the unique approaches many researchers take to solving this problem.

Protecting World Leader Using Facial Speaking Pattern Against Deepfakes

This paper proposes a dual stream learning facial and speaking patterns method to protect celebrities against deepfakes, which achieves better performance and has a higher resistance to video compression than state-of-the-art detection models.

Detecting Deepfake Videos with Temporal Dropout 3DCNN

A Temporal Dropout 3Dimensional Convolutional Neural Network (TD3DCNN) is proposed to detect deepfake videos and can enhance the representation and generalization ability, avoiding model overfitting and improving detecting accuracy.

Deepfake Video Detection Based on EfficientNet-V2 Network

This paper proposes using a new network of EfficientNet-V2 to distinguish the authenticity of pictures and videos, and highlights the superior performance of the new network by comparing the existing detection network with the actual training and testing results.

The Creation and Detection of Deepfakes

This article explores the creation and detection of deepfakes and provides an in-depth view as to how these architectures work and the current trends and advancements in this domain.

A Review of Image Processing Techniques for Deepfakes

This study aims to highlight the recent research in deepfake images and video detection, such as deepfake creation, various detection algorithms on self-made datasets, and existing benchmark datasets, as well as how to counter the threats from deepfake technology and alleviate its impact.



FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals

It is shown that biological signals hidden in portrait videos can be used as an implicit descriptor of authenticity, because they are neither spatially nor temporally preserved in fake content.

Exposing DeepFake Videos By Detecting Face Warping Artifacts

A new deep learning based method that can effectively distinguish AI-generated fake videos from real videos is described, which saves a plenty of time and resources in training data collection and is more robust compared to others.

DeepFake Detection by Analyzing Convolutional Traces

This work focuses on the analysis of Deepfakes of human faces with the objective of creating a new detection method able to detect a forensics trace hidden in images: a sort of fingerprint left in the image generation process.

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection

The results obtained in the empirical evaluation show that additional efforts are required to develop robust facial manipulation detection systems against unseen conditions and spoof techniques, such as the one proposed in this study.

Deepfake: A Survey on Facial Forgery Technique Using Generative Adversarial Network

Working of Deepfake technique along with how it can swap faces with maximum precision in the video has been presented and its advantages and drawback have been listed.

Deepfake Video Detection Using Recurrent Neural Networks

  • David GueraE. Delp
  • Computer Science
    2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)
  • 2018
A temporal-aware pipeline to automatically detect deepfake videos is proposed that uses a convolutional neural network to extract frame-level features and a recurrent neural network that learns to classify if a video has been subject to manipulation or not.

Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints

The first study of learning GAN fingerprints towards image attribution and using them to classify an image as real or GAN-generated is presented, showing that GANs carry distinct model fingerprints and leave stable fingerprints in their generated images, which support image attribution.

FaceForensics++: Learning to Detect Manipulated Facial Images

This paper proposes an automated benchmark for facial manipulation detection, and shows that the use of additional domain-specific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.

Do GANs Leave Artificial Fingerprints?

It is shown that each GAN leaves its specific fingerprint in the images it generates, just like real-world cameras mark acquired images with traces of their photo-response non-uniformity pattern.

Detecting GAN-Generated Imagery Using Saturation Cues

The structure of the generating network of a popular GAN implementation is analyzed, and it is shown that the network’s treatment of exposure is markedly different from a real camera, including effective discrimination between GAN imagery and real camera images used to train the GAN.