Deepfake Videos in the Wild: Analysis and Detection

@article{Pu2021DeepfakeVI,
  title={Deepfake Videos in the Wild: Analysis and Detection},
  author={Jiameng Pu and Neal Mangaokar and Lauren Kelly and Parantapa Bhattacharya and Kavya Sundaram and Mobin Javed and Bolun Wang and Bimal Viswanath},
  journal={Proceedings of the Web Conference 2021},
  year={2021}
}
AI-manipulated videos, commonly known as deepfakes, are an emerging problem. Recently, researchers in academia and industry have contributed several (self-created) benchmark deepfake datasets, and deepfake detection algorithms. However, little effort has gone towards understanding deepfake videos in the wild, leading to a limited understanding of the real-world applicability of research contributions in this space. Even if detection schemes are shown to perform well on existing datasets, it is… 

Figures and Tables from this paper

A Novel Method of Deepfake Detection

The findings show that the detection are generally domain-specific tasks, however that using Transfer Learning considerably improves the model performance parameters, whereas convolutional RNN gives sequence detection advantage.

The MeVer DeepFake Detection Service: Lessons Learnt from Developing and Deploying in the Wild

The MeVer DeepFake detection service is introduced, a web service detecting deep learning manipulations in images and video and shows that the service performs robustly on the three benchmark datasets while being vulnerable to Adversarial Attacks.

An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detection

A fake talking face detection framework FTFDNet is proposed by incorporating audio and visual representation to achieve more accuratefake talking face videos detection and an audio-visual attention mechanism (AVAM) is proposed to discover more informative features, which can be seamlessly integrated into any audio- visual CNN architectures by modularization.

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

A comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of Deepfake detection, with more than 318 research papers carefully surveyed is provided.

Event‐Based Training in Label‐Limited Regimes

  • L. Linville
  • Computer Science
    Journal of Geophysical Research: Solid Earth
  • 2021
This work develops a way to leverage distributional information across a set of sensors in the absence of comprehensive annotation as a domain‐informed regularization term applied during gradient‐based learning.

Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors

It is concluded through detailed experimentation that unimodals, addressing only a single modality, video or audio, do not perform well compared to ensemble-based methods, whereas purely multimodal-based baselines provide the worst performance.

FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset

A novel Audio-Video Deepfake dataset (FakeAVCeleb) is proposed that not only contains deepfake videos but respective synthesized cloned audios as well and proposes a novel multimodal detection method that detects deep fake videos and audios based on this dataset.

References

SHOWING 1-10 OF 57 REFERENCES

LightFace: A Hybrid Deep Face Recognition Framework

A review of face recognition has been done and the description of the developed lightweight hybrid high performance face recognition framework enables to switch face recognition models among state-of-the-art ones.

DeepFakes: a New Threat to Face Recognition? Assessment and Detection

This paper presents the first publicly available set of Deepfake videos generated from videos of VidTIMIT database, and demonstrates that GAN-generated Deep fake videos are challenging for both face recognition systems and existing detection methods.

MesoNet: a Compact Facial Video Forgery Detection Network

A method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face.

The DeepFake Detection Challenge Dataset

Although Deep fake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real "in-the-wild" Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos.

Use of a Capsule Network to Detect Fake Images and Videos

A capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning, uses many fewer parameters than traditional convolutional neural networks with similar performance.

Celeb-DF: A New Dataset for DeepFake Forensics

This work presents a new DeepFake dataset, Celeb-DF, for the development and evaluation of DeepFake detection algorithms, generated using a refined synthesis algorithm that reduces the visual artifacts observed in existing datasets.

Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations

It is shown that relatively simple visual artifacts can be already quite effective in exposing such manipulations, including Deepfakes and Face2Face.

FaceForensics++: Learning to Detect Manipulated Facial Images

This paper proposes an automated benchmark for facial manipulation detection, and shows that the use of additional domain-specific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.

WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection

WildDeepfake is a small dataset that can be used, in addition to existing datasets, to develop and test the effectiveness of deepfake detectors against real-world deepfakes, and proposes two (eg. 2D and 3D) Attention-based Deepfake Detection Networks (ADDNets) to leverage the attention masks on real/fake faces for improved detection.

Face Recognition: Too Bias, or Not Too Bias?

A human evaluation to measure bias in humans is done, which supports the hypothesis that an analogous bias exists in human perception.
...