ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

@article{He2021ForgeryNetAV,
  title={ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis},
  author={Yinan He and Bei Gan and Siyu Chen and Yichun Zhou and Guojun Yin and Luchuan Song and Lu Sheng and Jing Shao and Ziwei Liu},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={4358-4367}
}
  • Yinan He, Bei Gan, Ziwei Liu
  • Published 9 March 2021
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
The rapid progress of photorealistic synthesis techniques have reached at a critical point where the boundary between real and manipulated images starts to blur. Thus, benchmarking and advancing digital forgery analysis have become a pressing issue. However, existing face forgery datasets either have limited diversity or only support coarse-grained analysis.To counter this emerging threat, we construct the ForgeryNet dataset, an extremely large face forgery dataset with unified annotations in… 
ForgeryNet - Face Forgery Analysis Challenge 2021: Methods and Results
TLDR
Methods and results in the ForgeryNet: Face Forgery Analysis Challenge 2021 which employs the ForgeriesNet benchmark are reported, and the top ranked solutions are analyzed.
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
TLDR
A Multi-modal Multi-scale TRansformer (M2TR), which uses a multi-scale transformer that operates on patches of different sizes to detect the local inconsistency at different spatial levels, which outperforms state-of-the-art Deepfake detection methods.
Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization
TLDR
This work introduces a content driven audio-visual deepfake dataset, termed as Localized Audio Visual DeepFake (LAV-DF), explicitly designed for the task of learning temporal forgery localization and demonstrates the strong performance of the proposed method for both tasks of temporal forgeries localization and deepfake detection.
Countering Malicious DeepFakes: Survey, Battleground, and Horizon
TLDR
A comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of Deepfake detection, with more than 318 research papers carefully surveyed is provided.
DeepFake MNIST+: A DeepFake Facial Animation Dataset
TLDR
A new human face animation dataset is proposed, called DeepFake MNIST+1, generated by a SOTA image animation generator, which includes 10,000 facial animation videos in ten different actions, which can spoof the recent liveness detectors.
Few-shot Forgery Detection via Guided Adversarial Interpolation
TLDR
This work addresses the few-shot forgery detection problem by designing a comprehensive benchmark based on coverage analysis among various forgery approaches, and proposing Guided Adversarial Interpolation (GAI), and enhancing the discriminative ability against novel forgery approach via adversarially interpolating the artifacts of the minority samples to the majority samples under the guidance of a teacher network.
DeepFakes Detection: the DeeperForensicsDeeperForensics Dataset and Challenge
TLDR
This chapter presents the on-going effort of constructing DeeperForensics-1.0, a large-scale forgery detection dataset, to address the challenges above, and describes the detailed challenge information and summarize the winning solutions to take a closer look at the current status and possible future development of real-world face forgery Detection.
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
TLDR
This paper harnesses the natural correspondence between the visual and auditory modalities in real videos to learn temporally dense video representations that capture factors such as facial movements, expression, and identity, and shows that this method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments.

References

SHOWING 1-10 OF 65 REFERENCES
BMN: Boundary-Matching Network for Temporal Action Proposal Generation
TLDR
This work proposes an effective, efficient and end-to-end proposal generation method, named Boundary-Matching Network (BMN), which generates proposals with precise temporal boundaries as well as reliable confidence scores simultaneously, and can achieve state-of-the-art temporal action detection performance.
FaceFeat-GAN: a Two-Stage Approach for Identity-Preserving Face Synthesis
TLDR
FaceFeat-GAN is a novel generative model that improves both image quality and diversity by using two stages, unlike existing single-stage models that map random noise to image directly.
VoxCeleb2: Deep Speaker Recognition
TLDR
A very large-scale audio-visual speaker recognition dataset collected from open-source media is introduced and Convolutional Neural Network models and training strategies that can effectively recognise identities from voice under various conditions are developed and compared.
Looking to listen at the cocktail party
TLDR
A deep network-based model that incorporates both visual and auditory signals to solve a single speech signal from a mixture of sounds such as other speakers and background noise, showing clear advantage over state-of-the-art audio-only speech separation in cases of mixed speech.
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues
TLDR
This work proposes a novel Frequency in Face Forgery Network (F3-Net), taking advantages of two different but complementary frequency-aware clues, and applies DCT as the applied frequency-domain transformation to introduce frequency into the face forgery detection.
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
TLDR
A novel single-shot, multi-level face localisation method, named RetinaFace, which unifies face box prediction, 2D facial landmark localisation and 3D vertices regression under one common target: point regression on the image plane.
The DeepFake Detection Challenge Dataset
TLDR
Although Deep fake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real "in-the-wild" Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos.
DeepFaceLab: A simple, flexible and extensible face swapping framework
TLDR
This paper details the principles that drive the implementation of DeepFaceLab and introduces the pipeline of it, through which every aspect of the pipeline can be modified painlessly by users to achieve their customization purpose, and it's noteworthy that Deep faceLab could achieve results with high fidelity and indeed indiscernible by mainstream forgery detection approaches.
Detecting CNN-Generated Facial Images in Real-World Scenarios
TLDR
This work presents a framework for evaluating detection methods under real-world conditions, consisting of cross-model, cross-data, and post-processing evaluation, and evaluates state-of-the-art detection methods using the proposed framework.
Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
TLDR
An approach for face image generation of virtual people with disentangled, precisely-controllable latent representations for identity of non-existing people, expression, pose, and illumination and introduces contrastive learning to promote disentanglement.
...
...