A Paper List for Deepfake-Detection Research. Given a video w or w/o audio,decide it's true or false, False audio,True video/False video,True audio/False video,False audio/True audio,True video.
The videos fault includes ：Swap face, remove entity, adding entity, tempory edit(Inter-Frame Manipulations), Background/Color cahnge , text changed(last five categories from Videoshame)
The audio fault includes : audio manipulation,audio changed/added
These models use unimodel features to do detect,either video features or audio features.
| Title | Venue | Code | Demo |
|---|---|---|---|
| VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Traces | WACV,2024 | github | - |
| E3: Ensemble of Expert Embedders for Adapting Synthetic Image Detectors to New Generators Using Limited Data | CVPR Workshop,2024 | github | - |
These models use both video and audio features to do detect false video or false audio. And actually, the main task in this way is how to efficiently learn the audio-vision features , a subset of multimodal features learning
| Title | Venue | Code | Demo |
|---|---|---|---|
| AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection | CVPR,2024 | - | - |
| Recurrent Convolutional Structures for Audio Spoof and Video Deepfake Detection | TSP,2020 | github | - |
| PVASS-MDD: Predictive Visual-audio Alignment Self-supervision for Multimodal Deepfake Detection | TCSVT,2023 | - | - |
| Not made for each other– Audio-Visual Dissonance-based Deepfake Detection and Localization | ACM MM,2020 | github | - |
| Joint Audio-Visual Deepfake Detection | ICCV,2021 | - | - |
| FUSION AND ORTHOGONAL PROJECTION FOR IMPROVED FACE-VOICE ASSOCIATION | ICASSP,2022 | github | - |
| CONTRASTIVE AUDIO-VISUAL MASKED AUTOENCODER | ICLR,2023 | github | - |
| AVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake | TIFS,2023 | github | - |
| Lost in Translation: Lip-Sync Deepfake Detection from Audio-Video Mismatch | CVPR Workshop,2024 | - | - |
Disharmonious sounds or videos, as well as their mismatches, can be regarded as anomalous segments in time series. Therefore, deepfake detection can be considered as a time series anomaly detection task
| Title | Venue | Code | Demo |
|---|---|---|---|
| Self-Supervised Video Forensics by Audio-Visual Anomaly Dete | CVPR,2023 | github | demo |
| Title | Venue |
|---|---|
| Generation and detection of manipulated multimodal audiovisual content:Advances, trends and open challenges | Information Fusion,2024 |
| Deepfake Generation and Detection: A Benchmark and Survey | arxiv,2024 |
| Title | Link | Source Link |
|---|---|---|
| Videosham | - | source |