thirdeye

An effort to ensure seeing remains believing

Thirdeye is a comprehensive system for deepfake video detection developed by Mahesha Kulatunga for a MSc Data Analytics dissertation while at The University of Warwick. This code base handles the preprocessing, training and evaluating required when creating neural networks for video classification. Included are 5 pre-trained 3D CNN architectures that can be used for uknown video classification. The following README outlines the basic functionality of the system.

Requirements

Thirdeye requires python3.6 or later. Hardware requirements for Thirdeye are more or less in-line with the requirements for TensorFlow. TensorFlow is tested and supported on the following 64-bit systems:

• Ubuntu 16.04 or later
• Windows 7 or later
• macOS 10.12.6 (Sierra) or later (no GPU support)
• Raspbian 9.0 or later

In addition, around 9GB disk space is required including saved models and the DFD dataset.

Dependencies

Python Version

• 3.6

Software

• FFmpeg

Python Packages

• opencv_python==4.0.0.21
• moviepy==0.2.3.5
• face_recognition==1.2.3
• matplotlib==3.0.2
• pandas==0.23.4
• scipy==1.2.1
• Keras==2.2.4
• tensorflow==1.12.0
• numpy==1.16.1
• scikit_learn==0.21.3

Set up

• Ensure all dependencies are satisfied.
• Navigate to the thirdeye directory in the command line.
• Run the example.py file to run an example classification of sample videos included in the data folder.

python example.py

Use the following code-names to select a network using set_network()

• providence_v1
• providence_v2
• odin_v1
• odin_v2
• horus

For further questions on the network designs or Thirdeye in general, please contact [email protected].

How do I train a network?

• Ensure training files are stored in the ./Data/TRAIN/DF_RAW and ./Data/TRAIN/REAL_RAW
• Ensure testing files are stored in the ./Data/TEST/DF_RAW and ./Data/TEST/REAL_RAW
• Import and create an instance of thirdeye.

import thirdeye  
t = thirdeye.Thirdeye()

• Perform preprocessing on training files.

t.perform_preprocessing()

• Set the network to any desired; choose from providence_v1, providence_v2, odin_v1, odin_v2 and horus.

t.set_network(<network>)

• Initiate training.

t.train()

• This can all be handled with a single command at system initialization.

t = thirdeye.Thirdeye(name=<network>, pre_p=True, force_t=True)

For access to the DFD dataset please contact [email protected].

How do I classify unknown videos?

• Ensure all unknown videos are stored in ./Data/UNKNOWN/UNKNOWN_RAW
• Import thirdeye and pick a network.

import thirdeye
t = thirdeye.Thirdeye(network=<network>)

• Carry out the classification

t.classify()