This is the supplementary source code for our paper Wavelet-Packet Powered Deepfake Image Detection, which is currently under review.
The plot above illustrates the fundamental principle. It shows an FFHQ and a style-gan-generated image on the very left. In the center and on the right, packet coefficients and their standard deviation are depicted. We computed mean and standard deviation values using 5k images from each source.
The latest code can be installed in development mode with:
$ git clone https://github.com/gan-police/frequency-forensics
$ cd frequency-forensics
$ pip install -e .
We utilize pre-trained models from the following repositories:
For our wavelet-packet computations, we build upon :
In the paper, we compare our approach to the DCT-method from:
We utilize three datasets that commonly appeared in previous work:
The following section of the README serves as a guide to reproducing our paper.
We work with images of the size 128x128 pixels. Hence, the raw images from the LSUN/CelebA data set have to be cropped
and/or resized to this size. To do this, run freqdect.crop_celeba
or freqdect.crop_lsun
, depending on the dataset.
This will create a new folder with the transformed images. The FFHQ dataset is already distributed in the required image
size.
Use the pretrained GAN-models to generate images. In case of StyleGAN, there is only a pre-trained model generating images of size 1024x1024, so one has to resize the GAN-generated images to size 128x128 pixels, e.g. by inserting
PIL.Image.fromarray(images[0], 'RGB').resize((128, 128)).save(png_filename)
into the ffhq-stylegan .
Store all images (cropped original and GAN-generated) in a separate subdirectories of a directory, i.e. the directory structure should look like this
source_data
├── A_original
├── B_CramerGAN
├── C_MMDGAN
├── D_ProGAN
└── E_SNGAN
For the FFHQ case, we have only two subdirectories: ffhq_stylegan/A_ffhq
and ffhq_stylegan/B_stylegan
. The prefixes
of the folders are important, since the directories get the labels in lexicographic order of their prefix, i.e.
directory A_...
gets label 0, B_...
label 1, etc.
Now, to prepare the data sets run freqdect.prepare_dataset
. It reads in the data set, splits them into a training,
validation and test set, applies the specified transformation (to wavelet packets, log-scaled wavelet packets or just
the raw image data) and stores the result as numpy arrays.
Afterwards run e.g.:
$ python -m freqdect.prepare_dataset ./data/ffhq_stylegan/ --log-packets
$ python -m freqdect.prepare_dataset ./data/ffhq_stylegan/
The dataset preparation script accepts additional arguments. For example, it is possible to change the sizes of the
train, test or validation sets. For a list of all optional arguments, open the help page via the -h
argument.
Now you should be able to train a classifier using for example:
$ python -m freqdect.train_classifier \
--data-prefix ./data/source_data_packets \
--calc-normalization \
--features packets
This trains a regression classifier using default hyperparameters. The training, validation and test accuracy and loss
values are stored in a file placed in a log
folder. The state dict of the trained model is stored there as well. For a
list of all optional arguments, open the help page via the -h
argument.
To plot the accuracy results, run:
$ python -m freqdect.plot_accuracy_results {shared, lsun, celeba} {regression, CNN} ...
For a list of all optional arguments, open the help page via the -h
argument.
To calculate the confusion matrix, run freqdect.confusion_matrix
. For a list of all arguments, open the help page via
the -h
argument.
This project is licensed under the GNU GPLv3 license
If you find this work useful please consider citing:
@misc{wolter2021waveletpacket,
title={Wavelet-Packet Powered Deepfake Image Detection},
author={Moritz Wolter and Felix Blanke and Charles Tapley Hoyt and Jochen Garcke},
year={2021},
eprint={2106.09369},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
This project has been supported by the following organizations (in alphabetical order):
- Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)
- Fraunhofer Cluster of Excellence Cognitive Internet Technologies (CCIT)
- Harvard Program in Therapeutic Science - Laboratory of Systems Pharmacology
This project has been funded by the following grants:
Funding Body | Program | Grant |
---|---|---|
DARPA | Young Faculty Award (PI: Benjamin M. Gyori) | W911NF2010255 |
This package was created with @audreyfeldroy's cookiecutter package using @cthoyt's cookiecutter-snekpack template.