csun22 / synthetic-voice-detection-vocoder-artifacts

This repository is related to our Dataset and Detection code from the paper: AI-Synthesized Voice Detection Using Neural Vocoder Artifacts accepted in CVPR Workshop on Media Forensic 2023.

Synthetic-Voice-Detection-Vocoder-Artifacts

LibriSeVoc Dataset

1. We are the first to identify neural vocoders as a source of features to expose synthetic human voices. Here are the differences shown by the six vocoders compared to the original audio:

2. We provide LibriSeVoC as a dataset of self-vocoding samples created with six state-of-the-art vocoders to highlight and exploit the vocoder artifacts. The composition of data set is shown in the following table: The source of our dataset ground truth comes from LibriTTS. Therefore, we follow the naming logic of LibriTTS. For example: 27_123349_000006_000000.wav, 27 is the ID of the reader, and 123349 is the ID of chapter.

Deepfake Detection

We propose a new approach to detecting synthetic human voices by exposing signal artifacts left by neural vocoders by modifying and improved the RawNet2 baseline by adding multi-loss, lowering the error rate from 6.10% to 4.54% on ASVspoof Dataset. This is the framework of the proposed synthesized voice detection method:

Paper & Dataset

For more details please read our paper: https://openaccess.thecvf.com/content/CVPR2023W/WMF/html/Sun_AI-Synthesized_Voice_Detection_Using_Neural_Vocoder_Artifacts_CVPRW_2023_paper.html

For more details please download our dataset: https://drive.google.com/file/d/1NXF9w0YxzVjIAwGm_9Ku7wfLHVbsT7aG/view

synthetic-voice-detection-vocoder-artifacts's Issues

Release of pre-trained models

Hello
Thanks for dataset, can you release pre-trained models for vocoder detection system?

Regards
Ajinkya

Train.py script

Where is the train.py script and other remaining scripts?

Name of vocoders training datasets

Hello,

Thank you for sharing this dataset.

Would it be possible to have more information on the generation of audios? In particular, the names of the vocoders training datasets used.

Thank you.

Tony

Clarification Needed on Intra-dataset vs Cross-dataset Evaluation Metrics in Paper

I have some questions regarding the evaluation metrics and results presented in Sections 4.4 and 4.5.

Intra-dataset Evaluation (Section 4.4)

The paper reports a very low EER of 0.19% on the WaveFake dataset using the RawNet2 model.

• To confirm my understanding, was this evaluation performed with the model being trained and tested on the same WaveFake dataset?

Cross-dataset Evaluation (Section 4.5)

On the other hand, the EER significantly increased to 26.95% when the model trained on the LibriSeVoc dataset was tested on the WaveFake dataset. This suggests poor generalization to unseen data.

• Are there any ongoing efforts to improve this aspect of the model, perhaps through domain adaptation techniques or exposure to a more diverse set of vocoders during training?

full script

Thank you for the awesome project. Could you provide a full script to me ([email protected]), I would like to do some experiment.

Thanks

Request for Training and Evaluation Code

I'm interested in your CVPR paper and would like to replicate your experiments. Could you please upload the training and evaluation code?

Hello, could you please provide the split files for training, dev, and test splits in Table 2?