theolepage / sslsv

Framework for training and evaluating self-supervised learning methods for speaker verification.

sslsv

Collection of self-supervised learning (SSL) methods for speaker verification (SV).

Resources

Models

• sslsv.model.ThinResNet34
  "Delving into VoxCeleb: environment invariant speaker recognition" (arxiv)
  Joon Son Chung, Jaesung Huh, Seongkyu Mun

Losses

• sslsv.losses.InfoNCE
  "Representation Learning with Contrastive Predictive Coding" (arxiv)
  Aaron van den Oord, Yazhe Li, Oriol Vinyals

• sslsv.losses.VICReg
  "VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning" (arxiv)
  Adrien Bardes, Jean Ponce, Yann LeCun

• sslsv.losses.BarlowTwins
  "Barlow Twins: Self-Supervised Learning via Redundancy Reduction" (arxiv)
  Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, Stéphane Deny

Datasets

VoxCeleb1 and VoxCeleb2 are used for our experiments and we rely on MUSAN and Room Impulse Response and Noise Database for data augmentation.

To download, extract and prepare all datasets run python prepare_data.py data/. The data/ directory will have the structure detailed below.

data
├── musan_split/
├── simulated_rirs/
├── voxceleb1/
├── voxceleb2/
├── trials
├── voxceleb1_train_list
└── voxceleb2_train_list

Trials and train lists files are also automatically created with the following formats.

• trials

  1 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00008.wav
  ...
  0 id10309/0cYFdtyWVds/00005.wav id10296/Y-qKARMSO7k/00001.wav
  

• voxceleb1_train_list and voxceleb2_train_list

  id00012 voxceleb2/id00012/21Uxsk56VDQ/00001.wav
  ...
  id09272 voxceleb2/id09272/u7VNkYraCw0/00027.wav
  

Please refer to prepare_data.py script if you want further details about data preparation.

Usage

Start self-supervised training with python train.py configs/vicreg_b256.yml.

To-Do

• DDP: adapt losses and supervised sampler
• Refactor evaluation (use AudioDataset class for handling test data)
• Documentation, comments, typing

Credits

Some parts of the code (data preparation, data augmentation and model evaluation) were adapted from VoxCeleb trainer repository.

sslsv's Issues

voxcele2

Hi dear, I didn't see any detail about voxcele2 in your article, why the repo use it?

Hi, which config can result in best eer in the article?

I try vicreg_b256_comp_2.yml config, but I seem cant get 8.47 eer, what is wrong? So I want to know best config ~

Supervised version

Hi Théo,

I was impressed by your sslsv project.
In the README.md, self supervised version is well introduced. (Maybe, " python train.py configs/vicreg_b256.yml ")
If i want to use the supervised version, how can I use the supervised version of contrastive learning?
I think if dist.is_available() and dist.is_initialized() are true, it's for self supervised learning. And if Distributed is false, it's for supervised learning. Is it right?

I'll be waiting for the reply.

ThinResNet34

Thanks for your open-source code. I found that your ThinResNet34 implementation is different from that in your paper. I wonder if you have a revised version.

Can you open the checkpoints?

Hi @theolepage

I want to try fine-tuning. If you can open the ckpt, it'll really help me a lot.

Thank you checking my question

sorry to bother you

Hi, I would like to know the training time of your entire experiment with specific equipment, can you tell me?

can‘t find the code about this paper，“Experimenting with Additive Margins for ContrastiveSelf-Supervised Speaker Verification“

Hi，may be I can‘t find the code about this paper，“Experimenting with Additive Margins for ContrastiveSelf-Supervised Speaker Verification”，I want to see the loss function in this paper，can you open this code？

sorry

sorry, I can't understand the code for using the annotated label in evaluate_label_efficient, it seems that the training process is the same as no label process. Can you talk some key about it?

a question about labeled data

Hi,dear. I read your paper says "Percentage of labeled data"，but I see your code is "the number label of per people". Are they referring to the same, do they only have 100 labels each?