fcaspe / ddx7

Differentiable FM Synthesis of Musical Instrument Sounds

Minimal Mistakes remote theme starter

Click Use this template button above for the quickest method of getting started with the Minimal Mistakes Jekyll theme.

Contains basic configuration to get you a site with:

• Sample posts.
• Sample top navigation.
• Sample author sidebar with social links.
• Sample footer links.
• Paginated home page.
• Archive pages for posts grouped by year, category, and tag.
• Sample about page.
• Sample 404 page.
• Site wide search.

Replace sample content with your own and configure as necessary.

---

Troubleshooting

If you have a question about using Jekyll, start a discussion on the Jekyll Forum or StackOverflow. Other resources:

• Ruby 101
• Setting up a Jekyll site with GitHub Pages
• Configuring GitHub Metadata to work properly when developing locally and avoid No GitHub API authentication could be found. Some fields may be missing or have incorrect data. warnings.

ddx7's Issues

issue about FAD

Hi,
Thanks for the wonderful work.
I am also trying to replicate the Frechet Audio Distance score, and I also used Google's original FAD repo, but I achieved different results.
Flute Violin Trumpet
Test Data 1.3814 0.3003 0.7313
DDX7（I_max=2) 8.2608 7.0066 5.451

I delete this line in test() to get the ref_train.wav and calculate FAD between
1, ref_train.wav and synth_test.wav
2, ref_train.wav and ref_test.wav

model = model.eval()
for k in self.loaders.keys():
if(k == 'train'): continue
print(k)

And I find that the FAD score I get is different from the results on the paper. So do you have any idea about the reason?

Replicating Frechet Audio Distance score for DDX7

Hi @fcaspe,

Thanks for the wonderful work on DDX7! I am trying to replicate the Frechet Audio Distance score for the violin corpus, but I find that the FAD score I get is different from the results on the paper.

My steps are as follows:

1. Follow Dataset Generation to get URMP processed, so locally I have dataset/train/violin/*.wav and data/train/violin/16000.h5
2. python train.py using default config.yaml and model tcnres_f0ld_fmstr. I am able to see checkpoints generated under runs/exp_test/testrun
3. python train.py again by setting mode: test in config.yaml. I get ref_test.wav and synth_test.wav.
4. For FAD calculation I am using this repo: https://github.com/gudgud96/frechet-audio-distance, based on a Torch VGGish and a pytorch-fid implementation.
   • for Original Test Set, I calculate distance for dataset/train/violin/*.wav and ref_test.wav
   • for Synthesized Test Set, I calculate distance for dataset/train/violin/*.wav and synth_test.wav

The results I get are not the same (and somehow a little weird..):

Dataset	FAD
Original Test Set	7.754
Synthesized Test Set	4.176
Synthesized Test Set (model checkpoint = 0)	42.48

The model checkpoint = 0 case makes sense, but seems like I have a lower score on synthesized test set. Would love to discuss with you about the details of how you calculated FAD, I am thinking if there might be some diff between our FAD implementation.

Another point to double check is: in trainer.py, is it that I should add self.model = self.load_checkpoint(self.model,-1) in the mode=='test' block to load the best model checkpoint for inference?

Thanks in advance!