Use librosa for inference.py instead of torchaudio about ast

[AlexJian1086]

WIth reference to the paper for the calculation of mel filterbank, I am using librosa.feature.melspectrogram() function to replace kaldi of pythorch given in inferency.py but I am not sure about how to replicate the parameters such as '25ms Hamming window every 10ms' and what would be hop_length, n_fft, win_length for librosa? Please provide the clarity.

[YuanGongND]

Hi there,

Matching outputs of Librosa and torchaudio is out of the scope of this repo, you should consult either librosa or torchaudio authors. It might be hard to make them exactly the same but I assume you should be able to get similar output with appropriate parameters. Or, you can train/fine-tune the model using the librosa generated spectrogram.

Specifically for librosa.feature.melspectrogram(), hop_length should be 10ms, win_length should be 25ms, window should be scipy.signal.windows.hann, sr should be 16,000, n_fft should be 128.

-Yuan

[AlexJian1086]

Ah okay, thank you for clarification.
Although what exactly should I fine-tune here to achieve the desired results as inference pipeline for audioset, I assume the window size, overlap, mel bin etc would still remain same as provided in paper?

Also fbanks calculated in torchaudio.compliance.kaldi.fbank is same as librosa.feature.melspectrogram() and python_speech_features.base.logfbank?

[YuanGongND]

So the best way is to train and test using the feature extracted by the same toolkit. For audio event classification, you can just reuse our window size, overlap, etc to save time for searching; if your task is significantly different from audio event classification, you can consider using your own parameters.

The output of different toolkits might be different, you need experiments to confirm if they are the same.

-Yuan