Data augmentation with Torchaudio about lhotse

[pzelasko]

I think a complete switch would require us to drop support for PyTorch 1.6 and earlier, unless torchaudio 0.7 works with older PyTorch releases? If that's the case, that would be undesirable.

Anyway, we should take care to make the data augmentation interface pretty generic, e.g. accepting augment_fn in relevant places that has a signature of Callable[[np.ndarray, int], np.ndarray] with arguments audio_samples and sampling_rate - the latter is required as some effects like SoX speed might require resampling afterwards. The augment_fn would return the augmented time-domain signal.

[mthrok]

Hi @pzelasko

Congrats on the first release 🎉
Regarding the versioning, torchaudio only supports the the version of PyTorch released at the same time. For torchaudio 0.7, it's PyTorch 1.7.0.

We are also interested in providing building blocks for augmentation. We can collaborate in there.

[pzelasko]

Thanks, Moto!

And thanks for the clarification. Looking at 0.7 docs, the example using class SoxEffectTransform(torch.nn.Module): confirms my intuition about moving towards the augment_fn API in Lhotse - it seems that would play well together, as you could pass such a nn.Module where such augment_fn is expected. The only difference I can see is that your transform would also return a sampling_rate, but I'm not sure if we need to/should support changing the sampling rate as part of data augmentation in Lhotse...?

[mthrok]

The reason why I added the output sample rate is some of the effects (rate, upsample, downsample and speed AFAIK) change the sampling rate and when I was writing the documentation, I wanted the example to be general enough to cover all the supported effects.

I think it makes sense to limit it to only return the tensor, if it's written for specific purpose. For example, I am thinking that rather than providing a general SoxEffectTransform, providing a transform that is specialized in solving a specific task, like RandomSpeedPerturbation (similar to the example in apply_effects_file function) might be more valuable.

If there are other common use cases of sox effects in training recipes, torchaudio can provide these too.

[pzelasko]

@mthrok can you check out if the torchaudio.sox_effects usage in PR #124 makes sense to you? Also, I've reported an issue in torchaudio where I found an inconsistent torchaudio behavior compared to WavAugment.