Batch sizes about deepspeech.pytorch

[ryanleary]

We filter the data used to under a certain length. If you look at the librispeech.py script, for example, you can see that there is a flag there for doing filtering when the manifest is created.

[SeanNaren]

Baidus' internal code is much more memory optimised than Torch (atleast for RNNs), allowing them to run much larger batch sizes per GPU. This has always been a crux of using RNNs! I'm honestly not sure where to exactly look to handle this, but since you have pascal cards this PR is something to look at. Persistent RNNs speed up RNNs a lot when using smaller batch sizes because there is less transfer of memory!

[ryanleary]

I've seen many references to their work on being efficient with memory. I think I'm just surprised that it's making a ~4x difference in batch size. Impressive (and good for them) I suppose. Will close if this behavior is roughly consistent with others are seeing.

[EgorLakomkin]

Yes, I have observed the same behaviour

[SeanNaren]

I'm going to tie #55 to this, with successful FP16 we should be allowed to double batch sizes (with slight performance loss due to cublas doing some funky stuff with pseudo FP16 that I was notified about)

[SeanNaren]

As explained here, sadly FP16 won't be viable for some time (atleast for training).

[haquynh1505]

@ryanleary @SeanNaren : Excuse me! I've been seeing you guys training on full set of Librispeech. I wonder if you do something to constrain the length of the audio files?
I'm also doing experiments on Librispeech (100h), and I just cannot train with the batch sizes as big as yours.
I know that in the dataset there are quite a lot files longer than 30 seconds. I don't know if this is the main the reason that keeps me from training with bigger batch size? Now I'm training with the batch size of 8, and it's extremely slow.
Did you guys do something with the dataset?
Thanks in advance!

[haquynh1505]

@ryanleary : Ok I see. Thank you so much!