Expected a 'cuda' device type for generator (related to speed issues?) about deq

[jerrybai1995]

Hi @polo5 ,

Thanks for your interest in our paper! Yes, MDEQ-Large does take a few hours to finish all epochs, so your calculation is correct. However, I should note that most (about 80%?) of the time was actually spent on boosting accuracy from 90% to ~93.5%. If you are just looking for a 90% accuracy, an MDEQ-Large should achieve that within 1.5-2 hours.

If you use an MDEQ-Tiny model and set AUGMENT in the config file to be True (while slightly increasing the # of epochs), you should also expect near 90% accuracy even more quickly. But of course, you are using a smaller model.

[jerrybai1995]

Also re: error: I haven't encountered this error for this repo before, but I'll check for sure for PyTorch 1.10!

[liu-jc]

I got this error as well when I use PyTorch 1.10. After changing to PyTorch 1.8.1, everything is fine. You can take a look at this issue. Seems related to a bug in pytorch 1.10.

[polo5]

Thanks @liu-jc.

This is an important issue then, since pytorch 1.10 is the recommended version for this repo. The other issue is that previous pytorch versions (<1.10) do work on Ubuntu but somehow don't work on Windows for this repo (I get some strange error-less interruption which looks like a segmentation fault). Oh well, if setting devices manually hasn't slowed down the code I'm happy with that solution.

[jerrybai1995]

The issue with PyTorch <1.10 is that the hook implementation currently used to implement O(1) memory (see e.g., https://github.com/locuslab/deq/blob/master/DEQ-Sequence/models/deq_transformer.py#L380) did not work before, and was a bug that PyTorch only recently fixed in 1.10 (see this issue). I will check this again recently and update on this thread.

I've never tried on Windows environment but I suspect it has something to do with the WSL?