Preactivation ResNets about recipes

[benanne]

Absolutely, this would be an excellent addition :) Perhaps it's best to keep it separate, since it's about a different paper? Of course we can make sure that they link to each other.

[FlorianMuellerklein]

Oh great, I'm glad that I could contribute! What is the best way to proceed? I do my training loops a little differently from Lasagne docs and the other examples. Should things like that be made consistent?

[f0k]

I do my training loops a little differently from Lasagne docs and the other examples. Should things like that be made consistent?

No, that's totally fine!

Looking at your train_nn.py, though, the model to use is hard-coded. Can you change this to allow specifying the model on the command line? Something simple like:

import sys
variant = sys.args[1] if len(sys.args) > 1 else 'normal'
depth = int(sys.argv[2]) if len(sys.args) > 2 else 18
if variant == 'normal':
    from models import ResNet_FullPreActivation as ResNet
elif variant == 'bottleneck':
    from models import ResNet_BottleNeck_FullPreActivation as ResNet
elif variant == 'wide':
    from models import ResNet_FullPre_Wide as ResNet
else
    print ('Unsupported model %s' % variant)
output_layer = ResNet(X, depth)

Also there should be instructions about where to download and put the CIFAR-10 dataset (or an automatic download from within the script as we do for the MNIST example in Lasagne).

[FlorianMuellerklein]

No problem, I'll get right on those changes.

Should we remove the wide ResNet though? It's from a different paper and in that paper they use different preprocessing and L2. It wouldn't be a reproduction of the results, but probably a more fair comparison.

[f0k]

Different L2 should be easy to fix, different preprocessing maybe not -- but it should be enough to add a disclaimer (i.e., add a comment that says this is not reproducing the wide resnets paper, but just provided for comparison).

[FlorianMuellerklein]

I updated my repo with a more informative README, system args added to train_nn.py and a comment in utils.py about where to put cifar-10.

[f0k]

Cool, looks nice and clean now! We can either just add this as a link to our wiki: https://github.com/Lasagne/Lasagne/wiki/3rd-party-extensions-and-code, or you can submit it as a subdirectory to Lasagne/Recipes/papers. In this case, I'd probably exclude the weight files, though, and ask @ebenolson to upload them instead so you can add the links to the README file. Also make sure to include your name in the README file and code if you submit it to Lasagne/Recipes.

[FlorianMuellerklein]

I'm happy to do either one, if you all have a preference then I'll do that. I think that it would be cool to see it in Lasagne/Recipes/Papers. I also just updated everything with wide-ResNet results, my wide-ResNet did slightly better than the paper. Perhaps due to the different training and preprocessing.

[f0k]

I think that it would be cool to see it in Lasagne/Recipes/Papers.

Sure, if you prefer that, just do it! It will probably be more visible there than in the wiki, but people googling for "wide resnet lasagne" can already find your repository. Just decide for yourself if it's worth the extra effort.

[f0k]

Was merged two weeks later in #74. Closing.