Sizings, questions, possible error in preresnet about fb.resnet.torch

[colesbury]

• The 7x7 convolution is zero-padded by 3 on each side, so a total padding of 6 on the width and 6 on the height. The max pooling is also padded (by 1 on each side). I think max pooling is padded with -inf instead of zero. The Torch documentation makes it sound like a total padding of 3, but I don't think that's how it's implemented. (@soumith, is this correct?)
• Yes, this is a bug -- thanks for catching it. Because of this, the ResNet-200 model has an extra batch norm & ReLU in the first layer, which it shouldn't.
• The extra copy is to make the -shareGradInput option work correctly with this model. This option re-uses the CUDA storages for gradInput when computing the backward pass to save memory, but the implementation is a fragile hack.

[soumith]

yes it's symmetric padding, 3 on each side.

[anibali]

@colesbury Why is there no nn.Copy for cifar10?

[colesbury]

@anibali, there should probably be a copy there too in case you run with -shareGradInput

[anibali]

Great, thanks. Just trying to figure out how everything works :)