only have bias but miss "act" about stylegans-pytorch

[yuuho]

My implementation also does activation.

For some reason they don't mention it in the paper, but in their implementation they do a process called gain. Gain is simply the process of multiplying the feature map by sqrt (2).
And my understanding is that the implementation is:

• StyleGAN1: LeakyReLU( conv(feature)*(2**0.5) + noise + bias )
• StyleGAN2: LeakyReLU( ( conv(feature) + noise + bias )*(2**0.5) )

I don't know the detailed effect of this process...

I've written it in nn.Sequential so the source code comparison can show this difference.
For this reason, I decomposed bias and act into separate nn.Modules.

stylegans-pytorch/packaged/run_pt_stylegan1.py

Line 194 in d916bc0

'Fconv8' : [ ['EqConv3x3',512, 512], ['amp'], ['noiseE',512, 8], ['bias',512], ['Lrelu'] ], 'adain8b' : [['adain',512]], 'toRGB_8' : [['EqConv1x1',512, 3], ['bias',3]],

stylegans-pytorch/packaged/run_pt_stylegan2.py

Line 215 in d916bc0

'Fconv8' : [['EqModConv3x3', 512, 512], ['noiseP', 8], ['bias',512], ['amp'], ['Lrelu'] ], 'toRGB_8' : [['EqModConv1x1',512, 3], ['bias',3]],

[Johnson-yue]

Thank you for such detailed explanation。 But I think there are some question here
I think the orig implementation is:

• StyleGAN2 : LeakyReLU( ( conv(feature) + noise + bias ) ) *(2**0.5) this is different from

yours:

• StyleGAN2 : LeakyReLU( ( conv(feature) + noise + bias ) *(2**0.5))

the difference is position of Gain !!
orig: ModConv2d -> add noise -> add bias -> lrelu -> multiply Gain
yours : ModConv2d -> add noise -> add bias -> multiply Gain -> relu

Although they are numerically equal

[Johnson-yue]

And I meaning your toRGB_* only bias and not have activation
I think it should be：
'toRGB_8' : [['EqModConv1x1',512, 3], ['bias',3], ['Lrelu'], ['amp']],

[Johnson-yue]

Sorry it is my mistake, you are right