marekrei / sdsn Goto Github PK

Hello,

I have recently been trying to re-implement your SDSN model with PyTorch to use it with the pre-trained weights you provide in this repository. However, as I was having a hard time to reproduce your results, I dug into your code a little bit and found a small mistake with the gating layer.

In the construct_network method, when you test if gating is defined as "first", "second" or "both" in the config, you use an elif statement on the second test instead of an if, which prevents the model of ever using gating on both the first and second word and limiting it to "second" when "both" is used in the configuration file.

Since because of this the pre-trained weights provided in this repository were trained with gating applied only on the second word, do you think we should expect better performances with the same approach during prediction, or do you believe we could observe an improvement by using gating on both words?