Documentation needed in this function as to which equation is implemented. The reference of the paper is helpful though. about texar

[ZhitingHu]

@Shrshore would you please fix? Besides:

• I'm wondering if it's better to implement it as PositionEmbedder (like WordEmbedder), rather than PositionEncoder?

• Make sure all modules inherits the base class ModuleBase

• Use 4-space indent (not 2)

[DevSinghSachan]

@Shrshore . would you also please update the documentation of Transformer Decoder, particularly, the module doc.
https://github.com/ZhitingHu/txtgen/blob/master/texar/modules/decoders/transformer_decoders.py

Also, the filename can be "transformer_decoder.py"

[haoransh]

Hi, @DevSinghSachan @ZhitingHu Sure! FYI, currently this file is copied from https://github.com/OpenNMT/OpenNMT-tf/blob/fad5bdbe6be99e8bb47b2edd81c9ff7c4f862a41/opennmt/layers/position.py. Since the functions defined here is to be used in the beam search module.

[ZhitingHu]

So it's not from T2T? Are these functionalities necessary for T2T beam search?

Let's try to implement by ourselves, or copy from T2T at most, not other libraries.

[haoransh]

fixed. I have moved the position encoding to the texar/core/layers.py, and modified the implementation to match the t2t. The implementation is t2t is in https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/layers/common_attention.py. While t2t doesn't support the parameterized positional encoder, maybe we could add this feature soon.