leviswind / pytorch-transformer Goto Github PK

Hi. Please i would like to add some features to this code that i have read on beam search. Like coverage penalty and length normalization. But i don't know where to start. Can you help please?

1st, backend.Embedding is removed in pytorch.
2nd, the tensors in this repo are not in the same device type, this really messed up the code, I tried to modify this device the tensors in the code stored on, but the more I modify, the more errors it raised, finally I have no choice but giving up read and deploy this code. I suggest you try to define you code purely on cpu, not adding many codes in this repo to clarify than you run some operations on cuda, this really mess this repo up if you didn't make it clear that all the operation is compatible with the storing device type of all the variables.

When I run the train.py，report this error

How to solve this problem？THKS

Traceback (most recent call last):
File "train.py", line 90, in
train()
File "train.py", line 81, in train
writer.export_scalars_to_json(hp.model_dir + '/all_scalars.json')
AttributeError: 'SummaryWriter' object has no attribute 'export_scalars_to_json'

How can i correct it, please?

Thank you for your excellent work,can eval.py infer on one sentence?

I want to know why using the parameter 'sinusoid'? And could I not set the parameter ‘maxlen’ and using varying length of the text between batch?

in if __name__ == '__main__':
outputs = position_encoding(num_units)(inputs)
should be
outputs = positional_encoding(num_units)(inputs)

hi, thanks for your implementation~
When I look up your code and compare with the tf-version by Kyubyong, I fond that the implementation of layer normalization in your repository is slightly differing with Kyubyong's.

followings are you code and K's code, and he added **0.5 in the denominator:

https://github.com/leviswind/pytorch-transformer/blob/master/modules.py#L69

https://github.com/Kyubyong/transformer/blob/master/modules.py#L36

Am I running the train file because of the pytorch version problem?