Custom embedding in Transformer about trax

[lukaszkaiser]

Yes, it involves writing your own embedding layer. For more details, please ask on the gitter chat.

[ithieund]

Hi @yashjakhotiya , did you have the solution?
Can you share it here?
Thank you very much.

[yashjakhotiya]

Hi. Yes, @ithieund. I took the hint from @lukaszkaiser and wrote my logic in the 7 lines after this - https://github.com/yashjakhotiya/Adversarial-Attacks-On-Transformer-Based-Malware-Detector/blob/9e3c09234324235242b0bf378f6a75416888f423/transformer.py#L107.

However I wasn't able to retain the abstraction and had to insert my custom layer into an implementation for transformers.

[ithieund]

Hi. Yes, @ithieund. I took the hint from @lukaszkaiser and wrote my logic in the 7 lines after this - https://github.com/yashjakhotiya/Adversarial-Attacks-On-Transformer-Based-Malware-Detector/blob/9e3c09234324235242b0bf378f6a75416888f423/transformer.py#L107.

However I wasn't able to retain the abstraction and had to insert my custom layer into an implementation for transformers.

Thank you.