[Faster transformer] having a guide on how to use weights from a Hugginface transfomer model (Roberta based) with faster transformer 3.1 about fastertransformer

[byshiue]

If the model architectures of BERT and Roberta are same, you should be able to run Roberta on FT.
You can refer this converter and check the weights are put correct in https://github.com/NVIDIA/FasterTransformer/blob/main/sample/pytorch/utils/encoder.py.

You can also try to run roberta with BERT model directly because we have showed that FT can use BERT model directly.

[byshiue]

Close this bug because it is inactivated. Feel free to re-open this issue if you still have any problem.

[pommedeterresautee]

Thanks for the link, it will be very helpful!

[ehuaa]

Thanks for the link, it will be very helpful!

@pommedeterresautee I'm encounter with the same issue to use weights from Huggingface Roberta model to use with Bert FT model, can you use Bert directly? Thanks