[Bug]: dear teachers, when i change the gelu on spu, the result always similar very much, even when i change the gelu at a worse format about spu

[tpppppub]

I am not familiar with GPT-2 model details but I don't find anywhere the huggingface gpt-2 model calling F.gelu. (https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt2/modeling_gpt2.py). Any reason to replace F.gelu if they are never used?

[zhangwaer]

I am not familiar with GPT-2 model details but I don't find anywhere the huggingface gpt-2 model calling F.gelu. (https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt2/modeling_gpt2.py). Any reason to replace F.gelu if they are never used?
thank you very much, i understand your teaching, thank you very much!! i will replace the actiation correctly.