Why Sigmoid isn't used in the LKA module? about van-classification

[MenghaoGuo]

Good advice.

Intuitively, I think that sigmoid affects gradient propagation and computation speed. However, we have not done relevant experiments in this area yet.
If we have time, we will do this experiment.

[K1nght]

Thanks a lot, I'm glad to hear from you about further improvement.

[YanShuang17]

@MenghaoGuo

Hi, We used your great backbone in our internal detection dataset and found two potential problems:

(1) When we export model to Libtorch and do inference with FP16, the model output is NAN, the source is the LKA, whose output value causes the overflow problem because of no normalization op (FP32 works well...).

(2) Training is unstable, when use the same optimizer config, VAN-base works well, but VAN-tiny/small can't converge.

FYI