trainning is too slow compared with MAE+VIT-Base about convnext-v2

[s9xie]

Thanks for your interests. For the experiments in our paper, we used JAX and TPU for pre-training (see some notes in appendix A.2).
This pytorch release relies on an external library (MinkowskiEngine) for the Sparse Conv implementation, the cuda kernel is not optimized (thus the 50% GPU utilization) which makes pre-training slower.

But anyways, ViT+MAE has a clear advantage in terms of the pre-training speed due to Transformers' native sparse processing capability. Fortunately, in reality we might afford a higher pre-training cost, and during deployment stage efficient ConvNet models might have advantages.

[smarengin]

Thank you for your kindly reply. :-)

[ritikajha]

On a DGX machine with 8 A100's how much time will it take to pretrain the base model.