CUDA Error: no kernel image is available for execution on the device (209) /tmp/build-via-sdist-nl8se4dx/flashinfer-0.0.4+cu118torch2.2/include/flashinfer/attention/decode.cuh: line 871 at function cudaFuncSetAttribute(kernel, cudaFuncAttributeMaxDynamicSharedMemorySize, smem_size) about flashinfer

[yzh119]

This is due to CUDA version mismatch, what's the CUDA version (and PyTorch CUDA version) on your device?

nvidia-smi
python -c "import torch; print(torch.version.cuda)"

[lucasjinreal]

Am using cuda118 ,

>>> import torch
torch>>> torch.__version__
'2.2.0+cu118'

installed with pip install flashinfer -i https://flashinfer.ai/whl/cu118/torch2.2/
my torch works ok (even though torch now using their own nvidia toolkit through pypi)

Doe sthey any miniml func can be test why the cuda is mismatch?