RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasGemmEx( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16F, lda, b, CUDA_R_16F, ldb, &fbeta, c, CUDA_R_16F, ldc, CUDA_R_32F, CUBLAS_GEMM_DFALT_TENSOR_OP)` about big-sleep

[BassemKaroui]

I had the same error, and thx to @anvarganiev I noticed that one tensor had the wrong shape

[apprehensivetarget]

I am gettind the same error.

[TheGildedGoo]

Having the same error utilizing a 1660 Super. Thinking it may have something to do with the GPU.

[apprehensivetarget]

I fixed it by reinstalling pytotch

[Rerumu]

Having this issue on Debian with my GTX 1660 Super. Reinstalling pytorch did not help.
After some testing I found it may be a memory issue. Setting cutout size to 17 allows it to run, but any higher produces the error.

[anvarganiev]

I had wrong tensor shape in NN architecture. In one place there was 32x0x6 tensor.

[0xBYTESHIFT]

Got this issue too, using the default CLI, not a .py file with my parameters. How to fix it?