jeffbolznv / vk_cooperative_matrix_perf

Sample benchmark demonstrating the VK_KHR_cooperative_matrix extension

vk_cooperative_matrix_perf is a sample/benchmark demonstrating performance of using the VK_KHR_cooperative_matrix Vulkan extension, and the associated GL_KHR_cooperative_matrix GLSL extension.

The benchmark queries the supported matrix multiply sizes and precisions from the Vulkan implementation, and runs a couple different shaders at various tiles sizes and reports the performance in teraflops.

Running this application requires an NVIDIA Turing or newer GPU, and a recent driver that supports the VK_KHR_cooperative_matrix extension.

Modifying and rebuilding the shaders requires a glslangValidator.exe with GL_KHR_cooperative_matrix support. This is included Vulkan SDK version 1.3.261.0 and newer.

vk_cooperative_matrix_perf's Issues

integer_cooperative_matrix does not work

I was trying to benchmark the performance of an NVIDIA TITAN RTX , nvidia driver version 440.100 (on Ubuntu 18.04.01)
with the code in this repo.

the function:
vkGetPhysicalDeviceCooperativeMatrixPropertiesNV
doesn't seem to store in the cooperativeMatrixProperties vector any integer element
for the AType field of the VkCooperativeMatrixPropertiesNV struct.

More specifically, if after context initialization and before any benchmark I write:

printf("What I am going to do \n");
for (uint32_t tt = 0; tt < TT_COUNT; ++tt) {
for (uint32_t i = 0; i < numCooperativeMatrixProperties; ++i) {
VkCooperativeMatrixPropertiesNV *cooperativeMatrixProps = &cooperativeMatrixProperties[i];
printf("\ncooperativeMatrixProps = %dx%dx%d A = %s B = %s C = %s D = %s scope = %s\n",
cooperativeMatrixProps->MSize,
cooperativeMatrixProps->NSize,
cooperativeMatrixProps->KSize,
componentTypeInfo[cooperativeMatrixProps->AType].typeName,
componentTypeInfo[cooperativeMatrixProps->BType].typeName,
componentTypeInfo[cooperativeMatrixProps->CType].typeName,
componentTypeInfo[cooperativeMatrixProps->DType].typeName,
scopeString[cooperativeMatrixProps->scope]);
}
}

I obtain the following output:

What I am going to do

cooperativeMatrixProps = 16x8x16 A = float16_t B = float16_t C = float16_t D = float16_t scope = subgroup
cooperativeMatrixProps = 16x8x8 A = float16_t B = float16_t C = float16_t D = float16_t scope = subgroup
cooperativeMatrixProps = 16x8x16 A = float16_t B = float16_t C = float32_t D = float32_t scope = subgroup
cooperativeMatrixProps = 16x8x8 A = float16_t B = float16_t C = float32_t D = float32_t scope = subgroup
cooperativeMatrixProps = 16x8x16 A = float16_t B = float16_t C = float16_t D = float16_t scope = subgroup
cooperativeMatrixProps = 16x8x8 A = float16_t B = float16_t C = float16_t D = float16_t scope = subgroup
cooperativeMatrixProps = 16x8x16 A = float16_t B = float16_t C = float32_t D = float32_t scope = subgroup
cooperativeMatrixProps = 16x8x8 A = float16_t B = float16_t C = float32_t D = float32_t scope = subgroup

No integer types for A.
Benchmarks are only performed on float types.
I did not modified nor recompiled the shaders.