nvpro-samples / vk_raytrace

Ray tracing glTF scene with Vulkan

VK_RAYTRACE

This project is a glTF 2.0 sample viewer using Vulkan ray tracing. It follows the ray tracing tutorial and combines all chapters into a single example.

The lighting equation is based on:

• glTF PBR reference implementation from Khronos.
• Disney PBR

The rendering pipeline can be switched from:

• Ray Tracing Pipeline: RayGen, Closest-Hit, Miss, Any-Hit model
• Ray Query: Compute shader using Ray Queries

Features

• Load glTF 2.0 files ascii and binary using tiny glTF
• Support for full node hierarchy
• Full implementation of metallic-roughness and specular-glossiness
• IBL lighting using importance light sampling
• Alpha blend and cut-out
• Texture transforms and samplers

Attributes

• Normal : create geometric normal when not present
• Texture coords : only set 0, project cube map when text coords are missing.
• Tangents : generate tangents and bitangent when missing.
• Color : default (1,1,1,1)

Extensions

• KHR_lights_punctual (some support)
• KHR_materials_pbrSpecularGlossiness
• KHR_materials_clearcoat
• KHR_materials_unlit
• KHR_texture_transform
• KHR_materials_anisotropy
• KHR_materials_transmission
• KHR_materials_sheen (some support)

Usage

Controls

Action	Description
LMB	Rotate around the target
RMB	Dolly in/out
MMB	Pan along view plane
LMB + Shift	Dolly in/out
LMB + Ctrl	Pan
LMB + Alt	Look around
Mouse wheel	Dolly in/out
Mouse wheel + Shift	Zoom in/out (FOV)
Space	Set interest point on the surface under the mouse cursor.
F10	Toggle UI pane.

Change glTF model

• Drag and drop glTF files (.gltf or .glb) into viewer

Change HDR lighting

• Drag and drop HDR files (.hdr) into viewer

Setup

git clone --recursive --shallow-submodules https://github.com/nvpro-samples/nvpro_core.git
git clone https://github.com/nvpro-samples/vk_raytrace.git

Other

Tags:

• raytracing, GLTF, HDR, tonemapper, picking, BLAS, TLAS, PBR material

Extensions:

• VK_KHR_RAY_QUERY_EXTENSION_NAME
• VK_KHR_RAY_TRACING_PIPELINE_EXTENSION_NAME
• VK_KHR_ACCELERATION_STRUCTURE_EXTENSION_NAME
• VK_KHR_SHADER_CLOCK_EXTENSION_NAME
• VK_KHR_MAINTENANCE3_EXTENSION_NAME
• VK_KHR_PIPELINE_LIBRARY_EXTENSION_NAME
• VK_KHR_DEFERRED_HOST_OPERATIONS_EXTENSION_NAME
• VK_KHR_BUFFER_DEVICE_ADDRESS_EXTENSION_NAME
• VK_EXT_SCALAR_BLOCK_LAYOUT_EXTENSION_NAME
• VK_EXT_DESCRIPTOR_INDEXING_EXTENSION_NAME
• VK_KHR_GET_MEMORY_REQUIREMENTS_2_EXTENSION_NAME
• VK_KHR_DEDICATED_ALLOCATION_EXTENSION_NAME
• VK_KHR_SWAPCHAIN_EXTENSION_NAME
• VK_KHR_GET_PHYSICAL_DEVICE_PROPERTIES_2_EXTENSION_NAME
• VK_EXT_DEBUG_UTILS_EXTENSION_NAME
• VK_KHR_SURFACE_EXTENSION_NAME

Links

• glTF format specification
• glTF Sample Models
• tiny glTF library
• Ray Tracer Tutorial

---

Test Scenes

Model	Link
	2 Cylinder Engine
	Alpha Blending
	Anisotropy
	Antique Camera
	Boom Box
	Carbon Fiber
	Clear Coat
	Corset
	Damaged Helmet
	Flight Helmet
	Metal Rough Spheres
	Punctual Light
	SciFi
	SpecGlossVsMetalRough
	Unlit
	Transmission
	Box Vertex Color

vk_raytrace's Issues

Which sun&sky model is used?

I cant find credits of used method in source codes

Incorrect clone command in README.md

The README.md incorrectly refers to the repo as "vk_raytracing":

git clone https://github.com/nvpro-samples/vk_raytracing.git

It should be "vk_raytrace" instead.

CMakeLists.txt Target Link Libraries Incomplete

The issue was: both libdl and pthreads were missing. I am not sure what the optimal route would be in the context of the overall tutorial/example system. But, I got things working with the following change in CMakeLists.txt circa line 151 (the Linkage section):

From:

 #####################################################################################
# Linkage
#
target_link_libraries(${PROJNAME} ${PLATFORM_LIBRARIES} shared_sources)

To:

#####################################################################################
# Linkage
#
find_package(Threads REQUIRED)
set(PROJECT_LIBRARIES ${CMAKE_DL_LIBS} -lpthread)
target_link_libraries(${PROJNAME} ${PLATFORM_LIBRARIES}  ${PROJECT_LIBRARIES} shared_sources)

is eStore image are float or int format

Hello.
My question is about final image.
Is used float or integer format for image data?

Unused functions in shaders/punctual.glsl

Hi! For which purposes is declared functions
vec3 getPunctualRadianceSubsurface()
vec3 getPunctualRadianceTransmission()
vec3 getPunctualRadianceClearCoat()
vec3 getPunctualRadianceSheen()
but not used?
How to plan use it?

Error Compiling the Shaders

Ran into this error compiling the shaders:

[ 53%] Linking CXX static library /home/chris/ray-tracing/bin_x64/libshared_sources.a
[ 53%] Built target shared_sources
[ 55%] Generating ../autogen/shaders/post.frag.h
/usr/bin/glslangValidator -o autogen/shaders/post.frag.h -V /home/chris/ray-tracing/vk_raytrace/shaders/post.frag --vn post_frag --target-env vulkan1.2
/home/chris/ray-tracing/vk_raytrace/shaders/post.frag
ERROR: Failed to open file: autogen/shaders/post.frag.h
[ 57%] Generating ../autogen/shaders/passthrough.vert.h
/usr/bin/glslangValidator -o autogen/shaders/passthrough.vert.h -V /home/chris/ray-tracing/vk_raytrace/shaders/passthrough.vert --vn passthrough_vert --target-env vulkan1.2
/home/chris/ray-tracing/vk_raytrace/shaders/passthrough.vert
ERROR: Failed to open file: autogen/shaders/passthrough.vert.h
[ 59%] Generating ../autogen/shaders/pathtrace.rahit.h
/usr/bin/glslangValidator -o autogen/shaders/pathtrace.rahit.h -V /home/chris/ray-tracing/vk_raytrace/shaders/pathtrace.rahit --vn pathtrace_rahit --target-env vulkan1.2
/home/chris/ray-tracing/vk_raytrace/shaders/pathtrace.rahit
ERROR: /home/chris/ray-tracing/vk_raytrace/shaders/pathtrace.rahit:88: 'ignoreIntersectionEXT' : undeclared identifier 
ERROR: /home/chris/ray-tracing/vk_raytrace/shaders/pathtrace.rahit:88: '' : compilation terminated 
ERROR: 2 compilation errors.  No code generated.

GLSL is installed:

AMDGPU Packages 
Installed Packages
glslc.x86_64  2020.4-1.fc33
Available Packages
glslc.x86_64  2020.2-1.fc33  
glslc.x86_64  2020.4-1.fc33  

GLSL Lang is @11.0

Installed Packages
glslang.x86_64 11.0.0-1.20200803.git5743eed.fc33

is VK_KHR_DEFERRED_HOST_OPERATIONS_EXTENSION_NAME needed?

Declared as needed in line

vk_raytrace/src/main.cpp

Line 108 in 558b878

contextInfo.addDeviceExtension(VK_KHR_DEFERRED_HOST_OPERATIONS_EXTENSION_NAME);

Where is used this extension in raytracer?
Or this extension is optional?

Hangs with non-beta NV driver v460.79 and Vulkan SDK 1.2.162.0

Hi there, just to let you know, this sample is hanging for me during startup after the debug statement "Create RtCore" in updatePipeline, specifically : vk_raytrace\rtx_pipeline.cpp (line 134).

The other vulkan KHR ray tracing samples seem to work fine with the mainline nvidia VK drivers, so I figured I'd try it here. No dice.

Supportted GPU

This works on Jetson nano or xavier?

#include <thread> -- header is missing

The thread include is missing in main.cpp:

In file included from /home/chris/lab/ray-tracing/vk_raytrace/main.cpp:32:
/usr/include/vulkan/vulkan.hpp:14218:5: note: declared here
14218 |     operator T const& () const & VULKAN_HPP_NOEXCEPT
      |     ^~~~~~~~
/home/chris/lab/ray-tracing/vk_raytrace/main.cpp: In function ‘int main(int, char**)’:
/home/chris/lab/ray-tracing/vk_raytrace/main.cpp:175:8: error: ‘thread’ is not a member of ‘std’
  175 |   std::thread([&] {
      |        ^~~~~~
/home/chris/lab/ray-tracing/vk_raytrace/main.cpp:46:1: note: ‘std::thread’ is defined in header ‘<thread>’; did you forget to ‘#include <thread>’?
   45 | #include "sample_example.hpp"
  +++ |+#include <thread>
   46 | 

how rayquery variant works?

Hello. Please explain how rayquery variant of renderer works.
Created acceleration structure and used or so?
Where is no fully compute renderer with self-made bvh, not use vulkan acceleration structure?

Change settings panel

Hello.
I cant find in code

• how to make panel fixed width in pixels
• how to start renderer with floating panel, not docked
• how to save panel states between renderer runs

https://github.com/Taehan-Kim/AppSlate.git

Commit

[Question] how "Remove banding" code works?

Please explain how "Remove banding" from shaders/post.frag works:
// Remove banding
uvec3 r = pcg3d(uvec3(gl_FragCoord.xy, 0));
vec3 noise = uintBitsToFloat(0x3f800000 | (r >> 9)) - 1.0f;
color = dither(sRGBToLinear(color), noise, 1. / 255.);

and give please example file to check this code.
I dont see differencies in many scenes

error compiling rgen shader:

I get a compilation error for just this shader:

1>Generating D:/poojan/projects/nvpro-samples/vk_raytrace/autogen/pathtrace.rgen.spv
1>C:/VulkanSDK/1.2.189.2/bin/glslangValidator.exe -g --target-env vulkan1.2 -o D:/poojan/projects/nvpro-samples/vk_raytrace/autogen/pathtrace.rgen.spv D:/poojan/projects/nvpro-samples/vk_raytrace/shaders/pathtrace.rgen
1>D:/poojan/projects/nvpro-samples/vk_raytrace/shaders/pathtrace.rgen
1>CUSTOMBUILD : error : D:/poojan/projects/nvpro-samples/vk_raytrace/shaders/traceray_rtx.glsl:46: 'no rayPayloadEXT/rayPayloadInEXT declared' : with layout(location = 0)
1>CUSTOMBUILD : error : D:/poojan/projects/nvpro-samples/vk_raytrace/shaders/random.glsl:34: '' : syntax error, unexpected UINT, expecting COMMA or SEMICOLON
1>CUSTOMBUILD : error : 2 compilation errors. No code generated.

Ofcourse, if I don't compile that shader, the sample runs (atleast the compute pipeline works as expected), so I suspect it's probably something different with my validator/version/sdk.

I twiddled around a lot with moving code, etc. No luck. I can't seem to understand what is so different than say the ray tracing gltf sample (all the samples except this one compile/run fine incidentally)

Any ideas?

black edges occur when zoom in

Hi, thanks for this greate project!

I encountered a problem which seems like a bug.

Take the robot-toon scene for example, when you zoom in to observe the model, some black lines will occur around the contour, as shown in the figure below.

This will also happen to other contours. Is that a bug?

How to build from source on MSYS2 MINGW64?

cmake configuration failed:

$ cmake ..
-- Building for: Ninja
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/msys64/mingw64/bin/cc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/msys64/mingw64/bin/c++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- -------------------------------
-- Processing Project vk_raytrace:
CMake Error at CMakeLists.txt:30 (find_path):
  Could not find BASE_DIRECTORY using the following files:
  nvpro_core/cmake/setup.cmake


-- Configuring incomplete, errors occurred!

strange output to console on Windows

---

←[0mVulkan Version:
←[0m - available: 1.3.268
←[0m - requesting: 1.3.0
←[0m______________________
←[0mUsed Instance Layers :
←[0m
←[0mUsed Instance Extensions :
←[0mVK_KHR_surface
←[0mVK_KHR_win32_surface
←[0mVK_EXT_debug_utils
←[0m____________________
←[0mDevices : 2
←[0m0: NVIDIA GeForce RTX 3060 Laptop GPU
←[0m - Compatible
←[0m1: Intel(R) UHD Graphics
←[0m - Missing extensions: ←[0mVK_KHR_acceleration_structure ←[0mVK_KHR_ray_tracing_pipeline ←[0mVK_KHR_deferred_host_operations ←[0m
←[0mCompatible physical devices found : ←[0m1
←[0mUsing Device:
←[0m - Device Name : NVIDIA GeForce RTX 3060 Laptop GPU
←[0m - Vendor : NVIDIA
←[0m - Driver Version : 551.92.0
←[0m - API Version : 1.3.271
←[0m - Device Type : Discrete GPU
←[0m________________________
←[0mUsed Device Extensions :
←[0mVK_KHR_swapchain
←[0mVK_KHR_shader_clock
←[0mVK_KHR_acceleration_structure
←[0mVK_KHR_ray_tracing_pipeline
←[0mVK_KHR_ray_query
←[0mVK_KHR_deferred_host_operations
←[0mVK_KHR_buffer_device_address
←[0m
←[0mCreate Offscreen←[0m --> (5.602 ms)
←[0mLoading HDR and converting C:/msys64/home/user/from-git/nv-feb24/bin_x64/Release/../../downloaded_resources//std_env.hdr
←[0m --> (57.192 ms)
←[0mLoading scene: C:/msys64/home/user/from-git/nv-feb24/bin_x64/Release/../../downloaded_resources//robot_toon/robot-toon.gltf←[0m --> (99.653 ms)
←[0m←[33m←[0mConvert to internal GLTF←[0m --> (61.516 ms)
←[0mCreate Buffers
←[0m - Create 18 Material Buffer←[0m --> (0.020 ms)
←[0m - Create 2 Textures, 2 Images←[0m --> (0.734 ms)
←[0m - Create 50 Vertex Buffers←[0m --> (94.095 ms)
←[0m ←[0m --> (6.171 ms)
←[0mCreate acceleration structure
←[0m BLAS(50)←[0mRT BLAS: reducing from: 64887040 to: 25686016 = 39201024 (60.41% smaller)
←[0m TLAS(52)←[0m --> (35.301 ms)
←[0mSwitching renderer, from 2 to 0
←[0mCreate RtxPipeline←[0m --> (17.305 ms)
←[0m

Cant link with nvml.lib(nvml.dll) with VS2022

Hello,

During compilation with VS2022 of vk_raytrace.exe I ended with an error I can't fix.

\vk_raytrace\out\build\x64-Debug\nvml.lib(nvml.dll) : error LNK2001: symbole externe non résolu __delayLoadHelper2

I tried to add

C:\Program Files\NVIDIA Corporation\NVSMI\nvml.dll
C:\Program Files\NVIDIA Corporation\NVSMI\nvml.lib

I have installed latest CUDA 12.2
https://developer.nvidia.com/gpu-deployment-kit says :
The GPU Deployment Kit is part of the CUDA Toolkit, in versions 8 and later, and is no longer available as a separate download.

I do not know how to proceed

cfgmgr32.h is a windows-specific header file preventing compilation in Linux

The cfgmgr.h is a windows-specific dependency preventing compilation on Linux.

[ 86%] Building CXX object CMakeFiles/vk_raytrace.dir/home/chris/lab/ray-tracing/shared_sources/imgui/imgui_camera_widget.cpp.o
[ 88%] Building CXX object CMakeFiles/vk_raytrace.dir/home/chris/lab/ray-tracing/shared_sources/imgui/imgui_impl_vk.cpp.o
In file included from /home/chris/lab/ray-tracing/vk_raytrace/sample_example.cpp:55:
/home/chris/lab/ray-tracing/vk_raytrace/nvml_monitor.hpp:19:10: fatal error: cfgmgr32.h: No such file or directory
   19 | #include <cfgmgr32.h>
      |          ^~~~~~~~~~~

bugs in pathtrace.glsl

I notice some problems in pathtrace.glsl. I am sorry I have no time to fix them myself. If these problems exist, I hope this issue post can help you out.

1. The shadow rays of point light sources are not handled correctly. The maximum distance of the shadow rays should not be 1e32 for them.
2. The environment map evaluation in PathTrace() does not have MIS weight.
3. The direct light contribution is accumulated after RR. So if RR is activated, the direct light of the last bounce may not be added.