cdgiessen / vk-module

Custom C++ Vulkan Bindings

Charles Custom C++ Vulkan Bindings

My (WIP) C++ bindings for the Vulkan Graphics API. These replace the existing C and C++ headers, so they are completely independent.

Contents

Currently there are projects

• vk-simple - C++ API that is like the C API with a handful of QoL improvements
• vk-module - (experimental) A Vulkan C++20 Module, and a different take on vulkan.hpp

vk-simple

NOTE: This is unfinished and not suitable for production use. Reasons:

• Doesn't compile because types from system headers aren't handled.
• Not updated to 1.2.170 which added 64 bit flags.
• No test suite to verify code quality. Will need CI setup as well.

Takes the current C headers and adds a few quality of life improvements while staying faithful to the original C API.

Improvements:

• Enums and Bitfields use enum classes, makes it impossible to use the wrong enum.
• sType parameters are initialized to their correct value.
• Structs always initialize all values to zero (except sType). Prevents uninitialized memory bugs.
• Add an API to automatically load function pointers with the following benefits:
  • Function pointers are known to be faster than using the exported Vulkan functions.
  • No compile time dependency on the Vulkan-Loader. It is loaded at runtime instead.
  • Automatically querying function pointers from extensions
  • Removes the need to use an external library such as volk to accomplish this task

Notes:

• This aims to be a production ready solution
• The goal of these bindings is to bridge the gap between the simplicity of the C API and the type safety of vulkan.hpp.
• I am not planning on adding additional features, but am open to simple QoL suggestions.

vk-module

NOTE: This is considered experimental and not suitable for production use.

vulkan.hpp offers many useful features while also containing several drawbacks. These bindings are my attempt at creating an expressive and convenient Vulkan interface in C++ while addressing what I consider to be vulkan.hpps shortcomings. While no API is perfect, vulkan.hpp is a vast improvement over the C API in developer quality of life and safety, which should not be understated.

The main concerns I have with vulkan.hpp are:

1. Poor function pointer loading interface & implementation
2. Slow compile times
3. Lack of (perceived) stability
4. Single file supports multiple C++ standards

None of these are deal breakers and as of writing the vulkan.hpp developers are exploring solutions to the most egregious in the list, slow compile times. For further details about why these are a problem and how I address them, look at the discussion.md in the misc folder.

Main differences between vk-module and vulkan.hpp

• C++20 only. No support for older standards. Features like modules, concepts, and three-way comparison aren't behind a macro guard.
• Streamlined function loading interface
  • Only one way to load and use functions which is easier to explain and doesn't have any performance overhead.
  • Supports multi-gpu without changing all the underlying code. (unless you chose to manually include the dispatcher in every function call)
• Use Expected<T> for error handling instead of exceptions. Functions which return VkResult in the C API now return Expected<T> and contain either the desired value or a vk::Result
• Independence from the C-API. Types are binary compatible with their C counterparts, allowing the API to work without also including vulkan.h first.
  • Because many popular libraries use a C interface, the vk_module_interop.h header exists to bridge the gap between the two. Currently it handles conversions between enums, bitfields, and handles.
  • Because of issues with forward declaration, handles are defined in the vk_module.h and is the only C API currently required for interop to work.
• Address Sanitizer (ASAN) suppression is built into the API. Some drivers are known to create false positives in ASAN which spam the log with a huge volume of reports. By selectively disabling ASAN before calling into the Vulkan API and then re-enabling it after, vk-module eliminates this headache.
• A much more opinionated API. There should be one way to do something and the API shouldn't be afraid to assert a 'correct' idea of how to use the API. The benefit of this is a smaller API that strives to be self-consistent. The hidden benefit is that the implementation is significantly simpler and shorter. vulkan.hpp is well over 90k lines of code, vk_module.h is 20k.
• Aim for solutions that give 80% of the benefit at 20% the cost, such as templates. They solve problems but also introduce more compilation overhead, so a conscious effort was made to use alternative solutions when it made sense.

Known Issues

• Does not support platform specific WSI related functions. An application must use SDL, GLFW, or equivalent library to create a VkSurfaceKHR
• Expected<T> is a homebrew solution and making error handling enjoyable in vulkan is tough. May change this later on
• Dependence on many standard libraries. No plan to remove or change this. Creating replacements for those standard library features just adds possible places for subtle bugs while introduction friction. (Ie, some say this is an issue, but I see the costs outweighing the benefits)
• pNext chains have no helper code. Need to write the StructuredChain replacement.
• API is big, fat, and wide. Very likely there are inconsistencies and errors lurking in the shadows.

History

Work began on this project in April 2020. These bindings started for 2 main reasons.

• C++20 has just been finalized which contains a new way to manage C++ code, modules.
• vulkan.hpp is probably the vulkan tool/project which almost everyone knows about and has the biggest range of opinions, from staunch supporters to dug-in detractors.

The discussion in various online chat groups led me to believe there may be a way to kill two birds with one stone, write C++20 module that are better Vulkan C++ bindings.

With that in mind, Work began on this project in April 2020. Over the spring and summer, the generator.py script was written to parse vk.xml and generate the vk-module bindings. It was a great learning project as it involved exploring python in detail and the xml document.

As the project currently stands (April 2021), I didn't know how wrong I was. Firstly, modules are still experimental or very recently released in C++20 compilers. Suffice to say, they aren't ready for primetime. Secondly, vulkan.hpp's problems are SIGNIFICANTLY more complicated than I could of imagined. There are many concerns the developers must balance: performance, safety, implementation complexity, interface simplicity, interoperability with the C API, then finding out just how little room there is to work with. Not to mention the moving target that is vk.xml which changes and breaks previous assumptions about what the bindings API design space is. This isn't to say these challenges aren't insurmountable, but is incredibly difficult to find a balance that doesn't leave me defeated in some aspect.

vk-module's Issues

Improve CMake file

Current CMakeLists.txt is geared to work on my system and my system only.
It should be rewritten so optional parts (like generation python script) properly check for python or download the other dependencies as necessary.
It should also be tested against using system vulkan-headers, and getting the vk.xml from the SDK/packages/manually downloaded headers.

Make DeviceFunctions store the Device handle

Currently, DeviceFunctions only loads the function pointers but doesn't store the Device handle, so no device functions will succeed.

Manually download vk.xml directly

The project requires the vk.xml and nothing else from the Vulkan-Headers repo.
It speeds up build time of the repo if the entire vulkan-headers repo isn't downloaded.

Add vk::Result to_string to main headers/module instead of in separate header

The ability to print vk::Result is the most useful to_string function, it only makes sense to include it in the main headers and not be separate.
Only difficulty is that now it must be documented that the vk::Result to string is inside the main header.

C++20 module shouldn't have such a reason to split them in the first place.

Add Mock Library for Vulkan C api

Create a library that implements all C api entry points, allow mocking of an application without needing a vulkan driver or loader.

Needs to allow specifying the return values of Vulkan functions.

Also might make more sense to be in a separate repository.

Similar to Mock-ICD but before the loader, not after.

Create proper C++20 Module version of bindings

Currently, there are C++17 headers but no module.

The interim step is to create C++20 headers for feature testing (std::span, default operator==).

This will require build system support to make it usable for many projects.

investigate: natvis & debug_utils set object name integration

It would be nice to see the debug utils handle in the debugger.
This could be accomplished by creating a hash map that the debug utils calls know about and modify.
Then natvis can call into that with the handle, then get the string from the hash map, and thus you get nice strings to go with your uint64_t values.

Low priority, but would be really cool!

Setup CI with github actions

Requires using the Mock Vulkan API library.
Will make tests run every time, ensuring changes don't break code.

Add std::span<T> overloads for C++20 module

Make overloads for any function which takes count+pointer argument pairs into functions with std::span. This allows for easy interoperation with std::vector & std::array.

Create full samples to highlight usage

Make InstanceFunctions take GlobalFunctions as a constructor argument, instead of Loader

DeviceFunctions take device & InstanceFunctions to construct, this make the InstanceFunctions inconsistent (which takes the loader, not global functions)

Use function for C++ enum to C enum conversion, possibly `c_enum()`

operator+ is terse but really confusing. static_cast<C_enum>(C++_enum) is too verbose.

Add allocator support for custom containers

It would be nice if users could specify a custom allocator to use for the various query functions.

Use memcmp() for C++17 structures with lots of members

Autogenerating hundreds of comparisons is a bit silly.
Also do this for union comparisons.

Extension functions promoted to core do not have non core version.

Extension functions which are aliases to now core functions do not have the alias version present.
This is a tricky issue, as the internal function in the loader may not be distinct (todo: double check that).

For example vkGetPhysicalDeviceImageFormatProperties2KHR doesn't exist, only vkGetPhysicalDeviceImageFormatProperties2. If the KHR version was added what should happen if the KHR version is called on a 1.0 driver that doesn't support the extension?

Add ASAN support

It would be really nice if the bindings did the necessary grunt work to disable asan checking inside the drivers, eliminating any driver or layer related false positives from application output.

`Loader& operator=(Loader && other) noexcept does not return a value`

move assignment operator should return *this;

Add Enumeration functions helpers

EnumerateInstanceExtensionProperties
EnumerateInstanceLayerProperties
EnumerateInstanceVersion
pfn_EnumeratePhysicalDevices
pfn_EnumerateDeviceExtensionProperties
pfn_EnumeratePhysicalDeviceGroups
pfn_EnumeratePhysicalDeviceQueueFamilyPerformanceQueryCountersKHR

Do not do EnumerateInstanceDeviceProperties - deprecated

Add version information into the headers directly.

Vulkan.h has macro defines to specify the major, minor, and patch version of the headers.
The bindings need to have similar functionality.

Also, the VK_MAKE_VERSION() macro needs to be made accessible.

Create helpers for functions which create multiple objects

vkCreateGraphicsPipelines and vkCreateComputePipelines take an array of pipeline create info's and should return a vector of pipelines. This is the last create/query function category but needs special consideration wrt how the len attributes are handled.

Make Loader initialization constructor use a tag, not a bool.

Cleaner and less "what does 'true'/'false' mean.

Make optional handles default constructed with nullptr

This way you don't have to specify an argument manually.
QueueSubmit() requires a Fence() to be provided currently, which is cumbersome.
Making it a pointer would allow actual nullptr values, but it may be simpler to have Fence fence = Fence() so the .get() function doesn't have to be conditional.

Note, there must be logic to guarantee the = default() only occurs if its the last argument in a function. Allocator Callbacks are in the middle, thus can't be defaulted.

Add builder pattern for creation functions

Create structs to hold all intermediate data and have .setXXX() member functions.
Anything that is passed by counter+pointer pairs will have a vector to store the data.
Return type is an expected like type, eg expected<NewValue, Error>.

Need to decide how to handle functions which can create multiple objects, and functions which aren't 'creation' but allocation (descriptor sets & command buffers).

Promote the Result enum to a Struct

There are many capabilities that would be very convenient. conversion to bool, allowing to be ignored (by an explicit function call that does nothing, for use in functions which [[nodiscard]], and more flexibility for interaction with the expected type.

Only downside would be that the enum values would have to be made class constants to preserve the current syntax. vk::Result::Success and vk::Result::EventSet` would need to continue to work.

Replace std::span with custom span

Reason are:
Allows C++17 users to use span - unify interface - really the only reason to do it. Having this be different between the two is silly.
span can be 'customized' to be better suited for vulkan, eg uint32_t size in most places. Make it a template argument which defaults to uint32_t.
Make an implicit conversion operator from std::span in C++20 mode for seamless interop.

Downsides:
Need to have vk::Span be as fleshed out as is reasonably possible. It can't have major deficiencies compared to std::span, lest it be a worse replacement.
More code = more tests needed

Properly #if-def guard for all types from extensions

Currently, structs & unions do not properly guard against the vulkan headers not containing newer extensions.

For example, PhysicalDevice4444FormatsFeaturesEXT will fail to compile if the header version isn't new enough to contain it.