Add profiling support to the HAL API for use in benchmarks about iree

[powderluv]

this was the year!!! thanks @benvanik

[benvanik]

1+ years on and progress is being made on this. New threading system will have statistics built-in and help me define the API we want to expose for this (likely something similar to feedback buffers).

[benvanik]

Maybe this year? :P
I think the first step is to get tracy recording Vulkan times and the task system appearing as a dedicated execution context in tracy as well. That will unblock performance investigations. I'm still not sure we have solid use cases for programmatic fine-grained profiling yet though those may be useful in parameter searches (though with all the usual caveats of applicability of timings they may not be).

[ThomasRaoux]

Do you have some pointers on what would be needed to do to have tracy recoding Vulkan times and what would the result look like? On Android we have pretty much no useful tools for profiling within a command buffer. I hacked up some timestamp queries to be able to get a breakdown for mobile Bert but it is obviously not a sustainable solution. So this is probably our best bet to be able to do at least some basic profiling on phones and I can help out with implementation.

[benvanik]

I've got an old set of changes that enables Vulkan in tracy that I'll revive and get working. The issue I ran into last time (and what prevented me from committing it) was that tracy could not at the time render disjoint or overlapping zones, meaning that if there was any asynchronous or overlapping execution it would pad every zone out such that they were perfectly nested. I remember seeing that fiber support was getting added to tracy (in some form), and if it has landed then we can use that to allow the out-of-order zones. Otherwise, the tracy support only produces useful results if there's single dispatches between global barriers such that no two dispatches ever overlap and that's not very useful for anything but microbenchmarks (which could still be useful, but not general-purpose and with all the caveats of applicability microbenchmarks on GPUs have).

[benvanik]

Fiber support doesn't seem to have landed, but what we really want is wolfpld/tracy#149 - that's how I accomplished this in wtf and it worked really well. Unfortunately it looks like it's not planned work so I'm not sure what to do there.

With the new feature allowing multiple GPU context tracks I can at least split up queues such that queues can overlap, but within each queue the numbers you'll be getting will not account for overlap :(

[benvanik]

I'm going to avoid doing any HAL work here and instead just add Vulkan support directly to the Vulkan HAL. When we want programmatic queries we'll need to add explicit APIs to the HAL but for just seeing timing in tracy we can avoid that.