traverse-research / hassle-rs

🦀 This crate provides an FFI layer and idiomatic rust wrappers for the new DirectXShaderCompiler library.

🌤 hassle-rs

This crate provides an FFI layer and idiomatic rust wrappers for the new DirectXShaderCompiler library.

• Documentation

Usage

Add this to your Cargo.toml:

[dependencies]
hassle-rs = "0.11.0"

Then acquire dxcompiler.dll on Windows or libdxcompiler.so on Linux directly from AppVeyor, or compile it from source according to the instructions in the DirectXShaderCompiler GitHub repository and make sure it's in the executable environment. See our support table below for specific compatibility notes on non-Windows OSes.

DxcValidator also requires dxil.dll which can be grabbed from any recent DXC release: https://github.com/microsoft/DirectXShaderCompiler/releases/latest More info: https://www.wihlidal.com/blog/pipeline/2018-09-16-dxil-signing-post-compile/

Supported DXC versions on non-Windows

Outside of Windows (e.g. Unix) the emulated COM API needed its fair share of fixes to match the layout on Windows. This results in repetitive API breakage that is hard to detect from within hassle-rs: be sure to math the hassle-rs release below to a minimum DXC commit to prevent runtime failures outside of Windows!

Since hassle-rs	DXC release	Git commit
0.10.0	v1.7.2212	https://github.com/microsoft/DirectXShaderCompiler/commit/47f31378a9b51894b0465b33ac1d10ce6349a468
0.5.1 (if using intellisense)	release-1.6.2012	https://github.com/microsoft/DirectXShaderCompiler/commit/2ade6f84d6b95bfd96eec1d6d15e3aa3b519d180

When compiling on MacOS with clang, or Linux with gcc, be sure to have at least https://github.com/microsoft/DirectXShaderCompiler/commit/af14220b45d3ce46e0bad51ce79655e41d07c478 (also included in release-1.6.2012).

Interesting DXC commits pertaining Unix support

These patches have had an effect on hassle-rs compatibility over time:

• [Linux] WinAdapter: Remove virtual dtors from IUnknown to fix vtable ABI
• Linux: Implement prefix-counted BSTR allocation in SysAllocStringLen
• [linux-port] Support full IID comparison on GCC

Usage examples

Compile HLSL into SPIR-V

let spirv = compile_hlsl(
    "shader_filename.hlsl",
    code,
    "copyCs",
    "cs_6_5",
    &vec!["-spirv"],
    &vec![
        ("MY_DEFINE", Some("Value")),
        ("OTHER_DEFINE", None)
    ],
);

Compile HLSL into DXIL and validate it

let dxil = compile_hlsl("test.cs.hlsl", test_cs, "main", "cs_6_5", args, &[]).unwrap();
let result = validate_dxil(&dxil); // Only a Windows machine in Developer Mode can run non-validated DXIL

if let Some(err) = result.err() {
    println!("validation failed: {}", err);
}

License

Licensed under MIT license (LICENSE or http://opensource.org/licenses/MIT)

Contributions

• Graham Wihlidal
• Tiago Carvalho
• Marijn Suijten
• Tomasz Stachowiak
• Manon Oomen

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, shall be licensed as above, without any additional terms or conditions.

Contributions are always welcome; please look at the issue tracker to see what known improvements are documented.

hassle-rs's Issues

[feature request] Serving the so with the library?

Is there any possibility of providing the so and dll binaries directly (similar to how kayija has the binary) instead of asking the user to collect it and install it in the right location?

Given that dxc is now open source I am not sure why the so needs to be manually collected by the user.

Replace com-rs dependency with com

com-rs has been deprecated in favor of com.

Linking fails on MacOSX (missing -luuid)

I'm trying (in a non-embark-related side project) to use this from a build.rs on a Macbook Air M1, but any use of hassle-rs causes a linker error about a missing -luuid when trying to build the build.rs:

error: linking with cc failed: exit status: 1
|
= note: "cc" "-arch" "arm64" "/Users/hrydg/dev/rgb/target/debug/build/rgb-dca230a1aa5b7e54/build_script_build-dca230a1aa5b7e54.10m9mxptggm1u1hd.rcgu.o" "/Users/hrydg/dev/rgb/target/debug/build/rgb-dca230a1aa5b7e54/build_script_build-dca230a1aa5b7e54.14wrnmxnrm9y2a17.rcgu.o" "/Users/hr

[SNIP]

libcompiler_builtins-56bb43a121401f7f.rlib" "-luuid" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/hrydg/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/hrydg/dev/rgb/target/debug/build/rgb-dca230a1aa5b7e54/build_script_build-dca230a1aa5b7e54" "-Wl,-dead_strip" "-nodefaultlibs" "-undefined" "dynamic_lookup"
= note: ld: library not found for -luuid
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Basically, seems like some library that hassle depends on is trying to link with -luuid, which apparently is not a thing on Mac (libuuid is header-only, apparently).

(note: just thought it might be good to report, you probably should not give this any kind of priority, heh)

My thoughts about `hassle_rs::compile_hlsl`

I just looked into hassle_rs::compile_hlsl and I saw that the compiler is recreated every time the function is called. I though about using a thread local to store the DxcLibrary and DxcCompiler and just take the compiler for the current thread (as Dxc is not thread safe). Just had the idea and wanted to ask what your thoughts are on it.

This is more of a discussion than an issue, but I'm posting it here because I can't create a discussion :)

If you agree with me on the idea, I can of course make a pull request!

Static Linking

Is there something preventing statically linking the DirectXShaderCompiler?

My understanding is that hassle doesn't actually build the compiler, but only provides an FFI interface, so statically linking would require building it as a library.

I've also read that shader validation is performed by the GPU/Driver, so is that why getting around the official dll's is not possible?

"Borrow" `Dxc` (lifetime) inside `DxcCompiler` and `DxcLibrary`

update: This is due to dropping the Dxc object while holding onto the DxcCompiler and DxcLibrary objects and trying to use them after Dxc was dropped.

I've been trying to get this to work with wgpu, but I'm stuck trying to get it to create the DxcBlobEncoding to pass to DxcCompiler.compile.

It fails with code 0xc0000005: Invalid access to memory location. (os error 998) when calling

let blob = dxc_library
    .create_blob_with_encoding_from_str(&source);

The exception is in https://docs.rs/crate/com-rs/0.2.1/source/src/macros.rs in the innermost part of

        impl $iface {
            $($(#[$fn_attr])*
            pub unsafe fn $func(&self, $($i: $t),*) -> $rt {
                ((*self.vtable).$func)(self $(,$i)*)
            })*
        }

Passing the same &source to hassle_rs::utils::compile_hlsl works.

edit:edit: Link to WIP where the crash occurs

Why is `dynlib` used as macOS library suffix?

I saw that the DXC native library file is named libdxcompiler.dynlib instead of libdxcompiler.dylib, what's the reason for this? I have never seen the type dynlibon MacOS

Where should libdxcompiler.so be placed?

The documentation mentions you should have the file on your environment, but merely placing it at the root of the project does not seem to allow the crate to find, it. It would be helpful for the error message to print the absolute path to where it is trying to find the .so/

Readme has outdated information about where to get dxil.dll

The downloads on the release tab of https://github.com/Microsoft/DirectXShaderCompiler seem to now include a more up to date version of dxil.dll than what WindowsSDK includes.

(left: SDK, right: latest github release (from releases, not CI)

Linux - search common locations for libdxcompiler.so

Hi! I'm using the directx-shader-compiler package on arch linux that just installs libdxcompiler.so to /usr/lib/libdxcompiler.so, but hassle-rs currently requires it be in the local directory so I have to copy it over. It'd be great for hassle-rs to check common locations (such as /usr/lib) for it first.

Ideally a tool or crate should be used to find libraries instead of hardcoding paths - I think some interface to ldconfig (is there a crate for this?) would work. https://stackoverflow.com/q/9922949 has some suggestions.

include macros

Write function-like procedural macros to invoke compile_hlsl and validate_dxil at compile time. I think most users of this library will want to compile their shaders to SPIR-V/DXIL at compile time because it's unlikely that players will have dxil.dll available.

To kick off the conversation, I suggest we follow in the footsteps of inline-spirv in regards to the usage patterns:

let spirv = include_hlsl!(
    "shader_filename.hlsl",
    "entry",
    "cs_6_5",
    &vec!["-spirv"],
    &vec![
        ("MY_DEFINE", Some("Value")),
        ("OTHER_DEFINE", None)
    ],
);

let dxil= validate_dxil!(inline_hlsl!(
    r#"
        // your inline HLSL code here
    "#,
    "entry",
    "cs_6_5",
    &vec![],
    &vec![
        ("MY_DEFINE", Some("Value")),
        ("OTHER_DEFINE", None)
    ],
));

I'm not sure what the API for validation should look like or whether it's feasible, but we should consider whether we will need to validate DXIL at compile time.

Automatic validation fails if `dxil.dll` isn't in the local folder.

Someone ran into this with wgpu (gfx-rs/wgpu#3433), and it turns out if you're pointing to dxil.dll outside of the local scope it'll silently fail to validate the shaders.

The solution seems to be passing -Vd in the compile flags and doing explicit validation.

Is it better to always do explicit validation, or should I avoid passing -Vd unless the user has set a custom dxil.dll path?

License Field Missing DXC's License

Thanks for this amazing library! As we're integrating this in wgpu, I noticed that the license is listed as only MIT.

Because this links in DXC the license field should probably be MIT AND NCSA.