embarkstudios / ash-molten Goto Github PK

This repo should be migrated to use Github Actions instead of Travis like all of our other repos.

@MaikKlein would it be possible to disable compiling the full C++ MoltenVK dependency when building with clippy, tests and for RLS?

As it can take 5+ min to compile MoltenVK on a MBP15 it would be nice to not have to build it for configurations where it is not used in practice.

Looks like it contains quite a few Apple M1 fixes as well as new SPIRV-Cross version which is good. cc @VZout @hrydgard

https://github.com/KhronosGroup/MoltenVK/releases/tag/v1.1.5

I'm trying to get vulkan, rust, and SDL2 up and running on iOS. I created an empty rust project to build as a staticlib. I added a dependency for ash-molten (not using the external feature). My lib.rs has a #[no_mangle] extern "C" function which my Xcode project calls. I am largely following the steps here: https://github.com/thombles/dw2019rust/tree/master/modules

When I trigger a build, I get this error:

/Users/pmd/dev/rust/ios_test/target/aarch64-apple-ios/debug/build/ash-molten-4ea38210b25f3328/out/MoltenVK-v1.0.41/External/glslang/External/spirv-tools/tools/reduce/reduce.cpp:33:18: error: 'system' is unavailable: not available on iOS
  int res = std::system(nullptr);
                 ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.2.sdk/usr/include/stdlib.h:190:6: note: 'system' has been explicitly marked unavailable here
int      system(const char *) __DARWIN_ALIAS_C(system);
         ^
/Users/pmd/dev/rust/ios_test/target/aarch64-apple-ios/debug/build/ash-molten-4ea38210b25f3328/out/MoltenVK-v1.0.41/External/glslang/External/spirv-tools/tools/reduce/reduce.cpp:41:21: error: 'system' is unavailable: not available on iOS
  int status = std::system(command.c_str());
                    ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.2.sdk/usr/include/stdlib.h:190:6: note: 'system' has been explicitly marked unavailable here
int      system(const char *) __DARWIN_ALIAS_C(system);
         ^
2 errors generated.
make[2]: *** [tools/CMakeFiles/spirv-reduce.dir/reduce/reduce.cpp.o] Error 1
make[1]: *** [tools/CMakeFiles/spirv-reduce.dir/all] Error 2
make: *** [all] Error 2
thread 'main' panicked at 'failed to fetchDependencies', /Users/pmd/.cargo/registry/src/github.com-1ecc6299db9ec823/ash-molten-0.5.0+41/build.rs:86:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

It looks like the spirv-tools are failing to build because they use std::system in the code which is not available in iOS. I'm not sure what to do here... is this code that would end up on device? Maybe this is an upstream change that recently broke this?

Currently we have a magic build.rs file that syncs down a specific git revision of MoltenVK and then runs MoltenVK's even more magic fetch_dependencies.sh script which syncs down another 6-7 C++ dependencies from git, and then it is able to build.

All of this does work, but it is a bit fragile and ran into a specific problem where our license scanner in cargo-about didn't detect the additional dependencies and licenses of these downloaded dependencies because they simply do not exist in the .crate file, only exist in the target directory after build.rs have been run - which is hard to scan and find.

@MaikKlein @Jake-Shadle think we should try and change to have an explicit update script that we run manually to get down MoltenVK and all its dependencies and check it in, instead of build.rs.

When a new pull request gets created the CI will fail on the first run on the pre-built step. Running the CI again it will succeed if no other issues are present with the PR.

Two issues

• Github actions doesn't have the latest version of xcode 11.5

• On OSX 10.15.4 and xcode 11.5 I get an unknown symbol in moltenvk ___isPlatformVersionAtLeast. I am not sure where it is coming from.

  After scavenging a few forum it appears to be defined in compiler-rt which is supposed to be re-exported by libSystem. But that is not the case on my system.

This is probably a question and not really an issue. Since I had build failures when ash-molten was grabbing and building its own copy of MoltenVK, I decided to try using a prebuilt binary from the Vulkan SDK using the external feature. However it wasn't clear to me what the best way of adding the library to the path would be. I did find a couple solutions that worked, but it took a while to figure it out and I'm wondering if there are better solutions.

Adding a -L via the RUSTFLAGS environment variable works
RUSTFLAGS=-L/Users/pmd/dev/sdk/vulkansdk-macos-1.2.131.2/MoltenVK/iOS/static cargo build

Setting LD_LIBRARY_PATH directly actually didn't work for me

I also tried adding this to my build.rs:
println!("cargo:rustc-link-search=all=/Users/pmd/dev/sdk/vulkansdk-macos-1.2.131.2/MoltenVK/iOS/static");

However, this didn't work I believe because ash-molten builds first and fails. Since the crate is pretty small, I tried copying/pasting code from this crate into my code and setting up the build.rs like this:

fn main() {
    println!("cargo:rustc-link-lib=framework=Metal");
    println!("cargo:rustc-link-lib=framework=AppKit");
    println!("cargo:rustc-link-lib=framework=QuartzCore");
    println!("cargo:rustc-link-lib=framework=IOKit");
    println!("cargo:rustc-link-lib=framework=IOSurface");
    println!("cargo:rustc-link-lib=dylib=c++");
    println!("cargo:rustc-link-lib=static=MoltenVK");
    println!("cargo:rustc-link-search=all=/Users/pmd/dev/sdk/vulkansdk-macos-1.2.131.2/MoltenVK/iOS/static");
    println!("cargo:rustc-link-lib=static=MoltenVK");
}

It's kind of ugly but ash-molten is small enough that copying/pasting the one lib.rs into my project does seem like a possible solution.

Very long term, I'm kind of thinking I will want to have SDL2 and MoltenVK source vendored so I can easily set breakpoints/add printfs to troubleshoot issues that might arise.. that might be another approach.

Any advice? Right now I lean towards the first method, although I just noticed that it seems to be doing a full build with every code change (including ash, sdl2, etc.)

Sorry for being long-winded, I'm thinking a bit out loud here in case someone has suggestions and for the benefit of anyone else that might try to do this :D

Currently one get a cryptic Objective C compile error if one is building with an unsupported Xcode, and with PR #26 we now require Xcode 12 (I think?) so more common to run into this. Would be much nicer if we can detect unsupported Xcode version in build.rs and fail there before starting the slow MoltenVK build.

Spawned off from #28 (comment)

Something you could take a quick look at @MarnixKuijs ?

That build script will just print a message that will never be forwarded to the user, because it also "succeeds" (since it doesn't panic, nor exit with a non-0 return code - either via fn main() -> ... or std::process::exit).

Is this intentional? cargo run, and many other uses of the library, will still fail with a linker error.

We download pre-built resources using the Github API. This however runs into Github's rate limit. This limit can be bumped by authenticating.

Currently it is quite commen for CI builds to fail due to this rate limit.

That was just released with nice Vulkan 1.1 support and Apple Silicon fixes

https://github.com/KhronosGroup/MoltenVK/releases/tag/v1.1.1

Details here: EmbarkStudios/rust-gpu#122

Apparently, according to nodejs/node-gyp#569, the "error"(warning?) can actually just be ignored and continue building as normal. I'm not sure exactly what needs to be changed here to do so.

Currently the Vulkan SDK doesn't include a pre-built MoltenVK library for the aarch64-apple-darwin target so we haven't uploaded it here for use with the pre-built feature. But we will want to have that fairly soon, either from a new Vulkan SDK (ideally) or build it ourselves.

In the meantime one will have to build MoltenVK from source and not use the pre-built feature

Embark uses a standard set of lints across all repositories. We just upgraded
to version 0.3 of these lints, and this repository needs to be updated to
the new set of lints with any warnings fixed.

Steps

1. Copy the contents of the lints.rs file from
   EmbarkStudios/rust-ecosystem and replace the old list of lints in the
   src/lib.rs and/or src/main.rs files with the new list.
2. Run cargo clippy
3. Update and fix any code that now triggers a new warning.

error: Build service could not start build operation: unknown error while handling message: Could not acquire lock at path `Path(str: "/Users/magnus/Library/Developer/Xcode/DerivedData/MoltenVKPackaging-bxdidhamrfqiwyhgsqzwwupeyhcv/Build/Intermediates.noindex/XCBuildData/PIFCache/PROJECT@v11_mod=1579705587.0288634_hash=8b05d8b6ed75d6f658c9c6bb804c0aad-Project.lock")`
** CLEAN FAILED **
4:33
Also get this warning:CMake Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.  MACOSX_RPATH is not specified for the following targets:   SPIRV-Tools-sharedThis warning is for project developers.  Use -Wno-dev to suppress it./Users/magnus/dev/ark/target/release/build/ash-molten-e1cf83056b3a6949/out/MoltenVK-v1.0.38/External/glslang/External/spirv-tools/utils/update_build_version.py:89: DeprecationWarning: 'U' mode is deprecated
  with open(changes_file, mode='rU') as f:
/Users/magnus/dev/ark/target/release/build/ash-molten-e1cf83056b3a6949/out/MoltenVK-v1.0.38/External/glslang/External/spirv-tools/utils/update_build_version.py:89: DeprecationWarning: 'U' mode is deprecated
  with open(changes_file, mode='rU') as f:
Cloning into 'Vulkan-Tools'...
Note: switching to '5824344b3e6783c2dc1953d22e6f1b162d2bca14'.

Unclear yet if that is related to ash-molten or even molten.

For rust-gpu we need various fixes in spriv-cross and MoltenVK.
I couldn't get a clear answer for their release schedule so I think adding a temporary "pre-release" binaries could be the way to go for now.

I can't really test this myself without #36 though.

Details: EmbarkStudios/rust-gpu#123

Device:

• OS: macOS 10.14.6

Additional context
This is pulling the moltenvk-1.1.0 branch

This repo no longer builds, does it try to always just get latest MoltenVK? We don't pin it to a specific version?

========== Building dependency libraries quietly. Please be patient on first build. ==========

still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
========== Done! ==========
xcodebuild -quiet -project "MoltenVKPackaging.xcodeproj" -scheme "MoltenVK Package (macOS only)" build
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.mm:19:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.h:24:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/GPUObjects/MVKDevice.h:26:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Vulkan/mvk_datatypes.hpp:23:
/Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/API/mvk_datatypes.h:238:1: error: unknown type name 'MTLTextureSwizzle'
MTLTextureSwizzle mvkMTLTextureSwizzleFromVkComponentSwizzle(VkComponentSwizzle vkSwizzle);
^
/Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/API/mvk_datatypes.h:241:1: error: unknown type name 'MTLTextureSwizzleChannels'
MTLTextureSwizzleChannels mvkMTLTextureSwizzleChannelsFromVkComponentMapping(VkComponentMapping vkMapping);
^
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.mm:19:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.h:24:
/Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/GPUObjects/MVKDevice.h:293:28: error: unknown type name 'MTLGPUFamily'
        bool getSupportsGPUFamily(MTLGPUFamily gpuFamily);
                                  ^
3 errors generated.
note: Using new build systemnote: Planning buildnote: Constructing build description

--- stderr