embarkstudios / ash-molten Goto Github PK
View Code? Open in Web Editor NEW๐ Statically linked MoltenVK for Vulkan on Mac using Ash ๐ฆ
Home Page: http://embark.rs
License: Apache License 2.0
๐ Statically linked MoltenVK for Vulkan on Mac using Ash ๐ฆ
Home Page: http://embark.rs
License: Apache License 2.0
This repo should be migrated to use Github Actions instead of Travis like all of our other repos.
@MaikKlein would it be possible to disable compiling the full C++ MoltenVK dependency when building with clippy, tests and for RLS?
As it can take 5+ min to compile MoltenVK on a MBP15 it would be nice to not have to build it for configurations where it is not used in practice.
Looks like it contains quite a few Apple M1 fixes as well as new SPIRV-Cross version which is good. cc @VZout @hrydgard
https://github.com/KhronosGroup/MoltenVK/releases/tag/v1.1.5
I'm trying to get vulkan, rust, and SDL2 up and running on iOS. I created an empty rust project to build as a staticlib. I added a dependency for ash-molten (not using the external
feature). My lib.rs has a #[no_mangle] extern "C" function which my Xcode project calls. I am largely following the steps here: https://github.com/thombles/dw2019rust/tree/master/modules
When I trigger a build, I get this error:
/Users/pmd/dev/rust/ios_test/target/aarch64-apple-ios/debug/build/ash-molten-4ea38210b25f3328/out/MoltenVK-v1.0.41/External/glslang/External/spirv-tools/tools/reduce/reduce.cpp:33:18: error: 'system' is unavailable: not available on iOS
int res = std::system(nullptr);
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.2.sdk/usr/include/stdlib.h:190:6: note: 'system' has been explicitly marked unavailable here
int system(const char *) __DARWIN_ALIAS_C(system);
^
/Users/pmd/dev/rust/ios_test/target/aarch64-apple-ios/debug/build/ash-molten-4ea38210b25f3328/out/MoltenVK-v1.0.41/External/glslang/External/spirv-tools/tools/reduce/reduce.cpp:41:21: error: 'system' is unavailable: not available on iOS
int status = std::system(command.c_str());
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.2.sdk/usr/include/stdlib.h:190:6: note: 'system' has been explicitly marked unavailable here
int system(const char *) __DARWIN_ALIAS_C(system);
^
2 errors generated.
make[2]: *** [tools/CMakeFiles/spirv-reduce.dir/reduce/reduce.cpp.o] Error 1
make[1]: *** [tools/CMakeFiles/spirv-reduce.dir/all] Error 2
make: *** [all] Error 2
thread 'main' panicked at 'failed to fetchDependencies', /Users/pmd/.cargo/registry/src/github.com-1ecc6299db9ec823/ash-molten-0.5.0+41/build.rs:86:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
It looks like the spirv-tools are failing to build because they use std::system in the code which is not available in iOS. I'm not sure what to do here... is this code that would end up on device? Maybe this is an upstream change that recently broke this?
Currently we have a magic build.rs
file that syncs down a specific git revision of MoltenVK and then runs MoltenVK's even more magic fetch_dependencies.sh
script which syncs down another 6-7 C++ dependencies from git, and then it is able to build.
All of this does work, but it is a bit fragile and ran into a specific problem where our license scanner in cargo-about
didn't detect the additional dependencies and licenses of these downloaded dependencies because they simply do not exist in the .crate
file, only exist in the target directory after build.rs
have been run - which is hard to scan and find.
@MaikKlein @Jake-Shadle think we should try and change to have an explicit update script that we run manually to get down MoltenVK and all its dependencies and check it in, instead of build.rs
.
When a new pull request gets created the CI will fail on the first run on the pre-built step. Running the CI again it will succeed if no other issues are present with the PR.
Two issues
Github actions doesn't have the latest version of xcode 11.5
On OSX 10.15.4
and xcode 11.5
I get an unknown symbol in moltenvk ___isPlatformVersionAtLeast
. I am not sure where it is coming from.
After scavenging a few forum it appears to be defined in compiler-rt
which is supposed to be re-exported by libSystem
. But that is not the case on my system.
This is probably a question and not really an issue. Since I had build failures when ash-molten was grabbing and building its own copy of MoltenVK, I decided to try using a prebuilt binary from the Vulkan SDK using the external
feature. However it wasn't clear to me what the best way of adding the library to the path would be. I did find a couple solutions that worked, but it took a while to figure it out and I'm wondering if there are better solutions.
Adding a -L via the RUSTFLAGS environment variable works
RUSTFLAGS=-L/Users/pmd/dev/sdk/vulkansdk-macos-1.2.131.2/MoltenVK/iOS/static cargo build
Setting LD_LIBRARY_PATH directly actually didn't work for me
I also tried adding this to my build.rs:
println!("cargo:rustc-link-search=all=/Users/pmd/dev/sdk/vulkansdk-macos-1.2.131.2/MoltenVK/iOS/static");
However, this didn't work I believe because ash-molten builds first and fails. Since the crate is pretty small, I tried copying/pasting code from this crate into my code and setting up the build.rs like this:
fn main() {
println!("cargo:rustc-link-lib=framework=Metal");
println!("cargo:rustc-link-lib=framework=AppKit");
println!("cargo:rustc-link-lib=framework=QuartzCore");
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=IOSurface");
println!("cargo:rustc-link-lib=dylib=c++");
println!("cargo:rustc-link-lib=static=MoltenVK");
println!("cargo:rustc-link-search=all=/Users/pmd/dev/sdk/vulkansdk-macos-1.2.131.2/MoltenVK/iOS/static");
println!("cargo:rustc-link-lib=static=MoltenVK");
}
It's kind of ugly but ash-molten is small enough that copying/pasting the one lib.rs into my project does seem like a possible solution.
Very long term, I'm kind of thinking I will want to have SDL2 and MoltenVK source vendored so I can easily set breakpoints/add printfs to troubleshoot issues that might arise.. that might be another approach.
Any advice? Right now I lean towards the first method, although I just noticed that it seems to be doing a full build with every code change (including ash, sdl2, etc.)
Sorry for being long-winded, I'm thinking a bit out loud here in case someone has suggestions and for the benefit of anyone else that might try to do this :D
Currently one get a cryptic Objective C compile error if one is building with an unsupported Xcode, and with PR #26 we now require Xcode 12 (I think?) so more common to run into this. Would be much nicer if we can detect unsupported Xcode version in build.rs
and fail there before starting the slow MoltenVK build.
Spawned off from #28 (comment)
Something you could take a quick look at @MarnixKuijs ?
Lines 375 to 378 in 09bed93
That build script will just print a message that will never be forwarded to the user, because it also "succeeds" (since it doesn't panic, nor exit with a non-0 return code - either via fn main() -> ...
or std::process::exit
).
Is this intentional? cargo run
, and many other uses of the library, will still fail with a linker error.
We download pre-built resources using the Github API. This however runs into Github's rate limit. This limit can be bumped by authenticating.
Currently it is quite commen for CI builds to fail due to this rate limit.
That was just released with nice Vulkan 1.1 support and Apple Silicon fixes
https://github.com/KhronosGroup/MoltenVK/releases/tag/v1.1.1
Details here: EmbarkStudios/rust-gpu#122
Apparently, according to nodejs/node-gyp#569, the "error"(warning?) can actually just be ignored and continue building as normal. I'm not sure exactly what needs to be changed here to do so.
Currently the Vulkan SDK doesn't include a pre-built MoltenVK library for the aarch64-apple-darwin
target so we haven't uploaded it here for use with the pre-built
feature. But we will want to have that fairly soon, either from a new Vulkan SDK (ideally) or build it ourselves.
In the meantime one will have to build MoltenVK from source and not use the pre-built
feature
Embark uses a standard set of lints across all repositories. We just upgraded
to version 0.3 of these lints, and this repository needs to be updated to
the new set of lints with any warnings fixed.
lints.rs
file fromsrc/lib.rs
and/or src/main.rs
files with the new list.cargo clippy
error: Build service could not start build operation: unknown error while handling message: Could not acquire lock at path `Path(str: "/Users/magnus/Library/Developer/Xcode/DerivedData/MoltenVKPackaging-bxdidhamrfqiwyhgsqzwwupeyhcv/Build/Intermediates.noindex/XCBuildData/PIFCache/PROJECT@v11_mod=1579705587.0288634_hash=8b05d8b6ed75d6f658c9c6bb804c0aad-Project.lock")`
** CLEAN FAILED **
4:33
Also get this warning:CMake Warning (dev):
Policy CMP0042 is not set: MACOSX_RPATH is enabled by default. Run "cmake
--help-policy CMP0042" for policy details. Use the cmake_policy command to
set the policy and suppress this warning. MACOSX_RPATH is not specified for the following targets: SPIRV-Tools-sharedThis warning is for project developers. Use -Wno-dev to suppress it./Users/magnus/dev/ark/target/release/build/ash-molten-e1cf83056b3a6949/out/MoltenVK-v1.0.38/External/glslang/External/spirv-tools/utils/update_build_version.py:89: DeprecationWarning: 'U' mode is deprecated
with open(changes_file, mode='rU') as f:
/Users/magnus/dev/ark/target/release/build/ash-molten-e1cf83056b3a6949/out/MoltenVK-v1.0.38/External/glslang/External/spirv-tools/utils/update_build_version.py:89: DeprecationWarning: 'U' mode is deprecated
with open(changes_file, mode='rU') as f:
Cloning into 'Vulkan-Tools'...
Note: switching to '5824344b3e6783c2dc1953d22e6f1b162d2bca14'.
Unclear yet if that is related to ash-molten or even molten.
For rust-gpu we need various fixes in spriv-cross and MoltenVK.
I couldn't get a clear answer for their release schedule so I think adding a temporary "pre-release" binaries could be the way to go for now.
I can't really test this myself without #36 though.
Details: EmbarkStudios/rust-gpu#123
Device:
Additional context
This is pulling the moltenvk-1.1.0
branch
This repo no longer builds, does it try to always just get latest MoltenVK? We don't pin it to a specific version?
========== Building dependency libraries quietly. Please be patient on first build. ==========
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
still building MoltenVK
========== Done! ==========
xcodebuild -quiet -project "MoltenVKPackaging.xcodeproj" -scheme "MoltenVK Package (macOS only)" build
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.mm:19:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.h:24:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/GPUObjects/MVKDevice.h:26:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Vulkan/mvk_datatypes.hpp:23:
/Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/API/mvk_datatypes.h:238:1: error: unknown type name 'MTLTextureSwizzle'
MTLTextureSwizzle mvkMTLTextureSwizzleFromVkComponentSwizzle(VkComponentSwizzle vkSwizzle);
^
/Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/API/mvk_datatypes.h:241:1: error: unknown type name 'MTLTextureSwizzleChannels'
MTLTextureSwizzleChannels mvkMTLTextureSwizzleChannelsFromVkComponentMapping(VkComponentMapping vkMapping);
^
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.mm:19:
In file included from /Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/Commands/MVKMTLBufferAllocation.h:24:
/Users/johan/git/embark/ash-molten/target/debug/build/ash-molten-261694c0a8c71a6c/out/MoltenVK/MoltenVK/MoltenVK/GPUObjects/MVKDevice.h:293:28: error: unknown type name 'MTLGPUFamily'
bool getSupportsGPUFamily(MTLGPUFamily gpuFamily);
^
3 errors generated.
note: Using new build systemnote: Planning buildnote: Constructing build description
--- stderr
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.