Comments (23)
We are definitely interested in making example toolchains to work on all platforms :)
from buck2.
Internally we have quite different toolchains - but the entire divergence is in the toolchain rule, so if you just take _system_cxx_toolchain_impl and make it spit out whatever toolchain you want, complete with as many hardcoding as you want, that will work just fine.
from buck2.
Given that this does work internally, I expect the work to get this working externally is small. Roughly:
- Remove the
if
in https://github.com/facebook/buck2/blob/main/examples/prelude/rust/BUCK so these targets are enabled. - Fix the Rust and C++ toolchains, partly as you've observed.
I expect the diff to be a few lines long. @KapJI - do we target msvc by default for Rust?
from buck2.
(As it turns out, my knowledge ends at the boundaries of Rust rules on Linux/Mac platforms 😅)
from buck2.
Is clang++
on your windows path, or only your mingw/shell path? E.g. if you start a vanilla cmd
shell, is clang++
on your path then? We run things like clang++
directly through cmd
as that gives best compatibility, but if your .bashrc
or similar sets up clang++
it won't be found.
from buck2.
You can also run this command manually to debug this issue outside Buck:
cmd "/c" "buck-out\v2\gen\root\fb50fd37ce946800\__hello_world__\__linker_wrapper.bat" <other args>
from buck2.
Is clang++ on your windows path, or only your mingw/shell path?
I don't use mingw, I use native Windows. The above is in my normal shell (nushell), but also
〉cmd /c "clang++"
clang++: error: no input files
which is part of why it's so confusing!
You can also run this command manually to debug this issue outside Buck:
Thanks @KapJI! I thought I had tried this, but I must have gotten some of the arguments wrong.
> cmd /c "buck-out\\v2\\gen\\prelude\\fb50fd37ce946800\\python_bootstrap\\tools\\__win_python_wrapper__\\win_python_wrapper.bat" "buck-out\\v2\\gen\\prelude\\fb50fd37ce946800\\rust\\tools\\__rustc_action__\\__rustc_action__" python "buck-out\\v2\\gen\\prelude\\fb50fd37ce946800\\rust\\tools\\__rustc_action__\\rustc_action.py" "@buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\hello_world-link-diag.args"
WARN rustc_codegen_ssa::back::link Linker does not support -no-pie command line option. Retrying without.
error: linking with `buck-out\v2\gen\root\fb50fd37ce946800\__hello_world__\__linker_wrapper.bat` failed: exit code: 1
|
= note: "cmd" "/c" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\__linker_wrapper.bat" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\self-contained\\crt2.o" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsbegin.o" "C:\\Users\\steve\\AppData\\Local\\Temp\\rustccz58eA\\symbols.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\extras\\hello_world\\hello_world.hello_world.aeb6c6e0-cgu.0.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\extras\\hello_world\\hello_world.hello_world.aeb6c6e0-cgu.1.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\extras\\hello_world\\hello_world.hello_world.aeb6c6e0-cgu.2.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\extras\\hello_world\\hello_world.hello_world.aeb6c6e0-cgu.3.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\extras\\hello_world\\hello_world.hello_world.aeb6c6e0-cgu.4.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\extras\\hello_world\\hello_world.hello_world.aeb6c6e0-cgu.5.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\extras\\hello_world\\hello_world.1l3753u24tlzpzc9.rcgu.o" "-L" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link-deps-0" "-L" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-Wl,-Bstatic" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd-e363be82127e72d4.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libpanic_unwind-271c0a4c2400bd0e.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libobject-3b3a88ddf57ad9b8.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libmemchr-c38acbaaa0512e61.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libaddr2line-a777dde688506f47.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libgimli-00e812c5215e2bb4.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_demangle-9824443ffde90bb7.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd_detect-c9cae9f57d72c5d8.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libhashbrown-80b5e088fad27661.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libminiz_oxide-25b744457ec6a6b9.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libadler-b662208514509737.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_alloc-70e1db2cbff7c5e3.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libunwind-bc622eac43f92150.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcfg_if-da38528f9991ea5d.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liblibc-0217604e5fc185ea.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc-094368c19a10127d.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_core-9310325d5d5607bd.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcore-5c3fe6fc6388f93c.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcompiler_builtins-d765c9bc514400ee.rlib" "-Wl,-Bdynamic" "-lkernel32" "-ladvapi32" "-luserenv" "-lkernel32" "-lws2_32" "-lbcrypt" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-nostartfiles" "-L" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-L" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\self-contained" "-o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\static_pic\\hello_world.exe" "-Wl,--gc-sections" "-nodefaultlibs" "@buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\bin-pic-static_pic-link\\__hello_world-link_linker_args.txt" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsend.o"
= note: lld-link: warning: ignoring unknown argument '--dynamicbase', did you mean '-dynamicbase'
lld-link: warning: ignoring unknown argument '--disable-auto-image-base'
lld-link: warning: ignoring unknown argument '--high-entropy-va'
lld-link: warning: ignoring unknown argument '-Bstatic'
lld-link: warning: ignoring unknown argument '-Bdynamic'
lld-link: warning: ignoring unknown argument '--nxcompat', did you mean '-nxcompat'
lld-link: warning: ignoring unknown argument '--gc-sections'
lld-link: error: could not open 'gcc_eh.lib': no such file or directory
lld-link: error: could not open ':libpthread.a.lib': no such file or directory
lld-link: error: could not open 'mingwex.lib': no such file or directory
lld-link: error: could not open 'mingw32.lib': no such file or directory
lld-link: error: could not open 'gcc.lib': no such file or directory
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
given this error and the comment above, is it expected to be using this inside of mingw? This reads like lld being passed ld flags to me...
I poked around noticed that the -msvc
version of Rust isn't supported yet, so maybe that is actually my issue here. (I set up a rustup override to use x86_64-pc-windows-gnu
in this directory because of it) I'm still not 100% sure why invoking the wrapper myself produces a more useful error.
from buck2.
It seems the issue here is we don't support lld-link
on Windows. We use MSVC flavoured llvm-link
internally.
llvm-link
should work if cxx_toolchain
has linker_type = "windows"
.
Basically Windows support in Buck2 is quite limited right now, especially in OSS version, but we're working on improving it.
from buck2.
Basically Windows support in Buck2 is quite limited right now, especially in OSS version, but we're working on improving it.
Totally understand, that's partly why I'm poking at it, I'd like to help make it better, if possible :)
I would be happy to use whatever setup makes this actually work.
cxx.bzl
has this
def _system_cxx_toolchain_impl(ctx):
"""
A very simple toolchain that is hardcoded to the current environment.
"""
archiver_type = "gnu"
linker_type = "gnu"
if host_info().os.is_macos:
linker_type = "darwin"
elif host_info().os.is_windows:
linker_type = "windows"
archiver_type = "windows"
so it seems to be setting that unconditionally. But then later, as one of the arguments to CxxToolchainInfo
:
linker_flags = ["-fuse-ld=lld"] + ctx.attrs.link_flags,
so it feels like this is currently hard-coded to use lld. I'm guessing that this means there's a divergence between the prelude inside of the company? How would I set up MSVC flavored llvm-link?
Even beyond that though, yinz are building a rust project on Windows in CI https://github.com/facebook/buck2/blob/main/.circleci/config.yml#L201 so this should work. I'm gonna continue to poke at how that environment differs from mine, I think.
from buck2.
Yes, system_cxx_toolchain
rule is very basic version of toolchain configuration we created for open source. Internally we construct cxx_toolchain
rule which has much more control on different parameters:
https://github.com/facebook/buck2/blob/main/prelude/cxx/cxx_toolchain.bzl
from buck2.
Ah that makes sense, and yeah, if I build the no_prelude example, it builds just fine, so that all fits!
Given all that, that unsticks me in the moment, but I am unsure if I should just close this out, or if it's useful to track enhancements to that basic toolchain config. Are you all interested in enhancements there, or is the idea that each project should be mostly coming up with its own configurations, rather than using the prelude?
from buck2.
Yes, MSVC toolchain is the default one.
from buck2.
I spent some time over a video call with @davidbarsky, who was very helpful in showing me some things, but we weren't able to figure out the root cause.
I'd love to help fix up those toolchains, but given I'm sorta stuck, I don't know how. Current open questions/thoughts:
- I still am not sure why clang++ cannot be found. I'm assuming there's some sort of isolation doing its job a bit too well, but I'm unsure where or how to investigate this.
- if I run the command manually, it does find clang++, but:
- it appears that ld -style linker args are being passed to lld, causing warnings that they're being ignored. It's not clear to me how this is happening.
- it cannot find a few different
.lib
files, this is probably something wrong with my system, as I don't use the -gnu Rust often.
from buck2.
I ran into the same issue and made a comment over at https://github.com/facebook/buck2-prelude/issues/6
If you change the linker to link
in cxx.bzl
and change the _DEFAULT_TRIPLE
to the -msvc
targets, everything seems to work
from buck2.
Can confirm! Works for me as well. Thanks so much @kcking !
from buck2.
@kcking @steveklabnik - can you confirm exactly how you reproduce this? E.g. what pieces of the Rust toolchain you put on the PATH, how you get clang and which version etc? I'd like to replicate. Once you have that, does compiling the C++ examples on Windows also work?
from buck2.
E.g. what pieces of the Rust toolchain you put on the PATH
Just the stuff that rustup does normally, .cargo/bin
.
how you get clang and which version etc?
Unlike many targets, this doesn't invoke the linker through the compiler, You don't use clang in this configuration, you're using link.exe
, the MSVC linker. This is the default for the MSVC target. People used to have to download it and install it, but IIRC recently, rustup will fetch it for you, so you don't have to do that either.
Once you have that, does compiling the C++ examples on Windows also work?
I haven't tried it yet but will later today.
from buck2.
So yes, this still leads to a "clang not found" error:
cxx_binary(
name = "main",
srcs = ["main.cpp"],
link_style = "static",
)
#include<iostream>
int main() {
std::cout << "Hello, world!" << std::endl;
}
invoking buck2 build //...
:
Still cannot find clang
, still works in my shell though. Tricky!
from buck2.
Oh, as for installing clang: I usually use vcpkg
, but for some reason it had issues building, so I used scoop
, another package manager. All of the llvm tools live in C:\Users\steve\scoop\apps\llvm\current\bin\
, which is in my PATH.
from buck2.
To get rust working, I overrode the linker to be link
on windows in cxx.bzl
. When trying @steveklabnik's cpp example, I get a similar not found error, but for link
:
Local command returned non-zero exit code <no exit code>
Reproduce locally: `link "-fuse-ld=lld" /Brepro "/OUT:buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__maincpp__\\maincpp" "@buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__maincpp__\\maincpp.linker.argsfile"`
stdout:
stderr:
Spawning executable `link` failed: program not found
Build ID: 982cd683-cd62-49d7-809c-ac2c9d92fadb
Jobs completed: 10. Time elapsed: 0.4s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2)
BUILD FAILED
Failed to build 'root//:maincpp (prelude//platforms:default#fb50fd37ce946800)'
Rust is somehow able to find link
even though it isn't on my path by default.
~ ❯ link
link: The term 'link' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
Perhaps buck2
should copy how rust searches for the linker on windows (I couldn't find that code after searching for a few minutes), or the prelude should explicitly add the Visual Studio tools bin to its path.
from buck2.
Perhaps buck2 should copy how rust searches for the linker on windows
That sounds like a reasonable path forward.
(I couldn't find that code after searching for a few minutes),
I am not an expert, but I believe that is here:
(that is calling https://docs.rs/cc/latest/cc/windows_registry/fn.find_tool.html)
from buck2.
While the "change it to link
" hack did work for making a rust binary, it falls apart when trying to build a build script, ported over from a cargo project with reindeer. Which... is also a rust binary, so it's not immediately obvious to me what the difference is here.
Additionally, when building a different project (dtolnay/cxx
) that has a working Rust & C++ build on Linux, I get failures like the above, but also
clang++: error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'
So, I'm looking forward to @lmvasquezg's work mentioned over in #163 :)
from buck2.
It seems like at this point, it can find my clang++
, and #163 is a more general "MSVC issues" issue, so I'm going to close this one in favor of that one.
Thank you all so much for the help on this.
from buck2.
Related Issues (20)
- load starlark from target output HOT 7
- Option to include remote execution actions in chrome-trace
- Build source file owned by another BUILD file HOT 1
- Status of `compatible_with` and alternatives HOT 6
- How do I call a rule impl explicitly? HOT 1
- Per-PACKAGE transitions HOT 7
- Shadow prelude rules globally HOT 4
- compile_command.json write to root directory HOT 7
- Template variables for `cxx_genrule` do not work with `system_cxx_toolchain`
- Windows System Libraires HOT 4
- namespacing rules
- How to get an `artifact`'s parent directory? HOT 2
- Will Buck2 work with Sun Grid Engine for remote execution? HOT 3
- How to select cmake binary based on host platform instead of target platform?
- The "Getting Started" link is broken :-( HOT 1
- Print target when selects fail to match on toolchain targets
- Default target platform for 'buck2 run'
- Check performance of tset artifactvalues cache HOT 3
- Zsh tab completion error
- feature request - provide access to stdout for successful build actions
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from buck2.