compiling without -fPIC fails on NixOS about git2-rs

[alexcrichton]

This may be a bug in libgit2's build system? It looks like this is building a bundled copy of libz and maybe the flags aren't being propagated down?

We should definitely pass -fPIC for everything other than 32-bit.

[carlosmn]

When the libgit2 build system decides to use the bundled libz, it does so by adding the libz source files to the overall list of source files so the same flags would apply for the libz files as they would for the rest of the library.

[alexcrichton]

@globin can you gist the full output of the build script? It should be located at target/debug/build/libgit2-sys-..../output

[twhitehead]

Ran into what I believe is the same thing building nixpkgs.rustPlatform against a slightly modified (nothing to do with rust) version of the 15.09 tag of nixpkgs.

I've attached the requested libgit2-sys.../output. I believe the key lines are

running: "cmake" ... -m64 -fPIC"
...
-- Found ZLIB: /home/nixbld/store/q1jlhbar12m5dr6fmwvpk7krsp9rz522-zlib-1.2.8/lib/libz.so (found version "1.2.8")
...
cargo:rustc-link-lib=static=z

From what I could determine it seems this means that

• CMake is correctly compiling everything with -fPIC
• CMake is using the dyanmic zlib library provided by the Nix zlib package
• Cargo is using the static zlib library provided by the Nix zlib package

Comparing the r-z-deflate.o object in liblibgit2_sys... with the deflate.o object in zlib.a confirms that they are the same (i.e., Cargo is taking the contents of zlib.a and sticking it in liblibgit2_sys... prefixed with r-z-).

While I do not understand the intent of this, I believe it is being triggered by the static bit in the cargo:rustc-link-lib=static=z line. This is output by the libgit2-sys-0.2.17/build.rs file. Specifically the last few lines of the main routine which read

if env::var("HOST") == env::var("TARGET") {
  prepend("PKG_CONFIG_PATH", dst.join("lib/pkgconfig"));
  if pkg_config::Config::new().statik(true).find("libgit2").is_ok() {
    return
  }
}

From the pkg_config documentation it seems this is being automatically generated from the output the pkgconfig program. Running it manually I get the following

$ PKG_CONFIG_PATH=$(pwd)/lib/pkgconfig:$PKG_CONFIG_PATH pkgconfig --static --libs --cflags libgit2
-I/tmp/cargo/cargo/target/x86_64-unknown-linux-gnu/release/build/libgit2-sys-93cd2089ab768807/out/include -I/home/nixbld/store/q1jlhbar12m5dr6fmwvpk7krsp9rz522-zlib-1.2.8/include -I/home/nixbld/store/zw3asr0y63qj5z4626xkc4959dxp6zrq-openssl-1.0.1p/include -I/home/nixbld/store/xv1i53hll6fidv3fkzab6f3nrqvkmrh8-libssh2-1.6.0/include -L/tmp/cargo/cargo/target/x86_64-unknown-linux-gnu/release/build/libgit2-sys-93cd2089ab768807/out/lib -L/home/nixbld/store/q1jlhbar12m5dr6fmwvpk7krsp9rz522-zlib-1.2.8/lib -L/home/nixbld/store/zw3asr0y63qj5z4626xkc4959dxp6zrq-openssl-1.0.1p/lib -L/home/nixbld/store/xv1i53hll6fidv3fkzab6f3nrqvkmrh8-libssh2-1.6.0/lib -lrt -lgit2 -lssl -lcrypto -ldl -lssh2 -lssl -lcrypto -ldl -lcrypto -ldl -lz 

Looking through the pkg-config source it seems the static attribute gets attached to any library where a static version exists (i.e., a .a file) that is not rooted under /usr

    "-l" => {
        self.libs.push(val.to_string());
        if statik && !is_system(val, &dirs) {
            println!("cargo:rustc-link-lib=static={}", val);
        } else {
            println!("cargo:rustc-link-lib={}", val);
        }
    }

fn is_system(name: &str, dirs: &[PathBuf]) -> bool {
    let libname = format!("lib{}.a", name);
    let root = Path::new("/usr");
    !dirs.iter().any(|d| {
        !d.starts_with(root) && fs::metadata(&d.join(&libname)).is_ok()
    })
)

Cheers! -Tyson

[twhitehead]

By the way, here is the directory list from the zlib package under Nix as well. You can see how it has both the static (which Cargo is picking up on) and the dynamic (which libgit2 is picking up on) library.

$ ls -l /home/nixbld/store/q1jlhbar12m5dr6fmwvpk7krsp9rz522-zlib-1.2.8/lib/
total 252
-r--r--r-- 1 root nixbld 141398 Dec 31  1969 libz.a
lrwxrwxrwx 1 root nixbld     13 Dec 31  1969 libz.so -> libz.so.1.2.8
lrwxrwxrwx 1 root nixbld     13 Dec 31  1969 libz.so.1 -> libz.so.1.2.8
-r-xr-xr-x 1 root nixbld 107880 Dec 31  1969 libz.so.1.2.8
dr-xr-xr-x 2 root nixbld   4096 Dec 31  1969 pkgconfig

[alexcrichton]

Ah yes, it sounds like libz.a is being used from the system which isn't compiled with -fPIC, but this should instead use the local version instead.

I think, however, that this library should probably not get compiled against the system copy of libgit2 (e.g. pkg-config shouldn't be used to find libgit2 on nixos) because the system copy is almost guaranteed to be incompatible with the bundled one.

[twhitehead]

I'm not sure about globin's setup, but I don't believe mine is compiling against a system libgit2. Or at least that is my take from the fact that the cargo build procedure spits out the following line

Compiling libssh2-sys v0.1.25 (registry file:///dev/null)

and the associated target/debug/build/libgit2-sys-..../output file indicates that a bundled libgit2 is being built.

[twhitehead]

Turns out nix has a patch for the hard coded /usr in the pkg-config source. It was hard-coded to replace it with the default /nix/store though so it didn't work in my case as I'm not using the default. I've submitted an upstream pull request to use the value of $NIX_STORE instead. This resolves the issue for me.

[alexcrichton]

Oh right, pkg-config is used later on to learn what the dependencies of libgit2 are. I... suspect this crate should stop doing that. If we just encode them all manually I believe this shouldn't happen.

[alexcrichton]

Ok, I pushed alexcrichton@4d2b771 which I think should take care of this, thanks for the info @twhitehead!