Experimental branch fails to build with LibcWrapGenerator about appimagekit

[probonopd]

make AppRun works, make runrime fails.

[RazZziel]

memcpy@GLIBC_DONT_USE_THIS_VERSION_2.14 = problem with LibcWrapGenerator, try with cmake . -DUSE_LIBC_WRAP_GENERATOR=OFF

[probonopd]

Yes, I can confirm that. Also see #49

[probonopd]

Do you know how to pass in -U_FORTIFY_SOURCE?

[RazZziel]

Already doing it LibcWrapGenerator/cc (as you already noticed in the email)

Is memcpy a "too new" symbol?

Shouldn't be, but this error was already happening in Tristan's blog comments and in theory it was a fixed bug, but it seems to persist even though we seem to have the latest version of LibcWrapGenerator.vala

[RazZziel]

#33 (comment):

I've disabled LibcWrapGenerator by default in d36f076

[probonopd]

Well I think we can use it if we set --target 2.14. Or do you have some other rationale for entirely abandoning it?

[RazZziel]

Should --target 2.14 work better than the current --target 2.7?

Anyway, we're not abandoing it entirely, it's just set off by default; and my only rationale for doing so right now is that it's breaking CI in the experimental branch, so it's blocking any other change (like 3eae15e), and any further experiment should be moved to a feature branch until it's proven stable

[probonopd]

Yes, it allows symbols that were introduced between 2.7 and 2.14 to be used. Works for head.

[RazZziel]

Makes sense, will try it on a new branch. If it works, should I enable by default again? Do you think it's stable enough?

[RazZziel]

Indeed, compiling against 2.14 works but against 2.7 doesn't

I finally understood the problem checking LibcWrapGenerator's README, and I'm not sure using glibc 2.14 is desirable:

Now, in glibc 2.14, it was decided that many programmers used memcpy() in
cases where they actually desired the behaviour of memmove(), or their
code was found to be unsafe, so in the interest of creating a more stable glibc,
they have provided us with a new implementation of memcpy() which does that.
That new symbol is named 'memcpy@GLIBC_2.14' in the new glibc ABI.

If you update your system's glibc, or distribute that very same binary
on a system with glibc 2.16, glibc will still contain the old version of
memcpy() bound to the 'memcpy@GLIBC_2.2.5' symbol in it's lookup table.

And there you have forward binary compatibility for your program.

But what we want is the very opposite of the above. We want to use a modern
GNU toolchain on a modern system with a modern glibc, but we want to have
our programs backwards compatible for much older versions of glibc.

So if I understand correctly, doesn't forcing glibc 2.14 destroy the very purpose of using LibcWrapGenerator? See for instance https://blogs.gnome.org/tvb/2013/12/14/application-bundles-revisited/comment-page-1/#comment-3831:

The memcpy function should not fall under the DONT_USE_THIS_VERSION category but rather be directed to 2.2.5, so if you are targeting any libc version > 2.2.5 then this shouldn’t happen (but as I only have my own glibc to test against, I can’t be sure yet that it’s parsing objdump output properly).

[probonopd]

The way I read it, with the current code we have and with LibcWrapGenerator we can at least ensure that no later than 2.14 must be installed, so yes I still think it's worthwhile. It will also warn us if future code tries to use functionality that is not supported in 2.14. That alone makes it worthwhile imho.

[RazZziel]

Indeed 2.14 should be old enough, but I'm afraid the problem is LibcWrapGenerator is not working correctly, and may fail again in a future:

• My Archlinux is using glibc 2.22, but is able to target glibc 2.7, using memcpy() from 2.2.5.
• Travis VM is using glibc 2.16, and for some reason is not able to target glibc 2.7, and instead must target the glibc version with the next version of memcpy(), which is 2.14.
• Same happens on Fedora 23

This means, if memcpy() changes again in 2.23, LibcWrapGenerator will fail again on us when targeting glibc 2.14 on some machines with glibc 2.23 or higher installed, regardless of whether glibc 2.14 supports the features we're using or not, just like it's doing right now when targeting glibc 2.7 on Travis or Fedora.

I mean, for now we can take advantage of these two version of glibc LibcWrapGenerator is taking us back (2.16 -> 2.14), but it's not really working on some machines, so it's not stable tech we can rely on, as it will eventually break the build on some machines unless these errors are tracked and fixed (if possible).

Tristan (creator of LibcWrapGenerator) once told me:

Ultimately though, this is not well supported by the maintainers of glibc, a much more stable approach to this is to just compile against the oldest version of glibc you intend to support - however this of course also has it's own complications (it's damn tricky to parallel install glibc and have your compiler use the alternative glibc).

[probonopd]

Oh wow. That stuff is really tricky... didn't know that. Do the same restrictions apply for https://github.com/ximion/limba/blob/master/contrib/licompile/ligcc? If I understand it correctly, it is following a very similar idea as LibcWrapGenerator.

[RazZziel]

No idea, but maybe we're facing a LibcWrapGenerator bug that the libgcc guys have figured out

[TheAssassin]

Is this really old issue still relevant? I don't think anyone uses LibcWrapGenerator any more.

[probonopd]

This was regarding the old master branch that has been stale for quite some time now, so closing. Nevertheless, LibcWrapGenerator can be be a solution in some cases if you need to build on a relatively new build system and want to target older systems. But using an old build system is a more robust solution, and is what we are using now.