Support musl libc about box64

[ptitSeb]

You need to use the ANDROID code path for everything malloc related.

[gamecss]

Thanks! now it don't stucks at box_malloc any more, but I encountered another problem, seem src/box64context.c:initAllHelpers isn't called, so init_pthread_helper and pthread_key_create not called too, then with a pthread_getspecific call, I get a SIGSEGV. However, I cannot search any refer to initAllHelpers. What should I do? Sorry. I missed collapsed content.
I moved initAllHelpers(context); below init_custommem_helper(context); to create pthread key before NewLibrarian. Is that ok?

[paulwratt]

@gamecss you only get notifications for posting not editing

However, I cannot search any refer to initAllHelpers. What should I do? Sorry. I missed collapsed content.
I moved initAllHelpers(context); below init_custommem_helper(context); to create pthread key before NewLibrarian. Is that ok?

EDIT:
also @gamecss do you have a list of symbols (names) for musl (ie compare them to glibc). My feeling of @ptitSeb response, is that Android is not a drop in replacement, so technically the only thing you need to do is match the calls, call names, symbol names, and since musl is a drop in replacement for glibc, you only have to make sure you are referencing the right library (or referencing the library in the right way)

NOTE: musl does change some naming conventions , ie I saw a musl patch yesterday that dropped "64" from one function name, which would cause an issue if applied, because glibc retains a 32bit code path, and hence needs both func and func64

Anyway, my point is, (especially based on @ptitSeb response) you should not need to do anything more than verify / change the references (and then test of coarse). ie most (but not all) musl call names are identical in glibc, ulibc, newlibc, dietlibc , etc, so most symbol names should be the same (which are the reference pointer to the actual function - "symbol table") programatically speaking

if you are just trying to run x86_64 musl compiled binaries on arm64/aarch64, it should be 1 for 1 identical, and you only have to verify / change the correct library name

[gamecss]

@ptitSeb Hello, developer! 😉 Is that ok?

[paulwratt]

@gamecss while you are waiting for a reply, you might want to re-read my post (sorry) i edited it :) (especially the last paragraph)

[paulwratt]

@gamecss I just read through the diff of your commits, and mostly it seems like you are on the right track, except in src/libtools/signals.c line 16 - that does not look right, and neither does line 1345/1348

same in src/libtools/threads.c line 82/84, also that initial // declare hidden functions does not look right either, should be more like the ones above .. = NULL;

EDIT: I say that last bit because the calls defined above yours and place holders for the real functions (hence they start with real_..).

[gamecss]

oops, they're typos
Should be #if !defined(ANDROID) && !defined(MUSL)

[paulwratt]

@gamecss overall I think you need to cleanup:
#if !defined(MUSL) && !defined(MUSL)
and change the use of:
#if !defined(ANDROID) && !defined(MUSL)
to
#ifndef (ANDROID || MUSL)

(yes they are functionally different, when not defined, as opposed to being defined and empty)

AND remember you are dealing with a library change not and OS change , so you probably shoud not be adding ifndef MUSL and ifdef MUSL at exactly the same places that the ANDROID path is. <= take this comment with a "grain of salt" , like I said, you seem to be on the right path, just keep cross-referencing docs & library sources

(Seb should reply in a couple of hours, its night time in Europe ATM, probably early morning)

EDIT: also how are you testing these changes, ie. start simple, don't try to get a full app working first off, if the changes allow printing to the console, then you are halfway there (and on the right path)

[paulwratt]

@gamecss does Musl really not have / use _pthread_unwind ? PThread is a seperate library, so it should be the same, and where these are unset (at lines 84,154, 402, 554, etc in src/libtools/threads.c) and some of them are Kernel calls if I am not mistaken.

[gamecss]

🤔 Seem functions like pthread_unwind_next only implemented in glibc

[ptitSeb]

ok, it really depend what you want to do:
If you want to run glibc x86_64 binary on musl aarch64, you need to code wrappers for missing glibc funciton in musl, because x86_64 program will use them.
If you want to run musl x86_64 program on musl aarch64, you can #ifdef out the missing functions (and you might need to add a couple of musl-exclusive ones).

Also, note that I have just changed a bit the "pthread cancel" handling. If you want to use glibc program, the current new method should work plug'n play, becasue it only use posix function now to emulated the glibc mecanism. If you want to run musl x86_64 program, you might have to create new wrapper to handle pthread_cleanup_push and pthread_cleanup_pop, as I assume those are actual function on musl and not macro like in glibc (that uses pthread_register_cancel, pthread_unregister_cancel and pthread_unwind_next).

[gamecss]

That's a weird thing.... In library_t *NewLibrary, through lib->type = LIB_UNNKNOW, still trigger SIGSEGV when trying print it as %s

[ptitSeb]

That's a weird thing.... In library_t *NewLibrary, through lib->type = LIB_UNNKNOW, still trigger SIGSEGV when trying print it as %s

I don't undersand what you are trying to do. LIB_UNKNOWN is an integer, it should not be printed as %s (or lib->type in fact), or you'll sure get a SEGFAULT there.

[gamecss]

oops sorry

[gamecss]

Thanks for your help! but I still have a little questions.

I moved initAllHelpers(context); below init_custommem_helper(context); to create pthread key before NewLibrarian.

Will this cause some problem?

.... as I assume those are actual function on musl and not macro like in glibc (that uses pthread_register_cancel, pthread_unregister_cancel and pthread_unwind_next).

In musl's pthread.h, pthread_cleanup_push(f, x) and pthread_cleanup_pop(r) are implemented as macro, too. But I couldn't found something like pthread_unwind_next and pthread_cond_clockwait. Should I just ignore them when compiling with MUSL?

Now I'm able to run some bascially command in busybox like ls and echo on sd625, but other commands often cause Error: Unsupported Syscall and crashes. Should I implement them?
(I couldn't run busybox ls though box64 on x86_64 platform. Got SIGSEGV after lots of symbol not found:

Error: Global Symbol ns_initparse not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5560 (0) in /bin/busybox
Error: Global Symbol res_mkquery not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5660 (0) in /bin/busybox
Error: Global Symbol ns_name_uncompress not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5670 (0) in /bin/busybox
Error: Global Symbol pivot_root not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5758 (0) in /bin/busybox
Error: Global Symbol ns_parserr not found, cannot apply R_X86_64_GLOB_DAT @0x1000c57a8 (0) in /bin/busybox
Error: Global Symbol ns_get32 not found, cannot apply R_X86_64_GLOB_DAT @0x1000c57e0 (0) in /bin/busybox
Error: Global Symbol ether_hostton not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5870 (0) in /bin/busybox
Error: Global Symbol swapoff not found, cannot apply R_X86_64_GLOB_DAT @0x1000c59e0 (0) in /bin/busybox
Error: Global Symbol crypt not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5b18 (0) in /bin/busybox
Error: Global Symbol swapon not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5ce0 (0) in /bin/busybox
Error: Global Symbol ether_aton_r not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5d20 (0) in /bin/busybox
Error: Global Symbol scanf not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5d40 (0) in /bin/busybox
Error: Global Symbol ns_get16 not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5e20 (0) in /bin/busybox
3464746|SIGSEGV @0x34e94d48 (???(./box64+0x34e94d48)) (x64pc=0x76322/???:"???", rsp=0x7f7fd8aa1c08), for accessing 0x76322 (code=1)

[gamecss]

Ok... I think it's really a disturb. giving up

[gamecss]

:( I'm so sorry

[ptitSeb]

sorry for what? there is no issue in trying things :)

[paulwratt]

TLDR; "no point in trying to beat Hussain Bolt in 100m dash, if you have not even learned to crawl yet, let alone walk"

Ok... I think it's really a disturb. giving up

Why give up so quick? Sometimes it can take weeks or even months to get things right, especially when you include testing.

But you should not be waiting on someone else before you proceed to the next step, especially taking into account timezone differences.

Like you said in OP, this is a learning experience for you. If you need someone to "hold your hand" while going through the initial stages (until you build your confidence up), then you should look to an online forum or discord, where there is someone "on call" (preferably in the same timezone ) to answer basic questions.

You already got further than most other people have, because no one has (publicly) tried to use Musl instead of Glibc with Box64.

You are "blazing a trail" that no one else has, and that in itself implies "being alone in the process".

So far you have not done anything I would not have tried, which implies you are on the right track, except waited for a reply when you are unsure about an approach (ie. " .. is this OK")

You can't get impatient with this process, otherwise you will fail. But here failure should be "exhausted all avenues", not "unsure if approach is correct" or "unsure of what to do next".

The only other thing I can say about your process, in relation to this repo and this "Issues" thread, (especially right before you make a new post in this thread) is to update your branch of your fork of Box64 on GitHub with your current changes, as much as possible, otherwise anyone else observing can not see exactly what you have done, or what you are trying to achieve. And verify your own changes by reading the diff (can more quickly catch potential problems)

Also if you feel the need to wait on some info (or a response) before proceeding down a particular line of action / approach, then create another branch where you can then pursue the "OK, they said No" approach, that way you don't have to wait, and you might discover some other insight, which means you can also go back to your original branch and try that theory out before getting an answer to your question.

I personally would like to see this completed, at least for command line tools, but ATM I am unable to assist / verify your approach, outside of reading the diff of your branch online via the GitHub web interface, and even then I only know "an overview level" of what you are trying to do, so I can not advise, only question and suggest .

If you do want to try "the hand holding approach", then I suggest you try asking on a Musl related forum somewhere, there is no doubt in my mind that someone who already knows Musl has also thought about Box64 too (even on RISCV possibly)

Apart from actually using Box64-Musl , I think this process will help others try Newlibc, uclibc, DietLibc .. etc. The other side of this exercise with Musl binaries on a Box64-Musl, is getting Glibc binaries working, and getting Musl binaries working on Box64 (which uses Glibc call translation). But everything in this paragraph depends on someone getting Musl working first ..

EDIT: On a final not, when you are ready to try a pull request or merge , this can make it easier to question, comment and fix your code, because of the integrated Issues + PR comment system on the GitHub web interface, also there is a way that easily allows your patch to be integrated into the current upstream repository (used for SerenityOS PR's - its a very specific "rebase" command).

[paulwratt]

Thanks for your help! but I still have a little questions.

I moved initAllHelpers(context); below init_custommem_helper(context); to create pthread key before NewLibrarian.

Will this cause some problem?

No need to wait for an answer tho this, proceed and test, you will know soon enough if it "Will this cause some problem?"

.... as I assume those are actual function on musl and not macro like in glibc (that uses pthread_register_cancel, pthread_unregister_cancel and pthread_unwind_next).

In musl's pthread.h, pthread_cleanup_push(f, x) and pthread_cleanup_pop(r) are implemented as macro, too. But I couldn't found something like pthread_unwind_next and pthread_cond_clockwait. Should I just ignore them when compiling with MUSL?

Again, no need to wait for an answer, simply proceed with ignoring them, until something comes up that "implies" you need to take another look at the issue.

Now I'm able to run some bascially command in busybox like ls and echo on sd625, but other commands often cause Error: Unsupported Syscall and crashes. Should I implement them? (I couldn't run busybox ls though box64 on x86_64 platform. Got SIGSEGV after lots of symbol not found:

Error: Global Symbol ns_initparse not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5560 (0) in /bin/busybox
Error: Global Symbol res_mkquery not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5660 (0) in /bin/busybox
Error: Global Symbol ns_name_uncompress not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5670 (0) in /bin/busybox
Error: Global Symbol pivot_root not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5758 (0) in /bin/busybox
Error: Global Symbol ns_parserr not found, cannot apply R_X86_64_GLOB_DAT @0x1000c57a8 (0) in /bin/busybox
Error: Global Symbol ns_get32 not found, cannot apply R_X86_64_GLOB_DAT @0x1000c57e0 (0) in /bin/busybox
Error: Global Symbol ether_hostton not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5870 (0) in /bin/busybox
Error: Global Symbol swapoff not found, cannot apply R_X86_64_GLOB_DAT @0x1000c59e0 (0) in /bin/busybox
Error: Global Symbol crypt not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5b18 (0) in /bin/busybox
Error: Global Symbol swapon not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5ce0 (0) in /bin/busybox
Error: Global Symbol ether_aton_r not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5d20 (0) in /bin/busybox
Error: Global Symbol scanf not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5d40 (0) in /bin/busybox
Error: Global Symbol ns_get16 not found, cannot apply R_X86_64_GLOB_DAT @0x1000c5e20 (0) in /bin/busybox
3464746|SIGSEGV @0x34e94d48 (???(./box64+0x34e94d48)) (x64pc=0x76322/???:"???", rsp=0x7f7fd8aa1c08), for accessing 0x76322 (code=1)

Are those not part of libBusyBox64.so ? (or what ever the linked library for BusyBox is called). Or are they part of Musl that are not in Glibc? (different name?)

Like I said earlier, "How are you testing your changes / fixes" - start simple. If you really want to use BusyBox on Alpine to start out, then make sure you compile BusyBox --with-standalone (or whatever switch compiles all BusyBox modules as stand alone binaries).

why do I say this: things like ether_aton_r, swapon, crypt and swapoff are not part of a normal ls commad process

you end up overloading yourself with required fixes that are totally unrelated to what you are testing, BUT , they are fixes you are going to have to make at some point.

Ok... I think it's really a disturb. giving up

this is not a practical solution, please fix this in /human/body/head or /human/mind/motivation

[gamecss]

I really thanks for your reply! About ether_aton_r, swapon, crypt and swapoff -- They're actually a part of musl, I can grep them in ld-musl-x86_64.so.1. About Error: Global Symbol error, I can think out only three reasons:

• These symbol doesn't linked, so box64 cannot find them
• Missing some EXPORT sth my_something function
• Missing some GO(sth, ......)

For the first possible reason, -lc linking options is already applied. But for the second and third possible reason, I can't do anything useful.

• What does the second paramater of GOs like GO(...., pFp_t) means?
• When and how should I write them?
  And is the really reason in these reasons?

These are lots of problems. I can't find documents about how to do. Of course, I'm able to test if they're works, but first I need to write some code. That's why I cannot do anything and have to give up. These should be done by someone knows a lot about musl & box64 & c, but not me. I prefer giving up compared to disturbing more and more people. However, thank you very much for your help! :)
(And sorry for my poor English, too)