mbachry / exxo Goto Github PK

when I build exxo and has error:
config.status: executing buildir commands
config.status: skipping top_srcdir/Makefile - not created
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing include commands
config.status: executing src commands
Traceback (most recent call last):
File "/usr/local/lib/python3.4/runpy.py", line 170, in _run_module_as_main
"main", mod_spec)
File "/usr/local/lib/python3.4/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/exxo_build/exxo-master-0125/exxo/bootstrap.py", line 229, in
main()
File "/home/exxo_build/exxo-master-0125/exxo/bootstrap.py", line 225, in main
b.bootstrap()
File "/home/exxo_build/exxo-master-0125/exxo/bootstrap.py", line 206, in bootstrap
self.install_pyrun()
File "/home/exxo_build/exxo-master-0125/exxo/bootstrap.py", line 184, in install_pyrun
self.render_setup_file()
File "/home/exxo_build/exxo-master-0125/exxo/bootstrap.py", line 153, in render_setup_file
with tmpl.open() as fp:
File "/usr/local/lib/python3.4/pathlib.py", line 1073, in open
opener=self._opener)
File "/usr/local/lib/python3.4/pathlib.py", line 947, in _opener
return self._accessor.open(self, flags, mode)
File "/usr/local/lib/python3.4/pathlib.py", line 326, in wrapped
return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: '/home/exxo_build/exxo-master-0125/build/egenix-pyrun-2.1.1/PyRun/Runtime/Setup.PyRun-2.7.tmpl'

and i change bootstrap.py
def render_setup_file(self):
fn = 'Setup.PyRun-{}'.format(self.python_major_version)
- tmpl = self.pyrun_dir / 'Runtime' / (fn + '.tmpl')
+ tmpl = self.pyrun_dir / 'Runtime' / (fn)

is this bug?

Hi, what's the reason you chose the Maven convention over the Python one? I like my project roots as tidy as possible, and yet another directory is an eye-sore. 😉

As I see in patch in function import_solib you extract extension shared object file and load it, however it's insufficient if module is linked with bundled libraries. This means that multiple manylinux1 wheels are not supported, for example you may look at numpy - there extension modules have RPATH pointing to package root .libs directory:

 $ readelf -a numpy/core/multiarray.so | grep RPATH
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../.libs]

And after extraction that module will not find required library at RPATH and will look at system-wide libraries, where it may not find what it wants.

If I'm wrong and exxo handles this issue, please correct me.

One solution may be extracting all .so files keeping their relative structure, another - recursively determine all dependencies and extract only them. Or you may extract them to RUNPATH directory of pyrun.

Also I'd like to suggest creating import hook in Python and adding it to sys.meta_path instead of patching zipimport.

Download Exxo is self-hosting. You can download it [here](https://bintray.com/artifact/download/mbachry/exxo/exxo-0.0.7.tar.xz). The archive contains just one file: exxo binary you should put somewhere in your PATH.
The linked tar.gz is not reachable.

Hello guys!

This project is cool!

The unicode of python2.7 is ucs2, and how to change it to ucs4?

I change PYTHON_VERSION_MAP in bootstrap.py, and has error:

make: *** No rule to make target '/tmp/exxo-master/build/egenix-pyrun-2.1.1/PyRun/Python-2.7.11-ucs4'. Stop.
Traceback (most recent call last):
File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/exxo-master/exxo/bootstrap.py", line 227, in
main()
File "/tmp/exxo-master/exxo/bootstrap.py", line 223, in main
b.bootstrap()
File "/tmp/exxo-master/exxo/bootstrap.py", line 204, in bootstrap
self.install_pyrun()
File "/tmp/exxo-master/exxo/bootstrap.py", line 170, in install_pyrun
self.pyrun_make(str(python_dir))
File "/tmp/exxo-master/exxo/bootstrap.py", line 115, in pyrun_make
cwd=str(self.pyrun_dir))
File "/usr/lib/python3.4/subprocess.py", line 561, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j1', '/tmp/exxo-master/build/egenix-pyrun-2.1.1/PyRun/Python-2.7.11-ucs4', 'PYTHONFULLVERSION=2.7.11']' returned non-zero exit status 2
root@IBM:/tmp/exxo-master#

This piqued my interest, and I was about to start trying it out, but then I noticed you haven't written any unit tests yet! I know you just started this recently, but if you're at all interested in getting support/code contributions, and adoption, you must add some test coverage!

I'm excited to see where you take this project. Seems like a good approach as an alternative to PEX.

This issue is more informational than bug report or feature request because I don't really know how to automatically detect such situation and I don't want to suggest building libraries ourselves from scratch.

To build pyrun linked statically against sqlite3 with icu support following changes were needed:

1. add libraries from icu-config --ldflags (-licui18n, -licuuc, -licudata) to _sqlite3 description in Runtime/Setup.PyRun-2.7
2. add -static-libstdc++ -static-libgcc to CXX variable in Makefile.pre and change LINKCC from CC to CXX - this is required because icu libraries are written in c++ and require its standard library, so linker has to know how to link statically against libstdc++. However if dynamic linking to libstdc++ is acceptable this paragraph is not needed and it's sufficient to add -lstdc++ to changes in first item.

I've noticed that this issue looks like blog post, sorry for abusing your tracker.

First, there is no need to link against termcap - readline and curses modules do not need it at linux, and this fails PyRun build on systems where libtermcap is not present. See exxo/patches/pyrun.diff where termcap is mentioned. And i see that you disabled termcap in ncurses setup. And it's required to link against libtinfo or there will be lots of undefined symbols during PyRun build.

Second, it's better to build ncurses with unicode support - --enable-widec and link against ncursesw. Also I'd like to ask why do you build your own ncurses and do not rely on system-wide like you do with readline, openssl, zlib, bzip2?

What differentiates exxo from pex? It is not clear from the README.

Just tried this out by trying to bundle crash.

Crash has the src in a separate src directory. And it's setup.py contains

package_dir={'': 'src'},
packages=find_packages('src'),

If python setup.py egg_info is executed the .egg-info file isn't in the same directory as setup.py but in src/<project>.egg-info

Anyway. The workaround with --main worked good enough. Cool stuff.

Once you settled the zip-safe challenges etc., it'd be nice if exxo could add a packaging envelope via an option or configuration value.

Rationale:

• Solves the "external shared object" dependency problem
• Since the package contents is so simple, automated packaging is too, and saves writing boring packaging tool config files (metadata is readily available from setup.py)

If you come around to this, I'd be willing to help with Debian (DEB/APT).

My use case involves a highly customized system that requires any program to be compiled with test -Wl,--dynamic-linker=/lib/ld-musl-x86_64.so.1 or it won't properly execute. (OpenWRT with BusyBox). In that case, may I know where I can add this flag in the exxo bootstrapping/build process?

For your reference, a binary compiled with Exxo would generate a "not found" error under BusyBox, and the ldd outputs shows this:

root@XNCB3B95FF859068:~/senpai# ldd SteelSenpai
        /lib64/ld-linux-x86-64.so.2 (0x55bcb8e11000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x55bcb8e11000)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x55bcb8e11000)
        libutil.so.1 => /lib64/ld-linux-x86-64.so.2 (0x55bcb8e11000)
        libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x55bcb8e11000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x55bcb8e11000)
Error relocating SteelSenpai: xdr_pointer: symbol not found
Error relocating SteelSenpai: __dcgettext: symbol not found
Error relocating SteelSenpai: clnt_perror: symbol not found
Error relocating SteelSenpai: clnt_pcreateerror: symbol not found
Error relocating SteelSenpai: __strncpy_chk: symbol not found
Error relocating SteelSenpai: __longjmp_chk: symbol not found
Error relocating SteelSenpai: __fprintf_chk: symbol not found
Error relocating SteelSenpai: xdr_bool: symbol not found
Error relocating SteelSenpai: __finite: symbol not found
Error relocating SteelSenpai: __printf_chk: symbol not found
Error relocating SteelSenpai: __rawmemchr: symbol not found
Error relocating SteelSenpai: __memcpy_chk: symbol not found
Error relocating SteelSenpai: __isinff: symbol not found
Error relocating SteelSenpai: xdr_opaque: symbol not found
Error relocating SteelSenpai: __vsnprintf_chk: symbol not found
Error relocating SteelSenpai: xdr_u_int: symbol not found
Error relocating SteelSenpai: xdr_bytes: symbol not found
Error relocating SteelSenpai: tmpnam_r: symbol not found
Error relocating SteelSenpai: __strcat_chk: symbol not found
Error relocating SteelSenpai: clnt_spcreateerror: symbol not found
Error relocating SteelSenpai: __vfprintf_chk: symbol not found
Error relocating SteelSenpai: __isnan: symbol not found
Error relocating SteelSenpai: clnttcp_create: symbol not found
Error relocating SteelSenpai: __isinf: symbol not found
Error relocating SteelSenpai: authdes_create: symbol not found
Error relocating SteelSenpai: __strcpy_chk: symbol not found
Error relocating SteelSenpai: xdr_enum: symbol not found
Error relocating SteelSenpai: __syslog_chk: symbol not found
Error relocating SteelSenpai: __open64_2: symbol not found
Error relocating SteelSenpai: __memset_chk: symbol not found
Error relocating SteelSenpai: __fread_chk: symbol not found
Error relocating SteelSenpai: __libc_clntudp_bufcreate: symbol not found
Error relocating SteelSenpai: authunix_create_default: symbol not found
Error relocating SteelSenpai: xdr_free: symbol not found
Error relocating SteelSenpai: xdr_string: symbol not found
Error relocating SteelSenpai: __sprintf_chk: symbol not found
Error relocating SteelSenpai: __realpath_chk: symbol not found
Error relocating SteelSenpai: __snprintf_chk: symbol not found
Error relocating SteelSenpai: clnt_create: symbol not found
Error relocating SteelSenpai: __memmove_chk: symbol not found
Error relocating SteelSenpai: host2netname: symbol not found
Error relocating SteelSenpai: __mbrlen: symbol not found
Error relocating SteelSenpai: _libc_intl_domainname: symbol not found