matthewbauer / nix-bundle Goto Github PK

View Code? Open in Web Editor NEW

627.0 627.0 47.0 1.42 MB

Bundle Nix derivations to run anywhere!

License: MIT License

Nix 31.61% Shell 24.81% Makefile 0.91% C 21.30% C++ 21.37%

nix

nix-bundle's People

Contributors

Stargazers

Watchers

nix-bundle's Issues

Unwanted output when running ./hello

I did the example from the front-page, but it printed some unwanted noise:

$ ./hello 
/tmp/tmpx-2017.04.26T09.58.27Z-c53661af/dat
Hello, world!

Faster startup by prelinking

prelink is a program that modifies ELF shared libraries and ELF dynamically linked binaries in such a way that the time needed for the dynamic linker to perform relocations at startup significantly decreases. Due to fewer relocations, the run-time memory consumption decreases as well (especially the number of unshareable pages). The prelinking information is only used at startup time if none of the dependent libraries have changed since prelinking; otherwise programs are relocated normally

According to the mailing list, this was not enabled in nixpkgs due to determinacy issues. That may not be an issue for bundles?

Here's also some discussion why it's not enabled by default in some distros: https://pagure.io/fesco/issue/1183, but again, those issues may not apply to nix-bundle?

Will need testing and benchmarking to see if it's worth it.

No way to pass command line arguments?

I tried to "package" mysql client with:

$ ./nix-bundle.sh mysql55 /bin/mysql

however it seems the resulting bundle doesn't pass through command line arguments.

mysql --version

Expected: mysql client prints its version number.
Actual: mysql client tries to connect to the default server.

crazy idea: bundled nixos installer

Some time ago I made an kexec-based installer image: https://gist.github.com/Mic92/4fdf9a55131a7452f97003f445294f97
Combined with nix-bundle an installer executable could be build, which can be executed on
every Linux, given enough RAM.

$ scp ./nixos-installer root@host:/tmp/nixos-installer
$ ssh root@host:/tmp/nixos-installer
# ... wait for reboot
$ ssh root@host
nixos>

I just leave this idea here as I have currently other projects, in case somebody wants to pick it up.

Not working on Centos 6.5

I am trying to build latest possible curl version using nix-bundle for centos 6.5.
The build was done on a Ubuntu 18.0 VM running on Windows. It works pretty fine where it was build, here are the outputs

rakesh@rakesh-VirtualBox:~/src/github.com/nix-bundle$ md5sum ./curl 
4eb3f636f1617114f9df298f761c7bc6  ./curl
rakesh@rakesh-VirtualBox:~/src/github.com/nix-bundle$ ./curl --version
curl 7.64.0 (x86_64-pc-linux-gnu) libcurl/7.64.0 OpenSSL/1.0.2r zlib/1.2.11 libssh2/1.8.0 nghttp2/1.36.0
Release-Date: 2019-02-06
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy 
rakesh@rakesh-VirtualBox:~/src/github.com/nix-bundle$ 
rakesh@rakesh-VirtualBox:~/src/github.com/nix-bundle$ ldd ./curl
    not a dynamic executable
rakesh@rakesh-VirtualBox:~/src/github.com/nix-bundle$

however when I use the same binary in a Centos machine running on aws, it doesn't work.
The outputs on Centos is as below:

root@ip-10-xxx-8-xxx tmp]# md5sum ./curl
4eb3f636f1617114f9df298f761c7bc6  ./curl
[root@ip-10-xxx-8-xxx tmp]# ./curl --version
unshare(): Invalid argument
[root@ip-10-xxx-8-xxx tmp]# 
[root@ip-10-xxx-8-xxx tmp]# ldd ./curl
    not a dynamic executable
[root@ip-10-xxx-8-xxx tmp]#

so the question is what exactly went wrong. Please let me know for any additional logs.

sha256 hash of appimagetool fails

Since you are pulling the continuous branch, the sha256 hash constantly changes, resulting in this failure:

trying https://github.com/probonopd/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   609    0   609    0     0   1121      0 --:--:-- --:--:-- --:--:--  1208
100  429k  100  429k    0     0   255k      0  0:00:01  0:00:01 --:--:--  636k
output path ‘/nix/store/d07pz2vzi2jk8s0g5ggrpnskx2lalgdq-appimagetool-x86_64.AppImage’ has sha256 hash ‘1irvbf0xnya16cyzpvr43jviq5ly3wl7b9753rji7d1hhxwb7b9r’ when ‘0l3hxp169dpyj3h38q9nsnh1cynam1j5zx8q362p93448rhm7d0y’ was expected
cannot build derivation ‘/nix/store/vpwsgskawi3x1s8ybaz6mw8760lqbbz0-appimagekit.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/n9p8plxfsgvd92rkg5npgaa2xfmibq4f-appimage.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/n9p8plxfsgvd92rkg5npgaa2xfmibq4f-appimage.drv’ failed
./nix2appimage.sh failed. Exiting.

Can the hash check be skipped?

Provide a small(er than Firefox) graphical app as an example

Firefox is a big beast (and potentially not the safest to run, see #53), I thought I would find a smaller package to demonstrate graphical apps.

Here are some patches to apply to the README (I'm sorry I was too lazy to setup a whole fork and RSA keys, etc. I hope you don't mind.)

I also found a game to showcase SDL compat as well. (As a wannabe indie game developer who doesn't want to bother with packaging for every Linux distro out there, I'm very interested in SDL support. I thought others might be too.)

Cannot run AppImages on NixOS

I can run arbitrary AppImages on NixOS with appimage-run tool. But appimages generated by nix-bundle fail:

$ ./nix2appimage.sh emacs
AppImage created at Emacs-x86_64.AppImage
$ appimage-run ./Emacs-x86.64.AppImage
unshare(): Operation not permitted

Even though kernel.unpriveleged_userns_apparmor_policy = 1

Update README.md

Unfinished from #48, the current interface should be documented.

nix-user-chroot Error

Hi, I would like to package a Nix app (seth) to a non-NixOs docker image (i.e alpine) with nix-bundle.sh.

Here's my Dockerfile:

FROM nixos/nix as buildSeth

RUN adduser -D sethservice && \
    chown -R sethservice /nix && \
    mkdir -p /etc/nix && echo 'sandbox = false' > /etc/nix/nix.conf && \
    apk --no-cache add ca-certificates curl git

USER sethservice
RUN curl https://nixos.org/nix/install | sh && source $HOME/.nix-profile/etc/profile.d/nix.sh && \
    echo "export PATH=$PATH:$HOME/.nix-profile/bin" >> $HOME/.profile && \
    source $HOME/.profile && \
    git clone --recursive https://github.com/dapphub/dapptools $HOME/.dapp/dapptools && \
    nix-env -f $HOME/.dapp/dapptools -iA seth

# Check it works
RUN source $HOME/.profile && seth help

RUN source $HOME/.profile && \
    git clone --branch v0.3.0 https://github.com/matthewbauer/nix-bundle $HOME/nix-bundle && \
    ./home/sethservice/nix-bundle/nix-bundle.sh /home/sethservice/.nix-profile/bin/seth /home/sethservice/seth

nix-bundle.sh seems to start copying the derivation paths but there is an error:

building '/nix/store/jyfzqbhik2riby6i5mp016fsljdj39yb-nix-user-chroot-2c52b5f.drv'...
unpacking sources
unpacking source archive /nix/store/yhvhnlv9l52vdb7i18q0hpkniyc4r35i-nix-user-chroot
source root is nix-user-chroot
patching sources
configuring
no configure script, doing nothing
building
build flags: SHELL=/nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash
g++ -o nix-user-chroot -DNIX_PATH='"nixpkgs=https://github.com/NixOS/nixpkgs/archive/master.tar.gz"' -DNIX_SSL_CERT_FILE='/no-cert-file.crt' -DENV_HOME='' -DENV_PATH='""' main.cpp
main.cpp: In function 'int main(int, char**)':
<command-line>: error: expected primary-expression before '/' token
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: error: 'no' was not declared in this scope
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: error: 'cert' was not declared in this scope
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: note: suggested alternative: 'creat'
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: error: 'file' was not declared in this scope
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: note: suggested alternative: 'time'
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
main.cpp:202:14: warning: ignoring return value of 'ssize_t write(int, const void*, size_t)', declared with attribute warn_unused_result [-Wunused-result]
         write(fd_setgroups, "deny", 4);
         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
main.cpp:217:10: warning: ignoring return value of 'int chdir(const char*)', declared with attribute warn_unused_result [-Wunused-result]
     chdir("/");
     ~~~~~^~~~~
make: *** [Makefile:6: nix-user-chroot] Error 1
builder for '/nix/store/jyfzqbhik2riby6i5mp016fsljdj39yb-nix-user-chroot-2c52b5f.drv' failed with exit code 2
cannot build derivation '/nix/store/gbxmdyrxvqfn9m4mnsyd5hn0c0izsspb-startup.drv': 1 dependencies couldn't be built
copying path '/nix/store/4251hlmqm101gkvhc4a0n7n8baifv4hq-scientific-0.3.6.2' from 'https://cache.nixos.org'...
cannot build derivation '/nix/store/yd3jb4pcpq7hncz76nywkd43rq24p2b5-arx.drv': 1 dependencies couldn't be built
error: build of '/nix/store/yd3jb4pcpq7hncz76nywkd43rq24p2b5-arx.drv' failed
./home/sethservice/nix-bundle/nix-bundle.sh failed. Exiting.

Do you have any idea where it may come from ?
I am using latest nixos/nix with nix-bundle version v0.3.0.

Thank you

Make CLI similar to nix build

I think that the command line interface for nix-bundle should be similar to nix build. That would improve learnability and usability. For reference, this is the output of nix build --help:

$ nix build --help
Usage: nix build <FLAGS>... <INSTALLABLES>...

Summary: build a derivation or fetch a store path.

Flags:
      --arg <NAME> <EXPR>       argument to be passed to Nix functions
      --argstr <NAME> <STRING>  string-valued argument to be passed to Nix functions
      --dry-run                 show what this command would do without doing it
  -f, --file <FILE>             evaluate FILE rather than the default
  -I, --include <PATH>          add a path to the list of locations used to look up <...> file names
      --no-link                 do not create a symlink to the build result
  -o, --out-link <PATH>         path of the symlink to the build result

Examples:

  To build and run GNU Hello from NixOS 17.03:
  $ nix build -f channel:nixos-17.03 hello; ./result/bin/hello

  To build the build.x86_64-linux attribute from release.nix:
  $ nix build -f release.nix build.x86_64-linux

Note: this program is EXPERIMENTAL and subject to change.

Of course there would need to be an additional option, for example --exec, to specify the executable. $ nix-bundle hello /bin/hello would then become:

$ nix-bundle -f '<nixpkgs>' hello --exec /bin/hello`

-f defaults to ./default.nix, so if one has a project with a default.nix, one can simply bundle it via:

$ nix-bundle --exec /bin/executablename

Reasonable options to support are:

--arg <NAME> <EXPR>
--argstr <NAME> <STRING>
-f, --file <FILE>
-I, --include <PATH>
A new option --exec as explained above. One could allow to omit this if there's only one executable in the built derivation.

I assume that most of these options can be supported simply by passing them on to the Nix tools used inside the nix-bundle script.

... is not allowed to refer to a store path

On current master (c5f2dfc), nix-bundle seems to be broken.

./nix-bundle.sh hello /bin/hello
errors out with:

the string '/nix/store/9pqfirjppd91mzhkgh8xnn66iwh53zk2-hello-2.10' is not allowed to refer to a store path (such as '!out!/nix/store/pyb210y8vqvd6maifaja58a3bl19wazs-hello-2.10.drv')

I assume this due to this change:#67
I didn't debug it any further.

Access to external files

I am trying to access external files from a bundled executable, and I am getting "File not found" errors. As I understand, the executable is in a virtual file system, so that it is understandable it not finding the files. Is there any solution for this?

Thanks!

Getting execvp errors when running binaries on the same machine

$ git clone https://github.com/matthewbauer/nix-bundle.git
$ cd nix-bundle
$ ./nix-bundle.sh -f channel:nixos-19.09 hello --bin /bin/hello
[1 copied (0.2 MiB), 13.3 MiB DL]
(with import ./default.nix {}; nix-bootstrap { target = "/nix/store/4w99qz14nsahk0s798a5rw5l7qk1zwwf-hello-2.10"; extraTargets = [ ]; run = "/bin/hello"; })
these derivations will be built:
  /nix/store/ix3s9xy4lb79g3h21m48xhg8z93kj251-startup.drv
  /nix/store/r30gwpi80v5p74pr82xwglckfmsrb3wd-maketar.drv
  /nix/store/qjb83jrdckr9yrm2xzjv5w5a7y1i7ydf-arx.drv
building '/nix/store/ix3s9xy4lb79g3h21m48xhg8z93kj251-startup.drv'...
building '/nix/store/r30gwpi80v5p74pr82xwglckfmsrb3wd-maketar.drv'...
tar: Removing leading `/' from member names
building '/nix/store/qjb83jrdckr9yrm2xzjv5w5a7y1i7ydf-arx.drv'...
Nix bundle created at hello.
$ ./hello 
execvp(/nix/store/4w99qz14nsahk0s798a5rw5l7qk1zwwf-hello-2.10/bin/hello): No such file or directory

Running on NixOS 19.09.1320.4ad6f1404a8.

Am I doing something wrong? README is somewhat outdated.

Generated AppImage has strange name

The Firefox AppImage gets generated in a strange place:

me@host:~/nix-bundle-master$ ls -lh /nix/store/jbwdybyyfs4y4yzdivwa205s82nhbzfv-appimage.AppImage 
-r-xr-xr-x 1 me me 127M Jan  1  1970 /nix/store/jbwdybyyfs4y4yzdivwa205s82nhbzfv-appimage.AppImage

That is a strange name.

If appimagetool would only be invoked with SOURCE as an argument, then it would determine the name of the AppImage from the application name in the desktop file.

Files are placed in /tmp rather than $HOME/.cache

On top of default.nix:

let
  arx' = haskellPackages.arx.overrideAttrs (o: {
    patchPhase = (o.patchPhase or "") + ''
      substituteInPlace model-scripts/tmpx.sh \
        --replace /tmp/ \$HOME/.cache/
    '';
  });

However, this does not work anymore since: solidsnack/arx@537e1bf . On the other hand, passing --tmpdir '$HOME/.cache' to arx also does not work, since arx expects a fully qualified path and thus rejects $HOME.

Strip binaries in bundles

Currently binaries generated by nix2appimage.sh are very large. This, in part, is due to the fact that they are not stripped.

Support macOS

This will be a meta issue on getting macOS/Darwin/XNU support for nix-bundle. AFAICT no reliable way to chroot exists no macOS for a user. Related links:

How to bundle a local derivation

I may be missing something obvious but I can't get nix-bundle to build a bundle for a derivation defined in the local directory.

On other comands like nix-env this works via -f .

Is this possible? How?

Cannot build VLC AppImage

Very interesting development. However, on xubuntu-16.04.1-desktop-amd64.iso I get:

sudo apt install curl
curl https://nixos.org/nix/install | sh
. /home/me/.nix-profile/etc/profile.d/nix.sh
wget https://github.com/matthewbauer/nix-bundle/archive/master.zip
unzip master.zip 
cd nix-bundle-master/
./nix2appimage.sh vlc

(...)

fetching path ‘/nix/store/y2j0z62i6d6dbkk1ql8nga5c0dz5px3w-v4l-utils-1.10.1-dev’...

*** Downloading ‘https://cache.nixos.org/nar/0q9n9bmid9kwv0cgqb99184vi8py6xq6dgyr4x1qbjq6q60f6ypy.nar.xz’ to ‘/nix/store/y2j0z62i6d6dbkk1ql8nga5c0dz5px3w-v4l-utils-1.10.1-dev’...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (56) SSL read: error:00000000:lib(0):func(0):reason(0), errno 104
/nix/store/5cpnwwnasypdi7p0av6qbaf52y99gmdz-xz-5.2.2-bin/bin/xz: (stdin): File format not recognized
error: unexpected end-of-file
download of ‘https://cache.nixos.org/nar/0q9n9bmid9kwv0cgqb99184vi8py6xq6dgyr4x1qbjq6q60f6ypy.nar.xz’ failed: No such file or directory
could not download ‘/nix/store/y2j0z62i6d6dbkk1ql8nga5c0dz5px3w-v4l-utils-1.10.1-dev’ from any binary cache
fetching path ‘/nix/store/y2j0z62i6d6dbkk1ql8nga5c0dz5px3w-v4l-utils-1.10.1-dev’ failed with exit code 1
fetching path ‘/nix/store/vhkz7c4cssk5lkj736qx3i6iwg6msvk2-libXt-1.1.5’...
killing process 14306
fetching path ‘/nix/store/9gxgmr0i0i4c1rz76vjaqg89bdr2fx3g-xz-5.2.2’...
cannot build derivation ‘/nix/store/228g8nmm1lkgx6xqjb9lpallqxsw327y-vlc.AppDir.drv’: 1 dependencies couldn't be built
killing process 14307
cannot build derivation ‘/nix/store/jz3x25aszaww3j3d9zgr37v5wi5d8fyw-appimage.AppImage.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/jz3x25aszaww3j3d9zgr37v5wi5d8fyw-appimage.AppImage.drv’ failed
cp: missing destination file operand after 'vlc'
Try 'cp --help' for more information.

Feature: Cross Compiling

I know the ReadMe explicitly states that the target architecture must match the host architecture. However, I thought I would experiment to see if it might be possible for nix-bundle to support cross-compilation in the future.

What I have so far is a working proof of concept which shows that is possible for nix-bundle to produce a runnable bundle for a different target architecture. At the moment, the target architecture is hard-coded and it requires a manual tweak to the startup script.

Changes can be found here:
nix-bundle
nixpkgs

What needs to be done:

Finish and merge #58606 (or similar) to strip cross-compiled packages (Not sure this is essential, but decompressing 200MB of unneeded packages is undesirable and really slows testing down.)
Put nix-user-chroot into nixpkgs. Again, not sure if this is essential, but it's the easiest way to have pkgsCross.<target-platform>.nix-user-chroot available
Either deduce or pass in the target platform and build nix-user-chroot for that platform (currently hardcoded)
I had to manually edit the startup script as follows:

LD_PRELOAD="\
  ./nix/store/28nb6zzh6mjj3706s0w4jyvvh5hb4dnb-armv7l-unknown-linux-gnueabihf-stage-final-gcc-7.4.0-lib/lib/libstdc++.so.6 \
  ./nix/store/2mm8gpnipz6i27s2314wm35kdih2kimm-glibc-2.27-armv7l-unknown-linux-gnueabihf/lib/libm.so.6 \
  ./nix/store/28nb6zzh6mjj3706s0w4jyvvh5hb4dnb-armv7l-unknown-linux-gnueabihf-stage-final-gcc-7.4.0-lib/lib/libgcc_s.so.1 \
  ./nix/store/2mm8gpnipz6i27s2314wm35kdih2kimm-glibc-2.27-armv7l-unknown-linux-gnueabihf/lib/libc.so.6" \
./nix/store/2mm8gpnipz6i27s2314wm35kdih2kimm-glibc-2.27-armv7l-unknown-linux-gnueabihf/lib/ld-linux-armhf.so.3 nix/store/lh0xjxrg149glgpxwq5sysg71f20z36l-nix-user-chroot-2c52b5f-armv7l-unknown-linux-gnueabihf/bin/nix-user-chroot -n ./nix -- nix/store/j5z84b29h6nzk19v4py0ki28w4kkmwwy-hello-2.10-armv7l-unknown-linux-gnueabihf/bin/hello $@

Presumably the bit that writes the script should be able to write these paths in, but I'm not sure what to put in ${} to get something like /nix/store/28nb6zzh6mjj3706s0w4jyvvh5hb4dnb-armv7l-unknown-linux-gnueabihf-stage-final-gcc-7.4.0-lib. I assume that the problem is that nix-user-chroot is expecting its libs to be in the usual absolute store paths, not relative to the current directory, so alternatively, it might be possible to either fix-up the library paths in nix-user-chroot binary so that it expects libs relative to itself, or to statically link it.

What do you think? Is this something that you'd want to support in nix-bundle?

AppImages made with nix2appimage.sh need nix installed

The idea behind AppImage is that they should run on target systems without extra dependencies besides what comes with the distribution in the default install. nix2appimage.sh does a great job at generating self-standing bundles, but currently requires ld-linux-x86-64.so.2 to be installed in /nix/store/amjgskg17wv125v9kahqdfxh8sx6mxgp-glibc-2.24/lib/.

The result is that on systems where nix is not installed, the AppImages generated by nix2appimage.sh refuse to run:

$ '/home/me/nix-bundle-master/Inkscape-x86_64.AppImage' 
Error: Error executing 'inkscape'; return code: -1

This is because the main executable inside the AppImage has its interpreter set to the one installed by nix in the host system, which is wrong. Instead it should point to the one bundled by nix2appimage.sh inside the AppImage.

# The binary has the interpreter set to the one in the system:
/tmp/.mount_ZedkdP/usr/bin/inkscape: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/amjgskg17wv125v9kahqdfxh8sx6mxgp-glibc-2.24/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped

# But there is an interpreter in the bundle, which should be used instead:
/tmp/.mount_ZedkdP/nix/store/amjgskg17wv125v9kahqdfxh8sx6mxgp-glibc-2.24/lib/ld-linux-x86-64.so.2

Doesn't work for large bundles

Trying to bundle tensorflow as a test:

nix-bundle python27Packages.tensorflow bin/tensorboard

generates a 1.5GB archive. When I try to run it, it runs out of memory.

I'm guessing nix-bundle isn't a good solution for these scenarios? Perhaps it's an issue with arx?

Cannot build nix-user-chroot

from a nix-shell -p nix-bundle:

these derivations will be built:
  /nix/store/a2lf72dhhpdp8bh75kcrr3if0zhfp6n9-nix-user-chroot-2c52b5f.drv
  /nix/store/k7x6avhbid5gmncsi004b63ghdg97c4n-startup.drv
  /nix/store/bmgmx633bgjzlkr8pw72vcsiql6jsb8c-maketar.drv
  /nix/store/03cxvg6rii4czxc0aq627z1s64h9x330-arx.drv
building '/nix/store/a2lf72dhhpdp8bh75kcrr3if0zhfp6n9-nix-user-chroot-2c52b5f.drv' on 'ssh://lilith'...
copying 1 paths...
copying path '/nix/store/yhvhnlv9l52vdb7i18q0hpkniyc4r35i-nix-user-chroot' to 'ssh://lilith'...
unpacking sources
unpacking source archive /nix/store/yhvhnlv9l52vdb7i18q0hpkniyc4r35i-nix-user-chroot
source root is nix-user-chroot
patching sources
configuring
no configure script, doing nothing
building
build flags: SHELL=/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23/bin/bash
g++ -o nix-user-chroot -DNIX_PATH='"nixpkgs=https://github.com/NixOS/nixpkgs/archive/master.tar.gz"' -DNIX_SSL_CERT_FILE='/no-cert-file.crt' -DENV_HOME='' -DENV_PATH='""' main.cpp
main.cpp: In function 'int main(int, char**)':
<command-line>: error: expected primary-expression before '/' token
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: error: 'no' was not declared in this scope
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: error: 'cert' was not declared in this scope
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: note: suggested alternative: 'creat'
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: error: 'file' was not declared in this scope
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
<command-line>: note: suggested alternative: 'time'
main.cpp:221:31: note: in expansion of macro 'NIX_SSL_CERT_FILE'
   setenv("NIX_SSL_CERT_FILE", NIX_SSL_CERT_FILE, 1);
                               ^~~~~~~~~~~~~~~~~
main.cpp:202:14: warning: ignoring return value of 'ssize_t write(int, const void*, size_t)', declared with attribute warn_unused_result [-Wunused-result]
         write(fd_setgroups, "deny", 4);
         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
main.cpp:217:10: warning: ignoring return value of 'int chdir(const char*)', declared with attribute warn_unused_result [-Wunused-result]
     chdir("/");
     ~~~~~^~~~~
make: *** [Makefile:6: nix-user-chroot] Error 1
builder for '/nix/store/a2lf72dhhpdp8bh75kcrr3if0zhfp6n9-nix-user-chroot-2c52b5f.drv' failed with exit code 2

This might be because I'm using a remote builder?

Crashes with "[...]-startup: No such file or directory"

Running the neuron bundle results in:

/tmp/tmpx-2f521268c1050713/dat/../run: line 1: ./nix/store/pn84j5vljjkxxgwk2fjfm3cvpg8rc2wr-startup: No such file or directory

To reproduce:

Install neuron using Nix
Create the bundle: ./nix-bundle.sh neuron $(which neuron)
Run the bundle: ./neuron --help
It will crash with the above error

Environment

20.09pre228622.029a5de0839 (Nightingale)

Use NixOS store when available

The NixOS store can be modified by users so we don't need a chroot.

Feature request: drop dependency on host /bin/sh

I think it would be cool if the dependency on host /bin/sh could be dropped. Then nix-bundle would truly have no external dependencies.

Perhaps the entry point could be something statically linked with musl?

Currently the front-page says "No external dependencies are required besides a compatible Linux kernel". So, to truly live up to that, it shouldn't use host /bin/sh :-)

Bundle .drv files

It would be nice to include the .drv file that each attribute is made from. That way the bundle could install parts of itself is something is missing in the registry. Also could enable NixOS support.

Gtk-Message: Failed to load module "canberra-gtk-module"

Getting Gtk-Message: Failed to load module "canberra-gtk-module" when running Inkscape generated using ./nix2appimage.sh inkscape. Also, the application looks ugly. How can Canberra be added to the "recipe"?

maketar fails with: [glibc locale] file changed as we read it

It doesn’t happen consistently ( :o ), but I’m quite often getting this when running nix-bundle:

tar: /nix/store/2qgjpsn1zkf0clvrrjympwf6ar2dx83r-glibc-2.27/share/i18n/locales/mai_NP: file changed as we read it
builder for '/nix/store/b4qxkfr51mplhfkl7i5dg4fvd8gwm2zw-maketar.drv' failed with exit code 1
cannot build derivation '/nix/store/00p5syq4hy5jv4pw843z191w8bi2kdsf-arx.drv': 1 dependencies couldn't be built

Or:

tar: /nix/store/2qgjpsn1zkf0clvrrjympwf6ar2dx83r-glibc-2.27/share/i18n/locales/cns11643_stroke: file changed as we read it

How is it even possible for this store path to change? :o

Re-running it, it works.

License

This looks pretty cool! I didn't see a license for this code - can you please add one?

Add nix-bundle to nixpkgs

Any plan to make this usable from within nixpkgs? Or is it already? Couldn't find it in nix-repl

Unable to bundle `xpra`?

~/nix-bundle$ nix-shell

[nix-shell:~/nix-bundle]$ ~/nix-bundle/nix-bundle.sh xpra /bin/xpra
error: attribute 'xpra' in selection path 'xpra' not found

Bundle non-nixpkgs attributes

How hard would it be to specify what nix file to use and which attribute to package outside nixpkgs?

cc @matthewbauer

Not working on first try

I tried nix-bundleing hello following the instructions, but it doesn't seem to work.

% nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/378662bbba6142ffee9fce02136f70d3d96209c2.tar.gz -p hello --run "./nix-bundle.sh hello \$(which hello)" && echo "done bundling" && ./hello
Nix bundle created at hello.
done bundling
execvp(/nix/store/7984c712x8ksg6c26jdggg2zxf0ilymg-hello-2.10/nix/store/b705xbm1nchiwj6cfnkjwi0lnnr8hnri-hello-2.10/bin/hello): No such file or directory

The nixpkgs version being used is a recent NixOS-16.09 version.

Cannot access any environment variables

From the source of AppRun.c I'd expect a number of environment variables to be available, in particular $USER. However:

$ echo $USER
nathan
$ nix-bundle bash /bin/bash
Nix bundle created at bash.
$ ./bash
bash-4.4$ echo $USER

bash-4.4$

Statically compile AppImage

Right now AppImage does dynamic linking to some libraries like FUSE and SquashFS. Ideally we could statically compile everything. This will increase our bundle size a little but will mean we don't need 'test-appimage.nix'.

Most likely we will have to slightly modify appimagetool to get this to work.

Nix 2 compat

Looks like on a NixOS system using Nix 2, we need to set NIX_REMOTE=daemon (as with any use of Nix 1.11 on Nix 2), or perhaps switch the 'nix' used to 'nixUnstable'?

Anyway would be good to ensure this worked with the upcoming 18.03 release :).

Thanks!

Create .desktop files on the fly

See 43f0f04#commitcomment-20773045

Basically, some applications don't have good .desktop files, so we may need to create them custom.

Faster startup using lz4?

I think this would require a custom archiver that can bootstrap lz4 decompression

Executing an AppImage bundled with nix2appimage.sh failing with Segmentation fault.

nix-bundle/AppRun.c

Line 248 in ffa3349

LOAD_ENV_VAR(HOSTNAME);

nix-bundle/AppRun.c

Line 250 in ffa3349

LOAD_ENV_VAR(LC_ALL);

Executing AppImages bundled with nix2appimage.sh was failing because those two environment variables were not set.

./nix2appimage.sh fails: /nix/store/.../bin/appimagetool: No such file or directory

$ ./nix2appimage.sh vlc
these derivations will be built:
  /nix/store/ryslsrgg7l1n8sr4v47vnvdfbw68v9h8-appimage.drv
building path(s) ‘/nix/store/q45h3wcl0xclsr8x4hkn3zh93kzb5lf4-appimage’
/nix/store/p4avr9pw3sa68ivryc490fvqmnws0zyv-stdenv/setup: line 853: /nix/store/kichfn29xdmzj8d4wg18d28fw9hkcxf2-appimagekit/bin/appimagetool: No such file or directory
builder for ‘/nix/store/ryslsrgg7l1n8sr4v47vnvdfbw68v9h8-appimage.drv’ failed with exit code 127
error: build of ‘/nix/store/ryslsrgg7l1n8sr4v47vnvdfbw68v9h8-appimage.drv’ failed
./nix2appimage.sh failed. Exiting.

`nix-run htop` causes sed error "unterminated `s' command"

[cody@nixos:~]$ realpath $(which nix-run)
/nix/store/9mcn9nnavrxbhwbacwlalxln0qbx0iyr-nix-bundle-0.3.0/bin/nix-run

[cody@nixos:~]$ nix-run htop
sed: -e expression #1, char 24: unterminated `s' command

[cody@nixos:~]$ nix-run hello
Hello, world!

Appimage function

I find myself doing this to wrap arbitrary packages (including GTK ones) with appimage.

let
  appimage_src = drv : exec : with self;
    self.stdenv.mkDerivation rec {
      name = drv.name + "-appdir";
      env = buildEnv {
        inherit name;
        paths = buildInputs;
      };
      src = env;
      inherit exec;
      buildInputs = [ drv ];
      usr_fonts = buildEnv {
        name = "fonts";
        paths = [noto-fonts];
      };
      buildCommand = ''
        source $stdenv/setup
        mkdir -p $out/bin
        cp -rL ${drv}/* $out/
        chmod +w -R $out/

        mkdir -p $out/share/fonts
        cp ${usr_fonts}/share/fonts/* $out/share/fonts -R

        mkdir -p $out/share/icons
        touch $out/share/icons/${drv.name}.png

        mkdir -p $out/share/applications
        cat <<EOF > $out/share/applications/${drv.name}.desktop
        [Desktop Entry]
        Type=Application
        Version=1.0
        Name=${drv.name}
        Path=${exec}
        Icon=$out/share/icons/${drv.name}
        Exec=$exec
        Terminal=true
        EOF
        chmod +w -R $out/
        '';
        system = builtins.currentSystem;
  };

in
  with (import (nix-bundle_src + "/appimage-top.nix"){nixpkgs' = nixpkgs_src_musl;});
    appimage (appdir {
      name ="gnuradio";
      target = appimage_src grc "/bin/hello";
    })

This needs some cleanup, but a function that does all this, and improving the CLI to do it too, would be a big jump in capability.

OpenGL: unable to open display

nix-bundle blender /bin/blender

Will fail with:

$ ./blender          
Unable to open a display
../run: line 1: 27203 Aborted                 ./nix/store/c1x6kjxjvbjzqzjcv38ni5mb9zpb7khx-nix-user-chroot-2b144e/bin/nix-user-chroot ./nix /nix/store/1fqm7lqkjdkcfdj1fz9b7n0xizznqy82-blender-2.79/bin/blender $@

On nixos 17.09.1997.1a8a95e879 (Hummingbird) using the packaged nix-bundle.

nix bundle'd executables do not work, fail with nix-user-chroot: No such file or directory

Original issue: NixOS/nix#4426

Describe the bug

I used nix bundle to build a standalone executable; however the executable doesn't work

Steps To Reproduce

$ nix bundle nixpkgs#hello
$ ldd hello
	not a dynamic executable
$ ./hello world
./nix/store/m6mcsf66rn5hc6s8n8yfb8rknxzvcwqh-startup: line 2: ./nix/store/j0rfffzsih095w4nvl6cp0zbmdiaycfs-nix-user-chroot-2c52b5f/bin/nix-user-chroot: No such file or directory

Expected behavior
the hello executable starts up as expected

nix-env --version output

nix (Nix) 2.4pre20201201_5a6ddb3

Additional context

Add any other context about the problem here.

example of nix2appimage.sh does not work because of icon format

Example of the readme:

$  ./nix2appimage.sh vlc        
these derivations will be built:
  /nix/store/s62ip4ga26m6ik4caa0m1gfhk9fsqvjp-AppRun.drv
  /nix/store/185c2bf8ksbnkw0f3j8nqm1aw0jip0a6-vlc.AppDir.drv
  /nix/store/wwaj40280gad8cjs2r3nn3pzshxpm9k6-appimage.drv
building '/nix/store/s62ip4ga26m6ik4caa0m1gfhk9fsqvjp-AppRun.drv'...
building
/nix/store/y7w6j74ipqnybprfdb66k7hhjcdxzmvc-AppRun.c: In function ‘main’:
/nix/store/y7w6j74ipqnybprfdb66k7hhjcdxzmvc-AppRun.c:134:45: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 253 [-Wformat-truncation=]
  134 |     snprintf(desktop_file, LINE_SIZE-1, "%s/%s", appdir, namelist[0]->d_name);
      |                                             ^~
/nix/store/y7w6j74ipqnybprfdb66k7hhjcdxzmvc-AppRun.c:134:5: note: ‘snprintf’ output 2 or more bytes (assuming 257) into a destination of size 254
  134 |     snprintf(desktop_file, LINE_SIZE-1, "%s/%s", appdir, namelist[0]->d_name);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/yacn11z90sqa1yxwpb5cl1sncac9sahc-AppRun
shrinking /nix/store/yacn11z90sqa1yxwpb5cl1sncac9sahc-AppRun/bin/AppRun
patchelf: cannot find section '.dynamic'. The input file is most likely statically linked
strip is /nix/store/7gagq119aqpfc54sj3s313ng0279mv9s-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/yacn11z90sqa1yxwpb5cl1sncac9sahc-AppRun/bin
patching script interpreter paths in /nix/store/yacn11z90sqa1yxwpb5cl1sncac9sahc-AppRun
checking for references to /build/ in /nix/store/yacn11z90sqa1yxwpb5cl1sncac9sahc-AppRun...
patchelf: cannot find section '.dynamic'. The input file is most likely statically linked
building '/nix/store/185c2bf8ksbnkw0f3j8nqm1aw0jip0a6-vlc.AppDir.drv'...
building '/nix/store/wwaj40280gad8cjs2r3nn3pzshxpm9k6-appimage.drv'...
WARNING: appstreamcli is missing, please install it if you want to use AppStream metadata
WARNING: gpg2 or gpg is missing, please install it if you want to create digital signatures
/nix/store/a6qmsiybsn4ghj41nymhsqbddd3blxbq-vlc.AppDir/vlc.AppDir/vlc.desktop: warning: key "Keywords" in group "Desktop Entry" is a reserved key for KDE
Using architecture x86_64
/nix/store/a6qmsiybsn4ghj41nymhsqbddd3blxbq-vlc.AppDir/vlc.AppDir should be packaged as VLC_media_player-x86_64.AppImage
vlc{.png,.svg,.svgz,.xpm} defined in desktop file but not found
For example, you could put a 256x256 pixel png into
/nix/store/a6qmsiybsn4ghj41nymhsqbddd3blxbq-vlc.AppDir/vlc.AppDir/usr/share/icons/hicolor/256x256/apps/vlc.png
builder for '/nix/store/wwaj40280gad8cjs2r3nn3pzshxpm9k6-appimage.drv' failed with exit code 1
error: build of '/nix/store/wwaj40280gad8cjs2r3nn3pzshxpm9k6-appimage.drv' failed
./nix2appimage.sh failed. Exiting.

but the icon actually exists

$  ls /nix/store/a6qmsiybsn4ghj41nymhsqbddd3blxbq-vlc.AppDir/vlc.AppDir/usr/share/icons/hicolor/256x256/apps/
vlc.png

current master of nix-bundle
Nix-info:

system: "x86_64-linux"
host os: Linux 5.8.14, NixOS, 20.09beta1083.51aaa3fa1b6 (Nightingale)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.3.7
channels(root): "nixos-20.09beta1083.51aaa3fa1b6, nixos-hardware, nixos-unstable-21.03pre246543.24c9b05ac53"
channels(symphorien): "home-manager-20.09"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

libgtk3-nocsd.so breaks nix-user-chroot

apt-get install gtk3-nocsd appears to entirely break nix-user-chroot, because we are using a dynamic binary with relative paths for the interpreter and rpath

so it still tries to obey:

user@msi:~/Downloads/tmp-to-extract-daedalus-testnet-installer$ echo $LD_PRELOAD
/usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0

and then it fails to find:
./nix/store/8klfw9p4m353canp95ps4pmmdjfr4rb8-nix-user-chroot-2c52b5f/bin/nix-user-chroot: error while loading shared libraries: libdl.so.2: cannot open shared object file: No such file or directory

unsetting LD_PRELOAD or switching to a pure static binary will fix things

Alternate commands

The proposed interface in the RFC exposed a -c, --command to allow for program names other than the default. I got something like this working, but not sure if there's a better way.

PROGRAM=mb-util nix bundle nixpkgs#mbutil --bundler .#defaultBundler

{
  description = " description";
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs";
    nix-bundle.url = "github:matthewbauer/nix-bundle";
  };
  outputs = { self, nixpkgs, nix-bundle  }:
    {
    bundlers = {
      nix-bundle = { program, system }: let
        nixpkgs' = nixpkgs.legacyPackages.${system};
        nix-bundl = import nix-bundle { nixpkgs = nixpkgs'; };
        envProg = builtins.getEnv "PROGRAM";
        prog = if envProg == ""
              then program
              else "${builtins.dirOf program}/${envProg}";
        script = nixpkgs'.writeScript "startup" ''
          #!/bin/sh
          .${nix-bundl.nix-user-chroot}/bin/nix-user-chroot -n ./nix -- "${prog}"
        '';
          #${program} "$@"
      in nix-bundl.makebootstrap {
        targets = [ script ];
        startup = ".${builtins.unsafeDiscardStringContext script} '\"$@\"'";
      };
    };

    defaultBundler = self.bundlers.nix-bundle;
    }
    ;
}

Getting execvp errors when running binaries on another machine

I used nix-bundle to make various binaries on an Ubuntu 19.04 machine with nix installed:

cd~
git clone https://github.com/matthewbauer/nix-bundle.git
cd nix-bundle
nix-shell

cd /path/to/new/directory
~/nix-bundle/nix-bundle.sh nixpkgs.xorg.xrdb
~/nix-bundle/nix-bundle.sh nixpkgs.wmctrl
~/nix-bundle/nix-bundle.sh nixpkgs.terminator
~/nix-bundle/nix-bundle.sh nixpkgs.xpra

The problems are as follows:

On my Ubuntu machine, the static binaries only work if they are executed within the folder which they were generated. For example, if I switch from /path/to/new/directory to /path/to/new and run i.e. ./directory/terminator, then I get execvp errors like

execvp(/nix/store/y6m5r6pwa63vhmywmcmwkkbm15v44jm4-terminator-1.91/bin/terminator): No such file or directory

If I tarball and send the binaries to another machine (running ArchLinux), then none of them work. No matter what directory the binaries are ran in, I get execvp errors like:

$ ./terminator
execvp(/nix/store/y6m5r6pwa63vhmywmcmwkkbm15v44jm4-terminator-1.91/bin/terminator): No such file or directory
$ ./wmctrl -l
execvp(/bin/wmctrl): No such file or directory

Some of the errors mention /nix/* paths, while others don't. In case relevant: the Arch machine doesn't have nix installed.

matthewbauer / nix-bundle Goto Github PK

nix-bundle's People

Contributors

Stargazers

Watchers

Forkers

nix-bundle's Issues

Environment

Recommend Projects

Recommend Topics

Recommend Org

Jobs