rootless-containers / runrootless

rootless OCI container runtime with ptrace hacks (No root privileges nor SUID binaries (e.g. newuidmap) are required!)

runROOTLESS: rootless OCI container runtime with ptrace hacks

Quick start (No root privileges nor SUID binaries are required!)

Install

Requires: Go, runc

user$ go get github.com/rootless-containers/runrootless
user$ $GOPATH/src/github.com/rootless-containers/runrootless/install-proot.sh

Future version should install a pre-built PRoot binary automatically on the first run.

Usage

Create an example Ubuntu bundle:

user$ cd ./examples/ubuntu
user$ ./prepare.sh
user$ ls -1F
config.json
prepare.sh
rootfs/

Make sure the bundle cannot be executed with the regular runc:

user$ runc run ubuntu
rootless containers require user namespaces

Note that even with runc spec --rootless, you cannot execute apt:

user$ rm config.json
user$ runc spec --rootless
user$ sed -i 's/"readonly": true/"readonly": false/' config.json
user$ runc run ubuntu
# apt update
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (22: Invalid argument)
E: seteuid 100 failed - seteuid (22: Invalid argument)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
Reading package lists... Done
W: chown to _apt:root of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (22: Invalid argument)
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (22: Invalid argument)
E: seteuid 100 failed - seteuid (22: Invalid argument)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (1: Operation not permitted)
E: Method http has died unexpectedly!
E: Sub-process http returned an error code (112)_

With runrootless, you can execute apt successfully:

user$ ./prepare.sh
user$ runrootless run ubuntu
# apt update
# apt install -y cowsay
# /usr/games/cowsay hello rootless world
 ______________________
< hello rootless world >
 ----------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Other examples

CentOS:

user$ cd ./examples/centos
user$ ./prepare.sh
user$ runrootless run centos
sh-4.2# yum install -y epel-release
sh-4.2# yum install -y cowsay
sh-4.2# cowsay hello rootless world

Alpine Linux:

user$ cd ./examples/alpine
user$ ./prepare.sh
user$ runrootless run alpine
/ # apk update
/ # apk add fortune
/ # fortune

Arbitrary Docker image:

user$ cd ./examples/docker-image
user$ ./prepare.sh opensuse
user$ runrootless run opensuse
sh-4.3# zypper install cowsay
sh-4.3# cowsay hello rootless world

Arbitrary container image, using skopeo and umoci. umoci and runROOTLESS share emulated chown(2) information via user.rootlesscontainers xattr.

user$ cd ./examples/skopeo-umoci
user$ ./prepare.sh docker://ubuntu
user$ cd umoci-bundle
user$ runrootless run ubuntu

runROOTLESS can be also executed inside Docker container, but --privileged is still required ( opencontainers/runc#1456 )

host$ docker run -it --rm --privileged akihirosuda/runrootless
~ $ id
uid=1000(user) gid=1000(user)
~ $ cd ~/examples/ubuntu/
~/examples/ubuntu $ ./prepare.sh
~/examples/ubuntu $ runrootless run ubuntu
#

Environment variables

• RUNROOTLESS_SECCOMP=1: enable seccomp acceleration (unstable)

How it works

• Transform a regular config.json to rootless one, and create a new OCI runtime bundle with it.
• Bind-mount a static PRoot binary so as to allow apt/yum commands.
• Inject the PRoot binary to process.args.
• Invoke plain runC.

Known issues

• apt / dpkg may crash when seccomp acceleration is enabled: #4

Future work

OCI Runtime Hook mode

runROOTLESS could be reimplemented as a OCI Runtime Hook (prestart) that works with an arbitrary OCI Runtime. This work would need adding support for PTRACE_ATTACH to PRoot. Also, it would require YAMA to be disabled.

Reimplement PRoot in Go

This is hard than I initially thought...

runrootless's Issues

Hide `user.rootlesscontainers.*` xattrs from the container

Support `runc exec`

Currently, runc exec is not hooked.

Ideally we should use a single proot instance, but injecting another instance should be ok now

apk script fails due to chroot EPERM

/ # apk add zsh
(1/4) Installing ncurses-terminfo-base (6.0_p20170930-r0)
(2/4) Installing ncurses-terminfo (6.0_p20170930-r0)
(3/4) Installing ncurses-libs (6.0_p20170930-r0)
(4/4) Installing zsh (5.4.2-r0)
Executing zsh-5.4.2-r0.post-install
ERROR: zsh-5.4.2-r0.post-install: script exited with error 127
Executing busybox-1.27.2-r6.trigger
ERROR: busybox-1.27.2-r6.trigger: script exited with error 127
1 error; 17 MiB in 15 packages

strace:

openat(3, "var/cache/misc/busybox-1.27.2-r6.trigger", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC, 0755) = 8                                       
write(8, "#!/bin/sh\n\ndo_bb_install=\n\nfor i"..., 365) = 365                                                                          
close(8)                                = 0                         
rt_sigprocmask(SIG_BLOCK, ~[], [], 8)   = 0                                                                                             
fork(strace: Process 15 attached                                                                                                        
 <unfinished ...>                                                                                                                       
[pid    15] gettid()                    = 15                                                                                            
[pid    15] rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0            
[pid    15] umask(022)                  = 000                       
[pid    15] fchdir(3)                   = 0                         
[pid    15] chroot(".")                 = -1 EPERM (Operation not permitted)                                                            
[pid    13] <... fork resumed> )        = 15                                                                                            
[pid    15] exit_group(127)             = ?                         
[pid    13] rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0                                                                                
[pid    15] +++ exited with 127 +++

https://github.com/alpinelinux/apk-tools/blob/5f7dcadc691f4064464be02427fa6e7fd61888f1/src/database.c#L1930

This is not specific to runrootless/proot.
Plain rootless runc hits this issue as well.

proot: fix chown(42, 43); chown(44, -1)

expected result: 44:43
actual result: 44:0

#include <stdio.h>
#include <unistd.h>

int main(int ac, char *av[]){
  if (ac != 2) {
    fprintf(stderr, "usage: %s FILE\n", av[0]);
    return 1;
  }
  const char *path = av[1];
  if (chown(path, 42, 43) < 0) {
    perror("chown(42, 43)");
    return 1;
  }
  if (chown(path, 44, -1) < 0) {
    perror("chown(44, -1)");
    return 1;
  }
  return 0;
}

apt/dpkg fails intermittently, when seccomp acceleration is enabled

root@runc:~# dpkg-deb --fsys-tarfile /var/cache/apt/archives/htop_2.0.1-1ubuntu1_amd64.deb > /dev/null 
dpkg-deb: error: subprocess <decompress> was killed by signal (Segmentation fault)
root@runc:~# dpkg-deb --fsys-tarfile /var/cache/apt/archives/htop_2.0.1-1ubuntu1_amd64.deb > /dev/null 
root@runc:~# dpkg-deb --fsys-tarfile /var/cache/apt/archives/htop_2.0.1-1ubuntu1_amd64.deb > /dev/null 
root@runc:~# dpkg-deb --fsys-tarfile /var/cache/apt/archives/htop_2.0.1-1ubuntu1_amd64.deb > /dev/null 
dpkg-deb: error: subprocess <decompress> was killed by signal (Segmentation fault)

RFC: consider renaming the project (runROOTLESS -> runFULLYROOTLESS?)

The current project name runROOTLESS is confusing because the upstream runc supports rootless as well but in a different way.

RFC

cc @cyphar

Fake capabilities (e.g. CAP_CHOWN, CAP_DAC_OVERRIDE)

unsure what pre-req's are needed for this to work

So, I think this needs user namespaces enabled to work - which in my situation is thus not "fully rootless" - the user needs to convince the admins that this is reasonable (and thus takes time)

Assuming I'm correct, would it be reasonable to mention this in the README?

I'm also wondering if the fork of PRoot is necessary anymore, or if the changes could be merged upstream? @oxr463 ?

memo: benchmark

image	command	regular runc (root) (config)	runrootless	runrootless+seccomp
docker gentoo/stage3-amd64	emerge --sync	52s	1m43s	2m54s
ditto	emerge zsh (after emerge --sync)	2m1s	9m3s	(crashed quickly)
alpine	apk add gcc	1.4s	2.2s	2.0s
ditto	apk add openjdk8	3.1s	4.4s	3.14s
ditto	git clone https://github.com/torvalds/linux.git	6m38s	10m43s	(crashed quickly)

• PRoot overhead seems significant for emerge, especially during compiling packages
• For apk add, overhead is negligible
• Suggestion:
  -- Enable PRoot only during apk/apt/yum operation
  -- Disable PRoot for compilation

proot: can we separate it out into a separate project

I've been recommending people use the proot plugin that you've written, but it's a bit difficult for people to just use it because it's part of this project. Would you mind if we separated it into a separate project and then placed it inside https://github.com/rootless-containers?

As an aside, did you want to put this project in the rootless-containers organisation as well?

modify an OCI bundle rather than a whole new wrapper script

containers/buildah#386 (comment)

@cyphar:

@AkihiroSuda Have you considered making runrootless a tool that will modify an OCI bundle rather than a whole new wrapper script? In principle you would only need to touch the config.json since you also bind-mount PRoot into the container.

Yes, it should be easily possible.
What do you expect for CLI UX?

Does runrootless _convert look fine?

(I prepended an underscore so that it won't conflict with future version of runc, but no strong opinion)