k3s-io / k3s-root

K3s agent user space binaries

k3s-root

==========

NOTE: this repository has been recently (2020-11-18) moved out of the github.com/rancher org to github.com/k3s-io supporting the acceptance of K3s as a CNCF sandbox project.

---

k3s-root is based on https://github.com/buildroot/buildroot and provides the userspace binaries for rancher/k3s

Building

k3s-root is dapper powered, which means you should be able to simply make on your machine to compile. By default, make will compile for amd64. If you want to compile for other architectures, you can run commands like:

make ARCH=arm64

or alternatively, if you want to compile all architectures at once, you can run:

make all-build

If it is not the first time you build, delete the directories artifacts/ and dist/ to avoid problems when creating the symbolic link in the package step

sudo rm -r artifacts/ dist/

The default way of building this project is through container using dapper. If you want to tinker and have more control over the build process, it is more practical to use Vagrant with the existing Vagrantfile

Upgrading to new buildroot version

To upgrade to a new buildroot version, you must follow 4 steps:

1 - Modify the BUILDROOT_VERSION in scripts/download

2 - Check what is the busybox version in the new buildroot package. Then, upgrade the package/busybox.config by cloning the busybox project and then:

git checkout $busybox_version
cp $K3S_ROOT_PATH/package/busybox.config .config
make oldconfig
# Choose the new options
cp .config $K3S_ROOT_PATH/package/busybox.config

3 - Follow the same steps with the buildroot/ configurations. The command make oldconfig also works in the buildroot project

4 - Verify if the upgrade worked correctly by comparing the old tarball and the new one. If the same files are there, then you are set

k3s-root's Issues

s390x support

We are working on building k3s-root on s390x, since we need the binaries generated in k3s-root in order to build rke2 on s390x. Unfortunately, buildroot doesn't support static builds on s390x https://bugs.busybox.net/show_bug.cgi?id=14256
Would it be possible to use shared libraries instead of static?

Upgrade to buildroot LTS 2019.02

clarify that k3s-root is not pure "apache-2" licensed

My understanding is that the license declaration is just for the k3s-root specific part, not for the embedded components like busybox. we could be more careful and specify GPLv2?

False positives triggering on VirusTotal

scripts should verify checksum

To detect potential compromise, the scripts should verify checksum here

k3s-root/scripts/download

Line 9 in 38468f8

curl -fL https://buildroot.org/downloads/buildroot-${BUILDROOT_VERSION}.tar.bz2 | tar xvjf - -C /usr/src/buildroot --strip-components=1

k3s-root/scripts/package

Lines 25 to 40 in 38468f8

	# download pre-built binaries
	SLIRP4NETNS_VERSION="v1.1.8"
	FUSE_OVERLAYFS_VERSION="v1.3.0"
	uname_m="${BUILDARCH}"
	case "${BUILDARCH}" in
	"amd64")
	uname_m="x86_64" ;;
	"arm64")
	uname_m="aarch64" ;;
	"arm")
	uname_m="armv7l" ;;
	esac
	curl -o /source/artifacts/${BUILDARCH}/bin/slirp4netns -sSL https://github.com/rootless-containers/slirp4netns/releases/download/${SLIRP4NETNS_VERSION}/slirp4netns-${uname_m}
	chmod +x /source/artifacts/${BUILDARCH}/bin/slirp4netns
	curl -o /source/artifacts/${BUILDARCH}/bin/fuse-overlayfs -sSL https://github.com/containers/fuse-overlayfs/releases/download/${FUSE_OVERLAYFS_VERSION}/fuse-overlayfs-${uname_m}
	chmod +x /source/artifacts/${BUILDARCH}/bin/fuse-overlayfs

Crun support?

It would be cool if k3s supported crun https://github.com/containers/crun

Remove socat as it is not needed anymore

socat was removed from containerd some time ago containerd/cri#1470 and this change was merged into the k3s containerd fork k3s-io/containerd@11a78d9

As far as I know we were just using socat for that. If that is the case we can remove it https://github.com/k3s-io/k3s-root/blob/master/buildroot/config#L3563

Including k3s-root in vanilla buildroot build?

What is the preferred way of including k3s-root in Buildroot built Linux image? I imagine building the k3s-root separately and adding the produced binaries using BR2_ROOTFS_OVERLAY is one option. Is there a better way?

ebtables-legacy segfauls on s390x with SLE15-SP2

I just took binaries from https://github.com/k3s-io/k3s-root/releases/download/v0.10.0-rc.0/k3s-root-xtables-s390x.tar https://github.com/k3s-io/k3s-root/releases/download/v0.10.0-rc.0/k3s-root-s390x.tar and executed on s390x VM with SLE15-SP2:

./bin/aux/ebtables-legacy
Segmentation fault (core dumped)

kernel info:

sles@rancher-sles15sp2-jeos-ext4-devnet:~/k3s-root-release/k3s-root> uname -a
Linux rancher-sles15sp2-jeos-ext4-devnet 5.3.18-24.43-default #1 SMP Thu Dec 3 21:53:21 UTC 2020 (b48bf35) s390x s390x s390x GNU/Linux

just help:

strace ./bin/aux/ebtables-legacy --help
execve("./bin/aux/ebtables-legacy", ["./bin/aux/ebtables-legacy", "--help"], 0x3ffc307f3f8 /* 49 vars */) = 0
set_tid_address(0x10187d0)              = 29685
brk(NULL)                               = 0x2fef000
brk(0x2ff1000)                          = 0x2ff1000
mmap(0x2fef000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x2fef000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3ff948fd000
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

and list tables:

strace -f ./ebtables-legacy -L
execve("./ebtables-legacy", ["./ebtables-legacy", "-L"], 0x3ffe7cfe910 /* 45 vars */) = 0
set_tid_address(0x10187d0)              = 29793
brk(NULL)                               = 0x2fba000
brk(0x2fbc000)                          = 0x2fbc000
mmap(0x2fba000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x2fba000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3ff9befd000
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)