projectcalico / go-build

Calico go-build

Base image for doing golang builds for the various project calico builds.

Building the image

To build the image:

make image

The above will build for whatever architecture you are running on. To force a different architecture:

ARCH=<somearch> make image

Tagging

The image is tagged the version, e.g. v0.9 or latest. In addition, the given architecture is appended to the end. Thus, for example, the latest version on amd64 will be calico/go-build:latest-amd64.

The above tagging scheme keeps everything in a single image repository calico/go-build and prepares for using multi-architecture image manifests.

Cross building using go-build

Any supported platform can be built natively from its own platform, i.e.g amd64 from amd64, arm64 from arm64 and ppc64le from ppc64le. In addition, ppc64le and arm64 are supported for cross-building from amd64 only. We do not (yet) support cross-building from arm64 and ppc64le.

The cross-build itself will function normally on any platform, since golang supports cross-compiling using GOARCH=<target> go build.

docker run -e GOARCH=<somearch> calico/go-build:latest-amd64 sh -c 'go build hello.go || ./hello'

The above will output a binary hello built for the architecture <somearch>.

Cross-running Binaries binfmt

The Linux kernel has the ability to run binaries built for one arch on another, e.g. arm64 binaries on an amd64 architecture. Support requires two things:

1. Registering an interpreter that can run the binary for the other architecture along with configuration information on how to identify which binaries are for which platform and which emulator will handle them.
2. Making the interpreter binary available.

The interpreter must exist in one of two places:

• The container where you are running the other-architecture binary.
• The container where you run registration, if you pass the correct flag during registration. This is supported only from Linux kernel version 4.8+.

For example, if you registered the s390x emulator at /usr/bin/qemu-s390x-static, and then wanted to run docker run -it --rm s390x/alpine:3.10 sh on an amd64, it wouldn't work in the first method, because the new container doesn't have an emulator in it. However, if you followed the second method, it would work, since the kernel already found and loaded the emulator. This works even if you delete the registration container.

To register emulators, we run:

docker run -it --rm --privileged multiarch/qemu-user-static:register

or simply

make register

After the above registration, your system can handle other-architecture binaries. The above registration uses the first method, since all kernels that support binfmt support this method, while only kernels from version 4.8+ support the latter. While docker-for-mac and docker-for-windows both use supporting kernels, almost every CI-as-a-service does not.

Using binfmt in other Calico projects

To use binfmt in other projects:

1. Ensure you have run registration as above.
2. Copy the correct interpreter into the container in which you will run other-architecture commands. The COPY must be before any RUN command.

FROM calico/go-build:v0.16 as qemu

FROM arm64v8/golang:1.15.2-buster as base

# Enable non-native builds of this image on an amd64 hosts.
# This must be the first RUN command in this file!
# we only need this for the intermediate "base" image, so we can run all the apk and other commands
COPY --from=qemu /usr/bin/qemu-*-static /usr/bin/

# now we can do all our RUN commands
RUN apk --update add curl
# etc

Running a Binary

To run a binary from a different architecture, you need to use binfmt and qemu static.

Register qemu-*-static for all supported processors except the current one using the following command:

docker run --rm --privileged multiarch/qemu-user-static:register

If a cross built binary is executed in the go-build container qemu-static will automatically be used.

Testing Cross-Run

There is a Makefile target that cross-builds and runs a binary. To run it on your own architecture:

make testcompile

or

make testcompile ARCH=$(uname -m)

To test on a different architecture, for example arm64 when you are running on amd64, pass it an alternate architecture:

make testcompile ARCH=arm64

You should see the "success" message.

go-build's Issues

Dockerfile.amd64 vs Dockerfile.s390x differences

Team,

I am trying to understand why there is such a dissimilarity between Dockerfile used to build amd64 and s390x images.

I have built s390x image based on Dockerfile.amd64 and it seem to have worked for me. I also ran Calico test cases which passed as well.

Is there something I am missing here? Here is a diff file that seems to have done the job for me when building Calico 3.18.1 on s390x:

 diff Dockerfile.s390x Dockerfile.amd64
1c1
< FROM calico/bpftool:v5.3-s390x as bpftool
---
> FROM calico/bpftool:v5.3-amd64 as bpftool
3c3
< FROM s390x/golang:1.15.2-buster
---
> FROM amd64/golang:1.15.2-buster
33c33
<         libbpf-dev linux-headers-s390x  && \
---
>         libbpf-dev linux-headers-amd64  && \
52,58c52,53
< RUN git clone -b v1.0.1 https://github.com/fossas/fossa-cli.git $GOPATH/src/github.com/fossas/fossa-cli; \
<     cd $GOPATH/src/github.com/fossas/fossa-cli; \
<     make; \
<     cp $GOPATH/bin/fossa /usr/local/bin; \
<     chmod +x /usr/local/bin/fossa
< #RUN curl -L https://github.com/fossas/fossa-cli/releases/download/v${FOSSA_VER}/fossa-cli_${FOSSA_VER}_linux_s390x.tar.gz | tar zxvf - -C /usr/local/bin --extract fossa
< #RUN chmod +x /usr/local/bin/fossa
---
> RUN curl -L https://github.com/fossas/fossa-cli/releases/download/v${FOSSA_VER}/fossa-cli_${FOSSA_VER}_linux_amd64.tar.gz | tar zxvf - -C /usr/local/bin --extract fossa
> RUN chmod +x /usr/local/bin/fossa
61,67c56,57
< RUN git clone -b v2.3.0 https://github.com/vektra/mockery.git $GOPATH/src/github.com/vektra/mockery.git; \
<     cd $GOPATH/src/github.com/vektra/mockery.git; \
<     go get github.com/vektra/mockery/v2/.../; \
<     cp $GOPATH/bin/mockery /usr/local/bin/mockery; \
<     chmod +x /usr/local/bin/mockery
< #RUN curl -L https://github.com/vektra/mockery/releases/download/v${MOCKERY_VER}/mockery_${MOCKERY_VER}_Linux_x86_64.tar.gz | tar zxvf - -C /usr/local/bin --extract mockery
< #RUN chmod +x /usr/local/bin/mockery
---
> RUN curl -L https://github.com/vektra/mockery/releases/download/v${MOCKERY_VER}/mockery_${MOCKERY_VER}_Linux_x86_64.tar.gz | tar zxvf - -C /usr/local/bin --extract mockery
> RUN chmod +x /usr/local/bin/mockery
103c93
< # Enable non-native runs on s390x architecture hosts
---
> # Enable non-native runs on amd64 architecture hosts
114c104
< RUN curl -sSL https://github.com/estesp/manifest-tool/releases/download/${MANIFEST_TOOL_VERSION}/manifest-tool-linux-s390x > manifest-tool && \
---
> RUN curl -sSL https://github.com/estesp/manifest-tool/releases/download/${MANIFEST_TOOL_VERSION}/manifest-tool-linux-amd64 > manifest-tool && \

goveralls not found on v0.12 release

su-exec: goveralls: No such file or directory

Can't use a big UID

Hi there.

I have a big user id (1069552525) in my ubuntu system. (the uid was assigned by my system administrator). When I try to use go-build image with passing a LOCAL_USER_ID env variable, I get a warning message (the second line):

Starting with UID : 1069552525
adduser: number 1069552525 is not in 0..256000 range

The case in that alpine adduser command has a limit on user id.

The case when it is critical for me:
I try to build the felix project (https://github.com/projectcalico/felix). When I run make build LOCAL_USER_ID=$(shell id -u) is passing to the go-builder (https://github.com/projectcalico/felix/blob/master/Makefile#L215), but this uid is not acceptable for the alpine adduser command and it fallbacks to the 0 uid (root user, as we know). As a result, a vendor directory creates on behalf of the root user and this step https://github.com/projectcalico/felix/blob/master/Makefile#L285 is failed with permission denied error.

I have linked PR that fixes this problem.

Thanks.

Cross-compile calico for arm64 and ppc64le from amd64

Following the merge of #23 , which simplifies building go-build for arm64 on arm64 and ppc64le on ppc64le, this issue is intended as a tracking issue for cross-compile binaries and cross-build of docker images for arm64 and ppc64le (in addition to amd64, of course) from amd64.

Since every CI-as-a-service provides amd64 only, we want to be able to run the entire build and push from amd64.

Related: see #22

cc @caseydavenport @tomdee @fasaxc

Ideas and issues raised by the above people are in a follow-on comment.

please add a link to the dockerfile on hub

https://hub.docker.com/r/calico/go-build/ does not have a link to the dockerfile - of course I found it here, but having a link would be great

Cross compiles are slow due to rebuilding stdlib

Since we do cross compiles using the amd64 image, it'd speed up the downstream builds if we built the standard library for each of the target arches.

s390x Dockerfile needs base image updated

Most of our Dockerfiles have migrated over to Alpine v3.9 and Go 1.12 but s390x is still using s390x/golang:1.11.9-alpine3.8, which means the image is missing some security patches (e.g. this most recent one).

Any s390x users want to take this on? 😄

Move build image to golang v1.13

Relates to Homebrew/homebrew-core#44132

The homebrew is already using go 1.13 to build all the golang formula is possible, seems calicoctl can live well with the latest release.

Maybe it worth taking the base build image to the latest and greatest. :)

go-build build is failing, glibc is the problem.

Docker build of the go-build images are (all architectures) started failing today. This is the error.
<....>
---> Running in 7ae02d7469e8
The command '/bin/sh -c wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub' returned a non-zero code: 5
Makefile:37: recipe for target 'calico/go-build' failed

It can be fixed by updating the url to:
https://cdn.rawgit.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub
( as recommended on https://rawgit.com/)

However, why do we need glibc in the first place, alpine is based libc.musl? I discovered that we have been installing this same package for all arch, yet the lib installed in all cases is for x86-64 so it could not be working on anything except the amd image. Therefore it must not be needed.

I suggest we remove the glibc from all the Dockerfiles.

@fasaxc WDYT

why not upgrade to golang:1.19.5 for all platforms?

the pr 1.19.5 just update amd64.
why not update all others?
like

Dockerfile.s390x,
Dockerfile.ppc64le,
Dockerfile.armv7,
Dockerfile.arm64

New push-manifest target not working on semaphore

# Docker login to hub.docker.com required before running this target as we are using /home/runner/.docker/config.json holds the docker login credentials
docker run -t --entrypoint /bin/sh -v /home/runner/.docker/config.json:/root/.docker/config.json calico/go-build:latest-amd64 -c "/usr/bin/manifest-tool push from-args --platforms linux/amd64,linux/arm64,linux/ppc64le --template calico/go-build:latest-ARCH --target calico/go-build:latest"
FATA[0003] Inspect of image "calico/go-build:latest-arm64" failed with error: manifest unknown: manifest unknown 

Replace manifest-tool with `docker manifest` command

It's still experimental, but only required when releasing go-build. Still probably preferable to grabbing the tool from estep's github.

Pushing out the arm64 images to docker hub?

Are there plans to push out the alternate architecture images to the hub? The Makefile says that the image names are:

• calico/go-build: x86_64
• calico/go-build-arm64: arm64

However, the hub only has the calico/go-build (x86_64) image.

Will you be doing one of:

• Pushing calico/go-build-arm64 out?
• Creating a single multi-arch calico/go-build with a multi-arch manifest?

The docker manifest subcommand hasn't been merged in yet (see this ), but you can use the manifest-tool to do the same thing.