moby / qemu

Docker QEMU mirror with not yet upstreamed patches

The Moby Project

Moby is an open-source project created by Docker to enable and accelerate software containerization.

It provides a "Lego set" of toolkit components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts and professionals to experiment and exchange ideas. Components include container build tools, a container registry, orchestration tools, a runtime and more, and these can be used as building blocks in conjunction with other tools and projects.

Principles

Moby is an open project guided by strong principles, aiming to be modular, flexible and without too strong an opinion on user experience. It is open to the community to help set its direction.

• Modular: the project includes lots of components that have well-defined functions and APIs that work together.
• Batteries included but swappable: Moby includes enough components to build fully featured container systems, but its modular architecture ensures that most of the components can be swapped by different implementations.
• Usable security: Moby provides secure defaults without compromising usability.
• Developer focused: The APIs are intended to be functional and useful to build powerful tools. They are not necessarily intended as end user tools but as components aimed at developers. Documentation and UX is aimed at developers not end users.

Audience

The Moby Project is intended for engineers, integrators and enthusiasts looking to modify, hack, fix, experiment, invent and build systems based on containers. It is not for people looking for a commercially supported system, but for people who want to work and learn with open source code.

Relationship with Docker

The components and tools in the Moby Project are initially the open source components that Docker and the community have built for the Docker Project. New projects can be added if they fit with the community goals. Docker is committed to using Moby as the upstream for the Docker Product. However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.

The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful. The releases are supported by the maintainers, community and users, on a best efforts basis only. For customers who want enterprise or commercial support, Docker Desktop and Mirantis Container Runtime are the appropriate products for these use cases.

---

Legal

Brought to you courtesy of our legal counsel. For more context, please see the NOTICE document in this repo.

Use and transfer of Moby may be subject to certain restrictions by the United States and other governments.

It is your responsibility to ensure that your use and/or transfer does not violate applicable laws.

For more information, please see https://www.bis.doc.gov

Licensing

Moby is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

qemu's Issues

Docker-in-docker (dind) under QEMU emulation

This issue is to track the ability to run docker in docker under QEMU emulation.

QEMU does not implement all the needed syscalls and some syscalls have incorrect or incomplete implementations. Here is a non-exhaustive list of issues, that were discovered:

• incorrect signal handling in clone() when flags=0
• missing pivot_root syscall
• missing copy_file_range syscall
• missing keyctl syscall
• missing memfd_create syscall
• missing F_GET_SEALS and F_ADD_SEALS for fcntl syscall
• incorrect handling when running execve of /proc/self/exe (used by Go reexec packages)

Tonis and I took a stab at it and it resulted in 3 types of patches:

1. Ready-to-PR

• Fix signal handling in clone() when flags=0: tiborvass@dd245fa
  (should be upstreamed)
• Implement pivot_root syscall: tiborvass@671e0b3
  (can be upstreamed)
• exec /proc/self/exe with binfmt assumptions (for reexec):
  tiborvass@cd60352
  This assumes QEMU is setup with binfmt with the preserve (P) option. More difficult to upstream.

2. Stop-gap patches

• copy_file_range and keyctl YOLO implementations:
  tiborvass@0508cf0
  This works because downstream code has fallbacks that work with these patches. These syscalls need proper implementations.
• qemu detection hack:
  tiborvass@a3b4db2
  This can be used by downstream code to disable features that don't work yet.

3. Fix attempts that were not needed/exercised in the end

• Bypassing memfd_create:
  tiborvass@885229e
  tiborvass@7ff4d91
• Implementation of F_GET_SEALS and F_ADD_SEALS for fcntl:
  tiborvass@6617af5

Ping @adamparco @justincormack

Go incompatible with qemu

On an M1 Preview 7 I encountered a go runtime failure for one of of our tools:

runtime: unexpected return pc for runtime.asyncPreempt called from 0x767ec0
stack: frame={sp:0xc0003cbb70, fp:0xc0003cbcf8} stack=[0xc0003ca000,0xc0003cc000)
000000c0003cba70:  00000000004165ff <runtime.heapBits.forwardOrBoundary+111>  00000040279bae00
000000c0003cba80:  0020300000000000  0000004027ba8fff
000000c0003cba90:  0000000000000400  00000040279baf00
000000c0003cbaa0:  0020300000000000  0000004027ba8fff
000000c0003cbab0:  000000c0003cbb30  0000000000417266 <runtime.heapBits.initSpan+310>
000000c0003cbac0:  00000040279bae00  0000000000000100
000000c0003cbad0:  0000004027ba8fff  0000000000000400
000000c0003cbae0:  00000040279baf00  0020300000000000
000000c0003cbaf0:  0000004027ba8fff  0000000000000400
#011/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0002a1d74)
#011/usr/local/go/src/sync/waitgroup.go:130 +0x64
main.main()

Appears that all bets might be off for golang on qemu, it's not a tested/supported platform for go, and there are known incompatibilities:

golang/go#36981 (comment)

Error running Cypress in Docker specifying amd64 platform on Apple silicon

Running this on an Apple M1 machine:

mkdir -p cypress/integration
echo "describe('My Test',()=>{it('',()=>{})});" > cypress/integration/spec.js
docker run --platform linux/amd64 -it -v $PWD:/e2e -w /e2e cypress/included:6.0.0

Fails with this:

qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
[22:1217/170937.770835:FATAL:gpu_data_manager_impl_private.cc(445)] GPU process isn't usable. Goodbye.
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
The Test Runner unexpectedly exited via a exit event with signal SIGTRAP

I'm sorry I can't provide any great insights on this! But after discussion with people in the Docker community Slack, it was decided that I should report it as a bug on here. I hope it's useful info. Happy to test it further.

Qemu only : linux/arm/v7 'Command 'install' is not valid, try 'pecl help'

Resubmitting from docker-library/php#893 .

Slightly odd issue - if I try and build a multiarch image and use a pecl install redis as part of it - then it fails on the linux/arm/v7 build. Example Dockerfile :

FROM php:7.3-apache
RUN pecl install redis && docker-php-ext-enable redis

Running :

docker buildx build --pull --load --no-cache --platform linux/arm/v7 -t test-pecl -f Dockerfile .

Fails with

 => ERROR [2/2] RUN pecl install redis && docker-php-ext-enable redis      1.3s
------                                                                          
 > [2/2] RUN pecl install redis && docker-php-ext-enable redis:                 
#5 0.583 Command 'install' is not valid, try 'pecl help' 

Running :

docker buildx build --pull --load --no-cache --platform linux/amd64 -t test-pecl -f Dockerfile .

Works fine. The docker version in front of me is Docker version 19.03.2, build 6a30dfc. Let me know if there's any more info I can supply or something to dig into :-)

Oh - and running the php:7.3 image on an actual ARM machine also runs pecl install redis a-ok as noted on the original issue linked above.

Issue running elasticsearch on arm64

Running Docker Desktop on Apple M1. Reported on dev preview feedback

seventeen:~() $ docker run -it --rm -e "cluster.name=docker-cluster" -e "bootstrap.memory_lock=true" -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms512m -Xmx512m" docker.elastic.co/elasticsearch/elasticsearch:7.4.0
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Exception in thread "main" java.io.IOException: Cannot run program "/usr/share/elasticsearch/jdk/bin/java": error=0, Failed to exec spawn helper.
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
	at org.elasticsearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:100)
	at org.elasticsearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:79)
	at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:57)
	at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:89)
Caused by: java.io.IOException: error=0, Failed to exec spawn helper.
	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:319)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:250)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)

when i do it in context of my product, i get a segfault from qemu:

seventeen:app(develop!) $ ./app_dev.sh logs -f elasticsearch
Attaching to app_elasticsearch_1
elasticsearch_1  | qemu: uncaught target signal 11 (Segmentation fault) - core dumped
elasticsearch_1  | Exception in thread "main" java.io.IOException: Cannot run program "/usr/share/elasticsearch/jdk/bin/java": error=0, Failed to exec spawn helper.
elasticsearch_1  | 	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)

Issue running selenium/standalone-chrome on arm64

Running Docker Desktop on Apple M1. Reported on dev preview feedback

i just tried some containers, most of them work fine for me. one that's though misbehaving is the selenium one...
it's run like described in this repo: https://github.com/SeleniumHQ/docker-selenium via docker run -d -p 4444:4444 -p 5900:5900 -v /dev/shm:/dev/shm selenium/standalone-chrome:4.0.0-beta-1-prerelease-20201208 . connecting with a vnc viewer works well, when trying to launch the chrome application (right click on desktop -> applications -> network -> web browsing -> google chrome) there's nothing happening though.
log file in /var/log/supervisor/xvfb-stdout.log reports this only. just wanted to give you a shout, don't know anything specific about that image but i'd imagine it's used by some people for selenium testing.

Trace/breakpoint trap
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
[387:387:1212/163414.315133:ERROR:nacl_fork_delegate_linux.cc(329)] Bad NaCl helper startup ack (0 bytes)
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
[360:395:1212/163414.401966:ERROR:file_path_watcher_linux.cc(315)] inotify_init() failed: Function not implemented (38)
[360:399:1212/163414.827603:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
qemu: unknown option 'type=utility'
[360:413:1212/163415.286343:ERROR:bus.cc(393)] Failed to connect to the bus: Address does not contain a colon
[360:413:1212/163415.286761:ERROR:bus.cc(393)] Failed to connect to the bus: Address does not contain a colon
[360:413:1212/163415.493547:ERROR:bus.cc(393)] Failed to connect to the bus: Address does not contain a colon
[360:413:1212/163415.494039:ERROR:bus.cc(393)] Failed to connect to the bus: Address does not contain a colon
[360:397:1212/163415.748544:FATAL:gpu_data_manager_impl_private.cc(445)] GPU process isn't usable. Goodbye.
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped

Possibly related: https://bugs.launchpad.net/qemu/+bug/1890545

Error while building/running Java application on Docker Mac for ARM64

See docker/for-mac#3646 for details. Reported against Docker Desktop but the bug is probably in the qemu code.

Issue running yandex/clickhouse-server on arm64

Running Docker Desktop on Apple M1. Reported on dev preview feedback

docker run --ulimit nofile=262144:262144 yandex/clickhouse-server --platform amd64
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Instruction check fail. The CPU does not support SSSE3 instruction set.
Instruction check fail. The CPU does not support SSSE3 instruction set.
Instruction check fail. The CPU does not support SSSE3 instruction set.
Instruction check fail. The CPU does not support SSSE3 instruction set.
Instruction check fail. The CPU does not support SSSE3 instruction set.
/usr/bin/dirname: missing operand

Issue running varnishd

varnishd fails to run the VCC compiler of qemu x86_64 on my M1 mini running Developer Preview 7 (also happened in 5):

varnishd[5351]: -I file CLI command failed (106)
Running VCC-compiler failed, signal 5
VCL compilation failed

Reproducible withvarnishd -C -f /etc/varnish/default.vcl where default.vcl is any valid VCL file.

Issue running rabbitmq:3.7.14-management on arm64

Running under Docker for Mac, Preview5

% docker run --platform linux/amd64 rabbitmq:3.7.14-management
Unable to find image 'rabbitmq:3.7.14-management' locally
3.7.14-management: Pulling from library/rabbitmq
6abc03819f3e: Pull complete 
05731e63f211: Pull complete 
0bd67c50d6be: Pull complete 
78d7bb64d81b: Pull complete 
fa37476193df: Pull complete 
5293f1f07690: Pull complete 
5f4305b1c03c: Pull complete 
d4b9a7eb3d27: Pull complete 
53924b30a970: Pull complete 
aae6654997e3: Pull complete 
7b042b8ea02a: Pull complete 
Digest: sha256:31495751c19e5c315ba71e3b92427a8836f313f04fc7fb5a56e64bd2e22ba463
Status: Downloaded newer image for rabbitmq:3.7.14-management
sigaltstack: Invalid argument
qemu: uncaught target signal 6 (Aborted) - core dumped
sigaltstack: Invalid argument
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted

qemu 4.1 update

This is nearly out, we should look to update.

Errors while running container built on Docker Mac for armhf

Consider the following Dockerfile to be built for armhf as an example:

FROM debian:buster-slim

env USER root

RUN apt-get update \
 && apt-get install --no-install-recommends -y tightvncserver xfonts-base \
 && mkdir /root/.vnc \
 && echo test | vncpasswd -f >/root/.vnc/passwd \
 && chmod 600 /root/.vnc/passwd

CMD vncserver

If you then try to run the resulting image, either on Docker Mac or on Raspberry Pi, VNC won't start, complaining:

Fatal server error:
could not open default font 'fixed'

If the image were built on Raspberry Pi, there would be no error and VNC would start successfully. The image built on Mac can be fixed on Raspberry Pi by manually executing dpkg-reconfigure xfonts-base before launching vncserver (might be somehow related to fontconfig.) There is no fix that works in QEMU, however. ARM64 doesn't suffer from this issue.

Sorry for not having a simpler example, this one was taken from the real world scenario.

Bullseye is unusable on qemu-ed arm64 currently

Qemu core dumps in a number of occasions.

Below is a simple example installing libc-bin.

This is working fine with debian buster, so, something with bullseye (linkage?) is triggering (more of) these issues.

This happens in Docker for mac 3.3.3, and also on a linux host.

It seems this project (moby/qemu) is way behind upstream. Is there still a strong reason to rely on this fork?

docker run -ti docker.io/library/debian:bullseye-slim@sha256:9f8ab8961b279bcad1a1711fff7e65d1a579845fe57b8605798394f23fe8e902 bash

apt install libc-bin
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Recommended packages:
  manpages
The following packages will be upgraded:
  libc-bin
1 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Need to get 742 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main arm64 libc-bin arm64 2.31-12 [742 kB]
Fetched 742 kB in 1s (792 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 6953 files and directories currently installed.)
Preparing to unpack .../libc-bin_2.31-12_arm64.deb ...
Unpacking libc-bin (2.31-12) over (2.31-11) ...
Setting up libc-bin (2.31-12) ...
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)

Doing some test with latest upstream qemu (for a buildkit image).

QEMU issue with curl and SSL certificates on Debian

FROM    debian:buster-slim
RUN    apt-get update && apt-get install -y --no-install-recommends curl ca-certificates
RUN    curl https://www.google.com

docker buildx build --platform="linux/arm64,linux/amd64,linux/386" -f Dockerfile . <- works as expected

docker buildx build --platform="linux/arm/v7" -f Dockerfile . <- fails with curl: (60) SSL certificate problem: unable to get local issuer certificate

Same problem with linux/arm/v6.

This problem is not visible on a native arm machine - so, I'm guessing it's a QEMU issue.

Error running Elasticsearch 6.x

When running the container elasticsearch:6.4.0@sha256:2244676001fa9b12012ec1f7a51526a8d7384fcb02674e5ae29b6e637e363908

The following is reported
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=0
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

Segfault on python math.exp in ppc64le ubuntu:18.04 image

Cross-posted to StackOverflow and Ubuntu bug reporter.

When calculating the exponential of a negative number in python 3.6 with the Ubuntu 18.04 Docker image (ppcle64), I receive a segfault.

python3 -c "import math; math.exp(-1)"
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault

I am running this in Docker for Mac. Here are the steps to reproduce the error.

docker run --rm -it --platform linux/ppc64le ubuntu:18.04
apt-get update -qq && apt-get install -y -qq python3
python3 -c "import math; print(math.exp(0))" # 1.0
python3 -c "import math; print(math.exp(-0))" # 1.0
python3 -c "import math; print(math.exp(1))" # 2.718281828459045
python3 -c "import math; print(math.exp(-1))"

This error does not happen when using the debian:10 or alpine images. It also does not happen on a real ppc64le machine:

[jakubk@service0002 ~]$ uname -m
ppc64le
[jakubk@service0002 ~]$ python3 -c "import math; print(math.exp(-1))"
0.36787944117144233

Here is the output of docker version on my machine:

Client: Docker Engine - Community
 Version: 19.03.8
 API version: 1.40
 Go version: go1.12.17
 Git commit: afacb8b
 Built: Wed Mar 11 01:21:11 2020
 OS/Arch: darwin/amd64
 Experimental: true

Server: Docker Engine - Community
 Engine:
  Version: 19.03.8
  API version: 1.40 (minimum version 1.12)
  Go version: go1.12.17
  Git commit: afacb8b
  Built: Wed Mar 11 01:29:16 2020
  OS/Arch: linux/amd64
  Experimental: true
 containerd:
  Version: v1.2.13
  GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version: 1.0.0-rc10
  GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version: 0.18.0
  GitCommit: fec3683

Could this be a bug in Docker's qemu?