Discrepancy between crun and runc when disallowing access by default to devices with cgroups v1 about crun

[giuseppe]

I tried reproducing it using a Podman container created with podman run --privileged -v /root:/root --rm -ti fedora:39 bash but in both cases the inner container does not create a cgroup.

how have you created the outer Docker container?

Can you please verify the cgroup of the container process cat /proc/$PID_CGROUP/cgroup from the host in both cases?

[giuseppe]

I tried reproducing it using a Podman container created with podman run --privileged -v /root:/root --rm -ti fedora:39 bash but in both cases the inner container does not create a cgroup.

EDIT: I was looking at the wrong thing.

They both create a cgroup, but I see the same configuration:

# cat /sys/fs/cgroup/devices/machine.slice/libpod-bcf881874d62ce2cf2226eb8598e0a1dd2bc4d1ea96c9fa9e577872720aca34c.scope/container/runc-container/devices.list
b *:* m
c *:* m
c 1:3 rwm
c 1:5 rwm
c 1:7 rwm
c 1:8 rwm
c 1:9 rwm
c 5:0 rwm
c 5:2 rwm
c 10:200 rwm
c 136:* rwm

# cat /sys/fs/cgroup/devices/machine.slice/libpod-bcf881874d62ce2cf2226eb8598e0a1dd2bc4d1ea96c9fa9e577872720aca34c.scope/container/crun-container/devices.list
c *:* m
b *:* m
c 1:3 rwm
c 1:8 rwm
c 1:7 rwm
c 5:0 rwm
c 1:5 rwm
c 1:9 rwm
c 5:1 rwm
c 136:* rwm
c 5:2 rwm

[Madeeks]

Hi @giuseppe, thanks for your reply.
The outer container is created with a command like

> docker run --rm -it -v $(pwd):/oci-bundle --privileged fedora:39 bash

The Docker config I'm running on my laptop is

> docker info
Client:
 Version:    24.0.7-ce
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.11.2
    Path:     /usr/lib/docker/cli-plugins/docker-buildx

Server:
 Containers: 6
  Running: 1
  Paused: 0
  Stopped: 5
 Images: 274
 Server Version: 24.0.7-ce
 Storage Driver: btrfs
  Btrfs: 
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 oci runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8e4b0bde866788eec76735cc77c4720144248fb7
 runc version: v1.1.10-0-g18a0cb0f32bc
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: builtin
 Kernel Version: 5.14.21-150400.24.81-default
 Operating System: openSUSE Leap 15.4
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.06GiB
 Name: carbon
 ID: 7IY2:7RUT:5VJZ:QQKN:S75T:CZBI:VM4J:UHRR:JT5K:75EH:FCU5:IKA7
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: madeeks
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

I thought the behavior could be related to the cgroup driver, but I obtained the same results (i.e. different device allowed lists) when using runc/crun on a Ubuntu 20.04 VM using Docker with a cgroupfs driver.
I can also reproduce the results with Podman using both systemd and cgroupfs arguments to --cgroup-manager (in this case I ran rootful Podman since the option is not supported with rootless Podman and cgroups v1).

I'll keep digging in how my container engines are setting up cgroups in the outer containers.

[giuseppe]

I thought the behavior could be related to the cgroup driver, but I obtained the same results (i.e. different device allowed lists) when using runc/crun on a Ubuntu 20.04 VM using Docker with a cgroupfs driver.

do you get the same results with runc and crun?

[Madeeks]

Apologies for the ambiguous wording.

I get the same results on my OpenSUSE laptop and the Ubuntu 20.04 VM.
That is, on both systems the device cgroups produced by runc and crun are different (when the runtimes are started inside a Docker container).

The Docker cgroup driver is different between the 2 platforms: systemd on the OpenSUSE laptop, cgroupfs on the Ubuntu VM.

If I create the outside container with Podman, I still obtain crun/runc device cgroup differences.
This happens even when using different values for Podman's --cgroup-manager option.