Nydus-snapshotter is a non-core sub-project of containerd.
Pulling and unpacking OCI container image are time consuming when start a container. Nydus is a vendor-neutral project aiming at mitigating the problem. It designs a new container image oriented and optimized file system format with capability of on-demand read. For instructions on how to build nydus container image, please refer to nydusify conversion tool.
Just invoke make
and find output binary ./bin/containerd-nydus-grpc
make
Containerd provides a general mechanism to exploit different types of snapshotters. Please ensure your containerd's version is beyond 1.4.0.
Add nydus as a proxy plugin into containerd's configuration file which may be located at /etc/containerd/config.toml
.
# The `address` field specifies through which socket snapshotter and containerd communicate.
[proxy_plugins]
[proxy_plugins.nydus]
type = "snapshot"
address = "/run/containerd-nydus/containerd-nydus-grpc.sock"
Restart your containerd service making the change take effect. Assume that your node is systemd based, restart the service as below:
systemctl restart containerd
Find a suitable nydusd
release for you from nydus releases page.
nydusd-fusedev
is FUSE userspace daemon handling linux kernel fuse requests from /dev/fuse
frontend.
nydusd-virtiofs
is a virtiofs daemon handling guest kernel fuse requests.
Nydus is configured by a json file which is required now. Because nydus container images are likely stored in a registry, where auth has to be provided. Please follow instructions to configure nydus configure nydus making it work properly in your environment.
Nydus-snapshotter is implemented as a proxy plugin (containerd-nydus-grpc
) for containerd.
Assume your server systemd based, install nydus-snapshotter:
Note: nydusd
and nydus-image
should be found from $PATH.
make install
systemctl restart containerd
Or you can start nydus-snapshotter manually.
# `nydusd-path` is the path to nydusd binary
# `address` is the domain socket that you configured in containerd configuration file
# `root` is the path to nydus snapshotter
# `config-path` is the path to nydus configuration file
$ ./containerd-nydus-grpc \
--config-path /etc/nydusd-config.json \
--shared-daemon \
--log-level info \
--root /var/lib/containerd/io.containerd.snapshotter.v1.nydus \
--cache-dir /var/lib/nydus/cache \
--address /run/containerd-nydus/containerd-nydus-grpc.sock \
--nydusd-path /usr/local/bin/nydusd \
--nydusimg-path /usr/local/bin/nydus-image \
--log-to-stdout
Utilize containerd's ctr
CLI command to validate if nydus-snapshotter is set up successfully.
$ ctr -a /run/containerd/containerd.sock plugin ls
TYPE ID PLATFORMS STATUS
io.containerd.snapshotter.v1 nydus - ok
Containerd can start container with specified snapshotter, so legacy method like nerdctl
or ctr
needs to specify the nydus
snapshotter when start container.
A CLI tool ctr-remote is alongside. Use nydus ctr-remote
to pull nydus image or start container based on nydus image.
$ sudo ctr-remote image rpull ghcr.io/dragonflyoss/image-service/nginx:nydus-latest
fetching sha256:75002dfe... application/vnd.oci.image.manifest.v1+json
fetching sha256:5a42e21c... application/vnd.oci.image.config.v1+json
fetching sha256:eb1af2e1... application/vnd.oci.image.layer.v1.tar+gzip
# Start container by `ctr-remote`
$ sudo ctr-remote run --snapshotter nydus ghcr.io/dragonflyoss/image-service/nginx:nydus-latest awesome-nydus
# Start container by `nerdctl`
nerdctl --snapshotter nydus run ghcr.io/dragonflyoss/image-service/nginx:nydus-latest
NOTE: A potential drawback using CRI is that we can hardly specify snapshotter to nydus-snapshotter
. So we have to change containerd's default snapshotter in its configuration file and enable snapshot annotations like below:
[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter = "nydus"
disable_snapshot_annotations = false
Use crictl
to debug starting container via Kubernetes CRI. Dry run steps of using crictl
can be found in documents.
We can also use the nydus-snapshotter
container image when we want to put nydus staff inside a container. See the nydus-snapshotter exmple for how to setup and use it.
Nydus aims to form a vendor-neutral opensource image distribution solution to all communities. Questions, bug reports, technical discussion, feature requests and contribution are always welcomed!
Join our Slack workspace