seifrajhi / nerdctl

contaiNERD CTL - Docker-compatible CLI for containerd, with support for Compose, Rootless, eStargz, OCIcrypt, IPFS, ...

[⬇️ Download] [📖 Command reference] [❓FAQs & Troubleshooting] [📚 Additional documents]

nerdctl: Docker-compatible CLI for containerd

nerdctl is a Docker-compatible CLI for containerd.

✅ Same UI/UX as docker

✅ Supports Docker Compose (nerdctl compose up)

✅ [Optional] Supports rootless mode, without slirp overhead (bypass4netns)

✅ [Optional] Supports lazy-pulling (Stargz, Nydus, OverlayBD)

✅ [Optional] Supports encrypted images (ocicrypt)

✅ [Optional] Supports P2P image distribution (IPFS) (*1)

✅ [Optional] Supports container image signing and verifying (cosign)

nerdctl is a non-core sub-project of containerd.

*1: P2P image distribution (IPFS) is completely optional. Your host is NOT connected to any P2P network, unless you opt in to install and run IPFS daemon.

Examples

Basic usage

To run a container with the default bridge CNI network (10.4.0.0/24):

# nerdctl run -it --rm alpine

To build an image using BuildKit:

# nerdctl build -t foo /some-dockerfile-directory
# nerdctl run -it --rm foo

To build and send output to a local directory using BuildKit:

# nerdctl build -o type=local,dest=. /some-dockerfile-directory

To run containers from docker-compose.yaml:

# nerdctl compose -f ./examples/compose-wordpress/docker-compose.yaml up

See also ./examples/compose-wordpress.

Debugging Kubernetes

To list local Kubernetes containers:

# nerdctl --namespace k8s.io ps -a

To build an image for local Kubernetes without using registry:

# nerdctl --namespace k8s.io build -t foo /some-dockerfile-directory
# kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: foo
spec:
  containers:
    - name: foo
      image: foo
      imagePullPolicy: Never
EOF

To load an image archive (docker save format or OCI format) into local Kubernetes:

# nerdctl --namespace k8s.io load < /path/to/image.tar

To read logs (experimental):

# nerdctl --namespace=k8s.io ps -a
CONTAINER ID    IMAGE                                                      COMMAND                   CREATED          STATUS    PORTS    NAMES
...
e8793b8cca8b    registry.k8s.io/coredns/coredns:v1.9.3                     "/coredns -conf /etc…"    2 minutes ago    Up                 k8s://kube-system/coredns-787d4945fb-mfx6b/coredns
...

# nerdctl --namespace=k8s.io logs -f e8793b8cca8b
[INFO] plugin/reload: Running configuration SHA512 = 591cf328cccc12bc490481273e738df59329c62c0b729d94e8b61db9961c2fa5f046dd37f1cf888b953814040d180f52594972691cd6ff41be96639138a43908
CoreDNS-1.9.3
linux/amd64, go1.18.2, 45b0a11
...

Rootless mode

To launch rootless containerd:

$ containerd-rootless-setuptool.sh install

To run a container with rootless containerd:

$ nerdctl run -d -p 8080:80 --name nginx nginx:alpine

See ./docs/rootless.md.

Install

Binaries are available here: https://github.com/containerd/nerdctl/releases

In addition to containerd, the following components should be installed:

• CNI plugins: for using nerdctl run.
  • v1.1.0 or later is highly recommended. Older versions require extra CNI isolation plugin for isolating bridge networks (nerdctl network create).
• BuildKit (OPTIONAL): for using nerdctl build. BuildKit daemon (buildkitd) needs to be running. See also the document about setting up BuildKit.
  • v0.11.0 or later is highly recommended. Some features, such as pruning caches with nerdctl system prune, do not work with older versions.
• RootlessKit and slirp4netns (OPTIONAL): for Rootless mode
  • RootlessKit needs to be v0.10.0 or later. v0.14.1 or later is recommended.
  • slirp4netns needs to be v0.4.0 or later. v1.1.7 or later is recommended.

These dependencies are included in nerdctl-full-<VERSION>-<OS>-<ARCH>.tar.gz, but not included in nerdctl-<VERSION>-<OS>-<ARCH>.tar.gz.

Brew

On Linux systems you can install nerdctl via brew:

brew install nerdctl

This is currently not supported for macOS. The section below shows how to install on macOS using brew.

macOS

Lima project provides Linux virtual machines for macOS, with built-in integration for nerdctl.

$ brew install lima
$ limactl start
$ lima nerdctl run -d --name nginx -p 127.0.0.1:8080:80 nginx:alpine

FreeBSD

See ./docs/freebsd.md.

Windows

• Linux containers: Known to work on WSL2
• Windows containers: experimental support for Windows (see below for features that are currently known to work)

Docker

To run containerd and nerdctl inside Docker:

docker build -t nerdctl .
docker run -it --rm --privileged nerdctl

Motivation

The goal of nerdctl is to facilitate experimenting the cutting-edge features of containerd that are not present in Docker (see below).

Note that competing with Docker is not the goal of nerdctl. Those cutting-edge features are expected to be eventually available in Docker as well.

Also, nerdctl might be potentially useful for debugging Kubernetes clusters, but it is not the primary goal.

Features present in nerdctl but not present in Docker

Major:

• On-demand image pulling (lazy-pulling) using Stargz/Nydus/OverlayBD Snapshotter: nerdctl --snapshotter=stargz|nydus|overlaybd run IMAGE .
• Image encryption and decryption using ocicrypt (imgcrypt): nerdctl image (encrypt|decrypt) SRC DST
• P2P image distribution using IPFS: nerdctl run ipfs://CID . P2P image distribution (IPFS) is completely optional. Your host is NOT connected to any P2P network, unless you opt in to install and run IPFS daemon.
• Recursive read-only (RRO) bind-mount: nerdctl run -v /mnt:/mnt:rro (make children such as /mnt/usb to be read-only, too). Requires kernel >= 5.12, and crun >= 1.4 or runc >= 1.1 (PR #3272).
• Cosign integration: nerdctl pull --verify=cosign and nerdctl push --sign=cosign, and in Compose
• Accelerated rootless containers using bypass4netns: nerdctl run --label nerdctl/bypass4netns=true

Minor:

• Namespacing: nerdctl --namespace=<NS> ps . (NOTE: All Kubernetes containers are in the k8s.io containerd namespace regardless to Kubernetes namespaces)
• Exporting Docker/OCI dual-format archives: nerdctl save .
• Importing OCI archives as well as Docker archives: nerdctl load .
• Specifying a non-image rootfs: nerdctl run -it --rootfs <ROOTFS> /bin/sh . The CLI syntax conforms to Podman convention.
• Connecting a container to multiple networks at once: nerdctl run --net foo --net bar
• Running FreeBSD jails.
• Better multi-platform support, e.g., nerdctl pull --all-platforms IMAGE
• Applying an (existing) AppArmor profile to rootless containers: nerdctl run --security-opt apparmor=<PROFILE>. Use sudo nerdctl apparmor load to load the nerdctl-default profile.

Trivial:

• Inspecting raw OCI config: nerdctl container inspect --mode=native .

Similar tools

• ctr: incompatible with Docker CLI, and not friendly to users. Notably, ctr lacks the equivalents of the following nerdctl commands:

  • nerdctl run -p <PORT>
  • nerdctl run --restart=always --net=bridge
  • nerdctl pull with ~/.docker/config.json and credential helper binaries such as docker-credential-ecr-login
  • nerdctl logs
  • nerdctl build
  • nerdctl compose up

• crictl: incompatible with Docker CLI, not friendly to users, and does not support non-CRI features

• k3c v0.2 (abandoned): needs an extra daemon, and does not support non-CRI features

• Rancher Kim (nee k3c v0.3): needs Kubernetes, and only focuses on image management commands such as kim build and kim push

• PouchContainer (abandoned?): needs an extra daemon

Developer guide

nerdctl is a containerd non-core sub-project, licensed under the Apache 2.0 license. As a containerd non-core sub-project, you will find the:

• Project governance,
• Maintainers,
• and Contributing guidelines

information in our containerd/project repository.

Compiling nerdctl from source

Run make && sudo make install.

See the header of go.mod for the minimum supported version of Go.

Using go install github.com/containerd/nerdctl/cmd/nerdctl is possible, but unrecommended because it does not fill version strings printed in nerdctl version

Test suite

Running unit tests

Run go test -v ./pkg/...

Running integration test suite against nerdctl

Run go test -exec sudo -v ./cmd/nerdctl/... after make && sudo make install.

For testing rootless mode, -exec sudo is not needed.

To run tests in a container:

docker build -t test-integration --target test-integration .
docker run -t --rm --privileged test-integration

Running integration test suite against Docker

Run go test -exec sudo -v ./cmd/nerdctl/... -args -test.target=docker to ensure that the test suite is compatible with Docker.

Contributing to nerdctl

Lots of commands and flags are currently missing. Pull requests are highly welcome.

Please certify your Developer Certificate of Origin (DCO), by signing off your commit with git commit -s and with your real name.

Command reference

Moved to ./docs/command-reference.md

Additional documents

Configuration guide:

• ./docs/config.md: Configuration (/etc/nerdctl/nerdctl.toml, ~/.config/nerdctl/nerdctl.toml)
• ./docs/registry.md: Registry authentication (~/.docker/config.json)

Basic features:

• ./docs/compose.md: Compose
• ./docs/rootless.md: Rootless mode
• ./docs/cni.md: CNI for containers network
• ./docs/build.md: nerdctl build with BuildKit

Advanced features:

• ./docs/stargz.md: Lazy-pulling using Stargz Snapshotter
• ./docs/nydus.md: Lazy-pulling using Nydus Snapshotter
• ./docs/overlaybd.md: Lazy-pulling using OverlayBD Snapshotter
• ./docs/ocicrypt.md: Running encrypted images
• ./docs/gpu.md: Using GPUs inside containers
• ./docs/multi-platform.md: Multi-platform mode

Experimental features:

• ./docs/experimental.md: Experimental features
• ./docs/freebsd.md: Running FreeBSD jails
• ./docs/ipfs.md: Distributing images on IPFS
• ./docs/builder-debug.md: Interactive debugging of Dockerfile

Implementation details:

• ./docs/dir.md: Directory layout (/var/lib/nerdctl)

Misc:

• ./docs/faq.md: FAQs and Troubleshooting