English | 繁體中文

Scripts and notes for the tiny chestnut eGPU dock. Flash it between its two firmwares, keep eGPU mode working on Linux, and find out how much power your supply can actually deliver to the card.

chestnut is an eGPU dock from comma and the tiny corp: a PCIe x4 slot behind an ASMedia ASM2464PD USB4 bridge, 12 V in over XT60 or ATX, open-source firmware. Comma sells the bare dock as "tiny chestnut" and a bundle with an RX 9060. Because the firmware is open, the dock can be either a normal USB4 eGPU enclosure or, with tinygrad's firmware, a USB3 device that tinygrad drives directly, so the GPU works over plain USB3 on machines without USB4.

• shop: https://comma.ai/shop/chestnut
• firmware source: https://github.com/tinygrad/asm2464pd-firmware (submodule here, pinned to ed4e39b7, the version the dock ships with)

Tested only on one machine: CachyOS, kernel 7.2.2, KDE Wayland, ASUS ROG Flow X13 GV301RE (Ryzen 6900HS, Radeon 680M, RTX 3050 Ti), Sapphire Pulse RX 9060 XT OC 16 GB. The port quirks, amdgpu workarounds and power numbers below come from that setup and may not apply to yours. The WSL section is what usbipd-win and WSL2 allow, not something I have run end to end. Flashing firmware and pushing a supply to its limit can brick the dock or damage hardware. Your risk.

• Linux, any distro: everything. The dock-side scripts need bash, Docker (or python3 with venv support, git and libusb for the venv route) and sysfs; lsusb is optional. The host-side scripts need systemd (the udev rule launches systemd-run), amdgpu and setpci. The udev rules use TAG+="uaccess"; on a system without systemd-logind use MODE="0666" instead.
• Windows: the dock-side scripts through WSL2 and usbipd-win, see below. Nothing runs on Windows itself. The .gitattributes keeps the scripts LF even when cloned with Git for Windows.
• macOS: not supported. Docker Desktop on macOS has no USB passthrough and there is no udev. A venv with libusb from Homebrew might work for hwmon and flashing, untested.

Which firmware is in the dock's flash decides everything.

On stock firmware the dock enumerates as add1:0001 "USB 3.2 PCIe TinyEnclosure" (Thunderbolt name "Gopod Group Limited USB4 NVMe SSD Pro Enclosure"). On a port without USB4 it falls back to a plain USB3 device. That is how you flash back.

• Docker, or a local Python venv (see below). chestnut.sh, hwmon.sh and powertest.sh are wrappers around some Python. By default they run it in the chestnut-fw container, which holds tinygrad (e4_flash.py imports its libusb bindings, the stress test runs on it), clang (tinygrad's AMD runtime shells out to it), libusb and pyusb, and sdcc for build. The host then needs Docker, nothing else. tinygrad downloads the GPU's firmware blobs on first use; they are kept in a Docker volume named chestnut-cache. The scripts in scripts/host/ are plain shell and do not need Docker, except kfd_stress.sh, which runs stress.py in the container.
• The dock on a USB port for flashing and monitoring. A USB4 port and a real power supply for eGPU mode.
• Windows: WSL2 and usbipd-win.

git clone --recursive <this repo> chestnut-tools   # the submodule is needed for stock/flash/build, not for recover or hwmon
cd chestnut-tools
./chestnut.sh image                                # build the container once

The container by hand, if you would rather not go through chestnut.sh:

docker build -t chestnut-fw .                                        # same as ./chestnut.sh image
docker build -t chestnut-fw --build-arg TINYGRAD_REF=master .        # try a newer tinygrad
docker run --rm -it --privileged -v /dev/bus/usb:/dev/bus/usb -v "$PWD":/repo -v chestnut-cache:/root/.cache -w /repo chestnut-fw bash

Inside: the repo is at /repo, the firmware source at /repo/firmware/asm2464pd-firmware, tinygrad and sdcc are on the path. --privileged plus /dev/bus/usb is what gives the container the dock; chestnut-cache keeps tinygrad's downloaded GPU firmware between runs. For eGPU-mode compute add --device /dev/kfd --device /dev/dri instead of the USB mount (see kfd_stress.sh).

Without Docker:

./chestnut.sh venv                                 # .venv/ with tinygrad, pyusb, pyftdi. From now on the scripts run locally.
sudo scripts/host/install.sh                       # also installs the dock's USB access rule (70-chestnut-usb.rules); re-plug after

recover still runs as root (openpilot's flasher insists), the wrapper uses sudo for it. powertest.sh needs clang on the host (tinygrad's AMD runtime shells out to it). build needs sdcc installed on the host. CHESTNUT_DOCKER=1 forces the container even when a venv exists, CHESTNUT_LOCAL=1 forces local with whatever python3 is.

For flashing, in either direction, put the dock on a non-USB4 port with an ordinary USB-C cable. Flashing talks to the dock as a USB device: the tinygrad firmware is one, and the stock firmware falls back to one only when the port does not offer USB4. On a USB4 port the stock firmware brings up the PCIe tunnel instead and there is nothing to flash. The USB4 port and the USB4 cable are only for eGPU mode.

./chestnut.sh stock       # eGPU mode. Needs the tinygrad firmware running.
./chestnut.sh chestnut    # chestnut mode. From stock firmware (non-USB4 port), or from the ROM bootloader.

After flashing to chestnut mode: power-cycle the dock. Unplug USB-C, cut the 12 V input, and leave both off for a full 10 seconds before powering up and plugging back in. The board carries large capacitors, so a quick off/on does not actually reset the chip and the old firmware keeps running.

After flashing to stock: install the udev rule (below), then shut the laptop down completely, attach the powered dock to the USB4 port, and boot. Do not rely on hot-plugging it. On this laptop the USB4 port's high-speed side wedges after long uptime; a dock plugged into a wedged port produces no events at all, and the only fix we know is a cold boot with the dock attached. Shutting down first costs nothing and also matches the removal rule (see quirks).

Flashing works on a USB2 link too. The tinygrad firmware answers E4 reads of at most 64 bytes there, so chestnut.sh flash runs the upstream flasher through scripts/dock/e4_flash.py, which splits reads into 64-byte pieces and verifies with two full read-backs. It prints a note when the link is USB2. We hit that when the laptop's USB4 controller was wedged after long uptime: the dock linked at USB2 on any cable until a cold boot, after which it linked at 10 Gbit/s again. tinygrad compute still works over USB2, transfers are just slower.

WSL has no USB. Forward the dock from Windows, then the scripts work as on Linux.

# Windows, admin PowerShell, once
winget install usbipd
usbipd list                        # 3801:0001 = tinygrad fw, add1:0001 = stock fw. Non-USB4 port, ordinary USB-C cable
usbipd bind --busid <BUSID>
# every session
usbipd attach --wsl --busid <BUSID>

# WSL
lsusb | grep -iE '3801|add1'       # must show the dock
./chestnut.sh image
./chestnut.sh stock                # etc.

• Docker has to be the Linux daemon inside WSL (Docker Desktop with the WSL backend, or dockerd in the distro). The scripts need --privileged and /dev/bus/usb.
• USB/IP is slow for bulk transfers. Fine for flashing and hwmon, too slow for tinygrad compute.
• In eGPU mode Windows owns the card. The AMD Windows driver handles it like any Thunderbolt eGPU. WSL never sees the PCI device, so nothing in scripts/host/ applies.
• To flash back from stock firmware the dock has to be on a non-USB4 port. In USB4 mode there is no USB device to talk to.

chestnut.sh, hwmon.sh and powertest.sh share one container and scripts/dock/common.sh. hwmon and powertest need chestnut mode.

./hwmon.sh [interval] prints the dock's input V/A/W and fault flag from the INA231, with running min V and max W.

Flash layout: 0x000-0x0FF is the config page (USB and Thunderbolt identity, PD settings, identical on every dock), 0x100 onwards is the firmware image (length, body, magic, checksum, CRC). flash never touches the config page. recover rewrites the whole chip from firmware/config-page.bin and the tinygrad image. A chip with a broken image comes up as the ROM bootloader 174c:2463.

Images in firmware/ are checked against SHA256SUMS before flashing. tinygrad-ed4e39b7.bin is the shipped tinygrad build (same bytes as openpilot ships; ed4e39b7 is the source commit, not a serial number). stock-AS_USB4_231204_85_00_00.bin is the ASMedia image. chestnut.sh build rebuilds the tinygrad image from the submodule. The size differs slightly because of the compiler, the version string is the same. stock, flash and build need the submodule because e4_flash.py lives there. recover, hwmon.sh and powertest.sh do not.

Run this in chestnut mode, where a supply collapse only drops a USB device. It steps the card's power limit (AM_POWER_LIMIT, which sets the SMU PPT) from 60 to 170 W, runs an 8192x8192 fp16 matmul at each step, and logs the card's own power reading and the dock's INA231: card W, input min V, input avg W, input peak W, sag, TFLOP/s. It stops when the supply folds, when the rail sags more than 15 %, or when card power stops going up. The SAFE limit is the highest step where the card hit its limit with the rail within 8 % of idle.

./powertest.sh                                # --from 60 --to 170 --step 10 --secs 20 --n 8192
./powertest.sh --from 200 --to 200 --secs 30  # one run at the card's max

Numbers from the RX 9060 XT: input power averages about 1.05x card power, peaks about 1.9x.

Both fold under real load (diffusion, uncapped games). Size the supply for the peaks, about 2x the card power you want. For full card power use the ATX adapter that comes with the dock or a 12 V supply of 30 A or more. Stock firmware does not drive ATX PS_ON, jumper it. tinygrad firmware does. Over USB3 throughput is host-bound at about 12.6 TFLOP/s at any cap, so a 60 W cap costs nothing in chestnut mode.

install.sh installs two udev rules: the eGPU rule for stock-firmware mode, and the dock's USB access rule so your user can talk to it in chestnut mode without Docker. With Docker the container runs privileged and the second rule is not needed, but it is harmless.

sudo scripts/host/install.sh             # GPU 1002:7590, cap 120 W
sudo scripts/host/install.sh --cap 170   # --cap none for the firmware default, --gpu VVVV:DDDD for another card
scripts/host/install.sh --steam          # as your user. Steam .desktop override, see below
sudo scripts/host/install.sh --uninstall

The udev rule does three things when the card appears:

1. Keeps runtime PM off. amdgpu puts an idle eGPU into BACO after about 25 s and it never comes back over the tunnel. amdgpu turns runtime PM back on at the end of probe, so the rule fires a detached systemd-run from the DRM card event that re-writes power/control=on for 37 s (/usr/local/bin/egpu-hotplug.sh).
2. Caps the dock-to-card link at Gen3. Gen4 x4 gave BadTLP and then fatal AER with the card idle. The tunnel is about 3 GB/s anyway.
3. Sets the power cap. Its own line, edit it as you like. This card accepts 119 to 200 W.

Steam: set launch options to /path/to/chestnut-tools/scripts/host/gpu-select.sh %command%. It uses the eGPU if present, else the laptop dGPU. GPU=nvidia|egpu|igpu forces one. It sets VK_LOADER_DRIVERS_SELECT, MESA_VK_DEVICE_SELECT, DRI_PRIME and clears KDE's NVIDIA PRIME offload variables. Without install.sh --steam, KDE starts Steam with those offload variables and games never see the AMD Vulkan driver.

• Do not remove the eGPU while the system is up, not even with scripts/host/egpu-eject.sh. amdkfd keeps a stale node and /dev/kfd returns EINVAL until reboot. Worse, TTM keeps freed pages and kswapd oopses a few minutes later and the machine freezes. Attach once per boot. To disconnect, shut down with the dock attached, then unplug. Suspend counts as a removal.
• If the supply folds under load ("device lost from bus", GPU recovery failed), power off completely. Unplugging afterwards hangs amdgpu's teardown and the hot-plug slot with it. nvtop stuck in D state on a dead card can be killed with kill -9.
• The USB4 port wedges: after a bad shutdown, and also after a day or so of uptime. Symptoms: nothing plugged in produces any event, or a USB3 device only links at USB2. Cold boot with the dock attached. The link comes up somewhere between 6 and 100 s into boot.
• Do not remove and rescan the PCI tree. It leaves a stale KFD node, and on a live card it kills the card.
• nvtop shows the eGPU link as "GEN1 x1". That is the virtual tunnel root port. The real link speed is in sysfs on the dock's port.