This repository hosts my cheatsheet for Hetzner GPU Server Setup. Hopefully, I could also be useful for others ;)
I opened the discussion section so that other GEX44 users can discuss various things.
Update: I no longer have a Hetzner GPU Server, as I have built my own GPU Workstation at home, see this post. Thus, this repo here is no longer maintained and it is archived now.
In March 2024, Hetzner thanksfully released a new generation of GPU servers, after their first GPU server back in March 2017!
The GEX44 comes with an Intel® Core™ i5-13500, 64GB of RAM and a NVIDIA RTX™ 4000 with 20GB GPU-RAM.
I will use the GPU mainly for fine-tuning NLP models with the awesome Flair library. I am very excited about the new GEX44 server, as I've also used the old GPU server back in 2018 for releasing a lot of Flair Embeddings!
After using Ubuntu 22.04 (installed from rescue console) the following steps are done in the initial setup stage:
- Updating and upgrading all Ubuntu 22.04 packages
- Upgrade from Ubuntu 22.04 to 24.04
- Install CUDA and NVIDIA drivers
On commandline it looks like:
$ apt update && apt upgrade -y
$ reboot # To make sure ;)
$ do-release-upgrade -dFollow all upgrade instructions and then we have a working Ubuntu 24.04 installation. Now all NVIDIA-related stuff is installed:
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
$ dpkg -i cuda-keyring_1.1-1_all.deb
$ apt-get update
$ apt-get -y install cuda-toolkit-12-6After that we install NVIDIA drivers and reboot our nice GEX44:
$ apt-get install -y cuda-drivers
$ rebootNow let's see if it was working:
$ nvidia-smi
Wed Feb 12 21:27:38 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.15 Driver Version: 570.86.15 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX 4000 SFF Ada ... Off | 00000000:01:00.0 Off | Off |
| 33% 58C P8 7W / 70W | 2MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+Amazing! The GEX44 is now ready to fine-tune nice models 🎉
We create a new (unpriviliged) user and add it to the sudoers list:
$ useradd -m -g users -s /bin/bash stefan
$ passwd stefan # Set a nice password here
$ usermod -aG sudo stefanThen login with the newly created user.
PyTorch is installed in a fresh virtual environment (via venv), as this is the easiest way to get it running.
Possible alternatives are e.g. Anaconda or using Docker with NVIDIA support and e.g. an NVIDIA PyTorch image).
The following steps will install a perfectly working PyTorch installation with GPU support:
$ sudo apt install python3-venv
$ python3 -m venv venvs/dev
$ source venvs/dev/bin/activate
$ pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121Now let's test if CUDA is available via:
$ python3 -c "import torch; print(torch.cuda.is_available())"
TrueNow we can start installing more libraries and fine-tuning own models!