getumbrel / umbrel-os

umbrelOS for Raspberry Pi 4 (only). Covert your Raspberry Pi into a home server in one click. For other hardware, checkout https://github.com/getumbrel/umbrel

☂️ OS

Umbrel OS is the operating system of Umbrel Bitcoin and Lightning node. It's based on Raspberry Pi OS (formerly Raspbian) and uses pi-gen for customization.

🚀 Getting started

To run Umbrel on:

• Raspberry Pi 4 (recommended) — Download the latest Umbrel OS
• Raspberry Pi 3 (no longer supported) - Download Umbrel OS v0.2.10
• Any other hardware - Manually install Umbrel

Instructions:

1. Make sure you've everything ready:

   1. Raspberry Pi 4 with 4GB or 8GB RAM
   2. 1TB SSD
   3. SSD Enclosure
   4. 16GB+ microSD Card
   5. microSD Card Reader
   6. Ethernet Cable
   7. Case

2. Download the latest release of Umbrel OS, or build the image from source.

3. Use an image flasher such as Balena Etcher to flash the image to a microSD card.

4. Put the microSD card in your Raspberry Pi, attach an external SSD or HDD (to USB 3 port), connect the Pi to your router with an ethernet cable.

5. Turn on the Pi and open http://umbrel.local from any device connected to the same network (it might take a while for it to be accessible).

⚠️ All data on the external hard drive will be erased on first boot.

If you're running Umbrel OS on Bitcoin mainnet (default), the external SSD or HDD should be at least 750 GB in size (we recommend 1 TB+) so it can store the whole Bitcoin blockchain.

💻 SSH

SSH is enabled by default and you can use the following credentials to login to your Umbrel node.

• Hostname: umbrel.local
• User: umbrel
• Password: The same password you use to log in to the Umbrel dashboard

If you haven't yet run through the setup process, the password will be set to moneyprintergobrrr.

🛠 Build Umbrel OS from source

Don't trust. Verify.

Step 1. Clone this repo

git clone https://github.com/getumbrel/umbrel-os.git

Step 2. Switch to repo's directory

cd umbrel-os

Step 3. BUIDL!

sudo ./build.sh

After the build completes (it can take a looooooong time), the image will be inside deploy/ directory.

🔧 Advanced

Config variables

The config file has system defaults which are used when building the image and for automated builds.

• UMBREL_VERSION - To install the specific Umbrel version.

• GITHUB_USERNAME - To automatically login to your Umbrel without typing a password.

• UMBREL_REPO - A custom git repo used to download Umbrel. UMBREL_VERSION is ignored if this is set, but it is required to also set UMBREL_BRANCH.

• UMBREL_BRANCH - The git branch or commit SHA in the custom repo that should be checked out.

Other Raspbian-related stuff can be found in Raspbian's documentation which is still applicable.

---

⚡️ Don't be too reckless

Umbrel is still in an early stage and things are expected to break every now and then. We DO NOT recommend running it on mainnet with real money just yet, unless you want to be really #reckless.

❤️ Contributing

We welcome and appreciate new contributions!

If you're a developer looking to help but not sure where to begin, check out these issues that have specifically been marked as being friendly to new contributors.

If you're looking for a bigger challenge, before opening a pull request please create an issue or join our community chat to get feedback, discuss the best way to tackle the challenge, and to ensure that there's no duplication of work.

---

"Being open source means anyone can independently review the code. If it was closed source, nobody could verify the security. I think it's essential for a program of this nature to be open source." — Satoshi Nakamoto

getumbrel.com

umbrel-os's Issues

Generate bitcoind username for essential services (namely LND)

Generate bitcoind username for essential services (namely LND). Also invoicer might be useful too for middleware to accept / receive payments.

Wifi instead of Ethernet Cable?

Hi there,
I am currently running a raspiblitz with wifi and would like to switch to Umbrel to try it out. But before I kill my current node, I 'd like to know if it supports wifi. If yes, are there any insturctions on how to do the installation with wifi?
If no, I'd like to see that feature added.

Thanks for the great work you are doing!

Switch to the MIT License?

@nolim1t is there a specific reason that an Apache license for chosen for this repo? If not, let's switch to MIT for better consistency.

Bundle Umbrel Compose's Docker Images

Issue

Even though Umbrel OS ships pre-bundled with Umbrel Compose, it has to pull all the docker images of services defined in $HOME/docker-compose.yml from docker hub on the first-boot. This adds a significant delay in the initial setup, thus having the user wait indefinitely to access the dashboard at http://umbrel.local without any method of keeping them updated with progress, ETA, or status of what's going on.

Also, the first image pulls are always the longest as there are no cached intermediary images, plus as we add more services to Umbrel Compose, it will only increase the delay.

Solution

Add a step in stage 2 of image generation which pulls and bundles all the docker images of services defined in $HOME/docker-compose.yml. Now since no image pulls will be required on the first boot, the user would be able to instantly access their dashboard at http://umbrel.local.

Also, there is no benefit of pulling images on the first boot as we're not pulling the latest images anyway (since they are pulled in accordance with their version tags defined in $HOME/docker-compose.yml) so bundling them does no harm.

Debian bullseye

Debian bullseye launches in summer this year, and we should update to that release when RPi OS Bullseye is released (Docker base images and Umbrel OS). I've created this issue to track all the required changes and progress on that update.

• Update docker base images
• Update Umbrel OS to build bullseye
• Create distro update backend
• Create distro update frontend

I'm not sure how you want to implement that (Ship it as part of an OTA update/Extra update button just for OTA updates).

@lukechilds @mayankchhabra @louneskmt What do you think about this?

disk-partitioned status is present when it shouldn't (rc.local fix)

disk-partitioned status is present when it shouldn't.

Need to add some additional checks to /etc/rc.local

New install on Pi 4b 8gb, exit code 137 all docker containers

Hi

Installed Umbrel OS on a new Pi 4b, 8gb ram

During the process, all the docker containers just exit with the code (137)

umbrel@umbrel:~ $ docker ps -a
CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS                        PORTS               NAMES
3d9e2911a73e        getumbrel/middleware:v0.1.7          "docker-entrypoint.s…"   2 hours ago         Exited (137) 19 minutes ago                       middleware
4eeafda273b6        getumbrel/manager:v0.2.7             "docker-entrypoint.s…"   2 hours ago         Exited (137) 19 minutes ago                       manager
bd0811bae0ab        getumbrel/neutrino-switcher:v1.0.3   "/usr/local/bin/swit…"   2 hours ago         Exited (137) 19 minutes ago                       neutrino-switcher
fd50219b4433        getumbrel/electrs:v0.8.6             "electrs"                2 hours ago         Exited (137) 19 minutes ago                       electrs
25be5fff7417        nginx:1.17.8                         "nginx -g 'daemon of…"   2 hours ago         Exited (137) 19 minutes ago                       nginx
f53f150aea1b        lncm/bitcoind:v0.20.1                "bitcoind -zmqpubraw…"   2 hours ago         Exited (137) 19 minutes ago                       bitcoin
f237db53c50b        lncm/lnd:v0.11.1                     "/usr/local/bin/lnd"     2 hours ago         Exited (137) 19 minutes ago                       lnd
0976542770fd        getumbrel/tor:v0.4.1.9               "tor"                    2 hours ago         Exited (137) 19 minutes ago                       tor
a9952d75b054        getumbrel/dashboard:v0.3.13          "docker-entrypoint.s…"   2 hours ago         Exited (137) 19 minutes ago                       dashboard
fb6d33c9470e        getumbrel/frontail:v4.9.1            "/frontail/docker-en…"   2 hours ago         Exited (137) 19 minutes ago                       frontail

1TB SSD attached

Rebooting and it continues to sync, let me know if theres anything else I can provide but from a little bit of searching online it seems it's relating to docker running out of memory

—
Edit
!! just noticed it has wiped the progress of the bitcoin sync and started again, surely this is not right? It was already up to block 250,000 and now started again from 0 after a reboot

Remove wallet creation/unlock logic

Wallet creation/unlocking, seed storage, etc is handled by https://github.com/getumbrel/umbrel-manager now

Move box install into /etc/rc.local

Move box install into /etc/rc.local to have more separation of concern rather than be part of the build process

Create LND createwallet & unlock middleware

Add in a docker container for use within the images for creating a wallet and unlocking the middleware for this.

For temporary development, store secrets on the filesystem (#5 ). But eventually we want to encrypt them in some way.

[Enhancement] Required dependencies not installed after running "sudo ./build.sh"

Following the installation instructions on the Raspberry Pi 4B, I get an error that dependencies are missing.

Required dependencies not installed

This can be resolved on Debian/Raspbian systems by installing:
quilt qemu-user-static debootstrap zerofree bsdtar bc

Perhaps we could add the installation of these dependencies directly to build.sh shell script.

Establish a directory for secrets and have a script to generate them

Ideally $HOME/secrets for any generated rpc passwords and generated seeds.

Subtasks

• Create /etc/init.d script which generates secrets for comms between bitcoind and lnd (Add to image)
• Symlink /etc/init.d script into runlevel 2 /etc/rc2.d (suggestions welcome). Add this to the image
• Add in support for writing to config files into the scripts
• check that the config files are actually useful

Optional

Encrypt them (but also how do we use the key to decrypt them is something we need to work out)

Expected Result

When we flash the SD card and run for the first time we should see all the secrets in $HOME/secrets

Overclocking

The Raspberry Pi 400 runs at 1.8 GHz by default. How about doing the same overclocking on Umbrel OS by default? It might boost the initial bitcoin sync, and my RPi is running fine in a FLIRC case at 2.0 GHz, so it shouldn't be an issue. Also, with pi-gen it is possible to create two versions easily, one with and one without overclocking, so you could also provide both.

@mayankchhabra @lukechilds What do you think?

Wiki contains wrong password

https://github.com/getumbrel/umbrel-os/wiki/Box-System-Defaults still contains the old password

Extremely degraded performance on some USB 3.0 SSDs

Many people, including myself, have had a painful experience of using an external SSD on a RPi 4 where the read/write speeds turn out to be less than 100 kB/s.

Turns out that it's a known problem (and a pretty common one) where the Linux kernel has a non-exhaustive blacklist for devices known to be unreliable when using UAS [full details here].

The fix is pretty straightforward though, just need to add a single line to /boot/cmdline.txt, so I think it's worthwhile to patch it for all RPi 4 builds (including those whose drives aren't blacklisted).

Containerize Tor

Currently, Tor is installed on the system. We should containerize it and run it as a docker service as a part of https://github.com/getumbrel/umbrel-compose

Set up partition utility

Set up disk partitioning utility inside the image.

Subtasks

• Have a working script (maybe make /media/noma if we are using noma or have it set to /mnt/ssd)
• Test it out on /etc/init.d/ with start command
• Symlink it to /etc/rc2.d and test it out on reboot

Expected Result

EXT4 partition created

FAT32 formatted filesystems don't work

Steps

1. Plug in a device which is formatted fat32
2. Mount /dev/sda1 /mnt/data

Expected result

Filesystem mounted

Actual result

Not mounted

Solutions

• We nuke any filesystems not ext4 thats run through the partitioner (as in if you plug it in, you agree to have your stuff formatted). I like this solution as it should not induce bloat. Also our lack of drive on bootup script lets users skip partitioning, and they can plug in a drive AFTER.
• Add another setup stage

sudo apt-get install exfat-utils

so we have full support of the users chosen filesystem. Also good too

Install fail2ban

Install fail2ban to prevent SSH brute-force attacks. The default configuration should be fine as it is.

OTA Updates

Update Umbrel OS Over-The-Air 🚀

Add bash aliases for commonly-expected executables running inside Docker containers

A useful abstraction for RPi installs could be to bash alias commonly-expected commands like lncli to the location where they're currently accessible (docker exec -it lnd lncli). Executables people usually try to get at are:

• lncli
• bitcoin-cli
• lnd
• bitcoind

Add CLI Hardware wallet support

Add CLI Hardware wallet support so we can eventually support HW wallets and users do not have to keep their funds on the node.

Initially just allow CLI access so we can debug.

Would eventually lead to an increase of the value proposition of having the node, rather than doing using a hot wallet.

Reference material

• Python HWI interface
• Creating a HW wallet

Docker doesn't work in the latest image

Docker daemon doesn't run by default. When trying to run it manually, it throws the following error:

Tested on:
RPi 4 / 4GB, running Umbrel OS v0.0.2.8

Get NGINX configured and installed

Get NGINX configured and installed with the image

Create additional swap

Setup script to create additional swap

Use previously downloaded blockchain

Is it possible to use a previously downloaded blockchain?

I already have a node running and have the entire blockchain, in order to avoid the IBD on my very slow internet, I'd like to be able to that. Is this possible?

Thanks,
Tiago

HDD failure. Can I recover?

Title pretty much says it all. I have a funded node w/funded channel and the HDD died. Deader than dead. I will replace it in a few days, but it seems the recovery functionality hasn't been rolled out yet.

Can I recover? I have my channel backup.

There's only one channel, and I can get the remote node operator to force close it if need be. But if theres a way to recover, its preferable.

Upgrade OS to 64-bit

Raspberry Pi OS (formerly Raspbian) 64-bit has finally been released in beta! We should use it as our base image.

Remove Tor

Now that Tor is dockerized, we can remove it from the OS after getumbrel/umbrel-compose#28 is merged

See if we can utilize the attached script for getting the latest docker-compose files

See if we can utilize this script for getting the latest docker-compose skeleton.

Benefits

• More Modular install (less dependencies). We can also get rid of many build steps in PR #8
• Updated in one place

Login prompt on boot

Two users reported that Umbrel OS got stuck on login prompt in Telegram. I'm not sure why this happens, but this should not happen. After a manual start of Umbrel, Umbrel was working as normal.

Increase open files limit

Issue

The default limit of 1024 open files is too low if there are multiple clients using Umbrel simultaneously, and can result in the error: can't accept connection: too many open files.

Solution

(Verified locally on my RPi 4 running Umbrel OS)

Step 1. Update open files limits

Add the following to the end of /etc/security/limits.conf:

*    soft nofile 128000
*    hard nofile 128000
root soft nofile 128000
root hard nofile 128000

Step 2. Enable limits for daemon processes

Add the following to the end of /etc/pam.d/common-session:

session required        pam_limits.so

Step 3. Enable limits for non-interactive daemon processes

Add the following to the end of /etc/pam.d/common-session-noninteractive:

session required        pam_limits.so

Enable firewall

Install Uncomplicated Firewall — ufw — and close all ports except 80 (for Nginx) and 22 (to SSH).

[FEATURE REQUEST] Error page if Umbrel fails to boot

Add a "Something went wrong, probably your SSD isn't being detected page" that shows when Umbrel can't be loaded. I'd implement it using a web server that has that page that is automatically stopped when Umbrel is started

Curl doesn't work

Curl doesn't work out-of-the-box with many https URLs (including GitHub).

We need to add CURL_CA_BUNDLE globally for it to work.

export CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

bitcoin container stopped working / corrupted on rpi 3

Hey - it's me again.
After we discussed the password change issue I just left it running for a few weeks (not even trying to connect to the dashboard).
Yesterday I tried connecting to the dashboard, but it returned operational: false, unlocked: false for more than an hour - which is when I decided to try the old docker-compose stop - docker-compose start technique. Unsuccesfully.
It seems like something went wrong along the way with the bitcoin container (see screenshots below).

Repository Housekeeping Tasks. Great for a first issue

Stuff which makes a good open source environment

Stuff to do

• Establish commit policy
• Set up CI/CD
• Make first "Release" with CI/CD
• Set up wiki documentation
• Establish contributing guidelines and code of conduct

Sorry for the double post.

not sure how that happened

[Bug] 504 Gateway Time-Out Error shows plain HTML in Error Box

Hello everyone!

Sorry it took me a while, wanted to clarify and investigate the error before I submit an issue.
To pick up on the twitter conversation with @mayankchhabra

504 Gateway Time-out Error shows plain HTML in Error Box

Tasks

• Fix Error Message Display of Error Code "504".

PS: Should I just submit Issues for every finding I have in the future or should I make a spreadsheet and you pick whats important? Don't want to flood your issue list 😄

Error - Not allowed by CORS

Tried Umbrel today. Flashed the image to a new 32GB SD card. The system booted, and I set my name and password. Then I got the 24 word list and wrote them all down. After the last word I tried to continue, but get the error "Not allowed by CORS". I am stuck at the last word screen and can not continue.

Also tried to redo the setup, and skipping the 24 words by selection the option "Note down later", but that gives me the same error.

How to compile Umbrel OS for x86_64

Are there any steps available or how-to compile Umbrel OS to run on x86_64?

Log to RAM to prevent SD degradation

Not necessarily required right now, but definitely worth considering in the near future. https://github.com/azlux/log2ram

umbrel.local hostname cycling

I'm seeing my ability to access the dashboard or ssh in via umbrel.local fail every 1-2 days. There's no doubt a more graceful way to resolve, but after sudo reboot, I'm back in action for another couple of days.

As a baseline, I am able to consistently ssh into various other devices on my network using their hostnames.

The details are over my head, so I don't know if this is something peculiar to my local network or not. FWIW, I'm using a single eero wifi router/AP running the latest firmware.

[Bug] 504 gateway timeout loop on initial site load & request flooding -> RPI3 performance issues during sync

504 gateway timeout loop on initial site load & request flooding

setup

• Raspberry Pi 3 B+ & 500GB HDD
• UmbrelOS
• Brave Browser, Edge and Firefox
• Node is syncing

problem information

• Initial site load took years, many requests timed out.
• After initial site load, the UI floods the API with requests while not receiving back any information, opening many duplicate connections.
• The Dashboard eventually loads, continuing to throw timeout errors and duplicate request warnings.
• After not getting better I unplugged the Pi, initially after the restart all status request were performed without a problem. The dashboard loaded rather quickly.
• Shortly after, it began doing requests that hadn't been completed yet and therefore skipping double requests.
• Once the blocks that it already pulled where validated and it began pulling and validating new blocks, it also began throwing the 504 gateway timeout loop occasionally (not continuously like before).
• Trying a user activated request (change password) does always end in a 504 timeout.
• remark: also requests that do succeed range from 3 seconds to 38 seconds.

Problem seems (from my pov) to be that the system is overloaded during initial block sync and can't reliably answer api requests.

Support for RockPro64

Hey guys!

Absolutely love what you're building with Umbrel. I'm looking to get my own Umbrel node working with testnet on a RockPro64 from PINE64. I'm not super familiar with the differences between the OS needed for Raspberry Pis and RockPros, but I just wanted to let you know that I'd be down for helping with the testing if that's something you're looking at supporting in the future!

Let me know if you have time/ideas as to where to start for that. Should I just try the OS as-is and report back?

Cheers!

Failed building from source - Ubuntu 20.04

When building the image from source, an error message is displayed with the dependencies for Debian / Raspibian. I think those dependencies will also work for Ubuntu 18.04 but it isn't clear - it could be helpful to list the dependencies in the README.

Either way, on Ubuntu 20.04 (from Windows Subsystem for Linux), those dependencies don't work as there is no bsdtar binary in apt. I had to install libarchive-tools instead for it to start the build process.

The build failed with the following, not sure if it is because of using libarchive-tools (which I think includes the bsdtar binary):

I: Extracting liblzma5...
I: Extracting zlib1g...
I: Running command: chroot /home/billy/src/umbrel-os/work/2020-08-30-umbrel-os/stage0/rootfs /debootstrap/debootstrap --second-stage
chroot: failed to run command ‘/debootstrap/debootstrap’: Exec format error
rmdir: failed to remove '/home/billy/src/umbrel-os/work/2020-08-30-umbrel-os/stage0/rootfs/debootstrap': Directory not empty

bitcoin credentials is being created twice

bitcoin credentials is being created twice when box is being set up.

Need to take a look at this

Doesn't start back up after shutdown

I was having some issues with my network one day. So I decided to restart my network. In an attempt to do things correctly, I shut my Umbrel instance down using the Umbrel UI and then shut everything off in my network and restarted it. The pi turned back on. However, Docker Composer did not start the containers back up, and the karen script did not start again. Running the Docker containers was easy. However, after spending some time trying to figure out why the v0.3.1 update wasn't working for me, I figured out that the karen script wasn't running and was causing updates to not kick off. I'm not sure why these pieces are not starting back up when the pi is turned back on after a successful manual shutdown from the UI.

Generate custom torrc config with tor secrets

Use the following
tor --hash-password <password>

But we might need to have a default for storing secrets. Maybe use the LND unlock password to do this right now?

Move disk partitioner script to /usr/local/bin

Add disk partitioner script from contrib/ in getumbrel/umbrel to /usr/local/bin and update rc.local