sigp / lighthouse-docker

A docker-compose environment for running the Lighthouse Eth2 client

🚨 DEPRECATION NOTICE 🚨

As of August 2022 Sigma Prime will no longer maintain lighthouse-docker. This decision was made to better focus on development of the core client. We will continue to provide official Docker images.

If you are currently using lighthouse-docker you need to either migrate to the community-fork, or eth-docker. You must do this before the mainnet merge in September 2022.

1. Community-maintained fork of lighthouse-docker by @kimpers: https://github.com/kimpers/lighthouse-docker
2. Guide for migrating to eth-docker: https://hackmd.io/@sproul/lighthouse-docker-migration

The original README is preserved below.

---

Lighthouse Docker

Provides a docker-compose environment for running Lighthouse.

The following features are available:

• A beacon node exposing a HTTP API on port 5052.
• A validator client (optional) that connects to the beacon node.
• Wallet and validator key generation (optional).

Usage

$ docker-compose up

A .lighthouse directory will be created in the repository root which contains the validator keys, beacon node database and other Lighthouse files.

Configuration

The docker-compose file requires that a .env file be present in this directory. The default.env file provides a template and can be copied .env:

$ cp default.env .env

Failure to create an .env file will result in the following error:

ERROR: Couldn't find env file: /home/karlm/lighthouse-docker/.env

lighthouse-docker's Issues

[feature] Include grafana and prometheus

geth docker does not upgrade with docker-compose pull

When you need to upgrade version of the underlying software (lighthouse, geth), you need to pull their latest docker images. You can do that with docker-compose pull before your docker-compose up. But since geth has a custom Dockerfile in this setup, it would stay on the version when you've first run it.

Update to v1

Lighthouse recently updated to v1, but it looks like this repo hasn't yet been properly updated.

Current issues are :

• --testnet flag has been changed to --network.
• testnet still defaults to medalla whereas latest testnet should be pyrmont.
• Update "allowed values" for testnet in default-env file.

Add --auto-register on next Lighthouse update

The validator client should be run with --auto-register, once sigp/lighthouse#1116 is merged and lighthouse-docker is updated.

It should be as simple as adding it here:

lighthouse-docker/scripts/start-validator-client.sh

Lines 14 to 17 in c9daa37

	exec lighthouse \
	--debug-level $DEBUG_LEVEL \
	validator \
	--server http://beacon_node:5052

net API required

This will fail when a local geth node is run, as the local geth does not expose the "net" API, and the lighthouse beacon requires it to query the net version.

Change start-geth.sh to include --http.api "eth,net" to resolve this.

Update GitHub "About" text

The text currently reads "A docker-compose environment for testing Lighthouse". Maybe this should be updated to not spook off people wanting to run a mainnet configuration? For reference: #52 (comment)

geth exited with code 137 in docker

I am running a validator node in docker following this doc: https://lighthouse-book.sigmaprime.io/become-a-validator-docker.html. After several hours or days of continuous running, the geth container exited with code 137. I didn't modify docker-compose.yml.
The last lines of geth output:

......
INFO [08-08|14:48:02.961] Imported new chain segment               blocks=1    txs=0    mgas=0.000   elapsed=9.345ms     mgasps=0.000   number=3189431 hash="284075…262e90" dirty=80.33MiB
INFO [08-08|14:48:18.167] Imported new chain segment               blocks=1    txs=2    mgas=0.745   elapsed=265.624ms   mgasps=2.804   number=3189432 hash="07b3b0…6fe552" dirty=80.34MiB
INFO [08-08|14:48:31.308] Imported new chain segment               blocks=1    txs=0    mgas=0.000   elapsed=157.848ms   mgasps=0.000   number=3189433 hash="988379…5f551c" dirty=80.32MiB
lighthouse-docker_geth_1 exited with code 137

docker inspect shows it's not OOM killed:

[
    {
        "Id": "2bfbfb3308d8912af57faeca9191e5f8f5dea579d0b8bf33ef23b5d598ebbc56",
        "Created": "2020-08-06T08:39:08.769682725Z",
        "Path": "sh",
        "Args": [
            "/root/scripts/start-geth.sh"
        ],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 137,
            "Error": "",
            "StartedAt": "2020-08-07T15:15:52.177099387Z",
            "FinishedAt": "2020-08-08T14:48:41.231861137Z"
        },
        "Image": "sha256:92da0f6b35ff5bbc8e4f67fbc926fe2d9706fb906a5a41fab79dfeed081aac42",
        "ResolvConfPath": "/var/lib/docker/containers/2bfbfb3308d8912af57faeca9191e5f8f5dea579d0b8bf33ef23b5d598ebbc56/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/2bfbfb3308d8912af57faeca9191e5f8f5dea579d0b8bf33ef23b5d598ebbc56/hostname",
        "HostsPath": "/var/lib/docker/containers/2bfbfb3308d8912af57faeca9191e5f8f5dea579d0b8bf33ef23b5d598ebbc56/hosts",
        "LogPath": "/var/lib/docker/containers/2bfbfb3308d8912af57faeca9191e5f8f5dea579d0b8bf33ef23b5d598ebbc56/2bfbfb3308d8912af57faeca9191e5f8f5dea579d0b8bf33ef23b5d598ebbc56-json.log",
        "Name": "/lighthouse-docker_geth_1",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "lighthouse-docker_geth:/root/.ethereum:rw",
                "/root/lighthouse-docker/scripts:/root/scripts:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "lighthouse-docker_default",
            "PortBindings": {
                "30303/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "30303"
                    }
                ],
                "30303/udp": [
                    {
                        "HostIp": "",
                        "HostPort": "30303"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/f3146e2773bba7668cfbefc40e46b51020ea07e823f6f773e22b554e3cfc1f4f-init/diff:/var/lib/docker/overlay2/793330b1b67656d31260eff929c90f182303b518a49e3ba102e3aa91139bc485/diff:/var/lib/docker/overlay2/5e3ad7e8897d87f4ef50ce23d0001dbf1dad66f38eebc6ad76ff120e9ce5c0c7/diff:/var/lib/docker/overlay2/5749bd8cc15280bed9e2977f16bd7de6bf7738d768a60343ad0005beb43bb744/diff:/var/lib/docker/overlay2/83e638f83c2a1c6fc34fb17843b22e09e6d8e1cbfff4ff3c01f4f7d494c2e718/diff",
                "MergedDir": "/var/lib/docker/overlay2/f3146e2773bba7668cfbefc40e46b51020ea07e823f6f773e22b554e3cfc1f4f/merged",
                "UpperDir": "/var/lib/docker/overlay2/f3146e2773bba7668cfbefc40e46b51020ea07e823f6f773e22b554e3cfc1f4f/diff",
                "WorkDir": "/var/lib/docker/overlay2/f3146e2773bba7668cfbefc40e46b51020ea07e823f6f773e22b554e3cfc1f4f/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "lighthouse-docker_geth",
                "Source": "/var/lib/docker/volumes/lighthouse-docker_geth/_data",
                "Destination": "/root/.ethereum",
                "Driver": "local",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/root/lighthouse-docker/scripts",
                "Destination": "/root/scripts",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "2bfbfb3308d8",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "30303/tcp": {},
                "30303/udp": {},
                "8545/tcp": {},
                "8546/tcp": {},
                "8547/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "DEBUG_LEVEL=info",
                "START_GETH=true",
                "START_VALIDATOR=true",
                "VALIDATOR_COUNT=1",
                "VOTING_ETH1_NODE=http://geth:8545",
                "GRAFFITI=poapNzGdoK2362h4fSnOsfw/1C+BbdIB",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "sh",
                "/root/scripts/start-geth.sh"
            ],
            "Image": "lighthouse-docker_geth",
            "Volumes": {
                "/root/.ethereum": {},
                "/root/scripts": {}
            },
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "3d19f478838c9eb9d52515c17bb5e6930c19c761f6178223580bc70fd5414379",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "lighthouse-docker",
                "com.docker.compose.project.config_files": "docker-compose.yml",
                "com.docker.compose.project.working_dir": "/root/lighthouse-docker",
                "com.docker.compose.service": "geth",
                "com.docker.compose.version": "1.26.2"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "cce38c05c556f3d4cfd035d956e1412b2f638861fe44512ebd438c29a50ad9da",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/cce38c05c556",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "lighthouse-docker_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "2bfbfb3308d8",
                        "geth"
                    ],
                    "NetworkID": "efeb82b6dd3c05f8bc2ec0e6539579025a81035a0ec35c5ce0bc23062fa83548",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]

So what could be the reason? Thanks.

Docker user changed

The newest docker image uses the user lighthouse instead of root, which results in a Permission denied error when starting the validator and beacon node

beacon_node_1       | sh: 0: Can't open /root/scripts/start-beacon-node.sh
lighthouse-docker_beacon_node_1 exited with code 127

start-beacon-node.sh mismatch with recent lighthouse release: "--max-skip-slots" flag currently not implemented in lighthouse

This commit 6545d67 adds the following flag:

--max-skip-slots 700 

However, this flag breaks the "sigp/lighthouse:latest" released to DockerHub.

Three suggestions:

• improved versioning and releases of lighthouse to DockerHub, perhaps even with coordination
• the docker-compose.yml in lighthouse-docker could specify the exact tagged docker image
• synchronization of versioning / tagging of lighthouse-docker in GitHub, versioning of lighthouse client in Github, and versioning on DockerHub

Makefile:7: recipe for target 'repo' failed

To reproduce, I checked out this repo and ran make.

Successfully built 359ec2a8b7d1
Successfully tagged lighthouse-builder:latest
docker run --mount type=bind,source="/tmp/lighthouse-docker"/lighthouse/repo,target=/root/lighthouse --mount type=bind,source="/tmp/lighthouse-docker"/lighthouse/cargo-cache,target=/root/.cargo lighthouse-builder
error: could not find `Cargo.toml` in `/root/lighthouse` or any parent directory
Makefile:7: recipe for target 'repo' failed
make: *** [repo] Error 101

Race condition with geth container

If START_GETH is defined, there can be a race condition where the beacon starts before geth starts accepting API connections, which causes the beacon to fail.

Use separate volumes for the beacon node and validator client

A Discord user pointed out that we currently use ./lighthouse-data:/root/.lighthouse as a volume for both the beacon node and validator client container.

It would make sense to create separate volumes so the beacon node's container can't access validator keys.

./lighthouse-bn-data:/root/.lighthouse
./lighthouse-vc-data:/root/.lighthouse