GithubHelp home page GithubHelp logo

protolambda / eth2-val-tools Goto Github PK

View Code? Open in Web Editor NEW
38.0 4.0 10.0 203 KB

Some experimental tools to manage validators - use at your own risk

License: MIT License

Go 81.98% Shell 12.62% Python 5.40%

eth2-val-tools's Introduction

Validator management tools

Warning: Use at your own RISK, this is all very EXPERIMENTAL

Deposits

Optionally install ethereal, to run the exec_deposits.sh step.

Important: when installing, run the commands outside of the root directory of this repository, to not mix up the go modules.

# Install this assignments tool
go install .

# Move out of this dir
cd ..

# Install ethereal
GO111MODULE=on go install github.com/wealdtech/ethereal@latest

Steps:

  • eth2-val-tools mnemonic, twice: one for validator keys, one for withdrawal keys. Put them in the config.
  • . my_config.sh: central configuration with environment vars, see example_config.sh for an example
  • . build_deposits.sh: uses the mnemonics to generate deposit data for the configured range of accounts. (overwrites any existing deposit data file)
  • . exec_deposits.sh: executes deposit datas, making eth1 transactions

For automatic validator assignment, tracking and deployment, use the assign command of the Go module in this repo.

Commands

keystores

Builds keystores/secrets in every format, for a given mnemonic and account range.

Usage:
  eth2-val-tools keystores [flags]

Flags:
  -h, --help                     help for keystores
      --insecure                 Enable to output insecure keystores (faster generation, ONLY for ephemeral private testnets
      --out-loc string           Path of the output data for the host, where wallets, keys, secrets dir, etc. are written (default "assigned_data")
      --prysm-pass string        Password for all-accounts keystore file (Prysm only)
      --source-max uint          Maximum validator index in HD path range (excl.)
      --source-min uint          Minimum validator index in HD path range (incl.)
      --source-mnemonic string   The validators mnemonic to source account keys from.

mnemonic

Outputs a bare 256 bit entropy BIP39 mnemonic, or stops with exit code 1.

Create a random mnemonic

Usage:
  eth2-val-tools mnemonic [flags]

Flags:
  -h, --help   help for mnemonic

deposit-data

To quickly generate a list of deposit datas for a range of accounts.

Create deposit data for the given range of validators. 1 json-encoded deposit data per line.

Usage:
  eth2-val-tools deposit-data [flags]

Flags:
      --amount uint                   Amount to deposit, in Gwei (default 32000000000)
      --fork-version string           Fork version, e.g. 0x11223344
  -h, --help                          help for deposit-data
      --source-max uint               Maximum validator index in HD path range (excl.)
      --source-min uint               Minimum validator index in HD path range (incl.)
      --validators-mnemonic string    Mnemonic to use for validators.
      --withdrawals-mnemonic string   Mnemonic to use for withdrawals. Withdrawal accounts are assumed to have matching paths with validators.

pubkeys

List pubkeys of the given range of validators. Output encoded as one pubkey per line.

Example, list pubkeys (for a random new mnemonic), account range [42, 123):

eth2-val-tools pubkeys --validators-mnemonic="$(eth2-val-tools mnemonic)" --source-min=42 --source-max=123

Output

Eth2 clients structure their validators differently, but this tool outputs all the required data for each of them.

Prysm

Prysm is a special case, they are centric around the Ethdo wallet system. Instead of using the EIP 2335 key files directly, like all the other clients.

In the output directory, a prysm dir is placed, with the following contents:

  • keymanager_opts.json: JSON file describing accounts and their passphrases. And the "Location" part can be configured with --key-man-loc, which will point to some "wallets" directory: where the actual wallets can be found.
    • Prysm requires Account names listed in the JSON to be prefixied with the wallet name, separated by a /. Like Assigned/foobarvalidator.
    • Ethdo wallets are in the same big store, and only one directory in this store per wallet. The directory must be named as UUID, and in the directory there must be a file with the same UUID name to describe the wallet.
    • Ethdo key files in the wallet must also be named as a UUID, so that they can be parsed in the .Accounts() call
  • wallets: a directory which is an Ethdo store with a single non-deterministic wallet in it, covering all keys.
    • The wallet name is called Assigned, and the keys are Assigned/val_<pubkey here> (excluding < and >) The pubkey is hex encoded, without 0x.
    • The wallet also contains an index file and all other ethdo-specific things

Lighthouse

Lighthouse is key-centric, no wallets involved. Following EIP 2335.

The output is:

  • secrets directory, containing one file per validator. Named after the pubkey (hex-encoded, 0x prefix). Each file contains the passphrase for the voting-keystore.json of the validator.
  • keys directory (equivalent of .lighthouse/validators, containing one directory per validator. Named after the pubkey (hex-encoded, 0x prefix). Each directory contains a voting-keystore.json, an EIP 2335 keystore file, with path field set to empty string. The voting-keystore.json name is a requirement of Lighthouse.

Nimbus

Nimbus, a lot like lighthouse, expects a keys and secrets directory, which can be configured. Each keystore is named keystore.json instead of voting-keystore.json however. For ease of use, an additional nimbus-keys directory will be output, with this naming scheme.

Teku

Like Lighthouse, Teku is also key-centric, but requires you to be explicit about finding keys. I.e. you need the CLI options:

--encrypted-keystore-validator-file=foobar/key.json
--encrypted-keystore-validator-password-file=secrets/foobar

This matches lighthouse close enough, but is clumsy. To make this easier, a teku configuration file is output, with the validator mappings configured for you.

Lodestar

Lodestar is very similar to Lighthouse/Nimbus, but has 3 directories:

  --keystoresDir="{{keystores_relative_dir}}"
  --secretsDir="{{secrets_relative_dir}}"
  --validatorsDbDir="{{validators_db_relative_dir}}"

These directories are relative to the --rootDir directory. The keystores dir has pubkey-named directories, each with a voting-keystore.json. The secrets dir has pubkey-named files containing passwords, but the pubkey in the names are encoded without the 0x prefix. The validators-DB dir is unimportant, and can be left empty. This is managed by lodestar.

License

MIT, see LICENSE file.

eth2-val-tools's People

Contributors

djrtwo avatar protolambda avatar rauljordan avatar samlaf avatar skylenet avatar z3n-chada avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

eth2-val-tools's Issues

bls-eth-go-binary error when installing on ubuntu 20.04

Running into this error when trying to install on ubuntu 20.04.
Any idea what's going on?

go: downloading github.com/google/uuid v1.3.0
go: downloading github.com/herumi/bls-eth-go-binary v1.28.1
go: downloading github.com/protolambda/go-keystorev4 v0.0.0-20211007151826-f20444f6d564
go: downloading github.com/protolambda/zrnt v0.30.0
go: downloading github.com/protolambda/ztyp v0.2.2
go: downloading github.com/spf13/cobra v1.1.3
go: downloading github.com/tyler-smith/go-bip39 v1.1.0
go: downloading github.com/wealdtech/go-eth2-types/v2 v2.6.0
go: downloading github.com/wealdtech/go-eth2-util v1.6.4
go: downloading golang.org/x/sync v0.0.0-20190423024810-112230192c58
go: downloading golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
go: downloading golang.org/x/text v0.3.7
go: downloading github.com/kilic/bls12-381 v0.1.0
go: downloading github.com/protolambda/bls12-381-util v0.0.0-20210720105258-a772f2aac13e
go: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
go: downloading github.com/minio/sha256-simd v1.0.0
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/ferranbt/fastssz v0.0.0-20210905181407-59cf6761a7d5
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/wealdtech/go-bytesutil v1.1.1
go: downloading golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1
go: downloading github.com/holiman/uint256 v1.2.0
go: downloading github.com/klauspost/cpuid/v2 v2.0.9
go: downloading github.com/mitchellh/mapstructure v1.4.2
go: downloading gopkg.in/yaml.v2 v2.4.0
# github.com/herumi/bls-eth-go-binary/bls
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:9:28: undefined: SecretKey
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:9:40: undefined: Fr
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:13:26: undefined: Fr
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:13:31: undefined: SecretKey
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:19:28: undefined: PublicKey
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:19:40: undefined: G1
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:23:26: undefined: G1
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:23:31: undefined: PublicKey
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:29:23: undefined: Sign
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:29:30: undefined: G2
../go/pkg/mod/github.com/herumi/[email protected]/bls/cast.go:29:30: too many errors

install as a go module

It would be nice if we can install this software the same way we can with others eg.:

GO111MODULE=on go get github.com/protolambda/rumor

Any particular reason why this one we install only from a cloned repo via go install .?

Lodestar 0x prefix on secrets path

In legacy version of Lodestar

The secrets dir has pubkey-named files containing passwords, but the pubkey in the names are encoded without the 0x prefix. The validators-DB dir is unimportant, and can be left empty. This is managed by lodestar.

This is an old bug that has been patched for a while. To drop this legacy code that reads the password from either 0x prefixed or not named files this tool should be updated.

Lodestar can use the exact same logic as Lighthouse or 0x prefix the secret key filename

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.