dappnode / stakingbrain

StakingBrain

Staking Brain is a critical logical component for Staking in DAppNode. It provides a user interface and a Launchpad API that allow manual and automatic keystore management. Designed to support not only solo stakers, but also DVT/LSD technologies, Staking Brain streamlines the staking process for all users.

Within the Dappnode environment, Staking Brain is incorporated into the Web3Signer packages (gnosis, mainnet, prater, lukso and holesky). It ensures that user configurations for validators are reliably maintained. Please note that Staking Brain does not store keystores itself, but ensures their storage in the web3signer. It also maintains consistency between the validator service and web3signer service, as the validator must recognize all the pubkeys of validators whose keystores have been imported into the signer.

Each Web3Signer package includes four services:

1. ConsenSys Web3Signer

2. DB (postgres)

3. Staking Brain

4. Flyway (runs only once when the package starts and then stops)

The new functionalities that the StakingBrain brings are:

1. Individual management of fee recipients

2. Management of validator tags (e.g., "solo", "obol", "rocketpool")

3. New UX features, such as an advanced mode for detailed validator status and a light/dark mode switch

Development Guide

Follow these steps to develop with Staking Brain:

1. Connect to your DAppNode (which should be running an instance of web3signer and validator in a network, e.g., Prater).

2. Clone the repository:

git clone https://github.com/dappnode/StakingBrain

3. Set your current DAppNode staker config in packages/brain/.env :

NETWORK="prater"
_DAPPNODE_GLOBAL_EXECUTION_CLIENT_PRATER="goerli-erigon.dnp.dappnode.eth"
_DAPPNODE_GLOBAL_CONSENSUS_CLIENT_PRATER="prysm-prater.dnp.dappnode.eth"

A) Local development

4. Build and start development mode

yarn
yarn build
yarn start:dev

5.Access the KeyManager UI (e.g. Prater)

http://localhost/?network=prater&signerUrl=http://web3signer.web3signer-prater.dappnode:9000/

B) Docker development

4. Build and start docker development mode

yarn
yarn build
docker-compose -f docker-compose-dev.yml build --no-cache
docker-compose -f docker-compose-dev.yml up -d
docker logs -f brain (To watch status)

5. Find the container IP

docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' brain

6. Access the KeyManager UI (e.g. Prater)

http://<obtainedIP>

Development Recommendations

1. Connect to your DAppNode using SSH and stop the brain container of your Prater web3signer:

docker ps | grep brain
docker stop <brainContainerName>

Note: If you do not stop the brain container, the cron will remove your keystores every minute.

2. Install the SSH plugin in Visual Studio to develop in real time.

Note: You'll need to install your VS plugins in the dappnode to use them.

stakingbrain's Issues

Finish database module unit tests

Finish implementing the unit tests for the database module

Add linter to format code

Add linter to format source code

Review mobile view

Review mobile view and fix the possible issues found

Review standard API client error handling

Have a look at all the possible errors that can be thrown while making HTTP(s) requests and check that the way of handling the errors is the correct one.

Add "edit" option in the UI

The edit option will only allow to edit the fee recipients with the "solo" tag, at least for the moment

Set-up CI

Set-up Continuous Integration using gha:

• Source code compilation (build)
•  Docker builds
• Tests

Add License

Research for the most suitable License for this project and add it to the repository

Fix validator API for Teku (TLS)

Fix the validator API for Teku (using TLS)

Add header host

Host is required to perform requests to the web3signer

Implement dev mode in package `common`

Implement dev mode in package common

Add types to validator fee recipients

Add types to validator fee recipients

Implement `updatePubkeys` method in the database module

Implement updatePubkeys method in the database module to allow to update properties for a given pubkey

Implement db with `lowdb`

Implement db with lowdb library.

Consider implementing a `validatorStatus` to be render on the frontend

Consider implementing a validatorStatus to be render on the frontend similar to the already implemented web3signerStatus. Review which API call can be used to check the status.

Method to be implemented on validator API class. Depends on #22

Add missing DELETE keys from validator

We need to add the function deleteKeystores() in the validator API client file. It also needs to be tested with the 4 consensus clients.

Implement initial migration

Implement the initial migration in charge of getting the pubkeys, feeRecipients and tags to be written on the brain db. This is a critical step.

• The pubkeys must be fetched from web3signer API. Take into account the possible downtime.
• Consider adding to the signer service a healthcheck and to the brain service a deppends_on: signer to ensure sync between both services.
• The Fee recipients must be fetched from the Validator API.
• The tag will be a default solo
• ⚠️ Putting the default tag as solo may be dangerous when the migration has been executed un-intended

Add import web3signer keystores from API

The current implementation of POST keystores to web3signer supports imports from UI

export interface Web3signerPostRequest {
  keystores: File[];
  passwords: string[];
  slashingProtection: File | undefined;
}

It should also support imports from API with keystores in JSON format

Add to existing frontend all needed features

Add to the existing frontend all the needed features by the staking brain:

• Add fee recipient input and view
• Add tag input and view. Take into account to be read-only
• Add pubkey input for DVT protocols on the import page

Test all imports edge cases from UI

Test all the imports edge cases from the UI to test and polish the backend/frontend validators import functionality:

• Import 1 valid keystore, password, tag, and fee recipient. Expected result succeed
• Import multiple valid keystores, passwords, tags, and fee recipients. Expected result succeed
• Import wrong JSON file is allowed
• Import valid keystore, valid password, and either invalid tag or fee recipient. Expected result should not be allowed from the frontend
• Import a valid tag and fee recipient, and either an invalid keystore or password. Expected result should be allowed by the frontend and an error should be thrown by the backend and property shown on the backend

Parse correctly the signer response after DELETE

We are getting an error when parsing the JSON response after DELETE sent to signer (it returns the slashing protection):

Unexpected token \ in JSON at position 0

Implement logger

Implement a logger module to contemplate all the logs levels: info, error, debug and so on.

Set logger based on a process env LOG_LEVEL

Implement `useEffect` in `showWeb3SignerStatus`

The web3signerStatus must be re-fetched every short period of time. Use useEffect to achieve the needed behaviour.

Avoid cron being run while keys are being imported to the signer

When the cron is run while the keystores are being imported to the signer, those keys are then removed from the DB and the signer, so the import process is not completed correctly. This could also happen in other situations, like when the keys are removed from the signer.

An idea could be to implement 2 functions like startCron() and stopCron() that could avoid this two processes occurring simultaneously.

Add icons to show in UI for the tags

Add icons for Obol, solo staker, stakewise... To be shown in the UI. Moreover, the logo should show the tag string when hovering over it

Handle responses from fee recipient endpoint in validator API

GET and DELETE requests to the fee recipient endpoint of the validator return no data in the response, so we must handle the status code to know if the request was successful

Add to frontend-backend socket calls all the missing functionality

Right now the frontend-backend socket calls only perform requests against the Signer API. Add the missing functionality:

1. Signer API calls
2. Validator API calls
   2.1 On error WriteOnDb taking into account fee recipients
   2.2 On succeed WriteOnDb taking into account fee recipients

Implement Github container registry

Implement Github container registry to distribute the stakingbrain as a docker image

Bebug why the webpack bundle is not loggin top level application errors

Bebug why the webpack bundle is not loggin top level application errors

Add regex to import view inputs

Add regex to import view inputs, add proper tips

• Fee recipient must be a valid eth address

Add warning when changing fee recipient if tag!=solo

There are some protocols that have a fixed address for the fee recipient. Then, we won't allow them to be changed.

For solo stakers, the fee recipient can be freely changed with no warnings

For the rest of the protocols the fee recipient might depend on other factors, like Stakehouse. In this case, the fee recipient depends on the LSD network, so we need to add a warning when changing the fee recipient

Set file permissions to `db`

Use node JS permissions to handle which processes and modules has access to write/read the db.

Add private functions to the db class to handle the permissions. These functions must be called on write operations to ensure not to be written twice and on init and close functions.

Update Readme.md

Update Readme file to describe:

• How to setup the develop modes
• The envs required by the brain to work
• The docker images available
• ?

Add missing tls certificates for Teku mainnet and gnosis

Add missing tls certificates for Teku mainnet and gnosis to the folder packages/brain/tls

Don't allow setting the fee recipient to 0x00...000

Teku does not allow to perform fee recipient POST for burn address (0x00...00), so we might have to reject a user adding this fee recipient in the UI (and also in the launchpad API?)

Set-up CD

Implement a CD using gha to generate releases following semver standard. Assets to add to the release:

• Docker image
• Source code
• Validator and signer versions compatibility?
• Changelog

Throw error on BeaconchaApi and handle it on the front end

Throw the error on BeaconchaApi backend and handle it on the front end. This way we can handle the error based on our needs also in the backend

Implement `cron` module

Implement the cron module that will persist:

• pubkeys and fee recipients on the validator. Depends on #6
• pubkeys on localdb based on web3signer trueth

Implement gracefully shutdown

Implement a graceful shutdown to handle the signals: SHUTDOWN, SIGTERM, SIGKILL..

Node JS internal processes to close:

• UI API
• Launchpad API
• DB write/read operations
• Cron
• Socket
• ?

Implement webpack to bundle the backend (package `brain`)

Implement webpack to bundle the backend (package brain):

• Reduced the final image size
• Simplifies the final transpiled code to a single index.js file

Implement develop modes

Implement develop modes:

•  Standalone: this mode sets up all the elements required for the middleware to work (signer and validator) on a dev docker-compose. Harder to implmenet
• Non-standalone: this mode will work out of a docker container, running on the localhost. It will assume a previous set-up of signer and validator running. Easier to implement

Add check for fee recipient format in UI

The fields added to the UI for including a fee recipient must include the checks for format, not-empty fields, etc.

Implement Local Key Manager + Fee Recipient + Tag API

Implement an API to handle keystores + password + tag + Fee Recipient. Take as ref the Local Keymanager-API

• The API will listen to requests from staking protocols packages, such as Rocketpool and Stakehouse
• The API must redirect the POST requests:
  1. Keystores + passwords: web3signer
  2. Fee Recipient: validator
  3. Write on local db: pubkey + fee recipient + tag
• Consider implementing a DNS whitelist

Add to the standard JSON request the following fields:

{
  "keystores": [ "..."
  ],
  "passwords": [
    ".."
  ],
  "slashing_protection": "...",
  "tags": ["obol"],
  "feeRecipients": ["0x03F3..."],
  "pubkeys": ["0xe23..."]
}

Adjust same margin from items on the toolbar and main content

Adjust the items on the toolbar and the main content to fit the same margin

Implement validator API

Implement the validator API class to handle:

• Remote keys following this standard
• Fee recipients following this standard

For the usage of validator API take into account the following is required:

• Token API bearer
• Certificate for Teku

Client API`s classes should throw error on error

Afer moving the client API's from the frontend to the backend (see #17), the API methods must now throw an error instead of returning what is needed to the frontend. These methods will be used in different parts of the code and not only on the frontend. Look for another location to add the error messages needed for the frontend, such as in calls.

Implement `reloadPubKeys` and `reloadFeeRecipients` as modules

Implement reloadPubKeys and reloadFeeRecipients as modules to be reused in different parts of the source code:

• By the frontend
• By the cron daemon

The reloadPubKeys will be in charge of:

• Ensure the local db has the pubkeys loaded in the signer
• Ensure the validator has the pubkeys

The reloadFeeRecipients will be in charge of:

• Ensure the that for each pubkey, the feerecipient loaded in the validator corresponds with the one from the db

Deppends on #22

Fix error dialog when importing

When importing the key we need to add an option to close the dialog and also we need to show the errors that might be thrown in a "beautiful" way

Standard API must throw an error if http code !== 2XX

Standard API must throw an error if http code !== 2XX

Design and add all the types required

Think and implement all the types that will be required by this app:

• HTTP requests and errors to the signer
• HTTP requests and errors to the validator
• DB types:
  • tags
  • pubkeys
  • fee recipients
  • ...?
• Validator and signer status
• ?

Test all validator and signer API calls

Check each of the calls made from the brain both to the signer and to all 4 consensus clients