solana-labs / bench-tps-dos-test Goto Github PK

known host make ssh failed to access the instance

change time_start time_end time format to ISO03339 in dos report

[problem]
The "bank-process_transactions" datapoint, which was used by dos-report to calculate mean_tps, has been removed in v1.16.
solana-labs/solana#31398
[solution]
modify influx query similar to below
solana-labs/solana#31531

Problem

https://buildkite.com/solana-labs/bench-tps-dos-quic/builds/162
https://buildkite.com/solana-labs/bench-tps-dos-quic/builds/164

failed but they didn't delete those servers when exited.

Discord report time range has never changed. (slack report is ok)
It is alwasy "time range: 2022-09-27 06:08:31+00:00 ~ 2022-09-27 06:10:11+00:00"

[issue]
A request for a referenced mean_skip_rate right before the test so that we can compare the "stable" time with testing period.
[solution]
query the mean_skip_rate for 1 hr before bench-tps start

currently thread_batch_sleep_ms is a fix.
It is 10ms for QUIC and 1ms for UDP.
We need to make it adjustable in buildkite to be able to do some tests quickly.

Some validator submit the mainnet-beta value to testnet. This cause a significant outliner; therefore start_slot and end_slot in dos report are the value of mainnet-beta because it queries max value of start_time and end_time.
Instead of max value of optimistic_slot, the median value is suggested in slack "bench-tps-dos-result" channel.

buildkite is not able to show the log of bench-tps so this is not easy to debug
suggest to printout the log after remove the instance

We run multiple branch of solana on dos test but the report of bench-tps-dos-test only show commit number. It is hard to tell which report is for which branch.

current code build all binaries.
We can build only bench-tps to save time

Request from user who wants to see the bench-test during epoch change.
Current script can not start at specific time because it dynamically start the instances.
To solve this, if instances has started before the specific time, we can wait for the time specified to start the bench-tps test.

Buildkite failed due to
sudo apt-get install libssl-dev libudev-dev pkg-config zlib1g-dev llvm clang cmake make libprotobuf-dev protobuf-compiler

  | E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 4407 (unattended-upgr)
  | E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
  | + ret_pre_build='Your branch is up to date with '''origin/master'''.'
  | 🚨 Error: The command exited with status 100

error message in bench-tps
metrics disabled: SOLANA_METRICS_CONFIG: environment variable not found

running quic use "--use-tpu-client" make bench-tps failed because quic is the default in master branch.
--tpu-disable-quic to disable the quic
This means in different branches, master and v1.10, we need to use two flags to set the command line

[Problem]
Solana-bench-tps use quic as a default; howere TPU_USE_QUIC=false shows in log may lead to confusion.
[solution]
Remove TPU_USE_QUIC related code

The grafana time is as below.
gf_from=$(echo "scale=2;${start_time}1000-288001000" | bc)
gf_to=$(echo "scale=2;${stop_time}1000-288001000" | bc)
The 288001000 is here to adjust local time to UTC. (from previous script)
time here is UTC already so it does not need to minus 288001000

adding --simulate-mint-percent option for the bench-tps command-line in order to test high-priority tx in dos test