ava-labs / avalanche-ops Goto Github PK
View Code? Open in Web Editor NEWoperation toolkit for Avalanche nodes
License: Other
operation toolkit for Avalanche nodes
License: Other
As a public project, it would benefit users to have a CONTRIBUTING.md
doc that explains the nature of the project and how to contribute to it successfully. See https://github.com/operator-framework/rukpak/blob/main/CONTRIBUTING.md for an example of one.
Currently, we only tags VPC, ASG resources with user id, so for s3 bucket, it's hard to tell who created (and who forgot to clean up).
Use s3 put bucket tagging API to tag S3 buckets with sts user identity.
1) provision nodes
2) state sync Fuji
3) track custom subnet
4) deploy custom vm on a new blockchain on that subnet
We can get much higher TPS if gas fees are set right
[WORKER #25] failed transfer failed to send_transaction '(code: -32000, message: replacement transaction underpriced, data: None)'
Currently avalancheup-aws offers the ability to install a chain after using apply
to spin up a set of validators. When it comes to deleting the chain however, delete
only offers the ability to delete the entire network via the spec file. If a user wanted to delete a chain, but not delete the entire cluster, it could be useful. This could be done via adding to delete
or making a new, standalone command.
error[E0004]: non-exhaustive patterns: `ethers_core::types::Chain::Chiado` not covered
--> /Users/gyuho.lee/.cargo/registry/src/github.com-1ecc6299db9ec823/ethers-etherscan-1.0.0/src/lib.rs:81:29
|
81 | let api_key = match chain {
| ^^^^^ pattern `ethers_core::types::Chain::Chiado` not covered
|
note: `ethers_core::types::Chain` defined here
--> /Users/gyuho.lee/.cargo/registry/src/github.com-1ecc6299db9ec823/ethers-core-1.0.1/src/types/chain.rs:35:5
|
26 | pub enum Chain {
| --------------
...
35 | Chiado = 10200,
| ^^^^^^ not covered
= note: the matched value is of type `ethers_core::types::Chain`
help: ensure that all possible cases are being handled by adding a match arm with a wildcard pattern or an explicit pattern as shown
|
120~ }
121+ ethers_core::types::Chain::Chiado => todo!()
|
This is required for Hyper SDK dev net.
The dev-box should be able to run custom scripts (ie send transactions) on startup.
Currently it's in my private repo...
Eliminate the need for SSH + whitelisted-subnets + systemctl restart
for aws infra provisioning, why was this in implemented in rust as opposed to using existing cloud provisioning tooling (terraform and/or ansible)?
First of all avalancheup-aws
is a nice tool it helped me quickly setup nodes on AWS. While working on creating a new subnet and blockchain I noticed that the AvalancheGo
version that was installed 1.10.0
was less than the version I needed 1.10,2
. Once I pulled in a new binary and installed it things worked fine. Also during debug I updated the system unit file for AvalancheGo
.
The issue is when the system gets migrated I assume due to spot workflow.
AvalancheGo
I have not dug into this much but I assume this is related to how the state is managed in the spot instances. When a new VM is started it pulls the binary from s3 and probably the unit file as well.
A short section in the docs would be useful describing.
AvalancheGo
ref. #29
Does avalanche-ops support building resources on Azure instead of AWS? Many customers, particularly in the financial sector, prefer to use Azure. Does it make sense to generalize the tool and support multiple clouds?
Useful to support more static anchor nodes for bootstrapping test network
We just need addresses for x/c chains
When using avalancheup-aws apply
the end result is a list of endpoints and commands that can be run, for things like installing a new chain on the network. These commands are very helpful, but since they are only emitted to stdout, they can get lost in the terminal. I think writing the output of apply
to stdout and a file (with the same aops-
convention as the spec yaml file) could be helpful as a resource for users to retain access to all the auto-generated commands across terminals.
Use ava-labs/subnet-evm#357 to measure TPS
For instance,
# TYPE avalanche_2jo7eCZKHWG1qyYQ24SwBh8aB8h5bWhn7KCxYTk9PQTd2Tpzs_vm_eth_chain_txs_accepted counter
avalanche_2jo7eCZKHWG1qyYQ24SwBh8aB8h5bWhn7KCxYTk9PQTd2Tpzs_vm_eth_chain_txs_accepted 2971
# TYPE avalanche_2jo7eCZKHWG1qyYQ24SwBh8aB8h5bWhn7KCxYTk9PQTd2Tpzs_vm_eth_chain_txs_processed counter
avalanche_2jo7eCZKHWG1qyYQ24SwBh8aB8h5bWhn7KCxYTk9PQTd2Tpzs_vm_eth_chain_txs_processed 2971
thread 'tokio-runtime-worker' panicked at 'Integer overflow when casting to u64', /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/primitive-types-0.12.1/src/lib.rs:38:1
https://www.paradigm.xyz/2023/06/flood
Just like rustup
#342 does some QoL linting but there are still some unresolved clippy warnings.
Ideal set of steps:
#[allow(...)]
the code to have each warning accounted forAfter that we will have a fully linted codebase that is more production ready.
I would be nice to have a simple web UI or even a basic terminal only binary that manages the configuration files generated by avalancheup. This could allow for the configuration files to be managed in a centralized location and allow for easy tracking of which instances have been created/deleted.
MVP Functionality
dasconnor
has been showcasing http://backstage.io/ perhaps that would work here if we decide to use a web UI.
DEBUG[03-06|19:44:41.417] snow/engine/common/bootstrapper.go#224: Not enough votes received, restarting bootstrap... - Beacons: 2 - Failed Bootstrappers: 1 - bootstrap attempt: 85 (weight 200000000000000000, alpha 100000000000000001, failedBeaconWeight 100000000000000000)
DEBUG[03-06|19:39:06.881] snow/engine/common/bootstrapper.go#152: Received an Out-of-Sync GetAcceptedFrontierFailed - validator: 2bqPDyto7aHNfUvmKY7Ut2fnTgT9JvMJM - expectedRequestID: 2, requestID: 1
DEBUG[03-06|19:39:06.881] snow/networking/handler/handler.go#322: Finished handling sync message: get_accepted_frontier_failed
DEBUG[03-06|19:39:06.881] snow/networking/handler/handler.go#303: Forwarding sync message to consensus: (Op: get_accepted_frontier_failed, NodeID: NodeID-2bqPDyto7aHNfUvmKY7Ut2fnTgT9JvMJM, RequestID: 2)
DEBUG[03-06|19:39:06.881] snow/engine/common/bootstrapper.go#346: sent 2 more GetAccepted messages with 0 more to send
DEBUG[03-06|19:39:06.881] snow/networking/handler/handler.go#322: Finished handling sync message: get_accepted_frontier_failed
DEBUG[03-06|19:39:06.882] snow/networking/handler/handler.go#303: Forwarding sync message to consensus: (Op: accepted, NodeID: NodeID-AGzvo2xkGThWXu8JVYd9mVBqt2cf3iNM4, RequestID: 3, NumContainerIDs: 1)
DEBUG[03-06|19:39:06.882] snow/networking/handler/handler.go#322: Finished handling sync message: accepted
DEBUG[03-06|19:39:08.881] snow/networking/handler/handler.go#303: Forwarding sync message to consensus: (Op: get_accepted_failed, NodeID: NodeID-2bqPDyto7aHNfUvmKY7Ut2fnTgT9JvMJM, RequestID: 3)
DEBUG[03-06|19:39:08.881] snow/engine/common/bootstrapper.go#224: Not enough votes received, restarting bootstrap... - Beacons: 2 - Failed Bootstrappers: 1 - bootstrap attempt: 2 (weight 200000000000000000, alpha 100000000000000001, failedBeaconWeight 100000000000000000)
Just remove metrics collection service from avalanche-ops
When spinning up the dev-box in a region, currently the process to add the dev-box to the validator VPC is done manually. This could be automated as part of spinning up the dev-box. This would further enable the ability to run scripts automatically once the dev-box is created.
When running tools like avalancheup-aws frequently, many files get generated: spec files, prometheus rules, etc. These are all by default written to the user's home directory. It may make sense to move them into a separate hidden directory to not clutter the user's home directory, for example .aops
or .avalanche-ops
. Other avalanche tooling like avalanche-cli, avalanche-network-runner, and avalanchego all use a hidden home directory for their data.
This change would involve updating all the docstrings to point to the new directory.
AWS_VOLUME_PROVISIONER_BIN_PATH=/home/ubuntu/volume-manager/target/release/aws-volume-provisioner
AWS_IP_PROVISIONER_BIN_PATH=/home/ubuntu/ip-manager/target/release/aws-ip-provisioner
AVALANCHE_TELEMETRY_CLOUDWATCH_BIN_PATH=/home/ubuntu/avalanche-telemetry/target/release/avalanche-telemetry-cloudwatch
AVALANCHED_AWS_BIN_PATH=/home/ubuntu/avalanche-ops/target/release/avalanched-aws
AVALANCHEGO_BIN_PATH=/home/ubuntu/avalanchego/build/avalanchego
cd /home/ubuntu/avalanche-ops
/home/ubuntu/avalanche-ops/target/release/avalancheup-aws default-spec \
--arch-type amd64 \
--rust-os-type ubuntu20.04 \
--anchor-nodes 4 \
--non-anchor-nodes 4 \
--region ap-northeast-2 \
--instance-mode=on-demand \
--instance-size=4xlarge \
--ip-mode=elastic \
--metrics-fetch-interval-seconds 60 \
--ingress-ipv4-cidr 0.0.0.0/0 \
--upload-artifacts-aws-volume-provisioner-local-bin ${AWS_VOLUME_PROVISIONER_BIN_PATH} \
--upload-artifacts-aws-ip-provisioner-local-bin ${AWS_IP_PROVISIONER_BIN_PATH} \
--upload-artifacts-avalanche-telemetry-cloudwatch-local-bin ${AVALANCHE_TELEMETRY_CLOUDWATCH_BIN_PATH} \
--upload-artifacts-avalanched-aws-local-bin ${AVALANCHED_AWS_BIN_PATH} \
--upload-artifacts-avalanchego-local-bin ${AVALANCHEGO_BIN_PATH} \
--network-name custom \
--keys-to-generate 50 \
--enable-nlb \
--primary-network-validate-period-in-days 32
version: 2
id: aops-custom-202304-2TKPeY
aad_tag: avalanche-ops-aad-tag
resources:
identity:
account_id: '931867039610'
role_arn: arn:aws:sts::931867039610:assumed-role/jumpcloud-experimental-developer/[email protected]
user_id: AROA5R542S55AJJWVOGVF:[email protected]
region: ap-northeast-2
ingress_ipv4_cidr: 0.0.0.0/0
s3_bucket: avalanche-ops-202304-42ziec8wgg-ap-northeast-2
kms_symmetric_default_encrypt_key:
id: 95c87189-0e73-4462-ba31-3befb680888c
arn: arn:aws:kms:ap-northeast-2:931867039610:key/95c87189-0e73-4462-ba31-3befb680888c
ec2_key_name: aops-custom-202304-2TKPeY-ec2-key
ec2_key_path: /home/ubuntu/aops-custom-202304-2TKPeY-ec2-access.key
cloudformation_ec2_instance_role: aops-custom-202304-2TKPeY-ec2-instance-role
cloudformation_ec2_instance_profile_arn: arn:aws:iam::931867039610:instance-profile/aops-custom-202304-2TKPeY-instance-profile
cloudformation_vpc: aops-custom-202304-2TKPeY-vpc
cloudformation_vpc_id: vpc-0439425269f4ebc93
cloudformation_vpc_security_group_id: sg-04b6f70bcf8cfa4df
cloudformation_vpc_public_subnet_ids:
- subnet-035f4501874063a4f
- subnet-0ec6e08f0adbc5301
cloudformation_asg_anchor_nodes:
- aops-custom-202304-2TKPeY-anchor-amd64-01
- aops-custom-202304-2TKPeY-anchor-amd64-02
- aops-custom-202304-2TKPeY-anchor-amd64-03
- aops-custom-202304-2TKPeY-anchor-amd64-04
cloudformation_asg_anchor_nodes_logical_ids:
- aops-custom-202304-2TKPeY-anchor-amd64-01
- aops-custom-202304-2TKPeY-anchor-amd64-02
- aops-custom-202304-2TKPeY-anchor-amd64-03
- aops-custom-202304-2TKPeY-anchor-amd64-04
cloudformation_asg_non_anchor_nodes:
- aops-custom-202304-2TKPeY-non-anchor-amd64-01
- aops-custom-202304-2TKPeY-non-anchor-amd64-02
- aops-custom-202304-2TKPeY-non-anchor-amd64-03
- aops-custom-202304-2TKPeY-non-anchor-amd64-04
cloudformation_asg_non_anchor_nodes_logical_ids:
- aops-custom-202304-2TKPeY-non-anchor-amd64-01
- aops-custom-202304-2TKPeY-non-anchor-amd64-02
- aops-custom-202304-2TKPeY-non-anchor-amd64-03
- aops-custom-202304-2TKPeY-non-anchor-amd64-04
cloudformation_asg_nlb_arn: arn:aws:elasticloadbalancing:ap-northeast-2:931867039610:loadbalancer/net/aops-custom-202304-2TKPeY-nlb/7dc1e1fb90b53789
cloudformation_asg_nlb_target_group_arn: arn:aws:elasticloadbalancing:ap-northeast-2:931867039610:targetgroup/aops-custom-202304-2TKPeY-tg/5d50684ef30aac69
cloudformation_asg_nlb_dns_name: aops-custom-202304-2TKPeY-nlb-7dc1e1fb90b53789.elb.ap-northeast-2.amazonaws.com
cloudformation_asg_launch_template_id: lt-083db80a88191d265
cloudformation_asg_launch_template_version: '1'
cloudformation_ssm_install_subnet_chain: aops-custom-202304-2TKPeY-ssm-install-subnet-chain
cloudwatch_avalanche_metrics_namespace: aops-custom-202304-2TKPeY-avalanche
created_nodes:
- kind: anchor
machineId: i-004218e7af59e6e3c
nodeId: NodeID-9Cf6ZXc5xUjfS63ZZY6Qe5PtUdDWEE2Eg
proofOfPossession:
publicKey: 0xae84eb755fb323debc330252978275886c5558254dd61f76b3036c1eb67d72811fb7849a40e9878f8022c47aed1bf479
proofOfPossession: 0xb9e4a894161b54f25c8fb48b0fc181019728f92a78b263faaa6d711461cd2689b53c84b69c83a1258c3d4cf0c5224a6208490fa8f5f828a38b61296c7118c05503b35e16e967d7c
be23308afa2d211c1b5ceb8df797fbda492b66738010af26c
publicIp: 43.200.185.164
httpEndpoint: http://43.200.185.164:9650
- kind: anchor
machineId: i-0004f1cda22ee6d84
nodeId: NodeID-AGYHGNq3CZjuZ54fsJdrPiPvtChKTejZr
proofOfPossession:
publicKey: 0xb12778e3660bfc2f76741784f1c1e76ba58894178c88db1340879964bd175cb0d7ff7e3050bc22060164a89a9223ff97
proofOfPossession: 0xb654102321216e99686c7a0645093f48fecf7531efbbf402fda9810ada92782b9c6fe564555e3d947fb89f290a46850a0424f8a6f82636eea91bb8ea3b86ddd2fba79c3bf7216f5
eacd972419766e3c663fda293837e6f907888b2ab90435e43
publicIp: 43.201.208.149
httpEndpoint: http://43.201.208.149:9650
- kind: anchor
machineId: i-0779d859b49002dd8
nodeId: NodeID-5veLTLisivPaHokQYZpb7usoYw52u6hD8
proofOfPossession:
publicKey: 0x96b5c7c898c61f30c24bcc611d1cdf211091b2ba030452e29f02952d218cfa0b7c31306a141407da21e566dfa4faf914
proofOfPossession: 0x8eb7ce0d1c60256d6888cb1b8d6099f6a7a2f3f67fcaa856f8ab1dd775ebf6a2f79d6539e2d92d9e3449624791482a0a0baebebbb7ee1f7bb3bf65bcc9a4b4d3e1d061cad1dfea0
171d7893885ee546793e2b99f94b1ebe8d4659458ea0a5c36
publicIp: 52.78.19.151
httpEndpoint: http://52.78.19.151:9650
machine:
anchor_nodes: 4
non_anchor_nodes: 4
arch_type: amd64
rust_os_type: ubuntu20.04
instance_types:
- m5.4xlarge
- c5.4xlarge
instance_mode: on-demand
ip_mode: elastic
volume_size_in_gb: 300
avalanched_config:
log_level: info
use_default_config: false
enable_nlb: true
disable_logs_auto_removal: false
metrics_fetch_interval_seconds: 60
primary_network_validate_period_in_days: 32
prefunded_keys:
- key_type: hot
private_key_cb58: PrivateKey-ewoqjP7PxY4yr3iLTpLisriqt94hdyDFNgchSxGGztUrTXtNN
private_key_hex: 0x56289e99c94b6912bfc12adc093c9b51124f0dc54ac7a766b2bc5ccf558d8027
addresses:
1000000:
x: X-custom18jma8ppw3nhx5r4ap8clazz0dps7rv5u9xde7p
p: P-custom18jma8ppw3nhx5r4ap8clazz0dps7rv5u9xde7p
short_address: 6Y3kysjF9jnHnYkdS9yGAuoHyae2eNmeV
eth_address: 0x8db97C7cEcE249c2b98bDC0226Cc4C2A57BF52FC
h160_address: 0x8db97c7cece249c2b98bdc0226cc4c2a57bf52fc
AWS_VOLUME_PROVISIONER_BIN_PATH=/home/ubuntu/volume-manager/target/release/aws-volume-provisioner
AWS_IP_PROVISIONER_BIN_PATH=/home/ubuntu/ip-manager/target/release/aws-ip-provisioner
AVALANCHE_TELEMETRY_CLOUDWATCH_BIN_PATH=/home/ubuntu/avalanche-telemetry/target/release/avalanche-telemetry-cloudwatch
AVALANCHED_AWS_BIN_PATH=/home/ubuntu/avalanche-ops/target/release/avalanched-aws
AVALANCHEGO_BIN_PATH=/home/ubuntu/avalanchego/build/avalanchego
cd /home/ubuntu/avalanche-ops
/home/ubuntu/avalanche-ops/target/release/avalancheup-aws default-spec \
--arch-type amd64 \
--rust-os-type ubuntu20.04 \
--anchor-nodes 4 \
--non-anchor-nodes 4 \
--auto-regions 3 \
--instance-mode=on-demand \
--instance-size=4xlarge \
--ip-mode=elastic \
--metrics-fetch-interval-seconds 60 \
--ingress-ipv4-cidr 0.0.0.0/0 \
--upload-artifacts-aws-volume-provisioner-local-bin ${AWS_VOLUME_PROVISIONER_BIN_PATH} \
--upload-artifacts-aws-ip-provisioner-local-bin ${AWS_IP_PROVISIONER_BIN_PATH} \
--upload-artifacts-avalanche-telemetry-cloudwatch-local-bin ${AVALANCHE_TELEMETRY_CLOUDWATCH_BIN_PATH} \
--upload-artifacts-avalanched-aws-local-bin ${AVALANCHED_AWS_BIN_PATH} \
--upload-artifacts-avalanchego-local-bin ${AVALANCHEGO_BIN_PATH} \
--network-name custom \
--keys-to-generate 50 \
--enable-nlb \
--primary-network-validate-period-in-days 32
version: 3
id: aops-custom-202304-zeB7h6
aad_tag: avalanche-ops-aad-tag
resource:
regions:
- us-west-2
- ap-northeast-2
- eu-west-1
s3_bucket: avalanche-ops-202304-42ziec8wgg-us-west-2
ingress_ipv4_cidr: 0.0.0.0/0
regional_resources:
us-west-2:
region: us-west-2
ec2_key_name: aops-custom-202304-zeB7h6-ec2-key
ec2_key_path: /home/ubuntu/aops-custom-202304-zeB7h6-ec2-access.us-west-2.key
eu-west-1:
region: eu-west-1
ec2_key_name: aops-custom-202304-zeB7h6-ec2-key
ec2_key_path: /home/ubuntu/aops-custom-202304-zeB7h6-ec2-access.eu-west-1.key
ap-northeast-2:
region: ap-northeast-2
ec2_key_name: aops-custom-202304-zeB7h6-ec2-key
ec2_key_path: /home/ubuntu/aops-custom-202304-zeB7h6-ec2-access.ap-northeast-2.key
machine:
total_anchor_nodes: 4
total_non_anchor_nodes: 4
arch_type: amd64
rust_os_type: ubuntu20.04
instance_mode: on-demand
ip_mode: elastic
volume_size_in_gb: 300
regional_machines:
eu-west-1:
anchor_nodes: 2
non_anchor_nodes: 2
instance_types:
- c6a.4xlarge
- m6a.4xlarge
- m5.4xlarge
- c5.4xlarge
us-west-2:
anchor_nodes: 1
non_anchor_nodes: 1
instance_types:
- c6a.4xlarge
- m6a.4xlarge
- m5.4xlarge
- c5.4xlarge
ap-northeast-2:
anchor_nodes: 1
non_anchor_nodes: 1
instance_types:
- m5.4xlarge
- c5.4xlarge
upload_artifacts:
avalanched_local_bin: /home/ubuntu/avalanche-ops/target/release/avalanched-aws
aws_volume_provisioner_local_bin: /home/ubuntu/volume-manager/target/release/aws-volume-provisioner
aws_ip_provisioner_local_bin: /home/ubuntu/ip-manager/target/release/aws-ip-provisioner
avalanche_telemetry_cloudwatch_local_bin: /home/ubuntu/avalanche-telemetry/target/release/avalanche-telemetry-cloudwatch
avalanchego_local_bin: /home/ubuntu/avalanchego/build/avalanchego
prometheus_metrics_rules_file_path: /home/ubuntu/aops-custom-202304-zeB7h6-prometheus-metrics-rules.yaml
avalanched_config:
log_level: info
use_default_config: false
enable_nlb: true
disable_logs_auto_removal: false
metrics_fetch_interval_seconds: 60
primary_network_validate_period_in_days: 32
prefunded_keys:
- key_type: hot
It's possible to optimize the Rust CI by caching the cargo home directory, or specifically certain files under .cargo
, and then sharing those files between different CI jobs. This avoids the need to re-download dependencies during multiple jobs.
See https://doc.rust-lang.org/cargo/guide/cargo-home.html#caching-the-cargo-home-in-ci for more details.
This could potentially be done for all of our Rust projects. Not sure which have this implemented already.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.