Use the following two providers as invulnerables:

https://github.com/paseo-network/paseo-action-submission/blob/main/paseo-system-parachain/asset-hub/active/CoinStudio.yml
https://github.com/paseo-network/paseo-action-submission/blob/main/paseo-system-parachain/asset-hub/active/turboflakes.yml

Paseo Testnet Runtime Upgrade Checklist

Preparation Tasks

• Upgrade Proposal Review: Confirm that the core team agreed on the upcoming Runtime Upgrade.
• Impact Analysis Completed: Which would be the impact of this Runtime Upgrade?.
• PR Creation:: Run the script to backport the changes from the Polkadot Fellowship Runtime and solve all conflicts.
• Benchmarking: Run benchmarks according to Paseo hardware specs and upload them as a different PR targeting the previous PR mentioned.
• Chopsticks testing: Fork Paseo relay-chain and its system chains and commit the targeted runtime upgrade.
  • Epoch and Era changes: Make chopticks to produce the necessary amount of changes to change the Era and assure blocks are still produced.
  • XCM transfers: Make a teleport to the Paseo assethub and PAS reserved transfer to any available parachain.
• Try-Runtime: Run try-runtime and mke sure that no state inconsistences are reported.
• Polkadot Fellowship contact: In case of any issues detected during testing, contact any Polkadot Fellowship member to address the concern. If no issues this can just be checked.
• Runtime upgrade on test instance: Commit the Runtime upgrade on the Paseo test instance and leave it running for a whole day.

Execution

• DB Backup: Ask any infrastructure provider to produce a backup of the network before commiting the upgrade.
• Notify Stakeholders: Distribute the communication plan through all relevant channels.
• Schedule Upgrade Window: Decide on and communicate the upgrade window to all stakeholders. Use always CET Timezone for scheduling or communicating deadlines.
• Final Pre-Upgrade Check: Conduct a last-minute check to ensure all systems and stakeholders are ready.
• Commit Upgrade: Initiate the runtime upgrade process according to the planned procedure. Use always CET Timezone for scheduling or communicating deadlines.
• Monitor Deployment: Closely monitor the network for any immediate issues during the upgrade.

Validation

• Verify Network Stability: Check that the network returns to normal operation post-upgrade.
• Confirm Upgrade Success: Validate that the new runtime version is correctly implemented across the network.
• Stakeholder Feedback: Collect and review feedback from validators, developers, and users.

Post-Upgrade

• Announce Completion: Inform stakeholders that the upgrade has been successfully completed.
• Post-Upgrade Review: Conduct a review meeting to discuss the upgrade process, identify any issues, and document lessons learned.

Troubleshooting

• Issue Tracking: Log any issues encountered during the upgrade in a dedicated tracker.
• Resolve Critical Issues: Prioritize and address any critical issues that arise.
• Communicate Ongoing Issues: Keep stakeholders informed about any unresolved issues and the plans for resolution.

updating to use latest paseo.raw.json client keeps crashing due to added lines

Jan 26 15:36:06 pc01 systemd[1]: polkadot.service: Main process exited, code=exited, status=1/FAILURE
Jan 26 15:36:06 pc01 systemd[1]: polkadot.service: Failed with result 'exit-code'.
Jan 26 15:36:16 pc01 systemd[1]: polkadot.service: Scheduled restart job, restart counter is at 8.
Jan 26 15:36:16 pc01 systemd[1]: Started "Rotko Networks - pso01 validator - Polkadot service".
Jan 26 15:36:16 pc01 polkadot[1771315]: Error:
Jan 26 15:36:16 pc01 polkadot[1771315]:    0: Invalid input: Error parsing spec file: failed to parse: Invalid multihash size 33. at line 9 column 4
Jan 26 15:36:16 pc01 polkadot[1771315]: Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Jan 26 15:36:16 pc01 polkadot[1771315]: Run with RUST_BACKTRACE=full to include source snippets.
Jan 26 15:36:16 pc01 systemd[1]: polkadot.service: Main process exited, code=exited, status=1/FAILURE
Jan 26 15:36:16 pc01 systemd[1]: polkadot.service: Failed with result 'exit-code'.

works
default_chain_spec_dl_url: "https://raw.githubusercontent.com/paseo-network/runtimes/8cb10b1404f3e1ec79aa72f88d182f6a3f38b3a3/chain-specs/paseo.raw.json"
latest doesnt
#default_chain_spec_dl_url: "https://raw.githubusercontent.com/paseo-network/runtimes/main/chain-specs/paseo.raw.json"

root@pso06:/opt/polkadot# diff paseo.raw.json paseo.raw.json.crash
7a8,9
>     "/dns/paseo-bootnode.radiumblock.com/tcp/30333/p2p/12D3KooWADeaC8zag4Qrb4GosSn65MmfVZztRPMaBdgZnQqXRo",
>     "/dns/paseo-bootnode.radiumblock.com/tcp/30335/wss/p2p/12D3KooWeayZC8zag4Qrb4GosSn65MmfVZztRPMaBdgZnQqXRo",

To work on the Polkadot Runtime: https://github.com/polkadot-fellows/runtimes/releases/tag/v1.1.2

We should substitute the default protocol id with other thing than "dot"

The following specs we keep in the repository would need addressing:

• paseo-local* is using "dot" as the protocol id.
• asset-hub-local* doesn't even specify a protocol id.

I'd suggest regenerating the local versions after amending the chain-spec-generator code

Enact simple runtime upgrade to test the process.

The crate versions for the runtimes have not been updated accordingly to the runtime spec versions.
We now have a mismatch on versioning numbers.

Include pallet_sudo to paseo runtime and add its genesis configuration to the chain spec.

When this PR is merged and release we could set the NetworId::Paseo in the XCM config.

The paseo core team is looking to deploy the asset hub, but first we want to do a cycle of local testing.

Branch: https://github.com/paseo-network/runtimes/tree/hb-paseo-assethub-testing

Even having everything set and configured correctly in the assethub chain spec, the parachain does not produce blocks locally when it should.

Below are the steps to reproduce the error:

Build chain-spec-generator

cd chain-spec-generator ; cargo build --release --features=fast-runtime

Create raw chains specs:

./target/release/chain-spec-generator paseo-local --raw > ../chain-specs/paseo.local.raw.json

./target/release/chain-spec-generator asset-hub-paseo-local --raw   > ../chain-specs/asset-hub-local.raw.json

Start relay chain with zombienet:

 ./zombienet-macos spawn --provider native -d /tmp/relay paseo-local.toml

paseo-local.toml :

[settings]
timeout = 1000

[relaychain]
chain_spec_path = "./paseo.local.raw.json"
default_command = "./polkadot"

 [[relaychain.nodes]]
 name = "alice"
 validator = true
 ws_port = 10000
 extra_args = ["--force-authoring -lparachain=debug --unsafe-rpc-external --rpc-cors=all"]

 [[relaychain.nodes]]
 name = "bob"
 validator = true
 extra_args = ["--force-authoring -lparachain=debug --unsafe-rpc-external --rpc-cors=all"]

 [[relaychain.nodes]]
 name = "charlie"
 validator = true
 extra_args = ["--force-authoring -lparachain=debug --unsafe-rpc-external --rpc-cors=all"]

 [[relaychain.nodes]]
 name = "dave"
 validator = true
 extra_args = ["--force-authoring -lparachain=debug --unsafe-rpc-external --rpc-cors=all"]

[types.Header]
number = "u64"
parent_hash = "Hash"
post_state = "Hash"

Once the relay chain started, start both collators:

./polkadot-parachain \
--collator \
--force-authoring \
--chain asset-hub-local.raw.json \
--base-path /tmp/parachain/alice \
--port 40333 \
--rpc-port 8844 \
--discover-local \
-- \
--execution wasm \
--chain paseo.local.raw.json \
--port 30343 \
--rpc-port 9977 

./polkadot-parachain \
--collator \
--force-authoring \
--chain asset-hub-local.raw.json \
--base-path /tmp/parachain/bob \
--port 40334 \
--rpc-port 8845 \
--discover-local \
-- \
--execution wasm \
--chain paseo.local.raw.json \
--port 30344 \
--rpc-port 9978 

Inject the following keys:

./polkadot-parachain key insert --base-path /tmp/parachain/alice \
  --chain asset-hub-local.raw.json \
  --scheme Ed25519 \
  --suri 0xd3a656b0d6ccacd7428da7fda811e6e1c89cc26035e649cbf680f7a6de8aa25e \
  --password-interactive \
  --key-type aura


./polkadot-parachain key insert --base-path /tmp/parachain/bob \
  --chain asset-hub-local.raw.json \
  --scheme Ed25519 \
  --suri 0x120870f7aeb7ace021ac3539be456dd09ba4fb456b6f3afd2c02f79c132410c9 \
  --password-interactive \
  --key-type aura

These are the keys used for both aura keys:

   bin ./polkadot-parachain key generate --scheme ed25519
Secret phrase:       conduct kitchen clump peasant acquire detail diagram balcony barrel census subway dove
  Network ID:        substrate
  Secret seed:       0xd3a656b0d6ccacd7428da7fda811e6e1c89cc26035e649cbf680f7a6de8aa25e
  Public key (hex):  0x2f430c4b3caebdb85bfe2ec034152fb2f110749b94283262bfb0226161f0b1f5
  Account ID:        0x2f430c4b3caebdb85bfe2ec034152fb2f110749b94283262bfb0226161f0b1f5
  Public key (SS58): 5D8g2Quafm4VDfsrDBxR4YYKwkeX2aKuwV7rkNDFN7bfdBS8
  SS58 Address:      5D8g2Quafm4VDfsrDBxR4YYKwkeX2aKuwV7rkNDFN7bfdBS8
  
➜  bin ./polkadot-parachain key generate --scheme ed25519
Secret phrase:       useful hundred vault mass vessel budget wait dutch expect mystery level quiz
  Network ID:        substrate
  Secret seed:       0x120870f7aeb7ace021ac3539be456dd09ba4fb456b6f3afd2c02f79c132410c9
  Public key (hex):  0xb6948fc8c313e24313cf6a42227be8cbd96dc71e7449beb4be35ee61a942282c
  Account ID:        0xb6948fc8c313e24313cf6a42227be8cbd96dc71e7449beb4be35ee61a942282c
  Public key (SS58): 5GC6hoCtbURddcXwtEg1rh1xZaTxyGz8urNpBBLvnwbG93Eu
  SS58 Address:      5GC6hoCtbURddcXwtEg1rh1xZaTxyGz8urNpBBLvnwbG93Eu

Please note we are using the right scheme (ed25519) taking into account the Polkadot AssetHub was launched with aura key under ed25519.

Paseo Testnet Runtime Upgrade Checklist

Preparation Tasks

• Upgrade Proposal Review: Confirm that the core team agreed on the upcoming Runtime Upgrade.
• Impact Analysis Completed: Which would be the impact of this Runtime Upgrade?.
• PR Creation:: Run the script to backport the changes from the Polkadot Fellowship Runtime and solve all conflicts.
• Benchmarking: Run benchmarks according to Paseo hardware specs and upload them as a different PR targeting the previous PR mentioned.
• Chopsticks testing: Fork Paseo relay-chain and its system chains and commit the targeted runtime upgrade.
  • Epoch and Era changes: Make chopticks to produce the necessary amount of changes to change the Era and assure blocks are still produced.
  • XCM transfers: Make a teleport to the Paseo assethub and PAS reserved transfer to any available parachain.
• Try-Runtime: Run try-runtime and mke sure that no state inconsistences are reported.
• Polkadot Fellowship contact: In case of any issues detected during testing, contact any Polkadot Fellowship member to address the concern. If no issues this can just be checked.
• Runtime upgrade on test instance: Commit the Runtime upgrade on the Paseo test instance and leave it running for a whole day.

Execution

• DB Backup: Ask any infrastructure provider to produce a backup of the network before commiting the upgrade.
• Notify Stakeholders: Distribute the communication plan through all relevant channels.
• Schedule Upgrade Window: Decide on and communicate the upgrade window to all stakeholders. Use always CET Timezone for scheduling or communicating deadlines.
• Final Pre-Upgrade Check: Conduct a last-minute check to ensure all systems and stakeholders are ready.
• Commit Upgrade: Initiate the runtime upgrade process according to the planned procedure. Use always CET Timezone for scheduling or communicating deadlines.
• Monitor Deployment: Closely monitor the network for any immediate issues during the upgrade.

Validation

• Verify Network Stability: Check that the network returns to normal operation post-upgrade.
• Confirm Upgrade Success: Validate that the new runtime version is correctly implemented across the network.
• Stakeholder Feedback: Collect and review feedback from validators, developers, and users.

Post-Upgrade

• Announce Completion: Inform stakeholders that the upgrade has been successfully completed.
• Post-Upgrade Review: Conduct a review meeting to discuss the upgrade process, identify any issues, and document lessons learned.

Troubleshooting

• Issue Tracking: Log any issues encountered during the upgrade in a dedicated tracker.
• Resolve Critical Issues: Prioritize and address any critical issues that arise.
• Communicate Ongoing Issues: Keep stakeholders informed about any unresolved issues and the plans for resolution.

Paseo Testnet Runtime Upgrade Checklist

Preparation Tasks

• Impact Analysis Completed: Which would be the impact of this Runtime Upgrade?.
• Upgrade Proposal Review: Confirm that the core team agreed on the upcoming Runtime Upgrade.
• PR Creation:: Run the script to backport the changes from the Polkadot Fellowship Runtime and solve all conflicts.
• Try-Runtime: Run try-runtime and mke sure that no state inconsistences are reported.
• Chopsticks testing: Fork Paseo relay-chain and its system chains and commit the targeted runtime upgrade.
  • Epoch and Era changes: Make chopticks to produce the necessary amount of changes to change the Era and assure blocks are still produced.
  • XCM transfers: Make a teleport to the Paseo assethub and PAS reserved transfer to any available parachain.
• Polkadot Fellowship contact: In case of any issues detected during testing, contact any Polkadot Fellowship member to address the concern. If no issues this can just be checked.
• Benchmarking: Run benchmarks according to Paseo hardware specs and upload them as a different PR targeting the previous PR mentioned.
• Runtime upgrade on test instance: Commit the Runtime upgrade on the Paseo test instance and leave it running for a whole day.

Execution

• DB Backup: Ask any infrastructure provider to produce a backup of the network before commiting the upgrade.
• Notify Stakeholders: Distribute the communication plan through all relevant channels.
• Schedule Upgrade Window: Decide on and communicate the upgrade window to all stakeholders. Use always CET Timezone for scheduling or communicating deadlines.
• Final Pre-Upgrade Check: Conduct a last-minute check to ensure all systems and stakeholders are ready.
• Commit Upgrade: Initiate the runtime upgrade process according to the planned procedure. Use always CET Timezone for scheduling or communicating deadlines.
• Monitor Deployment: Closely monitor the network for any immediate issues during the upgrade.

Validation

• Verify Network Stability: Check that the network returns to normal operation post-upgrade.
• Confirm Upgrade Success: Validate that the new runtime version is correctly implemented across the network.
• Stakeholder Feedback: Collect and review feedback from validators, developers, and users.

Post-Upgrade

• Announce Completion: Inform stakeholders that the upgrade has been successfully completed.
• Post-Upgrade Review: Conduct a review meeting to discuss the upgrade process, identify any issues, and document lessons learned.

Troubleshooting

• Issue Tracking: Log any issues encountered during the upgrade in a dedicated tracker.
• Resolve Critical Issues: Prioritize and address any critical issues that arise.
• Communicate Ongoing Issues: Keep stakeholders informed about any unresolved issues and the plans for resolution.

Use the chain spec tool to generate paseo network.

wip by slava