Discussed in #7

Clicking to see the attachment doesnt do anything.
Tried removing the discord and joining again. No win there.

Suibase internally uses curl to talk with internal processes (e.g. sui-faucet).

Curl might not target the right process when the user has the 'http_proxy' environment variable set.

Suibase should keep full control on the target ip:<port>

This problem was reported by adu on chainMovers Discord:

https://discordapp.com/channels/1038616996062953554/1081836316330831892/1109372029720858705

What are the best practices for managing your keystores during move development?

How to setup the same keystore for local / dev / test ? Or is this a bad idea? I think it would make life easier as you can use one pvt key which you can also add to your chrome wallet for dev and test networks.

Using different git e-mails accounts on commits will result duplicated cookbook contributors

In particular, need to find a workaround because the logs that were directed to a file are now on stdout (looks mess up):

Example:

                             Object ID                              |  Gas Value
----------------------------------------------------------------------------------
2023-05-23T19:33:52.128059Z  INFO sui::client_commands:                              Object ID                              |  Gas Value
 0x524ec20a221f904cf539a257a4bfbb88a4f5fa6bba4407739eb830cc4c16913b | 30000000000000000
2023-05-23T19:33:52.128229Z  INFO sui::client_commands: ----------------------------------------------------------------------------------
 0x6075d0a77cd1f5f6b70dbf193741d337bb0aee109d33cb342360e5108dfabfac | 30000000000000000
2023-05-23T19:33:52.128272Z  INFO sui::client_commands:  0x524ec20a221f904cf539a257a4bfbb88a4f5fa6bba4407739eb830cc4c16913b | 30000000000000000
2023-05-23T19:33:52.128311Z  INFO sui::client_commands:  0x6075d0a77cd1f5f6b70dbf193741d337bb0aee109d33cb342360e5108dfabfac | 30000000000000000
 0x83607afb0abb1806ec01edb62bfdbe9c96ee1cce7326a774ff710d11a31956e8 | 30000000000000000
 0xb6137142439a0fcc144c67ff189f2d072bfcd586e21e00937857f49ffb7608a8 | 30000000000000000
2023-05-23T19:33:52.128359Z  INFO sui::client_commands:  0x83607afb0abb1806ec01edb62bfdbe9c96ee1cce7326a774ff710d11a31956e8 | 30000000000000000
 0xfe295bf599eef8ec9ed43399f656b00d9b9c03ffbace604d06dc81f32e4e8c61 | 30000000000000000
2023-05-23T19:33:52.128443Z  INFO sui::client_commands:  0xb6137142439a0fcc144c67ff189f2d072bfcd586e21e00937857f49ffb7608a8 | 30000000000000000
2023-05-23T19:33:52.128588Z  INFO sui::client_commands:  0xfe295bf599eef8ec9ed43399f656b00d9b9c03ffbace604d06dc81f32e4e8c61 | 30000000000000000

Change to demo-app because of Sui SDK API changes

Provide ability to have multiple-instances of localnet running

How do I go from my Suibase required base64 private key to 

TS SDK required pvt key which is the same as the pvt key in the chrome wallets.

const keypair = Ed25519Keypair.fromSecretKey(Buffer.from(privateKey, 'base64'));

The pvt key from the chrome wallets don't work on a regen.

Thx

When doing "localnet publish" following would be useful:

• the "lsui move publish" always have the --json option set.
• Save the output under published-data with a fix name (so SDK or user can retrieve it).
• Have that Json data readable through the Rust SuiBase Helper.

====

Note: Since the normal output is disabled with --json, the package-id.json will need to be identified from parsing that json output.

Each workdir can run a sui-base background process for additional services.

"devnet status" should answer seperatly:

• Is the sui-base service RUNNING/STOPPED
• Is the network RESPONDING/DISCONNECTED

Example of services:

• short-term: Automated devnet faucet RPCs to keep refueling client addresses
• long-term: Reliable account management (against multi-apps/thread equivocation problems)

Services will be configurable in the "sui-base.yaml"

Note: That background process will be written in Rust. Will not be specific to devnet, but generalize to run one instance per workdir.

(base) ➜  move git:(main) localnet start 
Starting localnet process
..............................(may take some time on slower system).....................
Sui process not responding. Try again? (may be the host is too slow?).

Will check if I get a localnet running using the suidocs.
Its not that I changed my laptop.

Will show error message:

localnet workdir not initialized

Do one of the following:
        localnet start (recommended)
        localnet create

Type "localnet --help" for more options```

Hello,

I had the impression that the local network was reset at every stop. The objects I created during start_stop_1 didn't appear in the suiexplorer after a new local network start. Now I figured out they do show up but only after 1min.
Any idea why they show up only after 1 min?

,KR GCB

RPC link information for every workdir will be provided by a .state/links JSON file.

The file provides:

• All the links configured by the user.
• Sui-base recommendation of which links to use first and second (for a retry).

Implementation details:
The file can change at anytime (using a safe atomic filesystem mv, not a risky in-place file/inode modifications). From a helper perspective, you just open/read/close the file, use the primary and secondary recommendation. Re-open the file again for a new set of recommendation.

This is continuation of phase I (#52)

===

At this point, all major proxy server internal components are "verified working" but each needs more refinement.

It has been verified that the proxy server does its job (can have multiple Sui apps use the proxy while it alternates between two remote servers).

===

What need to be completed in this second phase:

• integration with start/stop/status suibase scripts
• support proper fallback logic (right now monitors health, but just pick the first server that works).
• retry logic (but logic for per request JSON parsing and decision is in-place).
• better integration with suibase.yaml (hot-reload).
• JSON-RPC API (for stats + health status of the remote servers).

A Sui PR breaks the suibase scripts using keytool:
MystenLabs/sui#13196

--> easy fix by replacing "keytool base64-to-bytes" with base64 and xxd utils.

Will affect user trying to load private address from suibase.yaml

What are the best practices to deploy to mainnet?

How to use a hardware wallet and multi-sig?

Investigate. If implemented, should be configurable through sui-base.yaml:

https://forums.sui.io/t/running-local-gas-faucet-url/25773/4

I reviewed the docs from start to finish and took notes using hypothesis whenever something came up.

You can find my notes via https://hypothes.is/search?q=suibase.io.

I hope it can be of any use.

Thx for the toolset, much appreciated.

Look into a way to add a discord icon/link prior to the github icon on the top navbar:

This can probably be implemented with the "Using Slots" feature described here:
See https://theme-hope.vuejs.press/guide/advanced/replace.html

If not possible, then consider using our new HTML post-processing feature ( #35 )

Problem Statement
In the Rust demo-app, the URL used by SuiClient is currently setup for "localnet".

When trying to adapt the demo to work with an "active" sui-base workdir (can change from one execution to another), it is inconvenient because the app have to figure out how every possible "workdir name" translate to which URL.

Solution
Distinct configurable value like RPC_URL, WS_URL need to be provided in a sui-base.yaml in every workdir.

The configuration should be generated (once) with default, and allowed to be further modified by the developer.

Eventually, all this should end up to be "well documented" since it is about building a common foundation for many SDK to count on.

(I think Frank was suggesting something similar recently coming from a different angle).

This is for the new Suibase VSCode extension.

The Suibase GUI interface to streamline Move development with a "network viewer" (to watch/debug last published modules).

===

First phase of development is for a "proof-of-concept" to verify that the extension can:

• Find the Suibase installed on MacOS, WSL2 and Ubuntu.
• Read access to all suibase.yaml
• Call into sui clients for every workdir.
• Call into external bash script
• Have webview panel do JSON-RPC with the local suibase daemon.

A user requested to add to suibase the capability to download the latest repo and build binaries only (not start process, not create sui.keystore, no localnet genesis etc...).

This will be done with a new "build" command.

Example:
$ localnet build
... build everything normally needed for localnet ...
$ localnet build -p sui-node
... build what is done normally + additional options (will build one more binary here) ...

===

The command can obviously mix with the others, so a user can follow with a "start" after a "build".

Discussed in #3

Latest action items (may change):

• Add a cargobinsui cargobin workdir when detecting a Sui binary on .install and make it active (if no other workdir already active).
• Consider to detect and create the workdir (silently in background) in other places (e.g. in common script init code).
  * Add "asui base list" and "asui base set " to more conveniently switch workdir.

This feature is already completed for Linux and Windows WSL2 ( See #68 )
As of right now, the binary built by Mysten Labs does not always work on MacOS (occasional "illegal instructions" on my M1 setup).

Consequently this feature is currently disabled for MacOS (reverts to building from source).

(1) Move the sui-base-helper outside of demo-app

(2) Changes to follow Rust API guidelines:
https://rust-lang.github.io/api-guidelines/about.html

Object-based API (sui-base helper is already there, just putting this here for completion):
   https://rust-unofficial.github.io/patterns/patterns/ffi/export.html

I've noticed that 1 network workdir takes up around 30GB and that it has something to do with the target/debug folders of cargo.

A note in the docs next to the update statement about the size and using cargo clean or another automated process for managing the size could be useful I think.

sui-base helper should provide a mean to get client addresses "by name".

This is for test setup that need either multiple address at same time, or coordinate to have various software "parts" not collide using the same.

For now, only genesis address will have a name and they can't be changed.

Simple Implementation
Information will be available in a read-only JSON map:

~/sui-base/workdirs/<workdir name>/.state/dns

{
    "known": {
        "<address name\>": { "address": "0x..." },        
    }
}

Example:

{
    "known": {
        "sb-1-ed25519": { "address": "0x038fddd909e46fd8853a518a3c7518c67e51bc5a" },
        "sb-2-ed25519": { "address": "0x130e3a131cd0c5249114e3d0cc4ba78ff61dd482" },
        ...
        "sb-5-ed25519": { "address": "0x2c3feb9d53b3507377c81f2b5de87ea8ea6385d2" },
        "sb-1-secp256k1": { "address": "0x464e289e078ef5560ffa3d6ea0e61795780d9826" },
        "sb-2-secp256k1": { "address": "0x92f23d8a5a48dd0e2d4052f0a9e757fcb39fc85b" },
        ...
    }
}

String format is:
sb-<index>-<key type>

Future Plan (thought for fun):
User dynamic "DNS" is not plan unless there is a use-case for it AND a sui-base service daemon is implemented (to handle concurrency). Another potential variation would be a pre-funded address pool.

This is for the major Multi-Link RPC feature.

--> First step is a simple proof-of-concept Rust process to proxy JSON-RPCs to one or two servers (altneratively?).

--> See to re-use existing open-source tokio/tower JSON-RPC projects.

I have a sui repo setup where I pull down the latest (git pull)
I then build (cargo build --locked --bin sui --bin sui-faucet)
Then localnet start - Which works fine

I do this for testing delta's from the last build in pysui development.

I then do localnet stop

To test a branch in the repo I then do git checkout <BRANCH_NAME>
Do the build (works OK)

However; if doing locanet start again I received this today:

localnet start
Starting localnet process
..............................(may take some time on slower system)........................
Sui process not responding. Try again? (may be the host is too slow?).
fastfrank@~/suibase/workdirs/mnet/sui $ localnet status
localnet DOWN
---
localnet process : DEAD
faucet process   : DEAD
---
client version: sui 1.5.0-62d796190f
asui selection: [ localnet ]
set-sui-repo  : /Users/fastfrank/suibase/workdirs/mnet/sui
fastfrank@~/suibase/workdirs/mnet/sui $ localnet start
Starting localnet process
./Users/fastfrank/suibase/scripts/common/__globals.sh: line 1114: 81473 Segmentation fault: 11  "$SUI_BIN_DIR/sui" start --network.config "$NETWORK_CONFIG" >&"$CONFIG_DATA_DIR/sui-process.log"
...........^C

I killed it. I think this may be the daemon not acting nice.

(was "Add secp256k1 address for localnet")

Will make genesis of localnet both a new command, but also configurable with sui-base.yaml

On short term will just create 15 addresses. 5 of each key scheme.

Goal is to help the user distinguish if RPCs are failing because:

• bad request (user need to fix it for succeeding)
• the network (retry later when network is OK)

As of right now, the proxy does not reflect JSON-RPC level errors in its statistics (but it is still seen transparently as an error to the proxy caller of course).

The proxy interpret and act only on errors at lower protocol layers (IP, HTTP), but not at "JSON-RPC" level.

The enhancement is to interpret the JSON-RPC to update the statistic and better reflect the "end result".

Related info: MystenLabs/sui#12989

Problem reported by @FrankC01

suibase-daemon fails to start.

Problem started after recent commit (which enable proxy server feature by default).

===
After investigation, found that the command line 'flock' is not installed on MacOS.

flock(1) is the lock mechanism used by suibase to make sure there is never more than one suibase-daemon running.

Discussed in #8

objects.md from Code Snippets can't be loaded into a web page.

Some constants from js assets contain different type of quote ( e.g instead ' they contain ` )

python post_processing_html.py generated inside assets/objects.html-31c214c4.js :

const e=JSON.parse(' {"key": ...... ' ); export{e as data};

instead of

const e=JSON.parse(` {"key": ...... ` ); export{e as data};

"localnet start" will not work without this fix.

(the sui process will never "start")

implementation details

Sui-base depends on using the config.yaml to initialize the localnet.
The format changed in 0.31 (new gas_amount array).
The changes are enough that the sui process will fail to parse config.yaml (older format) and will not start.

"tsui client publish --gas-budget 100000000"
Unable to find package manifest in '/home/xxx/suibase/workdirs/testnet/logs/sui.log' or in its parents

Does publishing via the client not work anymore, and are you obliged to use "testnet publish"?

top of error message:
walue@walue-ABRA-A5-V15-4:~/suibase$ localnet start
Building suibase-daemon
Compiling suibase-daemon v0.0.2 (/home/walue/suibase/rust/suibase/crates/suibase-daemon)
error: linking with cc failed: exit status: 1
|
= note: LC_ALL="C" PATH="/home/walue/.rustup/toolchains/1.70-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:

bottom of errror message:
= note: some extern functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the -l flag to specify native libraries to link
= note: use the cargo:rustc-link-lib directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile suibase-daemon (bin "suibase-daemon") due to previous error
Error: Fail to build suibase-daemon

I installed Sui prerequisites. i'm getting this error when first time run localnet start.
Rustc version: rustc 1.72.0 (5680fa18f 2023-08-23)

~/sui-base$ localnet faucet --help
Error: Invalid hexadecimal address [--help]

Should help like this instead:

~/sui-base$ localnet faucet
http://0.0.0.0:9123
Address: 0x265254cffb43b73101579fecd21884d34336b3f23f61428a4e681bd96857ad66
Balance: 4000000000000

Usage: localnet faucet <ADDRESS 1> ... <ADDRESS n> | all

Do:

• a "cargo clean" while preserving the produced binaries.
• logs trimming (particularly meaningful for localnet).

Can be trig on command line in 2 ways:

• As an independent workdir command (e.g. "localnet clean-up")
• As an option to any other workdir command (e.g "localnet update --clean-up"

Building the Sui client binaries have two major pain points:

(1) lots of storage space (>25 GB) mostly because of the intermediate build files.
(2) building time from ~3 min to >1hour depending of the user setup.

Because of the build time, suibase choose to keep the intermediate build files (no "cargo clean" after build). In short, making a compromise that assumes devs prefer to save time over cost of additional storage.

A better solution is to eliminate the building step and replace with downloads of pre-compiled binaries. No storage, no build time (the Mysten Labs repo is still cloned locally).

The binaries are the official ones now being built from Mysten Labs.

===

This release will be for Linux and Windows WSL2/Ubuntu.

MacOS may follow up later (there are some issues with running the binary provided by Mysten Labs).

The "-" in the project causes unnecessary confusion for namespace/package name convention.

Therefore, a lot of renaming to "suibase" will be done.

Proposing the UX to be:

• User activate the pip they want suibase into.

• Type '~/suibase/pip-install'

• This builds the Python package for their machine (if not already done) and install it in the activated python environment.

• Calling the script again simply update the Suibase python package (as needed).

Took the idea from ( https://pyo3.rs/v0.18.3/ )

This method keeps the python package in-sync with the local suibase version.

Having some issues here with suibase.

steps taken:
~/suibase/update
Current version 0.1.5-1b9b167

devnet update
devnet_proxy │ http://0.0.0.0:44341 │ * │
devnet │ https://fullnode.devnet.sui.io:443

"dsui client gas" gives
Request timeout

I had done the standard sui binaries install Thu to final test and publish 0.15.0

Back to sui-base...
Here is the output with no local started:

fastfrank@~/sui-base $ localnet status
localnet STOPPED
client version: sui 0.29.1-d6ebeaa32
asui selection: [ cargobin ]

I assumed (because of the last line) I could do asui client gas:

fastfrank@~/sui-base $ asui client gas
The /Users/fastfrank/.cargo/bin/sui was not found.
Follow Mysten Lab procedure to install it:
 https://docs.sui.io/build/install#install-sui-binaries

Even though I can do standard:

fastfrank@~/sui-base $ sui client gas
[warn] Client/Server api version mismatch, client api version : 0.29.1, server api version : 0.29.2
                             Object ID                              |  Gas Value
----------------------------------------------------------------------------------
 0x0d7fe4b9e8a3341337eb137dce3d8ce83e83234fdf9e850d0290a32a2620550e |  119997677
 0x13d88577a38b4b9550ecb2eb84ea94e4727129759f525e84cd9d331c4fb64117 |  199991896
 0x16371a61f3d09c0ad97cd988e4414a6d09e7944cb97f36bb4f9db48f45601f94 |  200000000
 0x2274c7e7a1db7be505634c310fe9fb546a110398dc03f375f4843d0ace22cf05 |  200000000
 0x33dafd34e581bd1909fc7a0f8173180789ed0f22072f8e6d93613e968301e000 |   4000000
 0x5b8495d388be1cfdb083741db43d2f32821393eba39201ed06ca6a28ac7b2ffa |  200000000
 0x64ffb2dc69cbbdb083ee71f2f8095dea09d1ddfde262600bfb87bd3cb0630207 |  200000000
 0x9bd90725a3ede448064b8b5e61794868657e7d9ac5873fe6a970898efaa2625a |   4000000
 0x9ed1ee41fa18a9260eab9ae706536709d4b15f1d8eb9b0498152ab9c11a0bbe8 |  10000000
 0xb29fe3475a821c98c5720e8cc04fb83ab8fbd32e871e15356ca47a5c0914368a |  200000000
 0xc30e4340495a8a89e51cae24c9d2bebf5619ede5398836a32f7c5abdf7e812d3 |  200000000
 0xebffecfb60c87b347060b364151a39b7b62ec418d36692e5d8f18b406b97e151 |  200000000
 0xfad8a3c1485ba90017e8cd7dbe3d5170f7dbc700ee26b7ee02d107d0fb174430 |  200000000

Currently the sui-base workdir is keeping data only on the last publication (deleting all previous data at the beginning of the attempt).

Should at least keep some logs of the last successful publication...

===

Solution will probably be influence by coming package upgrade feature from Mysten.

Same as testnet/devnet... except:

• "TBD" branch ("main" until launch)
• "TBD" RPC to Mysten public fullnode (same as "testnet" until launch)

As like all other "remote public" workdirs, special care is taken to NEVER delete the keystore once created.

(Only localnet wipe out the keystore on "regen").