flow-hydraulics / flow-wallet-api Goto Github PK

Horizontal scaling support. Often times the container will be executed with multiple replicas in the cluster. A replica should not interfere with other replica’s operation. This can be achieved by using database locks on records that are being processed, and supporting redundant event processing across replicas. It’s also best to lock records (e.g. SELECT FOR UPDATE) when a replica is processing them (e.g. polling).

Transaction throttling. A MAX_TPS parameter should be supported that sets the maximum transactions-per-second (TPS) rate at which the service can submit transactions to Flow.

If a transaction fails (result.Error is set) the error message should be inserted into the database. The error message will provide valuable feedback to the developer about what went wrong. It’s also worth noting that transaction errors are often expected as part of the logic. Recording events resulting from a transaction is also a nice-to-have.

Hi, I was getting this error on building the wallet docker image on Windows.

 > [builder  8/10] RUN ./build.sh:
: not found/build.sh: line 1:
: not found/build.sh: line 3:
#13 0.313 malformed import path "main.go\r": invalid char '\r'

It is clear from the error message that it was occurring because of the \r EOL character in build.sh. I also tried Ubuntu using WSL.

Solution:

I fixed it using dos2unix tool. In short, removed \r from the file.

System Info

OS Name: Microsoft Windows 10 Pro
Version: 10.0.19043 Build 19043
Docker Desktop: v3.6.0 (67351)

Whenever multiple database operations are needed, a transaction should be used to ensure data consistency. Account creation is an example. Also, some database calls return errors but are ignored (e.g. s.db.InsertAccountToken).

This issue was originally opened here by @tehranifar on July 28, 2021: onflow/wallet-api#12

We expect the admin account to contain enough FLOW to compensate for all custodial accounts' transactions.

Some endpoints (like FT withdrawal) use the user account as the payer, thus running into problems when transactions fail and the storage falls below the minimum.

All outgoing transactions from wallet-api should use the admin account as payer.

From @uhbif19:

@tehranifar

It looks to me that all transactions send by hand are paid by user. Flow are taken from account after each transaction. Am I correct?

From @tehranifar:

Yes, designating the admin account as payer will automatically make it pay for user transactions.

From @uhbif19:

I am not sure what you mean. 'designating the admin account as payer' - is that something possible in Wallet API? Or your answer is about situation when working with a regular Flow client libruary?

SYNC_HEADER parameter should be a query string parameter like the others across all endpoints.

It’s best to separate all environment config variables into a top-level config.go file with comments for reference. Package-level config can be injected downstream.

Use environment variables for all config, including flags like disable-raw-tx and disable-ft.

From @tehranifar:

We expect the admin account to contain enough FLOW to compensate for all custodial accounts' transactions.
Some endpoints (like FT withdrawal) use the user account as the payer, thus running into problems when transactions fail and the storage falls below the minimum.
All outgoing transactions from wallet-api should use the admin account as payer.

More discussions found in the closed issue #152

From @tehranifar:

Currently, wallet-api only supports 1 worker pool with 100 capacity. This will make all the requests (sync and async) execute in a single-threaded manner (1 thread, 100 queue capacity). We've recently added functionality to support multiple proposer keys, so in order to leverage that we need to change the job system to support multiple concurrent job pools.

This issue was originally opened here by @m4nuC on July 8, 2021: onflow/wallet-api#9

Currently, it seems to pick up the default PORT variable which is not ideal in a multi-service environment

From @psiemens:

@m4nuC Would it help to have a prefix on the environment variable? e.g. FLOW_PORT

From @m4nuC:

@psiemens yes definitely. Maybe even FLOW_WALLET_PORT or FLOW_WALLET_API_PORT?

We need to queue transactions to allow simultaneous requests for transactions.

We should have one "job" queue per one admin key.

At first we use only 1 admin key. This will allow one job to process at a time.

When a user sends a request for example for an account creation we will respond with a job id. The user then polls our service with the job id until the job has completed.

Insert public key associated with an account in the database.

Avoid using single letter variable names like a, aa, b in the code.

WaitForSeal needs to handle another terminal state: TransactionStatusExpired. Also a configurable timeout needs to be in place to cap transaction polling.

Job pool capacity should be configurable through environment variables. Currently it’s hard-coded to 100.

A recent protocol change now forbids transactions from containing multiple signatures from the same account key.

In the event that an account is both authorizer and the payer, the account only needs to sign the transaction envelope.

This results in a "duplicate signers" error in the wallet API when sending a transaction from the admin/service account.

This error can be fixed by updating this line as follows:

for _, a := range authorizers {
  // If account is also the payer, it must only sign the envelope
  if a.Address == payer.Address {
    continue
  }

  if err := builder.Tx.SignPayload(a.Address, a.Key.Index, a.Signer); err != nil {
    return err
  }
  
  // ..
}

This issue was originally opened here by @davidhouseholter-hah on August 20, 2021: onflow/wallet-api#17

I added a new package to publish messages to a message queue. I still need to add the event bus name to the cfg file but I thought it was best to talk to members of the team about architecture. I would like this logic to use the JobService but currently, the service is not being used for creating or update.

The changes I have made currently: https://github.com/onflow/wallet-api/compare/main...davidhouseholter-hah:ampq_message_client?expand=1

Maintenance flag support for event processing during sporks. It’s good for the container to support a way to halt all calls to the Flow network.

There was a recent patch to the Flow network that rejects transactions with no gas limit set. This applies to contract deployments and all transactions. Currently, adding proposal keys fails:

[SERVER] 2021/08/31 16:36:27 main.go:95: Starting server (v0.5.0)...
[JOBS] 2021/08/31 16:36:28 jobs.go:119: [Job a42592f1-e2ee-47b2-a5bb-e4e08a880352] Error while processing job: client: rpc error: code = InvalidArgument desc = invalid transaction: transaction gas limit (0) is not in the acceptable range (min: 1, max: 9999)
[SERVER] 2021/08/31 16:36:28 main.go:155: client: rpc error: code = InvalidArgument desc = invalid transaction: transaction gas limit (0) is not in the acceptable range (min: 1, max: 9999)

Other Flow tools have been patched (for example for cli onflow/flow-cli#373 and onflow/flow-cli#374). We need to make sure that the max gas limit is set on all outgoing transactions from wallet-api, including adding proposal keys.

Seems like we need to patch transactions/service.go and add the limit to Create.

The max gas limit is 9999.

The code should support transient errors and retries. For example if there is a network error on calling SendTransaction a retry should be triggered.

https://github.com/eqlabs/flow-wallet-service/blob/main/keys/google/helpers.go#L52

We suggest using the same transactions service in the code to run transactions needed by other services (e.g. for ft and nft). This will make the database more useful, promote reusability, and enable recovery and error tracking for other operations as well.

This issue was originally opened here by @hichana on July 3, 2021: onflow/wallet-api#7

I realize the wallet-api is not production-ready yet, but it would be great to have some instructions on how to deploy the api on a cloud provider like GCP. The wallet-api seems pretty powerful so for some it might be wise to start preparing for using it in a real-world app.

Also, how where in the repo is the actual code that starts the server to run the api? I'm not familiar with docker so maybe this is what I'm missing here. I guess the docker image has all of this and is building and running it? Some help/guidance in the readme for this would be appreciated too. Thanks!

some instructions to deploy on Kubernetes, or even something easier (if possible) like Google Cloud Run would be nice

From @psiemens:

Thanks for the feedback, @hichana! I agree that there should be instructions for how to deploy to a cloud environment.

This is required for https://github.com/flow-hydraulics/fcl-wallet-adapter. The requirement is simply an endpoint that takes a built transaction and a custodial account ID as an input and outputs the proper signature.

Here's more context:

Upon approval, the FCL adapter frontend should submit the transaction to the adapter backend, which in turn passes it to the Wallet API for signing.
This sign endpoint can be fully trusted. That means that it will not validate transactions before signing, and can even take a simple byte array (e.g. hex string as input).

The code string checks : AuthAccount to decide whether a signature is needed. We think it’s best to always assume 1 signature is needed and specify it in the docs.

Refactor general project structure/directories. You can check out [INSERT LINK TO OUR SAMPLE REPO] as an example.

Support custom Cadence account creation code (for initial setup)

Operation recovery support across endpoints. Running a transaction and waiting for it to seal is time consuming, and service restarts in a container environment (e.g. Kubernetes cluster) are common. If a transaction is sent to the network and the container is restarted while the status is being polled, the operation won’t resume after restart. To fix this, it’s best to insert into the database with pending status when a request is received, and check/resume non-terminal records on service startup.

The proposer key sequence number is fetched from Flow with GetAccount. This has the side-effect of failing transactions during high load due to incorrect sequence numbers. You can use the master account’s key as proposer for other accounts if locking/throttling is planned for that account. It should be noted in the docs that handling higher load is in the works.

From @hichana

I realize the wallet-api is not production-ready yet, but it would be great to have some instructions on how to deploy the api on a cloud provider like GCP. The wallet-api seems pretty powerful so for some it might be wise to start preparing for using it in a real-world app.
Also, how where in the repo is the actual code that starts the server to run the api? I'm not familiar with docker so maybe this is what I'm missing here. I guess the docker image has all of this and is building and running it? Some help/guidance in the readme for this would be appreciated too. Thanks!

some instructions to deploy on Kubernetes, or even something easier (if possible) like Google Cloud Run would be nice

From @psiemens:

Thanks for the feedback, @hichana! I agree that there should be instructions for how to deploy to a cloud environment.

Metrics support.

status_string.go might need some refactoring for clarity (check out _Status_index)

The KeyManagementClients needs to be closed to avoid leaking goroutines;

• in keys/google/google.go; Generate and Signer functions

Currently tests fail when using Google KMS keys (FLOW_WALLET_DEFAULT_KEY_TYPE=google_kms)

Service should fail to start if dependencies fail to init (for example gorm.New()). More specifically, runServer() might return error so it can be handled top-level in main.go.

When making transactions, record the resulting events and insert them to the API response. Related to #87

This issue was originally opened here by @m4nuC on August 20, 2021: onflow/wallet-api#15

When sending a transaction it seems that only the string type is currently working.

ie.: this will work

{
  "code":"transaction(greeting: String) { execute { log(greeting.concat(\", World!\")) }}",
  "arguments":[{"type":"String","value":"Hello"}]
}

But this won't

{
  "code":"transaction(greeting: UInt64) { execute { log(greeting.concat(\", World!\")) }}",
  "arguments":[{"type":"UInt64","value":"Hello"}]
}

Error is:

Error while processing job: failed to decode value: invalid JSON Cadence structure

Dockerized “end-to-end” test suite (in addition to the current unit tests). The suite could be run without any explicit environment setup from the developer for better CI support. Can be created with docker compose.

Database migrations support.

When returning a struct and error from a function, it’s better to return a pointer to a struct and error so if an error occurs the result is nil. This will prevent downstream code to process an erroneous empty result if the error is not checked. Example: LatestBlockId function.

From @m4nuC:

Currently, it seems to pick up the default PORT variable which is not ideal in a multi-service environment

From @psiemens:

@m4nuC Would it help to have a prefix on the environment variable? e.g. FLOW_PORT

From @m4nuC:

@psiemens yes definitely. Maybe even FLOW_WALLET_PORT or FLOW_WALLET_API_PORT?

got a problem when running the docker in M1 chip, could anyone kindly give any advice?

 => [internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 424B                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                      0.0s
 => => transferring context: 327B                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/golang:alpine                                                                                       3.6s
 => [auth] library/golang:pull token for registry-1.docker.io                                                                                          0.0s
 => [builder  1/10] FROM docker.io/library/golang:alpine@sha256:4233211cc2bfac6e7f4320bf641499c72444f917e23e0e990863a7fba835d165                      13.9s
 => => resolve docker.io/library/golang:alpine@sha256:4233211cc2bfac6e7f4320bf641499c72444f917e23e0e990863a7fba835d165                                 0.0s
 => => sha256:4233211cc2bfac6e7f4320bf641499c72444f917e23e0e990863a7fba835d165 1.65kB / 1.65kB                                                         0.0s
 => => sha256:bcb4617937e8c16fa2f5df7993826eba78d3475eaf418efee55c15fc38ee7557 1.36kB / 1.36kB                                                         0.0s
 => => sha256:46689a2b71ade7902cbc35039764a41ea2f3d2e4449873d4f247be6befba5776 4.87kB / 4.87kB                                                         0.0s
 => => sha256:552d1f2373af9bfe12033568ebbfb0ccbb0de11279f9a415a29207e264d7f4d9 2.71MB / 2.71MB                                                         0.6s
 => => sha256:61e34390c2b14c7d6b4929bed3213d6d86bb9aa93c4ec135494e476afbb7c8ff 281.69kB / 281.69kB                                                     0.6s
 => => sha256:5d8f557407009b175f4d46f52d9e57a4123bc5118b57e0a250e983230d791682 153B / 153B                                                             0.7s
 => => extracting sha256:552d1f2373af9bfe12033568ebbfb0ccbb0de11279f9a415a29207e264d7f4d9                                                              0.2s
 => => sha256:01639cd8f0be66171cd30430b7303882d5c6cf0ecefaaf1f427c202784f295dc 104.32MB / 104.32MB                                                    10.4s
 => => sha256:3c2a038518afd8335adf88978717b53faa01e32fd0ee5f41708b64e922f75944 156B / 156B                                                             1.2s
 => => extracting sha256:61e34390c2b14c7d6b4929bed3213d6d86bb9aa93c4ec135494e476afbb7c8ff                                                              0.1s
 => => extracting sha256:5d8f557407009b175f4d46f52d9e57a4123bc5118b57e0a250e983230d791682                                                              0.0s
 => => extracting sha256:01639cd8f0be66171cd30430b7303882d5c6cf0ecefaaf1f427c202784f295dc                                                              2.9s
 => => extracting sha256:3c2a038518afd8335adf88978717b53faa01e32fd0ee5f41708b64e922f75944                                                              0.0s
 => [internal] load build context                                                                                                                      0.5s
 => => transferring context: 17.48MB                                                                                                                   0.5s
 => [builder  2/10] RUN apk update && apk add --no-cache   musl-dev   gcc   build-base   git                                                          11.5s
 => [builder  3/10] WORKDIR /build                                                                                                                     0.0s 
 => [builder  4/10] COPY go.mod .                                                                                                                      0.0s 
 => [builder  5/10] COPY go.sum .                                                                                                                      0.0s 
 => [builder  6/10] RUN go mod download                                                                                                               47.1s 
 => [builder  7/10] COPY . .                                                                                                                           0.1s 
 => ERROR [builder  8/10] RUN ./build.sh                                                                                                               9.0s 
------                                                                                                                                                      
 > [builder  8/10] RUN ./build.sh:                                                                                                                          
#13 1.592 # runtime/cgo
#13 1.592 gcc: error: unrecognized command-line option '-m64'
------
executor failed running [/bin/sh -c ./build.sh]: exit code: 2
ERROR: Service 'wallet' failed to build : Build failed

The FT/NFT deposit tracking feature should track deposits into all accounts that receive the configured FT/NFTs, not just the accounts controlled by the API.

This issue was originally opened here by @davidhouseholter-hah on August 20, 2021: onflow/wallet-api#16

I created a new key in my KMS and converted the PEM to Hex and then created an account on the testnet. When I configure the wallet-api to use this address on the testnet I get the following error:

invalid proposal key: public key 0 on account 0xxxxx does not have a valid signature: [Error Code: 1009] invalid envelope key: public key 0 on account 0xxxxx does not have a valid signature: signature is not valid
exit status 1

Liveness/readiness probe support.

Flow CLI doesn't need to be bundled. You can use the Flow emulator Docker image instead.

The token name included in the API URL is case sensitive.

/v1/accounts/0xf8d6e0586b0a20c7/fungible-tokens/FUSD/ # this works
/v1/accounts/0xf8d6e0586b0a20c7/fungible-tokens/fusd/ # this fails

IMO the url should be normalized to always be case insensitive, and the all-lowercase version (second example above) should be treated as the canonical default.