GithubHelp home page GithubHelp logo

onflow / flow-emulator Goto Github PK

View Code? Open in Web Editor NEW
82.0 33.0 43.0 4.74 MB

The Flow Emulator is a lightweight tool that emulates the behaviour of the real Flow network

Home Page: https://onflow.org

License: Apache License 2.0

Makefile 0.86% Go 98.66% Dockerfile 0.26% Shell 0.04% Cadence 0.17%
flow blockchain emulator

flow-emulator's Introduction


Logo

The Flow Emulator is a lightweight tool that emulates the behaviour of the real Flow network.
Read the docs»

Report Bug · Contribute



The Emulator

The emulator exposes a gRPC server that implements the Flow Access API, which is designed to have near feature parity with the real network API.

The Flowser Emulator Explorer

There is also a block explorer GUI for the emulator, that will help you speed up development when using the emulator.

Running

Configuration

The Flow Emulator can be run in different modes and settings, all of them are described in the table bellow.

Please note that if you will run the emulator using the Flow CLI you must use flags to pass configuration values and if you plan to run the emulator with Docker you must use the environment variables (Env) to pass configuration values.

Flag Env Default Description
--port, -p FLOW_PORT 3569 gRPC port to listen on
--rest-port FLOW_RESTPORT 8888 REST API port to listen on
--admin-port FLOW_ADMINPORT 8080 Admin API port to listen on
--verbose, -v FLOW_VERBOSE false Enable verbose logging (useful for debugging)
--log-format FLOW_LOGFORMAT text Output log format (valid values text, JSON)
--block-time, -b FLOW_BLOCKTIME 0 Time between sealed blocks. Valid units are ns, us (or µs), ms, s, m, h
--contracts FLOW_WITHCONTRACTS false Start with contracts like NFT and an NFT Marketplace, when the emulator starts
--service-priv-key FLOW_SERVICEPRIVATEKEY random Private key used for the service account
--service-sig-algo FLOW_SERVICEKEYSIGALGO ECDSA_P256 Service account key signature algorithm
--service-hash-algo FLOW_SERVICEKEYHASHALGO SHA3_256 Service account key hash algorithm
--init FLOW_INIT false Generate and set a new service account
--rest-debug FLOW_RESTDEBUG false Enable REST API debugging output
--grpc-debug FLOW_GRPCDEBUG false Enable gRPC server reflection for debugging with grpc_cli
--persist FLOW_PERSIST false Enable persistence of the state between restarts
--snapshot FLOW_SNAPSHOT false Enable snapshot support
--dbpath FLOW_DBPATH ./flowdb Specify path for the database file persisting the state
--simple-addresses FLOW_SIMPLEADDRESSES false Use sequential addresses starting with 0x1
--token-supply FLOW_TOKENSUPPLY 1000000000.0 Initial FLOW token supply
--transaction-expiry FLOW_TRANSACTIONEXPIRY 10 Transaction expiry, measured in blocks
--storage-limit FLOW_STORAGELIMITENABLED true Enable account storage limit
--storage-per-flow FLOW_STORAGEMBPERFLOW Specify size of the storage in MB for each FLOW in account balance. Default value from the flow-go
--min-account-balance FLOW_MINIMUMACCOUNTBALANCE Specify minimum balance the account must have. Default value from the flow-go
--transaction-fees FLOW_TRANSACTIONFEESENABLED false Enable variable transaction fees and execution effort metering
as decribed in Variable Transaction Fees: Execution Effort FLIP
--transaction-max-gas-limit FLOW_TRANSACTIONMAXGASLIMIT 9999 Maximum gas limit for transactions
--script-gas-limit FLOW_SCRIPTGASLIMIT 100000 Specify gas limit for script execution
--coverage-reporting FLOW_COVERAGEREPORTING false Enable Cadence code coverage reporting
--contract-removal FLOW_CONTRACTREMOVAL true Allow removal of already deployed contracts, used for updating during development
--skip-tx-validation FLOW_SKIPTRANSACTIONVALIDATION false Skip verification of transaction signatures and sequence numbers
--host FLOW_HOST Host to listen on for emulator GRPC/REST/Admin servers (default: All Interfaces)
--chain-id FLOW_CHAINID emulator Chain to simulate, if 'mainnet' or 'testnet' values are used, you will be able to run transactions against that network and a local fork will be created. Valid values are: 'emulator', 'testnet', 'mainnet'
--redis-url FLOW_REDIS_URL '' Redis-server URL for persisting redis storage backend ( redis://[[username:]password@]host[:port][/database] )
--start-block-height FLOW_STARTBLOCKHEIGHT 0 Start block height to use when starting the network using 'testnet' or 'mainnet' as the chain-id
--rpc-host FLOW_RPCHOST '' RPC host (access node) to query for previous state when starting the network using 'testnet' or 'mainnet' as the chain-id
--legacy-upgrade FLOW_LEGACYUPGRADE false Enable upgrading of legacy contracts
--computation-reporting FLOW_COMPUTATIONREPORTING false Enable computation reporting for Cadence scripts & transactions
--checkpoint-dir FLOW_CHECKPOINTDIR '' Checkpoint directory to load the emulator state from, if starting the emulator from a checkpoint
--state-hash FLOW_STATEHASH '' State hash of the checkpoint, if starting the emulator from a checkpoint

Running the emulator with the Flow CLI

The emulator is bundled with the Flow CLI, a command-line interface for working with Flow.

Installation

Follow these steps to install the Flow CLI.

Starting the server

Starting the emulator by using Flow CLI also leverages CLI configuration file flow.json. You can use the flow.json to specify the service account which will be reused between restarts. Read more about CLI configuration here.

You can start the emulator with the Flow CLI:

flow emulator

You need to make sure the configuration flow.json exists, or create it beforehand using the flow init command.

Using the emulator in a project

You can start the emulator in your project context by running the above command in the same directory as flow.json. This will configure the emulator with your project's service account, meaning you can use it to sign and submit transactions. Read more about the project and configuration here.

Using Emulator in Go

You can use the emulator as a module in your Go project. To install emulator, use go get:

go get github.com/onflow/flow-emulator

After installing the emulator module you can initialize it in the code:

var opts []emulator.Option
privKey, err := crypto.DecodePrivateKeyHex(crypto.ECDSA_P256, "")

opts = append(opts, emulator.WithServicePublicKey(
privKey.PublicKey(),
crypto.ECDSA_P256,
crypto.SHA3_256,
))

blockchain, err := emulator.NewBlockchain(opts...)

You can then access all methods of the blockchain like so:

account, err := blockchain.GetAccount(address) 

Rolling back state to blockheight

It is possible to roll back the emulator state to a specific block height. This feature is extremely useful for testing purposes. You can set up an account state, perform tests on that state, and then roll back the state to its initial state after each test.

To roll back to a specific block height, you can utilize below HTTP request: `` POST http://localhost:8080/emulator/rollback

Post Data: height={block height}


Note: it is only possible to roll back state to a height that was previously executed by the emulator.
To roll back to a past block height when using a forked Mainnet or Testnet network, use the
`--start-block-height` flag.

## Managing emulator state
It's possible to manage emulator state by using the admin API. You can at any point 
create a new named snapshot of the state and then at any later point revert emulator 
state to that reference. 

In order to use the state management functionality you need to run the emulator with persistent state:
```bash
flow emulator --persist

Create a new snapshot by doing an HTTP request:

POST http://localhost:8080/emulator/snapshots

Post Data: name={snapshot name}

Please note the example above uses the default admin API port

At any later point you can reload to that snapshot by executing:

PUT http://localhost:8080/emulator/snapshots?name={snapshot name}

You need to use the same value for name parameter.

The snapshot functionality is a great tool for testing where you can first initialize a base snapshot with seed values, execute the test and then revert to that initialized state.

You can list existing snapshots with:

GET http://localhost:8080/emulator/snapshots

Cadence Code Coverage

The admin API includes endpoints for viewing and managing Cadence code coverage.

In order to use this functionality you need to run the emulator with the respective flag which enables code coverage:

flow emulator --coverage-reporting

To view the code coverage report, visit this URL: http://localhost:8080/emulator/codeCoverage

To flush/reset the collected code coverage report, run the following command:

curl -XPUT 'http://localhost:8080/emulator/codeCoverage/reset'

Note: The above command will reset the code coverage for all the locations, except for A.f8d6e0586b0a20c7.FlowServiceAccount, which is a system contract that is essential to the operations of Flow.

To get better reports with source file references, you can utilize the sourceFile pragma in the headers of your transactions and scripts.

#sourceFile("scripts/myScript.cdc")

Cadence Computation Reporting

The admin API includes an endpoint for viewing the Cadence computation reports for scripts & transactions.

In order to use this functionality you need to run the emulator with the respective flag which enables computation reporting:

flow emulator --computation-reporting

To view the computation report, visit this URL: http://localhost:8080/emulator/computationReport

Running the emulator with Docker

Docker builds for the emulator are automatically built and pushed to gcr.io/flow-container-registry/emulator, tagged by commit and semantic version. You can also build the image locally.

docker run -p 3569:3569 -p 8080:8080 -e FLOW_HOST=0.0.0.0 gcr.io/flow-container-registry/emulator

The full list of environment variables can be found here. You can pass any environment variable by using -e docker flag and pass the valid value.

Custom Configuration Example:

docker run -p 3569:3569 -p 8080:8080 -e FLOW_HOST=0.0.0.0 -e FLOW_PORT=9001 -e FLOW_VERBOSE=true -e FLOW_SERVICEPRIVATEKEY=<hex-encoded key> gcr.io/flow-container-registry/emulator

To generate a service key, use the keys generate command in the Flow CLI.

flow keys generate

Emulating mainnet and testnet transactions

The emulator allows you to simulate the execution of transactions as if they were performed on the Mainnet or Testnet. In order to activate this feature, you must specify the network name for the chain ID flag as well as the RPC host to connect to.

flow emulator --chain-id mainnet --rpc-host access-008.mainnet24.nodes.onflow.org:9000
flow emulator --chain-id mainnet --rpc-host access-002.devnet49.nodes.onflow.org:9000

Please note, the actual execution on the real network may differ depending on the exact state when the transaction is executed.

By default, the forked network will start from the latest sealed block when the emulator is started. You can specify a different starting block height by using the --start-block-height flag.

You can also store all of your changes and cached registers to a persistent db by using the --persist flag, along with the other sqlite settings.

To submit transactions as a different account, you can use the --skip-tx-validation flag to disable transaction signature verification. Then submit transactions from any account using any valid private key.

Debugging

To debug any transactions sent via VSCode or Flow CLI, you can use the debugger pragma. This will cause execution to pause at the debugger for any transaction or script which includes that pragma.

#debugger()

Development

Read contributing document.

flow-emulator's People

Contributors

andriidiachuk avatar avcdsld avatar bluesign avatar chasefleming avatar codingone21 avatar dependabot[bot] avatar dsainati1 avatar dylantinianov avatar fxamacker avatar guitarheroua avatar illia-malachyn avatar janezpodhostnik avatar jordanschalm avatar joshuahannan avatar jribbink avatar jrkhan avatar kay-zee avatar keisukeyamashita avatar m-peter avatar m4ksio avatar mrbrianhobo avatar pattyshack avatar peterargue avatar psiemens avatar ramtinms avatar sideninja avatar sukantoraymond avatar supuns avatar tarakby avatar turbolent avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flow-emulator's Issues

Format the contracts that is deployed a little better

Instructions

Issue To Be Solved

In the new 0.14.0 version of the cli i saw the following message in the log
INFO[0000] 📜 Flow contracts FlowFees=0xe5a8b7f23e8b548f FlowServiceAccount=0xf8d6e0586b0a20c7 FlowStorageFees=0xf8d6e0586b0a20c7 FlowToken=0x0ae53cb6e3f42a79 FungibleToken=0xee82856bf20e2aa6

It would be nice if those addreses where broken up to be a little nicer.

Panic when running `flow emulator --contracts`

Problem

Getting error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x7d354b]

when running flow emulator --contracts on Ubuntu 20.04.3. The stack trace points to github.com/onflow/[email protected]/server/contracts.go:91

Steps to Reproduce

run:

flow emulator --contracts

on Ubuntu 20.04.3

Acceptance Criteria

The command should run with no errors

Context

Trying to address issue 17 in flow-js-testing. The plan is to add a parameter to emulator.start to optionally deploy the extra contracts (including FUSD) and allow for minting FUSD tokens

provide an interface to query Emulator server status

Issue To Be Solved

It is a commonly encountered issue that there are intermittent failures for tests backed by flow-js-testing library, per described in onflow/flow-js-testing#47.
It turns out the root cause is that there is not a reliable way for processes to check whether the emulator is fully started, hence tests could be run before HTTP/gRPC servers are ready, causing failures.

Suggest A Solution

Rather than relying on this logging line, there should be a reliable interface (i.e. maybe /status) provided by the emulator to reflect the status of the servers

Expose application metrics

Instructions

Issue To Be Solved

Currently, the admins metrics API /metrics only exposes basic metrics.

List of current exposed metrics
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 5.2686e-05
go_gc_duration_seconds{quantile="0.25"} 6.7553e-05
go_gc_duration_seconds{quantile="0.5"} 7.8253e-05
go_gc_duration_seconds{quantile="0.75"} 0.000100387
go_gc_duration_seconds{quantile="1"} 0.000213081
go_gc_duration_seconds_sum 0.002876219
go_gc_duration_seconds_count 30
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 15
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.17.1"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 4.964944e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 1.18419216e+08
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 5557
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 618792
# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.
# TYPE go_memstats_gc_cpu_fraction gauge
go_memstats_gc_cpu_fraction 0.0280862975148316
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 5.57108e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 4.964944e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 1.5245312e+07
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 8.8064e+06
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 34698
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 1.3631488e+07
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 2.4051712e+07
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.644847991698006e+09
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 653490
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 14400
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 16384
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 246432
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 344064
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 9.633152e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 2.929491e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 1.114112e+06
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 1.114112e+06
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 3.40324e+07
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 16
# HELP grpc_server_handled_total Total number of RPCs completed on the server, regardless of success or failure.
# TYPE grpc_server_handled_total counter
grpc_server_handled_total{grpc_code="Aborted",grpc_method="ExecuteScriptAtBlockHeight",grpc_service="access.AccessAPI",grpc_type="unary"} 0
grpc_server_handled_total{grpc_code="Aborted",grpc_method="ExecuteScriptAtBlockHeight",grpc_service="flow.access.AccessAPI",grpc_type="unary"} 0
grpc_server_handled_total{grpc_code="Aborted",grpc_method="ExecuteScriptAtBlockID",grpc_service="access.AccessAPI
grpc_server_handled_total{grpc_code="ResourceExhausted",grpc_method="SendTransaction",grpc_service="access.AccessAPI",grpc_type="unary"} 0
...
... too long, trimmed
...
grpc_server_started_total{grpc_method="GetLatestProtocolStateSnapshot",grpc_service="flow.access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="GetNetworkParameters",grpc_service="access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="GetNetworkParameters",grpc_service="flow.access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="GetTransaction",grpc_service="access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="GetTransaction",grpc_service="flow.access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="GetTransactionResult",grpc_service="access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="GetTransactionResult",grpc_service="flow.access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="Ping",grpc_service="access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="Ping",grpc_service="flow.access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="SendTransaction",grpc_service="access.AccessAPI",grpc_type="unary"} 0
grpc_server_started_total{grpc_method="SendTransaction",grpc_service="flow.access.AccessAPI",grpc_type="unary"} 0
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 0
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0

But I want to now the metrics that answers:

  • How many keys are in the dev wallet?
  • How many blocks?
  • etc

I want to enrich the observability on this emulator.

[Placeholder] Performance issues with large amount of transactions

Issue To Be Solved

I have noticed that when running the emulator locally and sending a lot (thousands) of large transactions, it seems to slow down over time.

I have no hard data to back this claim yet. I just want to open up this issue so if anyone else notices any performance issues they can 👍 the issue (or comment on it).

In case this is widely experienced. We should look into addressing the issue.

Support non-local databases

Instructions

Issue To Be Solved

Currently, the Badger is only supported as an DB and all the data are written to the local disk.
To use this emulator as a development network, in my company, we are planning to deploy this emulator and the dev wallet to the GKE cluster.

But since the Badger makes the workload stateful, we need to create a Statefulset with persistence volume to store the data.
It will make it easier to manage if the emulator can run in stateless manner because Kubernetes workloads are basically designed as stateless.

(Optional): Suggest A Solution

I saw this article Supporting digital ownership and decentralization with Dapper Labs and Google Cloud saying that the Dapper, inc has now partnerships with GCP.

Therefore, I think the best candidate will be Cloud Firestore, a NoSQL database from GCP.
I also considered Cloud Spanner but it's not KVS and it's quite expensive 💵

Implement `GetAccountAtBlockHeight`

Issue To Be Solved

The GetAccountAtBlockHeight function is currently unimplemented in the emulator Access API. This function is useful for teams that need to query historical account information.

panic("implement me")

Suggest A Solution

To implement this, we must first implement emulator.Blockchain#GetAccountAtBlock:

flow-emulator/blockchain.go

Lines 680 to 682 in 33fb8fa

func (b *Blockchain) GetAccountAtBlock(address sdk.Address, blockHeight uint64) (*sdk.Account, error) {
panic("not implemented")
}

It should be possible to create a veresion of emulator.Blockchain#getAccount thats support a custom height, rather than just the latest.

The key is the b.storage.LedgerViewByHeight function, which creates a view into the storage trie at a specific height.

flow-emulator/blockchain.go

Lines 662 to 677 in 33fb8fa

func (b *Blockchain) getAccount(address flowgo.Address) (*flowgo.Account, error) {
latestBlock, err := b.GetLatestBlock()
if err != nil {
return nil, err
}
view := b.storage.LedgerViewByHeight(latestBlock.Header.Height)
programs := programs.NewEmptyPrograms()
account, err := b.vm.GetAccount(b.vmCtx, address, view, programs)
if errors.Is(err, fvm.ErrAccountNotFound) {
return nil, &AccountNotFoundError{Address: address}
}
return account, nil
}

Default contract deployment doesn't work on docker

Problem

When running the emulator as a docker container and passing in FLOW_WITHCONTRACTS, none of the contracts are deployed. I see the following error in the logs.

ERRO[0000] ❗  Failed to deploy contracts                 error="not able to deploy contracts without set private key"

I'm not sure what is happening, since I am passing in the service private key.

Steps to Reproduce

Deploy the emulator with the following environment variables:
      - FLOW_VERBOSE=true
      - FLOW_SIMPLEADDRESSES=true
      - FLOW_PERSIST=true
      - FLOW_WITHCONTRACTS=true
      - FLOW_SERVICEPRIVATEKEY=7304f85bf218f33430d43edfce2179effd227012632f97632e82587ffce2604e
      - FLOW_SERVICEPUBLICKEY=f458682be04790265e552251c391e0915a12f538cf871c20e1980f9caaf81cf02da7a0929d1f7d0e9524819b1b305a1910d3aa382cb8bb045341337e599872dc
      - FLOW_SERVICEKEYSIGALGO=ECDSA_P256
      - FLOW_SERVICEKEYHASHALGO=SHA3_256

Acceptance Criteria

None

Context

Blocks my local usage of the emulator.

Cadence error messages are difficult to read

Issue To Be Solved

Cadence error messages are condensed onto a single line and don't include line numbers, making it difficult to use them for debugging.

Suggest A Solution

Given that the emulator has access to the underlying Cadence errors from each transaction/script, we could use the attached line number data to create a readable printout for each error. If possible, it'd be great to show a snippet of the code near the error location.

`SEGFAULT/SIGSEGV` when deploying core contracts by default via Docker

Problem

While using Docker Compose to deploy the emulator it seg faults (SIGSEGV) and aborts when trying to deploy core contracts by default via environment variables.

Steps to Reproduce

Use the following docker-compose.yml (truncated to exclude other services that aren't relevant):

version: '3'
services:
  flow-dev-wallet:
    image: ghcr.io/onflow/fcl-dev-wallet:latest
    ports:
      - '8701:8701'
    environment:
      - PORT=8701
      - FLOW_ACCESS_NODE=http://flow-blockchain-emulator:8081
      - FLOW_ACCOUNT_KEY_ID=0
      - FLOW_ACCOUNT_PRIVATE_KEY=e22f344c8d8218effc51025bb690bddd52721ed6bbf815b951fcbd48de2c6a5c
      - FLOW_ACCOUNT_PUBLIC_KEY=01910f53695c0b5220076f6496d0a1dcf7a4c202768efcffe86c86a04eea60f82c61cbb029f6651282c75086bf7d2246cec6795244e6518652f375fea808a407
      - FLOW_INIT_ACCOUNTS=0
      - FLOW_ACCOUNT_ADDRESS=0xf8d6e0586b0a20c7
      - FLOW_AVATAR_URL=https://avatars.onflow.org/avatar/
  flow-blockchain-emulator:
    image: gcr.io/flow-container-registry/emulator:0.27.2
    ports:
      - '8081:8081'
      - '3569:3569'
    environment:
      - FLOW_HTTPPORT=8081
      - FLOW_SERVICEPRIVATEKEY=e22f344c8d8218effc51025bb690bddd52721ed6bbf815b951fcbd48de2c6a5c
      - FLOW_SERVICEKEYSIGALGO=ECDSA_P256
      - FLOW_SERVICEKEYHASHALGO=SHA3_256
      - FLOW_WITHCONTRACTS=true

Note the use of the FLOW_WITHCONTRACTS in the environment variables for the flow-blockchain-emulator service. flow-blockchain-emulator is also pinned to version 0.27.2 which is presently the latest version of the Flow emulator that is published.

Run the following command:

docker-compose up -d

View logs of the flow-blockchain-emulator service and view seg fault:

docker-compose logs flow-blockchain-emulator

Attaching to super-octo-spork_flow-blockchain-emulator_1
flow-blockchain-emulator_1  | time="2021-12-13T17:22:22Z" level=info msg="⚙️   Using service account 0xf8d6e0586b0a20c7" serviceAddress=f8d6e0586b0a20c7 serviceHashAlgo=SHA3_256 servicePrivKey=e22f344c8d8218effc51025bb690bddd52721ed6bbf815b951fcbd48de2c6a5c servicePubKey=01910f53695c0b5220076f6496d0a1dcf7a4c202768efcffe86c86a04eea60f82c61cbb029f6651282c75086bf7d2246cec6795244e6518652f375fea808a407 serviceSigAlgo=ECDSA_P256
flow-blockchain-emulator_1  | time="2021-12-13T17:22:22Z" level=info msg="📜  Flow contract" FlowServiceAccount=0xf8d6e0586b0a20c7
flow-blockchain-emulator_1  | time="2021-12-13T17:22:22Z" level=info msg="📜  Flow contract" FlowToken=0x0ae53cb6e3f42a79
flow-blockchain-emulator_1  | time="2021-12-13T17:22:22Z" level=info msg="📜  Flow contract" FungibleToken=0xee82856bf20e2aa6
flow-blockchain-emulator_1  | time="2021-12-13T17:22:22Z" level=info msg="📜  Flow contract" FlowFees=0xe5a8b7f23e8b548f
flow-blockchain-emulator_1  | time="2021-12-13T17:22:22Z" level=info msg="📜  Flow contract" FlowStorageFees=0xf8d6e0586b0a20c7
flow-blockchain-emulator_1  | panic: runtime error: invalid memory address or nil pointer dereference
flow-blockchain-emulator_1  | [signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x7b092b]
flow-blockchain-emulator_1  | 
flow-blockchain-emulator_1  | goroutine 1 [running]:
flow-blockchain-emulator_1  | github.com/onflow/flow-go-sdk/crypto.InMemorySigner.Sign(...)
flow-blockchain-emulator_1  |   /go/pkg/mod/github.com/onflow/[email protected]/crypto/crypto.go:129
flow-blockchain-emulator_1  | github.com/onflow/flow-go-sdk.(*Transaction).SignEnvelope(0xc000642180, 0xc7200a6b58e0d6f8, 0x0, 0x188d7c0, 0xc001399920, 0x0, 0x0)
flow-blockchain-emulator_1  |   /go/pkg/mod/github.com/onflow/[email protected]/transaction.go:316 +0xb4
flow-blockchain-emulator_1  | github.com/onflow/flow-emulator/server.deployContract(0xc000641a40, 0x1507542, 0x4, 0xc0012e0000, 0x1cdf, 0x2000, 0x18b4c28, 0xc0001e6530)
flow-blockchain-emulator_1  |   /app/server/contracts.go:91 +0x37e
flow-blockchain-emulator_1  | github.com/onflow/flow-emulator/server.deployContracts(0xc000641a40, 0xc000000004, 0x1517d2c, 0x13, 0x0, 0x0)
flow-blockchain-emulator_1  |   /app/server/contracts.go:50 +0x528
flow-blockchain-emulator_1  | github.com/onflow/flow-emulator/server.NewEmulatorServer(0xc0001eb490, 0xc0000fe1e0, 0x150fde7)
flow-blockchain-emulator_1  |   /app/server/server.go:143 +0x945
flow-blockchain-emulator_1  | github.com/onflow/flow-emulator/cmd/emulator/start.Cmd.func1(0xc0002b0500, 0xc00013a740, 0x2, 0x2)
flow-blockchain-emulator_1  |   /app/cmd/emulator/start/start.go:177 +0x85e
flow-blockchain-emulator_1  | github.com/spf13/cobra.(*Command).execute(0xc0002b0500, 0xc00018c010, 0x2, 0x2, 0xc0002b0500, 0xc00018c010)
flow-blockchain-emulator_1  |   /go/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x2c2
flow-blockchain-emulator_1  | github.com/spf13/cobra.(*Command).ExecuteC(0xc0002b0500, 0xc0002b0500, 0x1297680, 0xc0000420b8)
flow-blockchain-emulator_1  |   /go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x375
flow-blockchain-emulator_1  | github.com/spf13/cobra.(*Command).Execute(...)
flow-blockchain-emulator_1  |   /go/pkg/mod/github.com/spf13/[email protected]/command.go:902
flow-blockchain-emulator_1  | main.main()
flow-blockchain-emulator_1  |   /app/cmd/emulator/main.go:46 +0x3b

Acceptance Criteria

No segmentation fault and that the following core contracts are deployed:

  • FUSD
  • NonFungibleToken
  • NFTStorefront

This is per this source:

func deployContracts(conf *Config, b *emulator.Blockchain) ([]DeployDescription, error) {

Context

Blocks verification of deploying our contracts that depend on the aforementioned core contracts with a Flow emulated environment. This makes local development more difficult/cumbersome.

Account storage inspection

Add storage inspection capability to the emulator and expose an API to interact with this feature. This would benefit multiple use cases:

  • Integrating this feature into the CLI would allow other developers to inspect what is stored in the account storage
  • Testing framework can use the storage inspection to assert the correctness of actions
  • The Flowser tool requires storage inspection
  • The playground backend could use this functionality instead of custom implementation

DoD:

  • Implement the feature
  • Release a new version
  • Update the version in the CLI

View Contract and Account States

Issue To Be Solved

Similar to having a block explorer, it would be even better (since it's local anyway) to be able to view all the contracts that have been deployed and be able to dig into each and see the state of all the variables.

Along with this, since Flow sequentially creates accounts, it should be possible to see all active accounts on the chain and their available storage paths (both private and public for local dev) + their contents. The state should be easy to reset.

Having these would make it very easy to verify E2E contract logic and give a lot of confidence to developers by being able to inspect very easily.

Solution

This could be done through the CLI or a GUI but would require the emulator to implement the functionality first. Will need to scope out how plausible the above is but I'm thinking in terms of a cache that can capture these value vs implementing them as on-chain queries.

Improve log formatting

Issue To Be Solved

There are a few improvements that could be made to the current logging format used by the emulator:

  • Add option to print logs as JSON -- this is better for systems that aggregate and index log messages
  • Use consistent format for human-readable logs. The current logs are a mixture of our own custom format and the default formatting applied by logrus. We should choose a format that maximizes readability for emulator users.

Transferring Tokens to a non-existent account kills the emulator.

Instructions

Please fill out the template below to the best of your ability and include a label indicating which tool/service you were working with when you encountered the problem.

Problem

Sending tokens to an account that doesn't exist in the emulator (which exists on the testnet) gives the below error log and doesn't work anymore.
(It is not forcibly terminated, but it maintains a pending state indefinitely)

Steps to Reproduce

ERRO[0011] Failed to commit block error="[Failure Code: 2000] unknown failure: transaction invocation failed when executing transaction: fatal error: storage error: storage address allocation failed: failed to store the key storage index: failed to update storage used by key #73746f726167655f696e646578 on account e8f6dc7f3c13c0a0: account e8f6dc7f3c13c0a0 storage used is not initialized or not initialized correctly"

Borrowing Capability from non existent address crashes emulator

@justjoolz commented on Fri Apr 08 2022

🐞 Bug Report

If you send a transaction that tries to borrow a capability from an account that doesn't exist, the cli and emulator hang Waiting for transaction to be sealed...⠸

ERRO[0007] Failed to commit block error="[Failure Code: 2000] unknown failure: transaction invocation failed when executing transaction: fatal error: storage error: storage address allocation failed: failed to store the key storage index: failed to update storage used by key #73746f726167655f696e646578 on account 0000000000000001: account 0000000000000001 storage used is not initialized or not initialized correctly"

Reproduction steps

Steps to reproduce the behaviour:

  1. Run transaction that:
    let nonExistingAddr = Address(0x01) let ref = getAccount(nonExistingAddr).getCapability(ExampleNFT.CollectionPublicPath).borrow<&{NonFungibleToken.CollectionPublic}>()!
  2. See error

Expected behaviour

Transaction should execute with an error and block should still seal

Specifications

  • System: macOS 12.0.1
  • Flow CLI: v0.33.0


@sideninja commented on Fri Apr 08 2022

Thank you for this report. This looks like a bug to me and I will further investigate and provide a fix with the next release.


@janezpodhostnik commented on Fri Apr 08 2022

Thank you for the report.

This is something that was also discovered on testnet and addressed in onflow/flow-go#2253. And included in the emulator in #143.

Just needs merging and releasing.


@sideninja commented on Wed May 11 2022

Issue moved to onflow/flow-emulator #148 via ZenHub

Flags for storage parameters

Flags for storage parameters

Some context

Storage parameters are:

  • amount of storage an account gets per FLOW
  • and the minimum account balance (a.k.a.: minimum storage reservation)

They are defined on the FlowStorageFees contract here https://github.com/onflow/flow-core-contracts/blob/9a02fe98f39ef8c8632be88cc32c0ac6862fee50/contracts/FlowStorageFees.cdc#L28

If they get updated, they get updated with a transaction, not a code change.

The emulator currently takes the value of storage parameters from constants defined in flow-go: https://github.com/onflow/flow-go/blob/20969da4737c850ddc3cd43afdb741d1bcedc2f8/fvm/bootstrap.go#L55

Problem definition

When storage parameters are changing, the emulator might not yet have the most recent version of flow-go (so storage constants are not updated yet), but users should be able to test how the change in storage parameters affects their code as soon as possible.

Solution

Introduce optional flags (or environment variables) that can be set to change the storage parameters.

Borrowing Capability from non existent address crashes emulator

@justjoolz commented on Fri Apr 08 2022

🐞 Bug Report

If you send a transaction that tries to borrow a capability from an account that doesn't exist, the cli and emulator hang Waiting for transaction to be sealed...⠸

ERRO[0007] Failed to commit block error="[Failure Code: 2000] unknown failure: transaction invocation failed when executing transaction: fatal error: storage error: storage address allocation failed: failed to store the key storage index: failed to update storage used by key #73746f726167655f696e646578 on account 0000000000000001: account 0000000000000001 storage used is not initialized or not initialized correctly"

Reproduction steps

Steps to reproduce the behaviour:

  1. Run transaction that:
    let nonExistingAddr = Address(0x01) let ref = getAccount(nonExistingAddr).getCapability(ExampleNFT.CollectionPublicPath).borrow<&{NonFungibleToken.CollectionPublic}>()!
  2. See error

Expected behaviour

Transaction should execute with an error and block should still seal

Specifications

  • System: macOS 12.0.1
  • Flow CLI: v0.33.0


@sideninja commented on Fri Apr 08 2022

Thank you for this report. This looks like a bug to me and I will further investigate and provide a fix with the next release.


@janezpodhostnik commented on Fri Apr 08 2022

Thank you for the report.

This is something that was also discovered on testnet and addressed in onflow/flow-go#2253. And included in the emulator in #143.

Just needs merging and releasing.

`DEV_WALLET` or `DEV_WALLET_PORT` is not respected on emulator

Instructions

Please fill out the template below to the best of your ability and include a label indicating which tool/service you were working with when you encountered the problem.

Problem

Even after configuring DEV_WALLET or DEV_WALLET_PORT which is written on the README.md, the emulator will not respect that config.

Steps to Reproduce

$ env DEV_WALLET=true go run ./cmd/emulator/
...
INFO[0000] 📜  Flow contract                              FlowServiceAccount=0xf8d6e0586b0a20c7
INFO[0000] 📜  Flow contract                              FlowToken=0x0ae53cb6e3f42a79
INFO[0000] 📜  Flow contract                              FungibleToken=0xee82856bf20e2aa6
INFO[0000] 📜  Flow contract                              FlowFees=0xe5a8b7f23e8b548f
INFO[0000] 📜  Flow contract                              FlowStorageFees=0xf8d6e0586b0a20c7
INFO[0000] 🌱  Starting gRPC server on port 3569          port=3569
INFO[0000] 🌱  Starting REST API on port 8888             port=8888
INFO[0000] 🌱  Starting admin server on port 8080         port=8080

Acceptance Criteria

Emulators to get values from env vars.

Context

The correct env var names are FLOW_DEVWALLETENABLED and FLOW_DEVWALLETPORT and the current readme is wrong.
I also want to use the FLOW_DEVWALLET=true not FLOW_DEVWALLETENABLED=false because I feel redundant adding ENABLED and inconsistent with the flag --dev-wallet .

Add BLS signature support

Context

Cadence supports BLS signature scheme (signature verification, public key validation).
Flow VM uses external dependencies (Relic library) to implement the Cadence BLS features. In order to keep the emulator free of external non-Go libraries, the emulator currently panics when Cadence BLS features are used.

Issue To Be Solved

  • either support Relic in the emulator: write scripts that allows compiling Relic static library in the included GOPATH.
  • or write a Go implementation of BLS (performance doesn't matter). This includes using Go packages for pairings and operations on BLS12-381.

Emulator GUI

What if the emulator shipped with a GUI?

This is just an idea for now -- let's use this issue to evolve it further.

I find a block explorer to be very useful during testnet development because it allows me to easily observe the state of the blockchain. There's currently no tool that makes it easy to observe the state of a locally-running emulator.

What's the minimal version of an emulator GUI? Could it be a light block explorer that consumes the Access API, or should it also have the ability to control (start, stop, configure) the emulator?

display Cadence log when a transaction fails

Issue To Be Solved

When I add log("my debug statement") in a transaction, the logging is not displayed in the Flow emulator when the transaction fails -- it s only displayed when the transaction passes. This causes some inconvenience for debugging a transaction.

(Optional): Suggest A Solution

Display the Cadence transaction log even if a transaction fails, to make debugging easier

Missing http://localhost:8701/api/

Instructions

Please fill out the template below to the best of your ability and include a label indicating which tool/service you were working with when you encountered the problem.

Problem

this requesting a not exisit api http://localhost:8701/api/
https://github.com/onflow/fcl-dev-wallet/blob/main/contexts/ConfigContext.tsx#L22

but the /api was removed since after version 0.3.4
https://github.com/onflow/fcl-dev-wallet/tree/main/pages
https://github.com/onflow/fcl-dev-wallet/tree/v0.3.4/pages

<what is the problem you've encountered?>

Steps to Reproduce

<share any logs/screenshots or steps to replicate>

Acceptance Criteria

Context

<what are you currently working on that this is blocking?>

Account keys always reported as un-revoked through gRPC API

Problem

When querying accounts via the emulator gRPC API, keys are always reported with revoked=False, even if they are in fact revoked.

I think this is happening because this conversion function does not include the Revoked field, which means it is always set to the zero value (false):

func SDKAccountKeyToFlow(key *sdk.AccountKey) (flowgo.AccountPublicKey, error) {
encodedPublicKey := key.PublicKey.Encode()
flowSignAlgo := SDKSignAlgoToFlow(key.SigAlgo)
flowPublicKey, err := flowcrypto.DecodePublicKey(flowSignAlgo, encodedPublicKey)
if err != nil {
return flowgo.AccountPublicKey{}, err
}
flowhashAlgo := SDKHashAlgoToFlow(key.HashAlgo)
return flowgo.AccountPublicKey{
Index: key.Index,
PublicKey: flowPublicKey,
SignAlgo: flowSignAlgo,
HashAlgo: flowhashAlgo,
Weight: key.Weight,
SeqNumber: key.SequenceNumber,
}, nil
}

Steps to Reproduce

From @janezpodhostnik

  • I created an account with 2 full weight keys.
  • I revoked key 0
  • I called getAccount to retrieve the keys
  • but none of the keys are revoked in the response
  • however if i try to sign a transaction with the revoked key it correctly fails.

Import NFT storefront from repository

Issue To Be Solved

The emulator deploys several contracts by default, e.g. FT, NFT, etc.

One of the contracts, the NFT storefront, is directly committed to / vendored in this repository in server/contracts/NFTStorefront.cdc.

However, the other contracts are imported as "normal" Go dependencies, as their source repositories provide the contract code in Go packages, e.g. https://github.com/onflow/flow-nft/blob/master/lib/go/contracts/contracts.go#L3.

Suggested Solution

Account key structs

Issue To Be Solved

We have 2 different key structs in codebase and they are pretty much same (except order of SeqNumber and Weight)

type sdk.AccountKey struct {
	Index          int
	PublicKey      crypto.PublicKey
	SigAlgo        crypto.SignatureAlgorithm
	HashAlgo       crypto.HashAlgorithm
	Weight         int
	SequenceNumber uint64
	Revoked        bool
}
type flow-go.AccountPublicKey struct {
	Index     int
	PublicKey crypto.PublicKey
	SignAlgo  crypto.SigningAlgorithm
	HashAlgo  hash.HashingAlgorithm
	SeqNumber uint64
	Weight    int
	Revoked   bool
}

I think it would be nice to get rid of one or at least make structures same.

execution error code 1055: [Error Code: 1055] authorization failed for account 179b6b1cb6755e31: authorizer account does not have sufficient signatures (0 < 1000)

Problem

The following transaction:

import ExampleNFT from f8d6e0586b0a20c7

transaction(nftIds: [UInt64]) {
    prepare(
        payerAccount: AuthAccount
        
        , toAccount: AuthAccount
    ) {
        let fromCollection = payerAccount.borrow<&ExampleNFT.Collection>(from: /storage/NFTCollection)
            ?? panic("Could not borrow ExampleNFT.Collection reference")
        
        var toCollection = toAccount.borrow<&ExampleNFT.Collection>(from: /storage/NFTCollection)
        if toCollection == nil {
            let newCollection = ExampleNFT.createEmptyCollection()
            toCollection = &newCollection as &ExampleNFT.Collection
            toAccount.save<@ExampleNFT.Collection>(
                <- newCollection,
                to: /storage/NFTCollection
            )
            toAccount.link<&{NonFungibleToken.CollectionPublic}>(
                /public/NFTCollection,
                target: /storage/NFTCollection
            )
        }
        
        for id in nftIds {
            let nft <- fromCollection.withdraw(withdrawID: id)
            toCollection.deposit(token: <- nft)
        }
    }
}

With the following parameters:
Screen Shot 2021-11-22 at 7 52 13 PM

fails with the following error:

execution error code 1055: [Error Code: 1055] authorization failed for account 179b6b1cb6755e31: authorizer account does not have sufficient signatures (0 < 1000)

There are two accounts at play here, the account that owns the nfts, and the account receiving the nfts. The owner account is the transaction proposer, as well as payer. The receiving account simply receives the NFTs - but they're both signing the transaction.

Add support for `Block.view`

Issue To Be Solved

The cadence built-in function getCurrentBlock() returns a Block struct that has the view field, but view is always zero.

Suggest A Solution

  • Make it so getCurrentBlock().view always returns the same value as getCurrentBlock().height

Context

Somewhat blocking epoch smart contract testing, because the contract uses views to measure the passage of time.

Multi-platform Docker build

Issue To Be Solved

Currently, emulator docker builds are not available for ARM64 (M1 macs). It would be nice for the Dockerfile to be updated to support multi-platform builds out of the box.

(Optional): Suggest A Solution

Update the Dockerfile to support multi-platform builds (see linked PR)
https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
Also need to specify the platforms to be built for in the CI Github action.
https://github.com/docker/build-push-action/blob/master/docs/advanced/multi-platform.md

(Optional): Context

Working on an M1 mac, and I would like to be able to pull in an ARM64 docker build without building the image myself.

Adding debugging feature for failed signatures

Issue To Be Solved

The emulator throws an error when it receives in an invalid transaction signature, but does not specify why the signature was invalid. Given that the emulator is a development tool, it should be able to perform an extra check to provide more diagnostics information to help the developer.

(Optional): Suggest A Solution

A transaction signature can be invalid for one of the following reasons:

  1. signature was generated with the wrong private key
  2. signature was generated by hashing the transaction with the wrong hash algorithm (e.g. on-chain account says SHA2, client used SHA3)
  3. transaction was mutated after the signature was generated (e.g. sign the transaction, then set the gas limit, but submit the old signature)

Scenario 1 is difficult to report without knowing the original private key.

Scenario 2 can be detected by attempting to verify the signature with each other hash algorithm.

Scenario 3 cannot be reliably detected, but the emulator can print out a log of the transaction structure that was signed, which the developer can then compare against the transaction that their client prepares.

The displayed computationUsed is twice as much as the actual usage

Instructions

When I was evaluating the gas consumption of the transaction, I found that the computationUsed exceeded 9999 and the transaction was still successful, and further exploration revealed that it was twice as the actual use.

Problem

The displayed computationUsed is twice as much as the actual usage

Steps to Reproduce

  1. Run flow emulator
  2. send any transaction, record the computationUsed
  3. set --gas-limt = computationUsed/2 - 1, send transaction again will fail
  4. set --gas-limt = computationUsed/2, send transaction again will success

flow version is Version: v0.30.2

Flow Emulator Start command in ReadMe

Instructions

Please fill out the template below to the best of your ability and include a label indicating which tool/service you were working with when you encountered the problem.

Problem

< In the ReadMe of this repository (onflow/flow-emulator), the command for 'Starting the server' is given as "$flow emulator start -init", but it throws error with the "-init" keyword. The server starts only with the command "$flow emulator start".>

Steps to Reproduce

<share any logs/screenshots or steps to replicate>

Acceptance Criteria

Context

< I was trying to start the flow emulator, to deploy my code. >

FEATURE: Auto create accounts and set network codeword

Issue To Be Solved

I was playing around the idea of "auto create accounts" on demand. So instead of getting error on non-existing account with some flag we can force emulator to create the account on demand. ( I made a POC on https://github.com/bluesign/flow-emulator/tree/emulator_auto_accounts ) which creates account with the details of service account (pubkeys etc)

But I want to take this a bit further and curious about your ideas about this.

What about if we also let emulator to use any network codeword, especially testnet? ( with deploying contracts to that correct network addresses )

Also this would allow to write tests for testnet only and also can run them in emulator. I am not sure about feature parity between testnet and emulator, but seems like this can make life a bit easier for developers. Usually main problem with testnet is resetting account state, this can remove the need for that, also can push people to use emulator more.

Also this will be a backup for the times where testnet is down.

This can alternatively be implemented with using account keys data from X network, but keeping state internally, but it is a bit more confusing.

Increment the proposer key sequence for failed transactions

Problem

The flow-emulator doesn't increment the proposer key sequence for failed transactions. The current behavior doesn't match the changes introduced in #602 which was released in flow v0.17.4. I only tested the two cases listed below but most likely there are more cases to cover.

Version: v.0.22.0

Steps to Reproduce

  1. Submit a transfer transaction with an amount larger than the sender balance. The transaction should fail and the key sequence should be incremented.
➜  flow transactions get fcfbfae19116a4c6c30646a430533900c.....

❌ Transaction Error
execution error code 1101: [Error Code: 1101] cadence runtime error Execution failed:
error: pre-condition failed: Amount withdrawn must be less than or equal than the balance of the Vault
   --> ee82856bf20e2aa6.FungibleToken:170:16
    |
170 |                 self.balance >= amount:
    |                 ^^^^^^^^^^^^^^^^^^^^^^
Status		✅ SEALED
ID		fcfbfae19116a4c6c30646a430533900c...
Payer		8913a7d76722a585
Authorizers	[8913a7d76722a585]
Proposal Key:
    Address	8913a7d76722a585
    Index	0
    Sequence	1

The key sequence remains 1 after the transaction is sealed.

➜  flow accounts get 8913a7d76722a585
Address	 0x8913a7d76722a585
Balance	 1.00100000
Keys	 1
Key 0	Public Key		 bcee61165014006481673cef8dbb204daf5f33e369....
	Weight			 1000
	Signature Algorithm	 ECDSA_secp256k1
	Hash Algorithm		 SHA3_256
	Revoked 		 false
	Sequence Number 	 1
	Index 			 0
  1. Submit a transfer transaction to an invalid address or an address that cannot receive the transfer token "not setup". The transaction should fail and the key sequence should be incremented.
➜  flow transactions get 1c06fd999d93c57313056dfae458...
❌ Transaction Error
execution error code 1101: [Error Code: 1101] cadence runtime error Execution failed:
error: panic: Could not borrow receiver reference to the recipient's Vault
  -->1c06fd999d93c57313056dfae458...:27:6
   |
27 | 			?? panic("Could not borrow receiver reference to the recipient's Vault")
   |       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Status		✅ SEALED
ID		1c06fd999d93c57313056dfae458...
Payer		700a49a02359d112
Authorizers	[700a49a02359d112]
Proposal Key:
    Address	700a49a02359d112
    Index	0
    Sequence	1

The sequence remains 1 after the transaction is exuted

➜  flow accounts get 700a49a02359d112
Address	 0x700a49a02359d112
Balance	 1.00100000
Keys	 1
Key 0	Public Key		 349c2dab77ebf344f971b5a23fc44...
	Weight			 1000
	Signature Algorithm	 ECDSA_secp256k1
	Hash Algorithm		 SHA3_256
	Revoked 		 false
	Sequence Number 	 1
	Index 			 0

Acceptance Criteria

Match the flow implementation by incrementing the proposer key sequence for all transactions, except the four cases mentioned in this announcement.

  • invalid transaction (transaction rejected by collection nodes)
  • any of the transaction signatures are not valid
  • the transaction is missing the signature(s) by the proposer
  • the proposal sequence number is invalid (not match with the one stored on the chain)

Context

N/A

crypto: check the emulator crypto usage and point it to flow-go

Context

The SDK used its own copy of the crypto code because it got open-sourced while flow-go was still private. That shouldn't be the case anymore as flow-go became public

Definition of done

Check the usage of crypto in the emulator and point it to flow-go/crypto if needed.

Emulator snapshot documentation

The snapshot feature of the emulator lacks better documentation. We should add documentation on usage of the feature and its APIs along with a simple example.

Run the emulator from a snapshot of Testnet or Mainnet

Issue To Be Solved

Some applications will want to build on top of other applications already deployed on Flow.
This is difficult, given currently there is no way to simulate live network contracts and their state besides manually deploying those contracts on the Emulator, and generating interactions to arrive at the desired starting state.

(Optional): Suggest A Solution

Ability to load a slice of time/block delineated protocol state (Testnet or Mainnet) into the Emulator using a cli command.

Unable to pass certain structures as arguments to scripts / transactions

We have an contract that has been working fine for a long time, and with the v0.28.4 version of the emulator it has stopped working. Specifically, structs exposed by the contract itself can no longer be passed into the contract as an argument for scripts and transactions. The contract can be found here. Specifically the TenantService.MetadataField struct cannot be used as an argument, but only when it contains a TenantService.Mime value. For instance, the following script:

    import TenantService from .....
    pub fun main(metadata: TenantService.MetadataField): TenantService.MetadataField {
        return metadata
    }

passing the following value to it fails:

{
  "value": {
    "id": "A.0x02.TenantService.MetadataField",
    "fields": [
      {
        "name": "type",
        "value": {
          "value": {
            "id": "A.0x02.TenantService.MetadataFieldType",
            "fields": [
              {
                "name": "rawValue",
                "value": {
                  "value": "1",
                  "type": "UInt8"
                }
              }
            ]
          },
          "type": "Enum"
        }
      },
      {
        "name": "value",
        "value": {
          "value": {
            "id": "A.0x02.TenantService.Mime",
            "fields": [
              {
                "name": "type",
                "value": {
                  "value": "text/plain",
                  "type": "String"
                }
              },
              {
                "name": "bytes",
                "value": {
                  "value": [
                    {
                      "value": "32",
                      "type": "UInt8"
                    },
                    {
                      "value": "33",
                      "type": "UInt8"
                    },
                    {
                      "value": "34",
                      "type": "UInt8"
                    },
                    {
                      "value": "35",
                      "type": "UInt8"
                    },
                    {
                      "value": "36",
                      "type": "UInt8"
                    }
                  ],
                  "type": "Array"
                }
              }
            ]
          },
          "type": "Struct"
        }
      }
    ]
  },
  "type": "Struct"
}

What's interesting is that the following script works:

    import TenantService from .....
    pub fun main(mime: TenantService.Mime): TenantService.MetadataField {
        return TenantService.MetadataField(TenantService.MetadataFieldType.MIME, mime)
    }

You'll see that you can pass a TenantService.Mime into the contract but you can't pass it as a value on the MetadataField object (although other value types do work).

Here's a link to the playground with the contract(s) deployed and a script that illustrates the problem Even the playground now has trouble with it. Attached is an error from the javascript console of the web browser (there is no visual feedback of the error other than the JS console on the playground):

Screen Shot 2021-11-10 at 5 17 02 PM

Default values for service key flags are not working

Problem

When running the emulator with a specified service private key using the --service-priv-key flag, the signature and hashing algo are requested even though they should have default values.

Error:

Must specify service key signature algorithm (e.g. --service-sig-algo=ECDSA_P256)

Steps to Reproduce

Run emulator with only --service-priv-key value set.

Acceptance Criteria

It should use default values for sig and hash algo flags.

Unlinking capability does not persist between transactions in emulator - v0.18.0

Problem

When unlinking a capability in a first transaction, I am still able to access the reference from this capability in a second transaction. At the end of the first transaction, the capability appears as unlinked.

This example is ran with the v0.18.0 of flow-cli.

Steps to Reproduce

In a new folder, run flow init.
Create two files:

  • unlink.cdc: it unlink the receiver for the flow token vault. It does not panic, the capability is unlinked.
import FungibleToken from 0xee82856bf20e2aa6
import FlowToken from 0x0ae53cb6e3f42a79

transaction() {
    prepare(acct: AuthAccount) {
        acct.unlink(/public/flowTokenReceiver)

        if acct.getCapability<&{FungibleToken.Receiver}>(/public/flowTokenReceiver).borrow() != nil {
            panic("This should not be possible")
        }
    }
}
  • check_unlink.cdc: it runs the same check as the previous transaction, but this time it panics.
import FungibleToken from 0xee82856bf20e2aa6
import FlowToken from 0x0ae53cb6e3f42a79

transaction() {
    prepare(acct: AuthAccount) {
        if acct.getCapability<&{FungibleToken.Receiver}>(/public/flowTokenReceiver).borrow() != nil {
            panic("This should not be possible")
        }
    }
}

Run the two transactions:
flow transactions send unlink.cdc --signer emulator-account
flow transactions send check_unlink.cdc --signer emulator-account

The second transaction panics but it should not.

Acceptance Criteria

The capability should be unlinked in the second transaction, and this transaction should not panic.

Add flag to toggle transaction & gas fees

Issue To Be Solved

Fees are optional in FVM, but this option isn't exposed to users of the emulator. This would be useful for developers who want to experiment with the emulator without worrying about fees.

Suggest A Solution

flow emulator start --disable-fees

or

flow emulator start --disable-tx-fees --disable-account-fees --disable-storage-fees

flow CLI panics when deploying contract via "flow accounts add-contract"

Instructions

Please fill out the template below to the best of your ability and include a label indicating which tool/service you were working with when you encountered the problem.

Problem

Go panics when deploying a specific contract via the CLI. When deploying all contracts using the same config, etc, the contract is successfully deployed.

Steps to Reproduce

When deploying a contract via flow accounts add-contract Emit ./contract.cdc, I get a panic:

❯ flow accounts add-contract Emit ./contract.cdc
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc7ceb2]

goroutine 1 [running]:
github.com/onflow/flow-cli/pkg/flowkit.(*Account).Address(...)
        github.com/onflow/flow-cli/pkg/flowkit/account.go:40
github.com/onflow/flow-cli/pkg/flowkit.addAccountContractWithArgs(0x0, 0x7b2911503d08, 0x4, 0xc0003d2090, 0x83, 0xc000301a70, 0x0, 0x0, 0x7939b2, 0xc0000d6000, ...)
        github.com/onflow/flow-cli/pkg/flowkit/transaction.go:112 +0x272
github.com/onflow/flow-cli/pkg/flowkit.NewAddAccountContractTransaction(...)
        github.com/onflow/flow-cli/pkg/flowkit/transaction.go:80
github.com/onflow/flow-cli/pkg/flowkit/services.(*Accounts).AddContract(0xc0000c93b0, 0x0, 0x7b2911503d08, 0x4, 0xc00040e000, 0x83, 0x283, 0x0, 0x0, 0x0, ...)
        github.com/onflow/flow-cli/pkg/flowkit/services/accounts.go:223 +0xd7
github.com/onflow/flow-cli/internal/accounts.addContract(0xc00033c1e0, 0x2, 0x2, 0x16e5e80, 0xc00033a1f0, 0x0, 0x0, 0x1416e52, 0x4, 0x0, ...)
        github.com/onflow/flow-cli/internal/accounts/contract-add.go:67 +0x205
github.com/onflow/flow-cli/internal/command.Command.AddToParent.func1(0x1f2a9a0, 0xc00033c1e0, 0x2, 0x2)
        github.com/onflow/flow-cli/internal/command/command.go:116 +0x395
github.com/spf13/cobra.(*Command).execute(0x1f2a9a0, 0xc000142030, 0x2, 0x2, 0x1f2a9a0, 0xc000142030)
        github.com/spf13/[email protected]/command.go:856 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000318500, 0xc00085bf20, 0x1, 0x1)
        github.com/spf13/[email protected]/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/[email protected]/command.go:897
main.main()
        github.com/onflow/flow-cli/cmd/flow/main.go:69 +0x425
  • Note that I get the same panic with or without the presence of a --network=emulator flag, and when I try to do anything that changes the state of the protocol and is a flow accounts... subcommand. i.e:
❯ flow accounts get f8d6e0586b0a20c7

Address  0xf8d6e0586b0a20c7
Balance  9999999999.99500000
Keys     1

Key 0   Public Key               c1d20f3a7abd3663c154ce241807634502db84df45b362dd355976c5cb84c2d07595bec43d2fddad808d7957fc971d32048b7491461d300e618b979073ad73a0
        Weight                   1000
        Signature Algorithm      ECDSA_P256
        Hash Algorithm           SHA3_256
        Revoked                  false
        Sequence Number          3
        Index                    0

Contracts Deployed: 3
Contract: 'Emit'  // <----- This was deployed AFTER this report, as shown below.
Contract: 'FlowServiceAccount'
Contract: 'FlowStorageFees'


Contracts (hidden, use --include contracts)

works, while:

❯ flow accounts remove-contract Emit --network=emulator
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc7cbc2]

goroutine 1 [running]:
github.com/onflow/flow-cli/pkg/flowkit.(*Account).Address(...)
        github.com/onflow/flow-cli/pkg/flowkit/account.go:40
github.com/onflow/flow-cli/pkg/flowkit.NewRemoveAccountContractTransaction(0x0, 0x7d9a37ceb625, 0x4, 0x0, 0x8, 0xc000187800)
        github.com/onflow/flow-cli/pkg/flowkit/transaction.go:89 +0x22
github.com/onflow/flow-cli/pkg/flowkit/services.(*Accounts).RemoveContract(0xc000342f60, 0x0, 0x7d9a37ceb625, 0x4, 0x0, 0x0, 0x0)
        github.com/onflow/flow-cli/pkg/flowkit/services/accounts.go:309 +0x85
github.com/onflow/flow-cli/internal/accounts.removeContract(0xc0000c6da0, 0x1, 0x2, 0x16e5e80, 0xc000333960, 0x0, 0x0, 0x1416e52, 0x4, 0x0, ...)
        github.com/onflow/flow-cli/internal/accounts/contract-remove.go:58 +0xbb
github.com/onflow/flow-cli/internal/command.Command.AddToParent.func1(0x1f2ac20, 0xc0000c6da0, 0x1, 0x2)
        github.com/onflow/flow-cli/internal/command/command.go:116 +0x395
github.com/spf13/cobra.(*Command).execute(0x1f2ac20, 0xc00018c030, 0x2, 0x2, 0x1f2ac20, 0xc00018c030)
        github.com/spf13/[email protected]/command.go:856 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc0001fb180, 0xc0006bff20, 0x1, 0x1)
        github.com/spf13/[email protected]/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/[email protected]/command.go:897
main.main()
        github.com/onflow/flow-cli/cmd/flow/main.go:69 +0x425

doesn't.

When I get the panic, no request makes it to the gRPC server, so there are no relevant logs thrown by the emulator.

However, when deploying the same contract via: flow project deploy --network=emulator, I get the following result:

❯ flow project deploy --network=emulator

Deploying 1 contracts for accounts: one

Emit -> 0xf8d6e0586b0a20c7 (b396ee6e6e5a29a67407c56ba54077b3f6abed57e9549649184c487f7971707b)


✨ All contracts deployed successfully

The flow config:

❯ cat ~/flow.json

        │ File: /home/drifter/flow.json
   1   │ {
   2   │     "emulators": {
   3   │         "default": {
   4   │             "port": 3569,
   5   │             "serviceAccount": "one"
   6   │         }
   7   │     },
   8   │     "networks": {
   9   │         "emulator": "127.0.0.1:3569",
  10   │         "mainnet": "access.mainnet.nodes.onflow.org:9000",
  11   │         "testnet": "access.devnet.nodes.onflow.org:9000"
  12   │     },
  13   │     "accounts": {
  14   │         "one": {
  15   │             "address": "f8d6e0586b0a20c7",
  16   │             "key": "82b049f5fc2a963445213b47b7720a7ab92dd9366fdc0393c05959ec1bd43de4"
  17   │         },
  18   │         "two": {
  19   │             "address": "0x01cf0e2f2f715450",
  20   │             "key": "29ab6141b0746311f019fce9d9ea1e03f6793db1242e62b731847a8080b36051"
  21   │         },
  22   │         "three": {
  23   │             "address": "0x179b6b1cb6755e31",
  24   │             "key": "40dfa1ee8bdd6eb7daed35dc6aaa9bba2261e53916888c362e5d41b614bc8a77"
  25   │         }
  26   │     },
  27   │     "contracts": {
  28   │     "Emit": "/home/drifter/code/dapps/3mittr-contracts/3mit/contract.cdc"
  29   │   },
  30   │     "deployments": {
  31   │     "emulator": {
  32   │       "one": [
  33   │         "Emit"
  34   │       ]
  35   │     }
  36   │   }
  37   │ }

and the contract code:

❯ cat ~/code/dapps/3mittr-contracts/3mit/contract.cdc
        │ File: /home/drifter/code/dapps/3mittr-contracts/3mit/contract.cdc
   1   │ pub contract Emit {
   2   │     pub let emit: String
   3   │
   4   │     init() {
   5   │         self.emit = "test"
   6   │     }
   7   │
   8   │     pub fun getEmit(): String {
   9   │         return self.emit
  10  │     }
  11  │ }

Acceptance Criteria

n/a

Context

Not blocking anything. Low priority.

❯ pfetch
      /\          drifter@simulacrum
     /  \         os     Artix Linux
    /`'.,\        host   Blade 15 Base Model (Early 2020) - RZ09-0328 5.04
   /     ',       kernel 5.12.7-hardened1-1-hardened
  /      ,`\      uptime 8h 55m
 /   ,.'`.  \     pkgs   795
/.,'`     `'.\    memory 4201M / 15871M

❯ flow version
Version: v0.25.1
Commit: b7fe92a05d9d82496a2572e4245e85df335654b2
❯ flow status

Status:          🟢 ONLINE
Network:         emulator
Access Node:     127.0.0.1:3569
❯ procs flow
 PID:▲ User    │ TTY   CPU MEM CPU Time │ Command
               │       [%] [%]          │
 7750  drifter │ pts/5 0.0 0.2 00:00:01 │ flow emulator --persist -v

Service private key is not correctly set when public key is present

Problem

Using flags to set private and public key is causing private key to be ignored and not set on the configuration object.

Steps to Reproduce

Pass both values --service-priv-key and --service-pub-key and check what was the private key that was actually set.

Acceptance Criteria

The private key should be set when passed using a flag.

Context

The code processing configuration private and public key is faulty as seen here:
https://github.com/onflow/flow-emulator/blob/master/cmd/emulator/start/start.go#L95-L118

	if len(conf.ServicePublicKey) > 0 {
		checkKeyAlgorithms(serviceKeySigAlgo, serviceKeyHashAlgo)

		servicePublicKey, err = crypto.DecodePublicKeyHex(serviceKeySigAlgo, conf.ServicePublicKey)
		if err != nil {
			Exit(1, err.Error())
		}
	} else if len(conf.ServicePrivateKey) > 0 {
		checkKeyAlgorithms(serviceKeySigAlgo, serviceKeyHashAlgo)

		servicePrivateKey, err = crypto.DecodePrivateKeyHex(serviceKeySigAlgo, conf.ServicePrivateKey)
		if err != nil {
			Exit(1, err.Error())
		}

		servicePublicKey = servicePrivateKey.PublicKey()
	} else {
		servicePrivateKey, serviceKeySigAlgo, serviceKeyHashAlgo = getServiceKey(
			conf.Init,
			serviceKeySigAlgo,
			serviceKeyHashAlgo,
		)
		servicePublicKey = servicePrivateKey.PublicKey()
	}

API for resetting the emulator state

Instructions

Please fill out the template below to the best of your ability.

Issue To Be Solved

As part of the emulator management API, it would be useful to have a feature that:

  • Fully resets the emulator state
  • Waits for the emulator to be ready to receive new commands

(Optional): Suggest A Solution

I am not so familiar with the emulator code, I imagine the storage components (memstore and badger) would need to be reset, or perhaps the storage task can be reset as a whole while emulator actions are paused.

(Optional): Context

We currently run the emulator in a k8s cluster for development, and use a small node app in the container to start/seed/reset the emulator. It would be much more efficient and convenient for resetting dev/test state if the emulator exposed a reset mechanism accessible from outside the container.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.