square / beancounter Goto Github PK

Hey guys,

I'm trying to run the beancounter tool but receive a Failed negotiate version error:

go run ./main.go compute-balance --type multisig --block-height 1438791 --debug

paste public key:
tpubDBrCAXucLxvjC9n9nZGGcYS8pk4X1N97YJmUgdDSwG2p36gbSqeRuytHYCHe2dHxLsV2EchX9ePaFdRwp7cNLrSpnr3PsoPLUQqbvLBDWv

Result is:

2020/10/19 19:27:58 connecting to testnet.hsmiths.com
2020/10/19 19:27:59 172.88.21.24:53012 <- {"id":1,"method":"server.features","params":[]}
2020/10/19 19:27:59 172.88.21.24:53012 -> {"jsonrpc": "2.0", "result": {"hosts": {"testnet.hsmiths.com": {"ssl_port": 53012, "wss_port": 53014}}, "pruning": null, "server_version": "ElectrumX 1.15.0", "protocol_min": "1.4", "protocol_max": "1.4.2", "genesis_hash": "000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943", "hash_function": "sha256", "services": ["ssl://testnet.hsmiths.com:53012", "wss://testnet.hsmiths.com:53014"]}, "id": 1}
2020/10/19 19:27:59 172.88.21.24:53012 <- {"id":2,"method":"server.version","params":["beancounter","1.2"]}
2020/10/19 19:28:00 172.88.21.24:53012 -> {"jsonrpc": "2.0", "error": {"code": 1, "message": "unsupported protocol version: 1.2"}, "id": 2}
2020/10/19 19:28:00 error on recv from 172.88.21.24:53012: server error (1: unsupported protocol version: 1.2)
panic: Failed negotiate version

goroutine 1 [running]:
github.com/square/beancounter/utils.PanicOnError(...)
	/Users/ilia/Sandbox/beancounter/utils/utils.go:13
main.doComputeBalance()
	/Users/ilia/Sandbox/beancounter/main.go:229 +0xbe5
main.main()
	/Users/ilia/Sandbox/beancounter/main.go:71 +0x1ca
exit status 2

Do you plan to support the v1.4 protocol?

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

We should go back to showing something like this:

+---------------+--------------------+-------------------+---------------------+-------+
| TOTAL BALANCE | LAST RECEIVE INDEX | LAST CHANGE INDEX |     REPORT TIME     | BLOCK |
+---------------+--------------------+-------------------+---------------------+-------+
|        123456 |                 28 |                11 | 26 Sep 18 13:38 PDT | 12345 |
+---------------+--------------------+-------------------+---------------------+-------+

We know which block number we want to reach and we know (roughly) what block numbers the transaction we are fetching belong to. We can thus implement a progress bar.

Small outputs cannot be used (because the fee per byte x bytes required to use the output is greater than the output) and are called dust.

Should we track dust and report the total balance in a wallet followed by the amount of dust? This would give a more accurate view of the actual usable balance.

At minimum would be great to have binaries built on Travis for a git tag. As a bonus we could auto-update Github Releases as well.

We have one or more bugs we still need to squash.

We have a bunch of unittests, but no integration tests.

We should probably think of integration testing at the same time as #43, as they both might require specific architectural changes.

In the electrum backend, we should retry blacklisted nodes (either using a time decay or when there are no other nodes left). Otherwise, we risk getting (and staying) disconnected from the network.

Currently, you can invoke beancounter with --type single-address but forget to set lookahead. Lookahead will default to 100, performing a ton of useless work.

how to check simple address 1NarpHgn2moL6Hes1kMPRLfPtgaM8aPxtm

It's possible for an electrum server to have stale data. When we connect to a server, we should do a height check.

We make a few assumptions which can lead us to miss transactions if there are too many. For example, we assume the electrum servers are not pruning transactions. We assume a single request is enough to get the full address history. Etc.

Instead of silently returning an incorrect balance, we should at least panic() if an address has too many transactions.

We should fail (instead of returning an incorrect) result if find-block lands on current-height - 11.

Otherwise, invocations such as ./beancounter find-block "2022-01-01 12:00:00 GMT" will return a different result (until we cross 2022).

• min confirmations doesn't take into account that current height is a confirmed block.
• lookahead is off-by-one

To ensure reproducible results, we filter some blocks:

When computing a balance:
Beancounter only computes balances up-to "current height" - 6 blocks. We discard the last 6 blocks + unconfirmed blocks.

When mapping a timestamp to a block number:
We can only find blocks in the range 6 to current height - 11 (because of the way we take the median).

When the backend is processing data and something goes wrong, the request is re-queued (with the hope that another node will successfully handle the work). This creates a potential infinite loop. You can see this happen if you try to process an address with a huge number of transactions using the Electrum backend (e.g. 18cBEMRxXHqzWWCxZNtU91F5sbUNKhL5PX has 12k transactions, too many for Electrum).

When we re-queue the work, we should track how many attempts we have done. If the number of attempts is too high, we should panic().

There's an issue with fixtures which makes it look like our cache is inconsistent. Figure out why this happens. We have a temporary workaround (reschedule things), but we should find the root cause.

Address deriver can be improved in many ways:

• don't assume m=1, n=1 for non-multisig addresses. Instead, take an explicit address type parameter.
• don't assume multisig == segwit. It should be two flags.
• support the full range of address types.

fixtures only work correctly if all the arguments are unchanged. We should store some kind of hash(params) where params contains the xpubs, single-address, m, lookahead, block-height, etc. and panic if there's a mismatch.

Given that the tool can take a few hours in some cases, we should support saving the state / resuming.

We need to be careful about not using outdated state if the parameters change. We could simply throw the entire state away, or be smarter and keep some partial state forever (e.g. tx data never changes).