meritlabs / merit Goto Github PK

The rpc call estimatesmartfee is not returning a fee estimate. Double check to make sure if it's because there is insufficient data in the test network or if there is a bug.

Blocks need to include transactions and referrals. Should be compatible with miners.

ParseHex seems like it's causing issues.

Wallet software should check if a referral code from cli input is contained in blocks or mempool.

ANV computation uses a DB which contains aggregate ANV values. The ReferralsViewDB can update a particular key's ANV and it's corresponding parents.

We need to hook the updates up when a block is connected to the chain.

#52

unlockwallet <workingCode> should work.

Trying to stay disciplined here, and add this note for later, despite my deep desire to do it.

Base58.cpp has come a long way since the original bitcore indexing code was written. Today, we are manually type-checking like below:
https://github.com/meritlabs/merit/blob/feature/latest-bitcore-indexing/src/rpc/misc.cpp#L671-L675

Because of boost::variant, we can drop this manual approach in favor of DecodeDestination and EncodeDestination in most places. With that said, it affects a lot of the core code that MWS relies on, so I want to be sure we refactor it when the time is right, and not rush it now.

The following RPC methods are affected:

• getaddressdeltas
• getaddressmempool
• getaddresstxids
• getaddressutxos

We should have a robust, yet elegant, way of handling easyReceive transactions when more than one person is trying to unlock the same Merit.

Intro

edgesRatio parameter was introduced to express cuckoo cycle search algorithm difficulty. It says how many edges to generate to look for a cycle. the more edges we add, the more probability to find a cycle of N-length.

Problems:

• cycle verification algorithm doesn't take into account difficulty, that was used to find a cycle. thus, one can rewrite miner to use the least difficulty, find a cycle and set edgesRatio value to required by network.
• original advanced mining algorithm doesn't support it at all unlike the simple one. we added support for decreasing difficulty so it can be set from 0 to 50%. setting it to values gt 50% requires more significant changes.

Possible solutions

• we simply add check for edges in a cycle to be less than required value built with edgesRatio.

    // for cases when difficulty can not be gt 50
    edgesRatio / MAX_EDGES_RATIO * (1 << EDGEBITS)
    // for cases when difficulty can be set up to 100
    edgesRatio / MAX_EDGES_RATIO * (2 << EDGEBITS)
  

  The problem is, that with advanced algo we provide gaps between "buckets" of edges, e.g. we generate edges: 0..13, 16..29, 32..45, 48..61 - number of edges is 56, but the max value if 61. in case of simple miner it would be successive 56 edges (from 0 to 55) and last one would match the requirement. so both set of edges math the requirement of number of edges, but max values do not match. In this way, we'd impose restrictions on miner implementation

• we can try to modify advanced algorithm for full support of all ranges of difficulty levels, but now it doesn't seem possible to me.

After conversation with @mempko decided to get rid of edgesRatio parameter (that is a base for difficulty control for cuckoo algorithm) as it's not possible to prove it's verity and when new block comes from the network, as stated in cuckoos whitepaper, it's not suitable for cryptos as it does not provide much granularity for precise control of difficulty (not sure it's true, as it can be set to greater range of values than in original paper, where working set expressed in percents and set to ~ 45-50%).

Suggestion

We need to try to rewrite mean miner (advanced one) to support 100% difficulty (easiness actually) to measure time/memory usage to make some adjustments to hashcash pow difficulty

We want to write referrals to the RefDb during a couple key events:

• When receiving a new block from a peer.
• When syncing blockchain for first time.
• Possibly when we get a new RefTransaction in our MemPool.

Sign referrals to ensure that you are the owner of the address that was beaconed

Example of varying responses below:

Command 1

$ m1 unlockwalletwithaddress mwcSEbFa8pYGsVhez6UQnsH2ui9Rquom9X 58094f46fbi
error code: -1
error message:
unlockwalletwithaddress: provided code does not exist in the chain (RPC)

Command 2

$ m1 unlockwalletwithaddress 5094f46fbi rwioearosiet
{
  "isvalid": false
}

It seems that rpcwallet is not the best place for it.

We want to continue to stay up-to-date with the latest and greatest from upstream. There are some really talented developers on the BC core team. This also helps us ensure that we are staying abreast of security and consistency enhancements in core.

The genesis referral needs to be in the refDb in order to be used for downstream validation of all other referrals.

Wallet unlock flow

1. wallet is locked when created. No key pairs are generated (keypool is empty)
2. unlockwallet is called with bare referral code(short (?) string that is shared publicly)
3. one key pair is generated alongside with referral code associated with pubkey generated. beacon transaction is sent to the network
4. beacon transaction propagates through the network and gets to referral mempool on all nodes
5. transaction got mined in the next (or not) block
6. mined block is propagated through the network
7. local node gets new block when mined and looks for its own beacon transactions
8. if found in the block(confirmed) it unlocks the wallet and generates all other key pairs for the keypool (TopUpKeyPool) with associated referrals and sends beacon transactions for these keys
9. if not found goto 6.

Tasks

1. Disable keypool creation on startup
2. Add unlockwallet rpc method and CWallet::Unlock(std::string codeIn) method, that generates keypair, referral with provided code and new pubkey, send beacon transaction to the network
3. Add processing of network messages - check for beacon transaction in new blocks when mined and the same while downloading blockchain and lookup for wallet's transactions and save them to local database and unlock the wallet if first beacon transaction if found

Blockchain fails to sync with the testnet cluster producing next messages:

2017-09-01 08:55:02 ProcessMessages(headers, 15581 bytes): Exception 'non-canonical ReadCompactSize(): unspecified iostream_category error' caught
2017-09-01 08:55:02 ProcessMessages(headers, 15581 bytes) FAILED peer=1
2017-09-01 08:55:02 ProcessMessages(headers, 15581 bytes): Exception 'non-canonical ReadCompactSize(): unspecified iostream_category error' caught
2017-09-01 08:55:02 ProcessMessages(headers, 15581 bytes) FAILED peer=2
2017-09-01 08:55:02 ProcessMessages(headers, 15581 bytes): Exception 'non-canonical ReadCompactSize(): unspecified iostream_category error' caught
2017-09-01 08:55:02 ProcessMessages(headers, 15581 bytes) FAILED peer=0

When a note receives a transaction, it should relay it to it's peers.

I've been wondering if we should do a belt-and-suspenders check in the selection of reward recipients to be sure that all their keys have been beaconed.

Obviously, by the point that they have significant ANV, theirs keys are most likely already beaconed. But a second check may not hurt, to be extra safe.

If we do so, I think we should do it after the selection has already been run. (To keep things fast, and since this will be a rare outcome.)

Pseudo logic:

• selector picks 5 winners.
• runs 'doubleChecker' to be sure all winners are beaconed.
  • finds that one is not. oops!
• re-samples to fill in that one slot with a new address

(All, done in a deterministic way, such as the way we have implemented.)

Slightly less work might be to sample N more than we need. Then just go down the list an extra X number of people, where X is the number of unbeaconed keys in the top list.

We must be sure we are showing the bytes of uints in the correct order.

Context

• Merit version 0.2.0 Windows x64
• Windows 10 - Version 1703 (Build15063.674)
• Intel Core i7-6700HQ CPU @ 2.60 GHZ.
• 32 GB Ram
• Using QT Desktop Wallet

Reproduction steps:

1. Cause error #193
2. Close application (force)
3. Re-open application.

Behavior:

• Blockchain will not finish synchronizing because it will fall into an 'ActivateBestChain failed" loop
  Output as follows:

2018-01-21 19:56:48 ERROR: ConnectBlock(): Referral 3c3b73daea6d16f2f4eec10944cdff86a8f03814f3905e68b0b88e79c92c52ca is already beaconed
2018-01-21 19:56:48 ERROR: ConnectTip(): ConnectBlock 8a4bea23c7fc13a2b1c13048a5712f939b8927b6e0d1c1dadac595f7706fd041 failed
2018-01-21 19:56:48 ERROR: ProcessNewBlock: ActivateBestChain failed:  (code 0)
2018-01-21 19:56:48 ERROR: ConnectBlock(): Referral 3c3b73daea6d16f2f4eec10944cdff86a8f03814f3905e68b0b88e79c92c52ca is already beaconed
2018-01-21 19:56:48 ERROR: ConnectTip(): ConnectBlock 8a4bea23c7fc13a2b1c13048a5712f939b8927b6e0d1c1dadac595f7706fd041 failed
2018-01-21 19:56:48 ERROR: ProcessNewBlock: ActivateBestChain failed:  (code 0)
2018-01-21 19:56:48 ERROR: ConnectBlock(): Referral 3c3b73daea6d16f2f4eec10944cdff86a8f03814f3905e68b0b88e79c92c52ca is already beaconed
2018-01-21 19:56:48 ERROR: ConnectTip(): ConnectBlock 8a4bea23c7fc13a2b1c13048a5712f939b8927b6e0d1c1dadac595f7706fd041 failed
2018-01-21 19:56:48 ERROR: ProcessNewBlock: ActivateBestChain failed:  (code 0)
2018-01-21 19:56:48 ERROR: ConnectBlock(): Referral 3c3b73daea6d16f2f4eec10944cdff86a8f03814f3905e68b0b88e79c92c52ca is already beaconed
2018-01-21 19:56:48 ERROR: ConnectTip(): ConnectBlock 8a4bea23c7fc13a2b1c13048a5712f939b8927b6e0d1c1dadac595f7706fd041 failed
2018-01-21 19:56:48 ERROR: ProcessNewBlock: ActivateBestChain failed:  (code 0)
2018-01-21 19:56:48 ERROR: ConnectBlock(): Referral 3c3b73daea6d16f2f4eec10944cdff86a8f03814f3905e68b0b88e79c92c52ca is already beaconed
2018-01-21 19:56:48 ERROR: ConnectTip(): ConnectBlock 8a4bea23c7fc13a2b1c13048a5712f939b8927b6e0d1c1dadac595f7706fd041 failed
2018-01-21 19:56:48 ERROR: ProcessNewBlock: ActivateBestChain failed:  (code 0)
2018-01-21 19:56:48 ERROR: ConnectBlock(): Referral 3c3b73daea6d16f2f4eec10944cdff86a8f03814f3905e68b0b88e79c92c52ca is already beaconed
2018-01-21 19:56:48 ERROR: ConnectTip(): ConnectBlock 8a4bea23c7fc13a2b1c13048a5712f939b8927b6e0d1c1dadac595f7706fd041 failed
2018-01-21 19:56:48 ERROR: ProcessNewBlock: ActivateBestChain failed:  (code 0)

Possible solutions:

• The daemon probably should not attempt to unlock the wallet until the chain has synchronized fully.
• Harden the unlock command to queue the unlock

When trying to configure with --enable-debug flag, compilation fails with:

Fatal error: can't close libmerit_server_a-validation.o: File is too big

Environment:
Latest Windows, Ubuntu in WSL, mingw, x64.

In misc.cpp, there is a CLI command `getaddressmempool.' If we create a transaction, then generate 1 or more blocks, this CLI method will still return results, even though the general mempool has already been cleared:

https://github.com/meritlabs/merit/blob/feature/latest-bitcore-indexing/src/rpc/misc.cpp#L712-L781

m1 getaddressmempool '{"addresses": ["mrjV25Df1h5iGinFUZJaLMYnEcXbxJgf33"]}'

[
  {
    "address": "mrjV25Df1h5iGinFUZJaLMYnEcXbxJgf33",
    "txid": "c51caae1246e3044518d49aa44a8f2539f3c44edd6d6aa06eb3249a4c6904764",
    "index": 0,
    "satoshis": 100000000,
    "timestamp": 1505868866
  }
]

m1 getrawmempool true

{
}

By default, bitcoin is is sending the change of a transaction to a reservedKey. This will fail validation.

We should ensure that the reserveKey is beaconed. There are two ways we can handle this:

• reservekey can be one of the keys that we've already beaconed when creating the wallet.
• We can beacon the reservekey just as soon as we pull it from the keypool.

See here for the relevant bits of code:
https://github.com/meritlabs/merit/blob/feature/latest-bitcore-indexing/src/wallet/wallet.cpp#L2925-L2934

https://github.com/meritlabs/merit/blob/feature/latest-bitcore-indexing/src/wallet/rpcwallet.cpp#L408-L419

There should be an easy RPC method to be sure that an address is valid.

Changes from bitcore indexing merge have broken the building of the UI on OSX. Needs fix.

Describe the issue

Following the instructions (Mac OS X Build Instructions and Notes), I received errors when performing steps 3 and 4.

Screenshots.

'make check' errors

'make deploy' errors

What version of merit-core are you using?

0.1.0

Machine specs:

• OS: MAC OS 10.11.6
• CPU: intel core 2 duo / 2GHz
• RAM: 4 GB 667 MHz DDR2
• Disk size: 500 GB
• Disk Type (HD/SDD): HD

Check that vin address was beaconed if it uses CNoDestination vout.
Not sure if it's even an issue. Should figure out how this case works for vin's. I guess some address/key, that is used to sign transaction should be beaconed

It would be helpful to include the beaconed/unlocked state in the response payload.

Referrals will need a genesis referral in the block.

Context

• Merit version 0.2.0 Windows x64
• Windows 10 - Version 1703 (Build15063.674)
• Intel Core iJ7-6700HQ CPU @ 2.60 GHZ.
• 32 GB Ram

Reproduction steps:

1. Dismiss "Blockchain Syncing" overlay.
2. Enter unlock code and click submit.

Behavior:

• Blockchain synchronization stops, as evidenced by logs.
• Wallet UI stops responding, as seen below.

Possible solutions:

• Do not let the user dismiss the blockchain synchronization screen.
• Do not let the user attempt the unlock until sync is complete.
• Allow user to submit unlock code, but do not attempt to publish to blockchain until synchronization is complete. (Eg, show a spinner, etc.)

See here for more context: https://github.com/meritlabs/merit-bitcoin/blob/b1fcd5df01a9f3f0ea99acd099a968c8ab76de34/src/net_processing.cpp#L1190-L1198

Do we go all the way back to the genesis each time? Or just check the last referral?

Reproduction steps as follows:

1. Create new lightwallet account.
2. Beacon "receive key"
3. Generate 1 block to confirm beacon.
4. Using CLI: generatetoaddress 103 <lightwalletaddress>
   (Money is received.)
5. Using CLI: generate 100
   (To ensure coinbase is mature)
6. Send transaction from LightWallet.
   (All appears fine)
7. Using CLI: generate 1

Receive error:
CreateNewBlock: TestBlockValidity failed: (code 0)

From step #6, transaction appears to be valid. With the following raw bytestream:
010000000159bc00f69f3a8df67188059e2b04099d94349a169fd3236c05dce80f79b6a0da030000006a4730440220656b110831c24058e2b76dd052154e4ca656adc9e20bcf1121122c8758094a4c02206d228334b731cc8e6e73e99bb2de9ff55c4108ba58a138c7d1cc79f60bf754d30121037b60b5c1e844a8a125b3bd31180a2838ea478ecff1f31ba93d7095f7faf7ba4dffffffff02001bb700000000001976a9143f39ba5088c6039f303ba64b6fa3aff7723e329b88ac70a05300000000001976a9149084e6183959cbbdac2627b493f3595042f91dd188ac00000000

Decoded, it looks like this:

{
  "txid": "914488f90dbf22cfb09b323220d6a56634944b809c2faa4c24d91dc92cb0416a",
  "hash": "914488f90dbf22cfb09b323220d6a56634944b809c2faa4c24d91dc92cb0416a",
  "version": 1,
  "size": 225,
  "vsize": 225,
  "locktime": 0,
  "vin": [
    {
      "txid": "daa0b6790fe8dc056c23d39f169a34949d09042b9e058871f68d3a9ff600bc59",
      "vout": 3,
      "scriptSig": {
        "asm": "30440220656b110831c24058e2b76dd052154e4ca656adc9e20bcf1121122c8758094a4c02206d228334b731cc8e6e73e99bb2de9ff55c4108ba58a138c7d1cc79f60bf754d3[ALL] 037b60b5c1e844a8a125b3bd31180a2838ea478ecff1f31ba93d7095f7faf7ba4d",
        "hex": "4730440220656b110831c24058e2b76dd052154e4ca656adc9e20bcf1121122c8758094a4c02206d228334b731cc8e6e73e99bb2de9ff55c4108ba58a138c7d1cc79f60bf754d30121037b60b5c1e844a8a125b3bd31180a2838ea478ecff1f31ba93d7095f7faf7ba4d"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 0.12000000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 3f39ba5088c6039f303ba64b6fa3aff7723e329b OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9143f39ba5088c6039f303ba64b6fa3aff7723e329b88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "mmHG28smdwFWjBqwpB8QGHqXbavYfKDpha"
        ]
      }
    },
    {
      "value": 0.05480560,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 9084e6183959cbbdac2627b493f3595042f91dd1 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9149084e6183959cbbdac2627b493f3595042f91dd188ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "mth6oea9vdu5yR9fBDWXCSaHyXYK3FY6RB"
        ]
      }
    }
  ]
}