Suspicion something non-deterministic is happening in the current VM about neo-vm

[jsolman]

Latest chain files with current neo core master code base are now syncing without issue.

[jsolman]

BTW in case anyone else hits this, I am fairly sure the issue was actually breaching the maximum open files limit for the process. Make sure to up those when syncing from chain file if using a very fast disk and running on a flavor of Linux with a low default file descriptor limit (1024 is common, but not enough).

[jsolman]

Maybe by examining common script calls made by ANcEb4Xw8ouJnk5e5Q1z6o92Z67pNrydQ2 and by AWnA7E3tNpc3vtYRWdxkHYiRBBu2mSsAYR before the block numbers given in my exception messages will give a clue as to what operations might have been non-deterministic. I wish I had more information already to go on, but I had a lot of other things going on and not able to give this issue a lot of time yet.

If anyone else has reproduced and is looking into it, comment here and let me know.

[igormcoelho]

@jsolman that's a very good initiative! Vitor and I have been discussion with Wingy and others regarding how to provide a more stable platform regarding storage. I'll take a look soon on this issue... you can use ```cs on github to put C# code (with highlighting and stuff) ;)

[jsolman]

I did use ``` Probably just not in the initial comment though; so it probably only emails the initial one that didn't have the usage of it, but I added it within a short time of the initial post.

[erikzhang]

I can't reproduce this issue. None of the nodes I ran encountered this problem. Includes all consensus nodes and seed nodes in both mainnet and testnet.

[jsolman]

What version of leveldb are they running? Maybe something is wrong with my leveldb. I’ll look into that first.

Update: I don't think anything is wrong with my leveldb

[jsolman]

MainNet Consesnus nodes are not running the latest VM. I have confirmed it even though no one seems to say what vm code the consensus nodes are actually running.

There are implications with this, as people running the latest VM that use a NEP8 enabled contract on MainNet that exchange NEP5 could end up with balances on their nodes != balances on other nodes and consensus nodes, but after the consensus nodes are updated and resync the chain from the beginning they will see the balances the same as the new nodes running now that support NEP8.

I think it is important to update current MainNet consensus nodes to the latest VM as a high priority.

@erikzhang Can you confirm that when you synced the chain on 2.9.0 and didn't experience the problem I'm reporting here, that it was built using v2.3.1.1 of the VM?

[jsolman]

One thing this indicates to me is the advantage that it would be to have a merkle tree backing the storage and adding the hash of storage into the block header. Or having hash of the full state related to the VM execution (could for instance verify the hash of all instructions executed in the block) -- this is related to the idea here: neo-project/neo#385

[jsolman]

Maybe for Neo 3.0 the gas consumed for a block during the contract execution can be included in the block header and any changes to vm can be explicitly enabled by block height.

This would be less expensive to calculate and way less resource intensive than the merkle tree of storage, etc, but merkle tree of storage allows light nodes to verify their storage retrieved using block headers; maybe want both.

[erikzhang]

The consensus nodes will be upgraded to 2.9 next week as planned.