Inconsistency in the results of function seqeunce execution about mythril

[norhh]

This is likely due to lack of exploration, did your code halt at or around the stipulated execution time? You can test it by making it slightly easier, like fixing the burn(uint256) in tx-4.

[qiana0223]

Hi Norhh,

The execution takes about 1 day. It does not exceed 360000s (100hours).
From the end part of the logging result, there are about 100 occurrences of z3 timeout.

If I want to fix burn() in tx-4, are the following parameters correct?

-v5
analyze
*/solidity_files/HoloToken.sol:HoloToken
--transaction-sequences
[[ ],[ ],[ ],[1117154408]]
--solv
0.4.18

Note that 1117154408 is the hash of burn().

[norhh]

Yes, you can empty out the rest [[],[],[],[1117154408]]
Maybe you can increase the execution timeout to a few hours

[qiana0223]

with the following parameters:

-v5
analyze
../smartExecutor/tests/testdata/solidity_files/HoloToken.sol:HoloToken
--transaction-sequences
[[],[],[],[1117154408]]
--solv
0.4.18
--execution-timeout
390000

I have encountered this error:

This error did not occur when I executed this contract with tx=4.

[qiana0223]

Here are the parameters used:

-v5
analyze
*/solidity_files/HoloToken.sol:HoloToken
--transaction-sequences
[[],[],[],[1117154408]]
--solv
0.4.18
--execution-timeout
390000

The following screenshot shows the partial results:

It still shows that no new instructions are found.

[qiana0223]

I have re-run with the same parameters as I was afraid I might encounter some error for the assertion violation. The partial results are shown above, which is different from what I showed before.

[norhh]

Based on reading through the contract, the number of covered Instructions being 0 is expected here, as first few transactions cover almost all instructions. So, it's rare for new transactions to cover new instructions. But the new transactions cover new program states.

[norhh]

The assertion error is actually due to the nondeterminism in Z3. I'm trying to reproduce it, But I don't seem to get it.

[norhh]

If you are auditing, the best method of using the tx-sequences is to guess some likely sequences which might lead to issues and try them.
If you are using it to build a tool, best method is to use some heuristics to choose suspicious tx-sequences

[qiana0223]

My goal: collect all the function sequences that can be executed to generate new program states (up to sequences of length 4).

Problem: I can not collect some of them as their executions fail in the context of exploring all sequences. If I check them individually, they can be executed successfully.

Since I want to collect all the function sequences for a great many contracts, it is impossible to manually verify each contract whether some valid sequences failed to be collected.

If the result of executing one function sequence independently is the same as the result of executing it with many others, then I do not need to manually check.

Is there a way to remove the manual check?

[norhh]

Using "new instructions discovered" isn't a good way to see if the execution failed midway.
As in this example, solc optimises bytecode to reuse most of the code, such as sub(), require, ....
Most of the code in the function burn() is reused and is also reachable with mythril, a best way to verify is to check with an assert(False) and check for exceptions. The exception is triggered with [[],[],[],[1117154408]], the location pointed by assert is also different because even the assert gets reused.

[norhh]

If you are working on an offline strategy which doesn't care much about performance, you can incrementally build all possible transaction sequences and run them through mythril

[qiana0223]

Actually, I use the global states generated at 'STOP' and 'RETURN' to make decisions, as they mean the successful execution of complete paths. I just use the number of new instructions found to explain this problem as this information is easily obtained and can explain the problem with the given contract.

Describe this problem in another way. When executing one function sequence individually, the execution successfully generates global states at 'STOP' or 'RETURN' at the last function in the sequence, thus new world states are generated. When executing this sequence among a great many other function sequences, no global states are generated at 'STOP' or 'RETURN' at the last function of this sequence.

I am wondering what causes this difference.

[norhh]

no global states are generated at 'STOP' or 'RETURN' at the last function of this sequence,
the probable reason is that the default search strategy is bfs, so it's bound to not fully explore the final transaction before the timeout.
When I tried running the above example for a long timeout with just the last function fixed which has an assert(false), it detects this assertion.

[qiana0223]

Yes. Due to the timeout, no global states are generated at 'STOP' or 'RETURN' at the last function of the sequence. Then why is there no timeout when this sequence is executed individually so that global states are generated at 'STOP' or 'RETURN'?

[norhh]

The growth is exponential. It requires searching ~6^4 states in the 4th transactions. Which requires a long timeout. Like multiple days to weeks.

[qiana0223]

So, the long timeout is required because more time is needed to explore the states. As I have seen the occurrences of the solver(z3) timeout, is the long timeout also caused by the solver?

[norhh]

Solver takes a long time to solve, so it's an additional contributor for the long execution times including the path explosion discussed earlier

[qiana0223]

Sorry, I am still not able to understand why a function sequence that can be executed to generate new world states (Global states at 'STOP' or 'RETURN) can not be executed to generate new world states among a great many other sequences. I agree that it would take a lot of time to reach the sequence among a huge collection of sequences. But once the sequence is reached, the execution of it would be the same as the case when it is executed alone. The solver timeout is the same, the sequence is the same, why the execution results are not the same?

[norhh]

But once the sequence is reached, the execution of it would be the same as the case when it is executed alone
The execution in this case halts before reaching that particular sequence