eth-sri / ilf Goto Github PK

View Code? Open in Web Editor NEW

148.0 11.0 32.0 4.84 MB

AI based fuzzer based on imitation learning

License: Apache License 2.0

Dockerfile 0.60% JavaScript 0.46% Go 7.13% Python 91.39% Solidity 0.43%

fuzzing blockchain smart-contracts testing symbolic-execution machine-learning imitation-learning

ilf's People

Contributors

Stargazers

Watchers

ilf's Issues

Training set and benchmark dataset

Hi, I'm wondering if you have released the training set that can reproduce the pretrained model, as well as the benchmark dataset you used for evaluation in the paper?

understanding logged data

Hi, I have three questions on interpreting logs generated by ILF.

I have tried the following command in Usage of README.md:

$ python3 -m ilf --proj ./example/crowdsale/ --contract Crowdsale --fuzzer imitation --model ./model/ --limit 2000

and then I obtained the following logs:

[2019-12-30 15:07:15,768][1577718435.768150][ilf.fuzzers.environment][INFO] {"tx_count": 0, "num_contracts": 1, "insn_coverage": 0.0, "block_coverage": 0.0, "Crowdsale": {"tx_count": 0, "insn_coverage": 0.0, "covered_insns": 0, "all_insns": 525, "block_coverage": 0.0, "covered_blocks": 0, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,768][1577718435.768257][ilf.fuzzers.environment][INFO] initial calls start
[2019-12-30 15:07:15,777][1577718435.777859][ilf.fuzzers.environment][INFO] {"tx_count": 1, "num_contracts": 1, "insn_coverage": 0.022857142857142857, "block_coverage": 0.034482758620689655, "Crowdsale": {"tx_count": 1, "insn_coverage": 0.022857142857142857, "covered_insns": 12, "all_insns": 525, "block_coverage": 0.034482758620689655, "covered_blocks": 2, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,778][1577718435.778866][ilf.fuzzers.environment][INFO] {"tx_count": 2, "num_contracts": 1, "insn_coverage": 0.10095238095238095, "block_coverage": 0.15517241379310345, "Crowdsale": {"tx_count": 2, "insn_coverage": 0.10095238095238095, "covered_insns": 53, "all_insns": 525, "block_coverage": 0.15517241379310345, "covered_blocks": 9, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,779][1577718435.779850][ilf.fuzzers.environment][INFO] {"tx_count": 3, "num_contracts": 1, "insn_coverage": 0.1180952380952381, "block_coverage": 0.1896551724137931, "Crowdsale": {"tx_count": 3, "insn_coverage": 0.1180952380952381, "covered_insns": 62, "all_insns": 525, "block_coverage": 0.1896551724137931, "covered_blocks": 11, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,780][1577718435.780659][ilf.fuzzers.environment][INFO] {"tx_count": 4, "num_contracts": 1, "insn_coverage": 0.13523809523809524, "block_coverage": 0.22413793103448276, "Crowdsale": {"tx_count": 4, "insn_coverage": 0.13523809523809524, "covered_insns": 71, "all_insns": 525, "block_coverage": 0.22413793103448276, "covered_blocks": 13, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,781][1577718435.781489][ilf.fuzzers.environment][INFO] {"tx_count": 5, "num_contracts": 1, "insn_coverage": 0.1523809523809524, "block_coverage": 0.25862068965517243, "Crowdsale": {"tx_count": 5, "insn_coverage": 0.1523809523809524, "covered_insns": 80, "all_insns": 525, "block_coverage": 0.25862068965517243, "covered_blocks": 15, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,782][1577718435.782350][ilf.fuzzers.environment][INFO] {"tx_count": 6, "num_contracts": 1, "insn_coverage": 0.16952380952380952, "block_coverage": 0.29310344827586204, "Crowdsale": {"tx_count": 6, "insn_coverage": 0.16952380952380952, "covered_insns": 89, "all_insns": 525, "block_coverage": 0.29310344827586204, "covered_blocks": 17, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,782][1577718435.782432][ilf.fuzzers.environment][INFO] initial calls end
...
[2019-12-30 15:07:21,973][1577718441.973119][ilf.fuzzers.environment][INFO] {"tx_count": 74, "num_contracts": 1, "insn_coverage": 0.8114285714285714, "block_coverage": 0.896551724137931, "Crowdsale": {"tx_count": 74, "insn_coverage": 0.8114285714285714, "covered_insns": 426, "all_insns": 525, "block_coverage": 0.896551724137931, "covered_blocks": 52, "all_blocks": 58, "bugs": {"Leaking": ["withdraw"]}}}
[2019-12-30 15:07:22,076][1577718442.076097][ilf.fuzzers.environment][INFO] {"tx_count": 75, "num_contracts": 1, "insn_coverage": 0.8114285714285714, "block_coverage": 0.896551724137931, "Crowdsale": {"tx_count": 75, "insn_coverage": 0.8114285714285714, "covered_insns": 426, "all_insns": 525, "block_coverage": 0.896551724137931, "covered_blocks": 52, "all_blocks": 58, "bugs": {"Leaking": ["withdraw"]}}}
[2019-12-30 15:07:22,197][1577718442.197205][ilf.fuzzers.environment][INFO] {"tx_count": 76, "num_contracts": 1, "insn_coverage": 0.8114285714285714, "block_coverage": 0.896551724137931, "Crowdsale": {"tx_count": 76, "insn_coverage": 0.8114285714285714, "covered_insns": 426, "all_insns": 525, "block_coverage": 0.896551724137931, "covered_blocks": 52, "all_blocks": 58, "bugs": {"Leaking": ["withdraw"]}}}
...

[Question 1]
For each transaction log, how can I know which function has been invoked with what argument values?

Does ILF have options for obtaining such information? The section 2.1 in the CCS '19 paper seems to suggest that ILF has this feature.

[Question 2]
Are bugs in the logs accumulated ones or not?

Specifically, for example, is the bug in tx_count 75 the one that was already found in tx_count 74 or the one that is newly found in tx_count 75?

[Question 3]
There is a log initial calls end after tx_count 6. What does initial calls end mean?

Environment

I used docker image in this git repository.

Using ILF on EVM bytecode as opposed to Solidity?

Is it possible to run ILF on EVM bytecode as opposed to Solidity? If so, how can this be done?

Can ilf be extended for fuzzing webassembly applications?

Encountered errors when Docker build

Hi,

Hey, I am trying to install ILF from a local clone by following the instructions in the README file, but encountered some errors.
I am using a Macbook with Apple M1 Pro chip, the operating system is macOS Monterey 12.4.
Any suggestion is appreciated, thanks in advance!

(base) ➜  ilf git:(master) docker build -t ilf .
[+] Building 60.0s (38/44)                                                      
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 2.19kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
...

 => ERROR [33/40] RUN pip3 install -r requirements.txt --no-cache-dir     11.7s
...
#0 5.200 Collecting persistent (from -r requirements.txt (line 7))
#0 5.422   Downloading https://files.pythonhosted.org/packages/ea/22/7fd2e6926f86d080c2dca58b93de3701e56914021e2c6ea8bac151657bde/persistent-4.9.3.tar.gz (134kB)
#0 11.50     Complete output from command python setup.py egg_info:
#0 11.50     c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
#0 11.50      #include <ffi.h>
#0 11.50               ^~~~~~~
#0 11.50     compilation terminated.
...

The info on some dependencies that may related can be seen below:

(base) ➜  ilf git:(master) pip3 install cffi
Requirement already satisfied: cffi in /Users/rr/anaconda3/lib/python3.10/site-packages (1.15.1)
Requirement already satisfied: pycparser in /Users/rr/anaconda3/lib/python3.10/site-packages (from cffi) (2.21)
(base) ➜  ilf git:(master) xcode-select --version
xcode-select version 2395.
(base) ➜  ilf git:(master) brew info libffi
==> libffi: stable 3.4.4 (bottled), HEAD [keg-only]
Portable Foreign Function Interface library
https://sourceware.org/libffi/
/opt/homebrew/Cellar/libffi/3.4.4 (17 files, 724.8KB)
  Poured from bottle using the formulae.brew.sh API on 2023-07-20 at 15:02:19
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/libffi.rb
License: MIT

I will paste the whole output in the comment. Thanks!

docker image build failed

I try to build the docker image, however I failed due to the docker keep complaining there is module pip3. Can anyone help?

invalid opcode

   Deploying 'Migrations'
   ----------------------
net_version
eth_blockNumber
eth_getBlockByNumber
eth_getBlockByNumber
eth_gasPrice
eth_sendTransaction

  Transaction: 0x07f42b812e1081c2bf3c47f08bddae1173511dcb7eee1c4b5103a81099ca88a6
  Contract created: 0xd1de1b9e45605d53fb74e2ec6fee19f9d7696273
  Gas usage: 1000000000
  Block number: 1
  Block time: Fri Dec 08 2023 08:37:48 GMT+0000 (Coordinated Universal Time)
  Runtime error: invalid opcode

   > transaction hash:    0x07f42b812e1081c2bf3c47f08bddae1173511dcb7eee1c4b5103a81099ca88a6
eth_getTransactionReceipt
eth_getCode
eth_call
   ✖ Blocks: 0            Seconds: 0
 *** Deployment Failed ***

"Migrations" hit an invalid opcode while deploying. Try:
   * Verifying that your constructor params satisfy all assert conditions.
   * Verifying your constructor code doesn't access an array out of bounds.
   * Adding reason strings to your assert statements.


Exiting: Review successful transactions manually by checking the transaction hashes above on Etherscan.


Error:  *** Deployment Failed ***

If an opcode such as PUSH0 is included in the latest version of the evm, the above error occurs.

/ilf/ethereum/evm/opcode.py

I modified the file above, but the error still occurs.
Will it work if I modify all python sources that process opcode in the /ilf directory?
Or do I need to modify other dependency packages?

Docker installation fails on OS X

Hey all,

docker build -t ilf . fails on OS X 12.2.1 during the pip installation of requirements:

 > [30/31] RUN pip3 install -r requirements.txt --no-cache-dir:
#34 3.899 Collecting pycryptodome>=3.4.7 (from -r requirements.txt (line 1))
#34 5.453   Downloading https://files.pythonhosted.org/packages/a6/e7/a9f558ff1ddef20469cf500e13493bcff1a968714a4881cc0460e6597ed6/pycryptodome-3.14.1-cp35-abi3-manylinux1_x86_64.whl (2.0MB)
#34 8.422 Collecting pysha3>=1.0.1 (from -r requirements.txt (line 2))
#34 8.864   Downloading https://files.pythonhosted.org/packages/32/2a/b93e0c6d90c7c45e2fab35d7ef349e8c5bd7387a048e961b041fd9521556/pysha3-1.0.2-cp36-cp36m-manylinux1_x86_64.whl (127kB)
#34 9.076 Collecting graphviz (from -r requirements.txt (line 3))
#34 9.674   Downloading https://files.pythonhosted.org/packages/9d/fb/886e8ec7862989afc0c35d15813b6c665fe134cc6027cdde2fa300abe9a2/graphviz-0.19.1-py3-none-any.whl (46kB)
#34 9.746 Collecting numpy (from -r requirements.txt (line 4))
#34 14.15   Downloading https://files.pythonhosted.org/packages/45/b2/6c7545bb7a38754d63048c7696804a0d947328125d81bf12beaa692c3ae3/numpy-1.19.5-cp36-cp36m-manylinux1_x86_64.whl (13.4MB)
#34 21.59 Collecting torch (from -r requirements.txt (line 5))
#34 22.90   Downloading https://files.pythonhosted.org/packages/a4/54/81b1c3c574a1ffde54b0c82ed2a37d81395709cdd5f50e59970aeed5d95e/torch-1.10.2-cp36-cp36m-manylinux1_x86_64.whl (881.9MB)
#34 176.5 Killed
------
executor failed running [/bin/sh -c pip3 install -r requirements.txt --no-cache-dir]: exit code: 137

Cannot start fuzzing

I have followed all instruments and have finished the environment configuration.
However, when I want to fuzz the example project, an error occurred, and I cannot fix it.
Maybe anyone could help me?

Environment: python3.5
Commend: python3 -m ilf --proj ./example/crowdsale/ --contract Crowdsale --fuzzer imitation --model ./model/ --limit 2000
Log:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/renardbebe/go/src/ilf/ilf/__main__.py", line 8, in <module>
    from .fuzzers import Environment
  File "/home/renardbebe/go/src/ilf/ilf/fuzzers/__init__.py", line 1, in <module>
    from .environment import Environment
  File "/home/renardbebe/go/src/ilf/ilf/fuzzers/environment.py", line 9, in <module>
    from .symbolic import PolicySymbolic
  File "/home/renardbebe/go/src/ilf/ilf/fuzzers/symbolic/__init__.py", line 1, in <module>
    from .policy_symbolic import PolicySymbolic
  File "/home/renardbebe/go/src/ilf/ilf/fuzzers/symbolic/policy_symbolic.py", line 12, in <module>
    from ...symbolic.symbolic import constraints
  File "/home/renardbebe/go/src/ilf/ilf/symbolic/symbolic/constraints.py", line 2, in <module>
    from ilf.symbolic.symbolic import svm_utils
  File "/home/renardbebe/go/src/ilf/ilf/symbolic/symbolic/svm_utils.py", line 2, in <module>
    import ethereum
  File "/home/renardbebe/go/src/ilf/ilf/lib/python3.5/site-packages/ethereum/__init__.py", line 11, in <module>
    from . import slogging  # noqa
  File "/home/renardbebe/go/src/ilf/ilf/lib/python3.5/site-packages/ethereum/slogging.py", line 6, in <module>
    from ethereum.utils import bcolors, is_numeric
  File "/home/renardbebe/go/src/ilf/ilf/lib/python3.5/site-packages/ethereum/utils.py", line 9, in <module>
    from py_ecc.secp256k1 import privtopub, ecdsa_raw_sign, ecdsa_raw_recover
  File "/home/renardbebe/go/src/ilf/ilf/lib/python3.5/site-packages/py_ecc/__init__.py", line 8, in <module>
    from py_ecc import bn128  # noqa: F401
  File "/home/renardbebe/go/src/ilf/ilf/lib/python3.5/site-packages/py_ecc/bn128/__init__.py", line 9, in <module>
    from .bn128_curve import (  # noqa: F401
  File "/home/renardbebe/go/src/ilf/ilf/lib/python3.5/site-packages/py_ecc/bn128/bn128_curve.py", line 13, in <module>
    from py_ecc.typing import (
  File "/home/renardbebe/go/src/ilf/ilf/lib/python3.5/site-packages/py_ecc/typing.py", line 70, in <module>
    GeneralPoint = Union[Point2D[Field], Point3D[Field]]
  File "/usr/lib/python3.5/typing.py", line 546, in __getitem__
    "Cannot subscript an existing Union. Use Union[u, t] instead.")
TypeError: Cannot subscript an existing Union. Use Union[u, t] instead.

I will be very appreciate if you can help me start ILF successfully! Thanks a lot!

False positive on detecting leaking contracts?

Hi, I have tested the following code using ILF.

As you can see, this Test contract does not have payable functions. Thus, the statement to.transfer(amount) cannot send positive amount of value(>0) to to (except for an edge case where another contract that has some Ethers is killed and send money to Test via selfdestruct instruction).

However, ILF says that this contract has leaking vulnerability. Could you explain why ILF flags this contract as vulnerable one?

contract Test{
  function test(address to, uint value) public {
    to.transfer(value);
  }
}

Error locked to thread -

Hi i have build your project manually in WSL. it works well on your example contract. Now I have add my simple contract into a example folder i.e. myproject (which is init/generated/compiled/migrated through truffle). I could also generate its transactions.js file. But when i run training data python3 -m ilf --proj ./example/myproject/ --contract MyContract --limit 2000 --fuzzer symbolic --dataset_dump_path ./train_data/MyContract.data, then i got this error.

/home/saarshah/.local/lib/python3.6/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=FutureWarning)
[2020-01-12 13:43:29,435][1578807809.435980][__main__][INFO] fuzzing start
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7f2ea3ca6769]

goroutine 17 [running, locked to thread]:
ilf/execution.convertAndSign(0xc42014d0e0, 0xc4201678f0, 0x900)
        /home/saarshah/go/src/ilf/execution/transaction.go:104 +0x259
ilf/execution.ReadTransactions(0xc42018cf60, 0x26, 0xc4201678f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/saarshah/go/src/ilf/execution/transaction.go:47 +0x381
ilf/execution.(*Backend).DeployContracts(0xc4200208c0, 0xc4200208c0, 0x0, 0x0)
        /home/saarshah/go/src/ilf/execution/backend.go:93 +0x171
ilf/execution.NewBackend(0xc4201438e0, 0x14, 0x14, 0x7f2ea3cbe17e, 0xc, 0x17)
        /home/saarshah/go/src/ilf/execution/backend.go:77 +0x314
main.SetBackend(0x7f2ea42f2248, 0x7f2ea4096288)
        /home/saarshah/go/src/ilf/export/execution.go:17 +0x4c
main._cgoexpwrap_962cb359037a_SetBackend(0x7f2ea42f2248, 0x0)
        _cgo_gotypes.go:71 +0x5c
Aborted (core dumped)

I just want to know to test/fuzz any new contract what procedures should i followed to get results ?

UPDATED:
here is my contract

pragma solidity ^0.5.0;

contract MyContract {
    
    uint256 public  totalSupply ; 
    mapping( address => uint256) public  balances ;
    address public owner;
    
    constructor() public payable {
        totalSupply = 6;
        owner = 0xD87d41Bbd765C488664725443260C0EE58E86941;
    }
    
    function () external payable{
        buyToken();
    }

    function buyToken() public payable {
        require(totalSupply >= (msg.value/1000000000000000000)*2);
        balances[msg.sender] += (msg.value/1000000000000000000)*2;
        totalSupply -=(msg.value/1000000000000000000)*2;

    }
    function getTotalSupply()public view returns  (uint256 ){
        // require(caller==owner)
        return totalSupply;
    }
       function setTotalSupply(uint256 newSupply) public {
        totalSupply = newSupply;
        
    }

}

I have followed this procedure

make a truffle project in example folder
copy paste above contract
truffle compile
truffle migrate
python3 script/extract.py --proj example/myproject/ --port 8545
python3 -m ilf --proj ./example/myproject/ --contract MyContract--limit 2000 --fuzzer symbolic --dataset_dump_path ./train_data/mycontract.data

How to use ILF on multiple contracts

Is it possible that we can use your tool on multiple contracts ? if yes, then for each contract , we must provide a truffle contract format ?
Another question:
I have download this Contract from etherscan.io and do all thing to make it as truffle contract . But when i ran script/extract.py , it show following deployment error

if i don't give constructor arguments in migration folder (02_deploy_contract.js) , it cause (and must show) deployment error as shown below:
"MarketPlace" -- Invalid number of parameters for "undefined". Got 0 expected 5!.
If i give just dummy arguments as per constructor definition , then it still can't be deployed and cause ( and also must show) deployment error as shown below;

Error:  *** Deployment Failed ***
"MarketPlace" hit a require or revert statement somewhere in its constructor. Try:
   * Verifying that your constructor params satisfy all require conditions.
   * Adding reason strings to your require statements.

So, keeping in view the above fact, my question is that
How could then we run any contract (s) automatically (as you claimed in your research paper), if we supposed to give all valid values as per contract logic ?

running ILF fuzzer in parallel

Hi, I am using ILF fuzzer using docker, following the build instruction in the readme.

If I would like to run multiple processes of ILF fuzzer in parallel, is it enough to simply execute multiple commands of ILF fuzzer? Or do I have to do some additional work (e.g., allocating different port number for each process)?

I guess that the ILF would perform fuzzing on the testnet. Then, if multiple processes are executed at the same time, the multiple processes may share the same port number for the testnet. I am curious if sharing the same port number among multiple processes of ILF fuzzer can affect results of ILF fuzzer.

Error when extracting deployment transactions of the contracts

When i try to extracting deployment transactions use follow code.
$ rm example/crowdsale/transactions.json # this line is good
$ python3 script/extract.py --proj example/crowdsale/ --port 8545 # this line cause the error

i run into follow error:
/home/cyma/.npm-global/lib/node_modules/truffle/build/4986.bundled.js:329
const isREPL = options?.isREPL ?? false; //default to not displaying REPL commands
^

SyntaxError: Unexpected token '.'
at wrapSafe (internal/modules/cjs/loader.js:915:16)
at Module._compile (internal/modules/cjs/loader.js:963:27)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
at Module.load (internal/modules/cjs/loader.js:863:32)
at Function.Module._load (internal/modules/cjs/loader.js:708:14)
at Module.require (internal/modules/cjs/loader.js:887:19)
at require (internal/modules/cjs/helpers.js:74:18)
at Object.webpack_require.f.require (/home/cyma/.npm-global/lib/node_modules/truffle/build/cli.bundled.js:608:28)
at /home/cyma/.npm-global/lib/node_modules/truffle/build/cli.bundled.js:538:40
at Array.reduce ()
/home/cyma/.npm-global/lib/node_modules/truffle/build/4986.bundled.js:329
const isREPL = options?.isREPL ?? false; //default to not displaying REPL commands
^

By the way, i can fuzz the example successfully.
$ python3 -m ilf --proj ./example/crowdsale/ --contract Crowdsale --fuzzer imitation --model ./model/ --limit 2000 #so, this line is good

error during docker build

Hi, I encountered an error during docker build (via the command docker build -t ilf .) at step 27/34:

Step 27/34 : RUN go get github.com/ethereum/go-ethereum
 ---> Running in 68d84bc112cd
# github.com/ethereum/go-ethereum/common/math
/go/src/github.com/ethereum/go-ethereum/common/math/integer.go:84:21: undefined: bits.Sub64
/go/src/github.com/ethereum/go-ethereum/common/math/integer.go:90:19: undefined: bits.Add64
/go/src/github.com/ethereum/go-ethereum/common/math/integer.go:96:12: undefined: bits.Mul64
The command '/bin/sh -c go get github.com/ethereum/go-ethereum' returned a non-zero code: 2

Could you help me to handle this error? I have tried on two different Ubuntu 18.04 machines.

How do you generate/get truffle projects from open-source contracts to generate train-data

Hi Jingxuan,

As introduced in README, ILF requires a complete truffle project to execute. Do you directly get the truffle projects from Etherscan or somewhere else, or you manually construct truffle projects from thousands of solidity code in a batch? Thanks for your patience.

Best,

Ruixin

How to train the ifl_w2v.pkl?

May I know how do you get the ifl_w2v.pkl? Do you directly use word2vec model on the training data?

Problem with docker build

As of today, during the pip install phase of docker build, it failed due to a package called coincurve, which is installed with the ethereum package. The problem is the newer version of coincurve. Simply add coincurve==12.0.0 before ethereum in the requirements.txt file will fix the problem.