tintinweb / smart-contract-sanctuary

🐦🌴🌴🌴🦕 A home for ethereum smart contracts. 🏠

^{Smart Contract Sanctuary - MultiRepo / Index
⚠️UPDATE: Repo layout changed! see #13 (v1-layout)}

Smart Contract Sanctuary

🐦🌴🌴🌴🦕 A home for ethereum smart contracts verified on Etherscan. 🏠

^{⇝ This is the index repository for the smart-contract-sanctuary. 🔖 Bookmark this repo.
⇝ Chain-specific sub-repos and the index are updated twice a day.
⇝ Expect a full, recursive check-out to take 2GB+ disk space.}

Usage

The repo is configured for use with git+ssh (much more stable and faster).

First time - clone the index and checkout all chain-specific sub repositories from scratch:

⇒  git clone --recursive --depth=1 [email protected]:tintinweb/smart-contract-sanctuary.git

_{also see https://git-scm.com/docs/git-submodule for more options}

Existing repository but submodules never initialized - checkout submodules and update all chain-specific sub repositories:

⇒  git submodule update --init --remote --depth=1 --progress

Existing repository with submodules - update all chain-specific sub repositories:

⇒  git submodule update --remote --progress

Layout

Folder	Description
_docs	autogenerated stats; static github page
<chain>/contracts	Chain specific smart contracts
↳ ethereum/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-ethereum
↳ arbitrum/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-arbitrum
↳ avalanche/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-avalanche
↳ bsc/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-bsc
↳ fantom/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-fantom
↳ polygon/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-polygon
↳ tron/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-tron
↳ optimism/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-optimism
↳ celo/contracts	Git SubModule 👉 https://github.com/tintinweb/smart-contract-sanctuary-celo
<chain>/utils	Chain specific support scripts

📂 <chain>/contracts

Contains smart contract sources for various networks, grouped by the first two chars of the contract address. Files are named in the format <address>_<source_unit_name>, e.g. 0f0c3fedb6226cd5a18826ce23bec92d18336a98_URToken.sol

Some contracts are listed in contracts.json, but the file-system may contain more files than what is listed in this summary. Rely on the folder/file structure for a full list. This repo used to auto submit contracts to 4byte.directory.

📂 <chain>/utils

Support scripts for various activies like dumping smart contracts from public sources (etherscan.io, etherchain.com)

requires: pip install -r requirements.txt

Dev Notes

To use List of Verified Contract addresses with an OpenSource license, you can download the csv file, add it to the util folder, and run parse_download_contracts_etherscan_io.py (with your etherscan API). This will add the new contracts to the appropriate folder

👩‍🔬 Data Science Tools

• 🧠 SolGrep - A scriptable semantic grep utility for solidity (crunch numbers, find specific contracts, extract data)
• 🐞 semgrep - Semgrep is a fast, open-source, static analysis tool for finding bugs and enforcing code standards at editor, commit, and CI time, and now supports Solidity! ❤️ Thanks @JoranHonig for contributing the semgrep solidity parser.
• 🌟 Sourcegraph - A powerful online code search service that can be used to search the sanctuary without cloning.

🎓 Citation

If you are using this dataset in your research and paper, here's how you can cite this dataset:

• APA6

Ortner, M., Eskandari, S. (n.d.). Smart Contract Sanctuary. Retrieved from https://github.com/tintinweb/smart-contract-sanctuary.

• LateX (Bib)

 @article{smart_contract_sanctuary, 
          title={Smart Contract Sanctuary}, 
          url={https://github.com/tintinweb/smart-contract-sanctuary}, 
          author={Ortner, Martin and Eskandari, Shayan}} 

smart-contract-sanctuary's Issues

I'd like to contribute

celo can't get compiler infomation and library file

1、compiler just display "Solidity"
2、maybe get library files is good

files terminated with null characters

As this is not valid input to solc, I suspect this has something to do with how files are stored in sanctuary. Example:

• https://github.com/tintinweb/smart-contract-sanctuary-ethereum/blob/1e69d37c1aa7b2c6dd7285f756bddaf6c129040d/contracts/mainnet/04/04e0e2dd8fdbca1d584927cdbdeb7a1314dff1f4_Bloktrade.sol

Thanks for this incredibly helpful project!

illegal unicode character in `[email protected]`

It prevents the contracts to be compiled by solc, since this character is illegal. I wonder if a regular space can be used here instead of U+00a0?

Thanks for this incredibly helpful project!

BSC contract data doesn't update for two month

`contracts.json` containing conflict markers

Example: https://github.com/tintinweb/smart-contract-sanctuary-ethereum/blob/015d0105102504dc8733a18c3543f87f1829a5e8/contracts/goerli/contracts.json

{"address": "0x5Beb560fF733930ecc9981cF42dA71b10a386f98", "name": "SimpleWallet", "compiler": "0.8.7", "balance": "0 Ether", "txcount": 0, "settings": "", "date": "1/25/2022"}
<<<<<<< HEAD
{"address": "0x90f3E1105E63C877bF9587DE5388C23Cdb702c6B", "name": "EntryPoint", "compiler": "0.8.7", "balance": "0.00896224999273575 Ether", "txcount": 0, "settings": "", "date": "1/25/2022"}
=======
{"address": "0x90f3E1105E63C877bF9587DE5388C23Cdb702c6B", "name": "EntryPoint", "compiler": "0.8.7", "balance": "0.018639257982839423 Ether", "txcount": 0, "settings": "", "date": "1/25/2022"}
>>>>>>> 961d3ebb069ebce88fd2c3d88aab9a5a3ddee53b
{"address": "0x9ebC47b397D50591C9e2A0b4Ee6a1343aBB92289", "name": "HelloWorld", "compiler": "0.8.0", "balance": "-", "txcount": 0, "settings": "", "date": "1/25/2022"}

Dump issue in contracts with some specific characters

I used download_contracts_etherscan_io.py script to dump mainnet solidity contracts.

In the dumped contract codes, I have noticed that single quotation character has been written in its HTML character encoding '. So, those contracts could not be parsed using Solidity AST Parser.

Please note that there are other rare characters that have been converted to HTML character code. The issue is not limited to single quotation character.

The script that I used to fix the contracts:

import sys
import os
#________________________________________________________________________________________
def main(argv):
    
    path = ''
    if len(argv) == 0 :
        print("usage: arg1=path")
    else:
        path = argv[0] # root path of solidity repo
    for f in list_files(path,'sol'):
        preprocess_contract(f)
#________________________________________________________________________________________
def list_files(directory, extension): # returns list of files with extension, flattened
    flist = []
    for (dirpath, dirnames, filenames) in os.walk(directory):
        k = (os.path.join(dirpath,f) for f in filenames if f.endswith('.' + extension))
        for f in k:
            flist.append(f)

    print(str(len(flist)) + " files found in " + directory + " with ." + extension + " extension")
    return flist
#________________________________________________________________________________________
def preprocess_contract(contract_path):
    with open(contract_path, "r") as fin:
        data = fin.read()
    fin.close()

    path2 = contract_path.replace("/mainnet/", "/mainnet3/")
    head, tail = os.path.split(path2)
    os.makedirs(head, exist_ok=True)

    data = data.replace(''',"'")
    
    with open(path2, "w") as fin:
        fin.write(data)
#________________________________________________________________________________________
if __name__ == '__main__':
    main(sys.argv[1:])

I just replaced "mainnet" with another foldername ("mainnet3" in this case) to keep inner directories.

The interval for BSC network could be reduced to capture more verified contracts

The twice-daily timed crawl does miss a lot of contracts deployed on BSC mainnet, because there are just too many contracts verified daily on it.

I record the contract list from the last page https://bscscan.com/contractsVerified/20 just one hour before the upcoming commit. Those contracts in list were quickly flushed out as more newly deployed contracts verifed.

 0x2c7a65b9a9837bbbfb9fb3db23d7e60d16dbde53	MethInu	Solidity	 0.7.4	0.020147424739582457 BNB	36	-	7/4/2022	-	MIT
 0x35Ad6219Cda49A02eDdd78eeCa102D8EAC97347b	MoneyBars	Solidity(Json)	 0.5.16	0 BNB	4		7/4/2022	-	-
 0xe8BD594027BD7575b645c67258Fe6e733D821072	Address	Solidity	 0.8.4	0 BNB	1		7/4/2022	-	MIT
 0xbb237c8b4d78a599e72e26c5bec9df298bf748b6	GROWSHIB	Solidity	 0.8.10	0 BNB	5	-	7/4/2022	-	MIT
 0x257b0337843fdc2fac4f22c18e2131ad5f2d400a	HappyCat	Solidity	0.8.15	0 BNB	10		7/4/2022	-	None
 0x14093Ca6B1d38635448E44F36Ea304C465616858	MelodyShiba	Solidity	 0.8.7	4 wei	44		7/4/2022	-	None
 0x7e66e061d38089b0bccf1c3bd875cb0cb3034e8c	PCSGetAround	Solidity(Multi)	0.8.15	0 BNB	2		7/4/2022	-	MIT
 0xdcbb7f2a3bbcdb8b85e666caccd1e76398cfb9d0	TokenMintERC20Token	Solidity	 0.5.0	0 BNB	7		7/4/2022	-	None
 0x137cbbf71cb5ec4983cc390e35b3452d6df8cb0f	ShirojiInuGirl	Solidity	 0.8.6	0 BNB	6		7/4/2022	-	MIT
 0xf37e9b9d7e636e3b5dd333119f0d915659b9bc0e	TOKENNAME	Solidity	 0.8.7	0 BNB	5		7/4/2022	-	MIT
 0x66a8d2105b7091ac4226e5ed867b237188379b47	HoneyLock	Solidity	 0.8.0	0 BNB	15	 	7/3/2022	-	MIT
 0x98E9f3062a3Ed9Af49b7D47Cd6519077586B8Dc4	TiFiZilla	Solidity	 0.8.7	0.000001835904402423 BNB	124	-	7/3/2022	-	Unlicense
 0x14f9333ca302903a3b274ab51cd80a3d0896e538	OmegaGrow	Solidity	 0.8.11	0 BNB	5		7/3/2022	-	MIT
 0xf39ca8083f6c9cec97aa47a0f87e517bcdce0f5a	BUSDOGE	Solidity	 0.7.6	0 BNB	6		7/3/2022	-	MIT
 0x6f02e419f78cfd7b075c7f06c34869555d77baff	RickRollInu	Solidity	0.8.15	0 BNB	11		7/3/2022	-	None
 0x9cbdb045ce6718c69f9c4a763dae88e543e8f73d	MONEYBANK	Solidity	 0.6.12	0 BNB	4		7/3/2022	-	MIT
 0x647A99368A5910Fd4740B0b97380FEE03b7Df2AF	BearMRKT	Solidity(Json)	 0.5.16	0 BNB	4		7/3/2022	-	-
 0x9dd1c5c6bb0c4e9ae62c2c7139bc56a2b0666d80	HappyCat	Solidity	 0.7.6	0.011452457101671165 BNB	54		7/3/2022	-	Unlicense
 0xFb1d94093864cCcCD033b7fEa1Ce929aCcDEeDF3	Coin	Solidity	 0.8.6	0 BNB	88	-	7/3/2022	-	MIT
 0x132F756506681BB8Decff1d54D14a40B79E0dBDc	FuckIncels	Solidity(Json)	 0.8.4	0 BNB	1		7/3/2022	-	-
 0xfbBf56D7BceC734Fb8e5809A4fAF1384aB6c07cD	ShadowX	Solidity	 0.8.4	0 BNB	7		7/3/2022	-	None
 0xbf6778EB6C17e98e6a67541ea861D53ea16BE762	Token	Solidity	 0.8.2	0 BNB	4	-	7/3/2022	-	None
 0xd8ed00f6cfa9b06032d7827fdf538cbf50bc7be2	ShibFloki2	Solidity	0.8.15	0 BNB	13		7/3/2022	-	None
 0x5cbc5cc05d89618d0242f2e0507b14f465007d2f	CloudGrow	Solidity	 0.8.0	0 BNB	7		7/3/2022	-	None
 0xe861ad8a51291392b8A95b3138c4c5DF1441dE98	BasicWizard	Solidity(Json)	 0.8.9	0 BNB	1	 	7/3/2022	-	MIT

None of the above are included in this repo because of loose timing. Not sure if bscscan allows slightly higher rates?

There are some Vyper contracts with .sol extension

Hey,

I recently parsed the entire sanctuary, and I was getting some random parsing errors. Turned out to be that there are some vyper files with .sol extension. For example: fantom/contracts/mainnet/ca/cabc8fC306fCAa4c05B58522B13756AE12eDd902_Vyper_contract.sol

I don't have the entire list at hand, and reconstructing it may take me ~10h, but I suspect all of them have Vyper in their name.

cleanup: make tree-sitter-solidity happy by fixing html encoded apostrophes in code

In many early contracts, we missed decoding HTML entities which resulted in invalid solidity code. As a result, parsers could not parse the file, and compilers failed to compile the solidity code.

In an attempt to fix this, we deploy the least invasive fix by manually replacing ' with '. This should make a lot of sources parseable, while some HTML entities might still not be properly decoded (only affects "early contracts").

Affected files: >25k source units (sorry, GitHub 🥳 massive PR's incoming )
Affected repository: Ethereum, BSC; other chains were not affected as we already used fixed scripts. Most files are probably around solidity 0.4 and 0.5'ish.

The following PRs attempt to fix this problem. tree-sitter-solidity parser is super happy about this change ;)

• ethereum: mainnet, rinkeby, ropsten (other networks were not affected) 👉 tintinweb/smart-contract-sanctuary-ethereum#1
• bsc: mainnet, testnet 👉 tintinweb/smart-contract-sanctuary-bsc#1

Add file list or getter script

Please make it somehow possible to access file list.
Either have an up to date csv, json (git merging works great when lines randomly spread out)

or a scraper python script to get matching file for contract address.

LiteLauncher?

I'm trying to figure out how a Balancer smartpool is deployed and I can see that the first txn is a call createLiquidityBootstrapPool and the only results on Github is in your santuary repo.

I'm trying to figure out what that is, do you have any idea?

Sorry for the issue, no other way to contact you, and thanks for your time?!

Some contracts are unparsable due to web-scraping the HTML

Found when attempting to parse the mainnet corpus.

1. symbol = "<span class="__cf_email__" data-cfemail="64292427">[email&#160;protected]</span>";
2. string public standard = 'Token 0.1';

Both are parse errors, seem to be related to #18 and #19.

Files missing for multifile submissions

the script only parsed the first contract from etherscan. in case the verified contract was a multi-file submission other contracts were not downloaded.

Incomplete collection of smart contracts

I think this project should be able to collect all the verified smart contract source code on explorers such as etherscan, but it does not.

E.g:
https://bscscan.com/address/0xa41bf81be90fe9666cd566a80c85871f41529aed#code
https://bscscan.com/address/0xd4C83BA0782f431C5640D7400D7Cb8c7c393dbe1#code

contracts.json add internal transactions

Hello, thank you for this great dataset.

I am looking to find which contracts are important when I am doing a review. A simple proxy is to look up ether value and number of transactions (both are in the contracts.json). Additionally it would be nice to see number of "internal transactions" (message calls) received by each contract.

Add quick note on README about approx repo size

Please provide a quick, round number on the README file saying how many gigabytes are necessary to clone this repo.

The git client aborts and you have to start over if your hard disk did not have enough space. So having an idea of how much room you need available can help you prepare.

I get an Internal JSON-RPC error when trying to deploy my BabyCAKE Fork

The error message redirects me to this function:
function excludeFromFees(address account, bool excluded) public onlyOwner {
require(_isExcludedFromFees[account] != excluded, "BABYCAKE: Account is already the value of 'excluded'");
_isExcludedFromFees[account] = excluded;

    emit ExcludeFromFees(account, excluded);

The Error messaging reads specifically:
"creation of MonkeyDDragon errored: Internal JSON-RPC error.
{
"code": 3,
"message": "execution reverted: MonkeyDDragon: Account is already the value of 'excluded'",
"data": "0x08c379a0000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000394d6f6e6b657944447261676f6e3a204163636f756e7420697320616c7265616479207468652076616c7565206f6620276578636c756465642751919291839183"
}

Please what do I do for the contract to be deployed successfully?

cleanup: html email protection produces invalid source code

also see: ethereum/contracts/mainnet/94/94Aa5Ed767b6E05353703398362BB7a4D25C73dc_CVAlejandro.sol

⚠️ Repo layout will change soon as this repository is becoming too large for github 🙄

Main Repo: https://github.com/tintinweb/smart-contract-sanctuary

And one sub-repo per chain (e.g.): https://github.com/tintinweb/smart-contract-sanctuary-tron

All repositories are going to be linked to the main repo as git submodules

How to check out?

git clone --recursive https://github.com/tintinweb/smart-contract-sanctuary.git