consensys / python-solidity-parser

An experimental Solidity parser for Python built on top of a robust ANTLR4 grammar 📚

python-solidity-parser

An experimental Solidity parser for Python built on top of a robust ANTLR4 grammar.

ⓘ This is a python3 port of the javascript antlr parser maintained by @federicobond. Interfaces are intentionally following the javascript implementation and are therefore not pep8 compliant.

Install

#> pip3 install solidity_parser
#> python3 -m solidity_parser <parse|outline> <path_to_contract.sol>   # print parse tree or sourceUnit outline

HowTo

import sys
import pprint

from solidity_parser import parser

sourceUnit = parser.parse_file(sys.argv[1], loc=False) # loc=True -> add location information to ast nodes
pprint.pprint(sourceUnit)  
# see output below

output:

{'type': 'SourceUnit',
 'children': [{'type': 'PragmaDirective',
               'name': 'solidity',
               'value': '^0.4.22'},
              {'type': 'ContractDefinition'},
               'baseContracts': [],
               'kind': 'contract',
               'name': 'SimpleAuction',
               'subNodes': [{'initialValue': None,
                             'type': 'StateVariableDeclaration',
                             'variables': [{'expression': None,
                                            'isDeclaredConst': False,
                                            'isIndexed': False,
                                            'isStateVar': True,
                                            'name': 'beneficiary',
                                            'type': 'VariableDeclaration',
                                            'typeName': {'name': 'address',
                                                         'type': 'ElementaryTypeName'},
                                            'visibility': 'public'}]},
...

Nodes

Parse-tree nodes can be accessed both like dictionaries or via object attributes. Nodes always carry a type field to hint the type of AST node. The start node is always of type sourceUnit.

Accessing AST items in an Object Oriented fashion

# ... continuing from previous snippet

# subparse into objects for nicer interfaces:
# create a nested object structure from AST
sourceUnitObject = parser.objectify(sourceUnit)

# access imports, contracts, functions, ...  (see outline example in __main__.py)
sourceUnitObject.imports  # []
sourceUnitObject.pragmas  # []
sourceUnitObject.contracts.keys()  # get all contract names
sourceUnitObject.contracts["contractName"].functions.keys()  # get all functions in contract: "contractName"
sourceUnitObject.contracts["contractName"].functions["myFunction"].visibility  # get "myFunction"s visibility (or stateMutability)

Generate the parser

Update the grammar in ./solidity-antlr4/Solidity.g4 and run the antlr generator script to create the parser classes in solidity_parser/solidity_antlr4.

#> bash script/antlr4.sh

python-solidity-parser's Issues

Request for guidance on how to access for function body

I am using the Solidity Parser developed by you. I am grateful for making it public.
How I can access the function body. Can you please guide me on how I can access AST from the function body.

Read the file using utf8

The parser fails reading solidity files. We found that opening the files with utf8 solves it.

def parse_file(path, start="sourceUnit", loc=False, strict=False):
with open(path, 'r', encoding="utf8") as f:

    return parse(f.read(), start=start, loc=loc, strict=strict)

Unlock.sol.txt

For example in the Unlock.sol file.

How

Missing spaces in PragmaDirective

Contracts such as the Ballot contract shown in the official documentation use pragma versions that follow the same syntax as npm:

pragma solidity >=0.7.0 <0.9.0;

According to the official documentation, this is allowed and the parser seems to support this. However, when parsing contracts like this, the value associated with the PragmaDirective doesn't have any spaces:

>>> sourceUnit['children'][0]
{'type': 'PragmaDirective', 'name': 'solidity', 'value': '>=0.7.0<0.9.0'}

Is there any reason for omitting the spaces? Unfortunately, this precludes the use of other modules that already parse version strings that follow the npm specification. This suggests that it would be better to keep the original format.

Does not it support contract inheritance and 'using' grammar?

I want to test my contract file, but it does not work if contract has a 'using' syntax or if contract inherit a contract.

stateMutability and Visibility can't invoke

I can not invoke the statemutability and visibility of individual functions in a contract. Could you tell me how?

parser fails on payable(address(variable))

The parser fails on this line: payable(address(variable)), for some variable.

address payable newLock = payable(address(proxy));
FAILS

address payable newLock = address(proxy);
PASS

About 1/6 of mainnet contracts can not be parsed using this parser.

I used smart contract sanctuary to dump all mainnet solidity contracts (32382 contracts at the moment of dumping without removing duplicates).

Then I iterated over it and used your parser. This spreadsheet is list of all contracts that could not be parsed. (detected using exception handler). About 5K of contracts could not be parsed.

Hope it will be helpful.

solidity_parser_mainnet_32382_errors.zip

The parser doesn't support receive / fallback functions

Because there is no 'function' prefix the parser fails whenever there is a fallback / recieve exists in the code. For example,

fallback() external payable {}

FAILS

while function fallback() external payable {}

PASS

Thanks

[a:b] indexing not supported

bytes32 txNamehash = bytes32(data[i][4:36]);

This line through the exception "Exception: unrecognized expression"

Can you please handle it? Thanks

Feature Request: Get object's raw source

Hi,

How are you? Thanks of all, thanks for doing the port of the tool, I was thinking of doing it myself and I found someone else did it 😄

I was wondering if there was a way to get the raw source of an objects, let's say a contract or function.

I was thinking in something like:

sourceUnitObject = parser.objectify(sourceUnit)
raw_text = sourceUnitObject.contracts['MyContract'].raw
print(raw_text)
>>'contract MyContract { ..'

Thanks!

How to walk the AST produced by parse or parse_file?

I just don't know the meaning that the second param named "callback_object" in function "visit" of parser.py in this module. How can I leverage this function to walk the AST produced by parse or parse_file? Or maybe other more useful methods to apply to this.

Parser can't handle the following line

(,,,,,,uint8 ranking) = Avastar(targetAsset()).getPrimeByTokenId(_tokenId);

It will be great if you can solve it :)
Thanks very much for your great job !!!

Parser can't handle call

Hi :)
I'm using your parser and turns that it can't handle .call expressions.

For example this line: (bool success, ) = savingsAccount.call{value: received}('');

Please fix it if possible :)
Thanks!

Python-solidity-parser failed to handle throw keyword

pragma solidity ^0.4.13;
contract someContract {
mapping(address => uint) balances;

  function deposit() payable {
        balances[msg.sender] += msg.value; 
  } 

function withdrawOkayish(uint amount) {
if(balances[msg.sender] >= amount) {
balances[msg.sender] -= amount; if(!msg.sender.send(amount)) { throw; }
}
}
function withdrawBad2(uint amount) {
if(balances[msg.sender] >= amount) {
balances[msg.sender] -= amount;
if(!msg.sender.call.gas(2500000).value(amount)()) { throw; }
}
}

}

It produces the following output

{'children': [{'name': 'solidity',
'type': 'PragmaDirective',
'value': '^0.4.13'},
{'baseContracts': [],
'kind': 'contract',
'name': 'someContract',
'subNodes': [{'initialValue': None,
'type': 'StateVariableDeclaration',
'variables': [{'expression': None,
'isDeclaredConst': False,
'isIndexed': False,
'isStateVar': True,
'name': 'balances',
'type': 'VariableDeclaration',
'typeName': {'keyType': {'name': 'address',
'type': 'ElementaryTypeName'},
'type': 'Mapping',
'valueType': {'name': 'uint',
'type': 'ElementaryTypeName'}},
'visibility': 'default'}]},
{'body': {'statements': [{'expression': {'left': {'base': {'name': 'balances',
'type': 'Identifier'},
'index': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'sender',
'type': 'MemberAccess'},
'type': 'IndexAccess'},
'operator': '+=',
'right': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'value',
'type': 'MemberAccess'},
'type': 'BinaryOperation'},
'type': 'ExpressionStatement'}],
'type': 'Block'},
'isConstructor': False,
'modifiers': [],
'name': 'deposit',
'parameters': {'parameters': [],
'type': 'ParameterList'},
'returnParameters': [],
'stateMutability': 'payable',
'type': 'FunctionDefinition',
'visibility': 'default'},
{'body': {'statements': [{'FalseBody': None,
'TrueBody': {'statements': [{'expression': {'left': {'base': {'name': 'balances',
'type': 'Identifier'},
'index': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'sender',
'type': 'MemberAccess'},
'type': 'IndexAccess'},
'operator': '-=',
'right': {'name': 'amount',
'type': 'Identifier'},
'type': 'BinaryOperation'},
'type': 'ExpressionStatement'},
{'FalseBody': None,
'TrueBody': {'statements': [';'],
'type': 'Block'},
'condition': {'isPrefix': True,
'operator': '!',
'subExpression': {'arguments': [{'name': 'amount',
'type': 'Identifier'}],
'expression': {'expression': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'sender',
'type': 'MemberAccess'},
'memberName': 'send',
'type': 'MemberAccess'},
'names': [],
'type': 'FunctionCall'},
'type': 'UnaryOperation'},
'type': 'IfStatement'}],
'type': 'Block'},
'condition': {'left': {'base': {'name': 'balances',
'type': 'Identifier'},
'index': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'sender',
'type': 'MemberAccess'},
'type': 'IndexAccess'},
'operator': '>=',
'right': {'name': 'amount',
'type': 'Identifier'},
'type': 'BinaryOperation'},
'type': 'IfStatement'}],
'type': 'Block'},
'isConstructor': False,
'modifiers': [],
'name': 'withdrawOkayish',
'parameters': {'parameters': [{'isIndexed': False,
'isStateVar': False,
'name': 'amount',
'storageLocation': None,
'type': 'Parameter',
'typeName': {'name': 'uint',
'type': 'ElementaryTypeName'}}],
'type': 'ParameterList'},
'returnParameters': [],
'stateMutability': None,
'type': 'FunctionDefinition',
'visibility': 'default'},
{'body': {'statements': [{'FalseBody': None,
'TrueBody': {'statements': [{'expression': {'left': {'base': {'name': 'balances',
'type': 'Identifier'},
'index': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'sender',
'type': 'MemberAccess'},
'type': 'IndexAccess'},
'operator': '-=',
'right': {'name': 'amount',
'type': 'Identifier'},
'type': 'BinaryOperation'},
'type': 'ExpressionStatement'},
{'FalseBody': None,
'TrueBody': {'statements': [';'],
'type': 'Block'},
'condition': {'isPrefix': True,
'operator': '!',
'subExpression': {'arguments': [],
'expression': {'arguments': [{'name': 'amount',
'type': 'Identifier'}],
'expression': {'expression': {'arguments': [{'number': '2500000',
'subdenomination': None,
'type': 'NumberLiteral'}],
'expression': {'expression': {'expression': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'sender',
'type': 'MemberAccess'},
'memberName': 'call',
'type': 'MemberAccess'},
'memberName': 'gas',
'type': 'MemberAccess'},
'names': [],
'type': 'FunctionCall'},
'memberName': 'value',
'type': 'MemberAccess'},
'names': [],
'type': 'FunctionCall'},
'names': [],
'type': 'FunctionCall'},
'type': 'UnaryOperation'},
'type': 'IfStatement'}],
'type': 'Block'},
'condition': {'left': {'base': {'name': 'balances',
'type': 'Identifier'},
'index': {'expression': {'name': 'msg',
'type': 'Identifier'},
'memberName': 'sender',
'type': 'MemberAccess'},
'type': 'IndexAccess'},
'operator': '>=',
'right': {'name': 'amount',
'type': 'Identifier'},
'type': 'BinaryOperation'},
'type': 'IfStatement'}],
'type': 'Block'},
'isConstructor': False,
'modifiers': [],
'name': 'withdrawBad2',
'parameters': {'parameters': [{'isIndexed': False,
'isStateVar': False,
'name': 'amount',
'storageLocation': None,
'type': 'Parameter',
'typeName': {'name': 'uint',
'type': 'ElementaryTypeName'}}],
'type': 'ParameterList'},
'returnParameters': [],
'stateMutability': None,
'type': 'FunctionDefinition',
'visibility': 'default'}],
'type': 'ContractDefinition'}],
'type': 'SourceUnit'}

I have found that it does not handle the throw keyword. Both if case block contains only ';'. I am not able to find a way to resolve this issue. Is there any way to resolve this issue.

Parsing the acquired object is not serializable

This code does not execute and cannot serialize the variable kk2, making it impossible to transfer this data between multiple python processes

from solidity_parser import parse_file,objectify
import pickle

if __name__ == "__main__":
    print("Hello World!")
    kk = parse_file(path="samples/simple.sol")
    kk2 = objectify(kk)
    k3 = pickle.dumps(kk2)
    print(k3)

Question: how to add line number to the parser AST?

Hi, I think this should be a common question.
How can I add the line number to the parser output?
Thanks!