ethereum / eth-abi Goto Github PK
View Code? Open in Web Editor NEWEthereum ABI utilities for python
License: MIT License
Ethereum ABI utilities for python
License: MIT License
In the following integration tests that test custom registrations, our method of testing is a bit stateful since it registers then unregisters coders for the custom data type. If one test fails, that could lead to a confusing failure in another test when the registry complains that a registration already exists for the custom type:
We could modify those tests to use a fixture registry instance that is created per test run. If we want to continue to do true integration testing via encode_single
and decode_single
, we will probably need to modify the API to support use of a custom registry instance.
installing / testing on python 3.8 give this error HypothesisWorks/hypothesis#1943
It is fixed in hypothesis, but not in a version supported by eth-abi per current requirements.
working properly with python 3.8
Allow hypothesis > 4 and see if it works...
>>> codec.is_encodable('bytes32', '')
True
This is surprising behavior.
dunno yet.
from eth_abi import is_encodable_type
is_encodable_type("uint25")
ABITypeError: For 'uint25' type at column 1 in 'uint25': integer size must be multiple of 8
This section may be deleted if the expectation is "don't crash".
False
Python version:
3.8.7 (default, Jan 24 2021, 11:57:42)
[GCC 9.3.0]
Operating System: Linux-5.6.0-1048-oem-x86_64-with-glibc2.29
pip freeze result:
alabaster==0.7.12
apipkg==1.5
appdirs==1.4.4
atomicwrites==1.4.0
attrs==20.3.0
Babel==2.9.0
backcall==0.2.0
bitarray==1.2.2
bleach==3.3.0
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
cryptography==3.4.6
cytoolz==0.11.0
dataclassy==0.7.2
decorator==4.4.2
distlib==0.3.1
docutils==0.16
eth-abi==2.1.1
eth-hash==0.3.1
eth-keyfile==0.5.1
eth-keys==0.3.3
eth-rlp==0.2.1
eth-typing==2.2.2
eth-utils==1.10.0
execnet==1.8.0
filelock==3.0.12
hexbytes==0.2.1
hypothesis==4.57.1
idna==2.10
imagesize==1.2.0
importlib-metadata==3.7.3
ipdb==0.13.7
ipython==7.21.0
ipython-genutils==0.2.0
jedi==0.18.0
jeepney==0.6.0
Jinja2==2.11.3
MarkupSafe==1.1.1
more-itertools==8.7.0
mypy-extensions==0.4.3
packaging==20.9
parsimonious==0.8.1
parso==0.8.1
pexpect==4.8.0
pickleshare==0.7.5
pluggy==0.13.1
prompt-toolkit==3.0.17
ptyprocess==0.7.0
py==1.10.0
pycparser==2.20
pycryptodome==3.10.1
Pygments==2.8.1
pyparsing==2.4.7
pytest==4.6.11
pytest-forked==1.3.0
pytest-xdist==1.34.0
pytz==2021.1
requests==2.25.1
rlp==2.0.1
SecretStorage==3.3.1
six==1.15.0
snowballstemmer==2.1.0
sortedcontainers==2.3.0
sphinxcontrib-serializinghtml==1.1.4
sphinxcontrib-websupport==1.2.4
toml==0.10.2
toolz==0.11.1
tox==2.9.1
traitlets==5.0.5
urllib3==1.26.4
virtualenv==20.4.3
watchdog==2.0.2
wcwidth==0.2.5
webencodings==0.5.1
zipp==3.4.1
Catch both NoEntriesFound
and ABITypeError
exceptions here:
Lines 460 to 463 in decaadc
Inspired by: ethereum/eth-account#57 (comment)
The is_encodable
function can raise an exception if the type is unknown. This is probably un-avoidable, but it does leave 3rd party libraries in an awkward position when they might be dealing with an unknown or invalid type string.
Not sure which of these is the right API but adding one or both of the following utilities.
True/False
(never raises exception)None
or raises eth_utils.ValidationError
Need to add support for Vyper decimal10
type. See here: ethereum/web3.py#700
Need to add support for it.
hey yall, question for you
in https://eth-abi.readthedocs.io/en/stable/decoding.html , i see
>>> decode_single('uint256', '0x0000000000000000000000000000000000000000000000000000000000003039')
12345
the first param is type, which is labeled uint256, but i see the 2nd param is a str
despite the first input being `uint256'
are the docs wrong?
for context, i am trying to use this lib to
ipdb> decode_single('uint256','0x1e688c1400000000000000000000000000000000000000000000000000000000000002b7000000000
000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000
0002e516d50413441646154745867423943596a394d5545505578745a4d5a6d334d7469656b676e444a695a793275557a00000000000000000
0000000000000000000' )
*** TypeError: The `data` value must be of bytes type. Got <class 'str'>
but i get a type error
pip install fails with the following output:
โ kinawardservice git:(master) โ sudo pip install ethereum-abi-utils --upgrade
The directory '/Users/amiblonder/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/amiblonder/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting ethereum-abi-utils
Downloading ethereum-abi-utils-0.4.5.tar.gz
Complete output from command python setup.py egg_info:
Installed /private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs/setuptools_markdown-0.2-py2.7.egg
Searching for pypandoc
Reading https://pypi.python.org/simple/pypandoc/
Best match: pypandoc 1.4
Downloading https://pypi.python.org/packages/71/81/00184643e5a10a456b4118fc12c96780823adb8ed974eb2289f29703b29b/pypandoc-1.4.tar.gz#md5=28d28cf8f1942abf680c040707cee55a
Processing pypandoc-1.4.tar.gz
Writing /tmp/easy_install-c6FpMb/pypandoc-1.4/setup.cfg
Running pypandoc-1.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-c6FpMb/pypandoc-1.4/egg-dist-tmp-Rubecs
Maybe try:
brew install pandoc
See http://johnmacfarlane.net/pandoc/installing.html
for installation options
---------------------------------------------------------------
zip_safe flag not set; analyzing archive contents...
pypandoc.__init__: module references __file__
!!! pandoc not found, long_description is bad, don't upload this to PyPI !!!
creating /private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs/pypandoc-1.4-py2.7.egg
Extracting pypandoc-1.4-py2.7.egg to /private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs
Installed /private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs/pypandoc-1.4-py2.7.egg
Searching for wheel>=0.25.0
Reading https://pypi.python.org/simple/wheel/
Best match: wheel 0.30.0
Downloading https://pypi.python.org/packages/fa/b4/f9886517624a4dcb81a1d766f68034344b7565db69f13d52697222daeb72/wheel-0.30.0.tar.gz#md5=e48f8f2329f1419572d93b68a63272a9
Processing wheel-0.30.0.tar.gz
Writing /tmp/easy_install-XliS_m/wheel-0.30.0/setup.cfg
Running wheel-0.30.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-XliS_m/wheel-0.30.0/egg-dist-tmp-5IAHAX
warning: no files found matching 'wheel/*.txt'
warning: no files found matching '*.py' under directory 'wheel/test'
warning: no files found matching 'wheel/test/test-1.0-py2.py3-none-win32.whl'
warning: no files found matching 'wheel/test/headers.dist/header.h'
warning: no files found matching 'wheel/test/pydist-schema.json'
no previously-included directories found matching 'wheel/test/*/dist'
no previously-included directories found matching 'wheel/test/*/build'
creating /private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs/wheel-0.30.0-py2.7.egg
Extracting wheel-0.30.0-py2.7.egg to /private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs
Installed /private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs/wheel-0.30.0-py2.7.egg
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-build-XVffAh/ethereum-abi-utils/setup.py", line 33, in <module>
'Programming Language :: Python :: 3.5',
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py", line 272, in __init__
_Distribution.__init__(self,attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 287, in __init__
self.finalize_options()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py", line 327, in finalize_options
ep.load()(self, ep.name, value)
File "/private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs/setuptools_markdown-0.2-py2.7.egg/setuptools_markdown.py", line 22, in long_description_markdown_filename
output = pypandoc.convert(markdown_filename, 'rst')
File "/private/tmp/pip-build-XVffAh/ethereum-abi-utils/.eggs/pypandoc-1.4-py2.7.egg/pypandoc/__init__.py", line 66, in convert
raise RuntimeError("Format missing, but need one (identified source as text as no "
RuntimeError: Format missing, but need one (identified source as text as no file with that name was found).
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-build-XVffAh/ethereum-abi-utils/
bytes32
arguments in ABIs are decoded as null-terminated unicode strings, although they often don't contain text data.
bytes32
should be represented as a 64-character hex string, instead.
http://news.nationalgeographic.com/content/dam/news/2016/07/19/slothlove/01_slothlove_leaves.jpg
Do as was done in ethereum/eth-typing#10
The way eth-abi
is structured, it cannot be extended without changes to the core library. This is problematic because new ABI types are likely to be created (as seen here).
We'll create a new ABIRegistry
class with the following functionality.
registry = ABIRegistry()
# register an encoder and a decoder for a static type.
registry.register("string", string_encoder, string_decoder)
# register only an encoder or decoder
registry.register_encoder("string", string_encoder)
registry.register_decoder("string", string_decoder)
# register a dynamic encoder/decoder
registry.register(lambda base, sub, arrlist: bool(arrlist), array_encoder, array_decoder)
# register an exact static type.
registry.register(("uint", "256", "[]", uint_encoder, uint_decoder)
# the main encode/decode API
encode_abi = registry.encode_abi
decode_abi = registry.decode_abi
encode_single = registry.encode_single
decode_single = registry.decode_single
Then, within the codebase, we make a default registry to expose all of the built-in encoders and decoders.
validation
The register
method should raise an exception if the same static type is registered twice.
determining which encoder/decoder to use
Internally, the ABIRegistry
will need to implement a dynamic version of the eth_abi.encoding.get_single_encoder
and eth_abi.decoding.get_single_decoder
. The logic for these functions should be as follows.
It appears that eth-abi considers a string with a fixed length to be a valid type:
https://github.com/ethereum/eth-abi/blob/master/eth_abi/utils/parsing.py#L57-L61
However, eth-abi handles fixed length strings no differently than dynamic-length strings:
https://github.com/ethereum/eth-abi/blob/master/eth_abi/encoding.py#L66-L67
https://github.com/ethereum/eth-abi/blob/master/eth_abi/decoding.py#L52-L53
Perhaps the type validation code should be updated at some point to throw an error for fixed length strings?
I get it that Solidity specifies string to be nothing but a dynamic sized array of bytes, but as a user of a python library, isn't it more intuitive to expect that I can pass a standard Python string to be encoded as a Solidity "string".
Currently this is the line #410 in eth_abi/encoding.py
class StringEncoder(BaseEncoder):
@classmethod
def encode(cls, value):
if not is_bytes(value):
raise EncodingTypeError(
"Value of type {0} cannot be encoded by StringEncoder".format(
type(value),
)
)
I would take a shot at the following:
if type(value) is not str:
raise EncodingTypeError(
"Value of type {0} cannot be encoded by StringEncoder".format(
type(value),
)
)
serialized_bytes = value.encode('utf-8')
#perform consequent operations on serialized_bytes
In any case I would like to understand more on the decision to accept only bytes, the challenges and considerations that were thought of in the first place.
We should add support for non-standard packed mode as described in the solidity ABI spec here: https://solidity.readthedocs.io/en/develop/abi-spec.html#non-standard-packed-mode
I imagine support could be added in the form of another encoding function which could be imported by doing from eth_abi import encode_packed
or something like it. There is no need for a corresponding decoding function since the format is ambiguous and not parsable.
The encode_packed
function would probably need to use a separate packed_registry
as well as separate packed coder classes which would be routed by that registry.
Version: 1.3.0
Python: 3.7
OS: osx/linux/win
result of pip freeze | grep eth
eth-abi==1.3.0
eth-account==0.3.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.0b3
eth-rlp==0.1.2
eth-typing==2.1.0
eth-utils==1.5.2
prometheus-client==0.3.1
pyethereum==1.0.0
Please include any of the following that are applicable:
import eth_abi
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python37\Lib\site-packages\eth_abi\__init__.py", line 3, in <module>
from eth_abi.abi import ( # NOQA
File "C:\Python37\Lib\site-packages\eth_abi\abi.py", line 4, in <module>
from eth_utils import (
File "C:\Python37\Lib\site-packages\eth_utils\__init__.py", line 5, in <module>
from .abi import ( # noqa: F401
File "C:\Python37\Lib\site-packages\eth_utils\abi.py", line 1, in <module>
from typing import Any, Dict
File "C:\Python37\Lib\site-packages\typing.py", line 1356, in <module>
class Callable(extra=collections_abc.Callable, metaclass=CallableMeta):
File "C:\Python37\Lib\site-packages\typing.py", line 1004, in __new__
self._abc_registry = extra._abc_registry
AttributeError: type object 'Callable' has no attribute '_abc_registry'
Fill this section in if you know how this could or should be fixed.
The CI build badge is showing red in the readme, because of this failure:
https://circleci.com/gh/ethereum/eth-abi/2086?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
cc @davesque
๐คทโโ๏ธ
When I was subclassing the eth-abi encoders in web3py, I noticed that in the Encoder inheritance tree, some of the validate_value
methods are class methods and some are instance methods. For example, BooleanEncoder
inherits from Fixed32ByteSizeEncoder
which inherits from FixedSizeEncoder
. BooleanEncoder
's validate_value
method is a class method and the FixedSizeEncoder
's validate_value
method is an instance method. Some of the other methods have the same problem.
I think it probably makes sense to make most of them class methods since they are not changing the internal state of the class, but haven't looked to make sure that's true for all of the methods/classes.
I'm not sure if anything is still using the ureal
and real
types but they seem pretty old and it would be nice to clear out some code.
Should delete code relating to those types.
I would expect that decoding an abi value of string
would return a str
-typed value.
In [1]: from eth_abi import decode_abi
In [2]: decode_abi(['string'], b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05mystr\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
Out[2]: (b'mystr',)
I expected ('mystr',)
Last step of decoding should be to call .decode('utf8')
on a bytes value if using a string decoder.
Calling an ERC20 balanceOf
will blow up if balance is 0.
_______________________________ test_historic_view_token_balance _______________________________
web3 = <web3.main.Web3 object at 0x7f2029a5b860>
address = '0xFBCe7c17608eBd5640313eCf4d2ff09B6726bAB9', normalizers = ()
function_identifier = 'balanceOf'
transaction = {'to': '0xFBCe7c17608eBd5640313eCf4d2ff09B6726bAB9'}, block_id = 5876634
contract_abi = [{'constant': True, 'inputs': [], 'name': 'totalSupply', 'outputs': [{'name': '', 'type': 'uint256'}], ...}, {'constan...s'}, {'name': 'guy', 'type': 'address'}], 'name': 'allowance', 'outputs': [{'name': '', 'type': 'uint256'}], ...}, ...]
fn_abi = {'constant': True, 'inputs': [{'name': 'src', 'type': 'address'}], 'name': 'balanceOf', 'outputs': [{'name': '', 'type': 'uint256'}], ...}
args = ('0xaAF3FFEE9d4C976aA8d0CB1bb84c3C90ee6E9118',), kwargs = {}
call_transaction = {'data': '0x70a08231000000000000000000000000aaf3ffee9d4c976aa8d0cb1bb84c3c90ee6e9118', 'to': '0xFBCe7c17608eBd5640313eCf4d2ff09B6726bAB9'}
return_data = HexBytes('0x'), output_types = ['uint256'], is_missing_code_error = False
msg = "Could not decode contract function call balanceOf return data b'' for output_types ['uint256']"
def call_contract_function(
web3,
address,
normalizers,
function_identifier,
transaction,
block_id=None,
contract_abi=None,
fn_abi=None,
*args,
**kwargs):
"""
Helper function for interacting with a contract function using the
`eth_call` API.
"""
call_transaction = prepare_transaction(
address,
web3,
fn_identifier=function_identifier,
contract_abi=contract_abi,
fn_abi=fn_abi,
transaction=transaction,
fn_args=args,
fn_kwargs=kwargs,
)
if block_id is None:
return_data = web3.eth.call(call_transaction)
else:
return_data = web3.eth.call(call_transaction, block_identifier=block_id)
if fn_abi is None:
fn_abi = find_matching_fn_abi(contract_abi, function_identifier, args, kwargs)
output_types = get_abi_output_types(fn_abi)
try:
> output_data = decode_abi(output_types, return_data)
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/web3/contract.py:1363:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
types = ['uint256'], data = HexBytes('0x')
def decode_abi(types, data):
if not is_bytes(data):
raise TypeError("The `data` value must be of bytes type. Got {0}".format(type(data)))
decoders = [
registry.get_decoder(type_str)
for type_str in types
]
decoder = TupleDecoder(decoders=decoders)
stream = ContextFramesBytesIO(data)
> return decoder(stream)
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/eth_abi/abi.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <eth_abi.decoding.TupleDecoder object at 0x7f2033b002b0>
stream = <eth_abi.decoding.ContextFramesBytesIO object at 0x7f2029cb2150>
def __call__(self, stream):
> return self.decode(stream)
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/eth_abi/decoding.py:118:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
args = (<eth_abi.decoding.TupleDecoder object at 0x7f2033b002b0>, <eth_abi.decoding.ContextFramesBytesIO object at 0x7f2029cb2150>)
kwargs = {}
@functools.wraps(fn)
def inner(*args, **kwargs):
> return callback(fn(*args, **kwargs))
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/eth_utils/functional.py:22:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <eth_abi.decoding.TupleDecoder object at 0x7f2033b002b0>
stream = <eth_abi.decoding.ContextFramesBytesIO object at 0x7f2029cb2150>
@to_tuple
def decode(self, stream):
for decoder in self.decoders:
> yield decoder(stream)
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/eth_abi/decoding.py:164:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <eth_abi.decoding.UnsignedIntegerDecoder object at 0x7f2029a73518>
stream = <eth_abi.decoding.ContextFramesBytesIO object at 0x7f2029cb2150>
def __call__(self, stream):
> return self.decode(stream)
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/eth_abi/decoding.py:118:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <eth_abi.decoding.UnsignedIntegerDecoder object at 0x7f2029a73518>
stream = <eth_abi.decoding.ContextFramesBytesIO object at 0x7f2029cb2150>
def decode(self, stream):
> raw_data = self.read_data_from_stream(stream)
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/eth_abi/decoding.py:186:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <eth_abi.decoding.UnsignedIntegerDecoder object at 0x7f2029a73518>
stream = <eth_abi.decoding.ContextFramesBytesIO object at 0x7f2029cb2150>
def read_data_from_stream(self, stream):
data = stream.read(self.data_byte_size)
if len(data) != self.data_byte_size:
raise InsufficientDataBytes(
"Tried to read {0} bytes. Only got {1} bytes".format(
self.data_byte_size,
> len(data),
)
)
E eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes. Only got 0 bytes
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/eth_abi/decoding.py:279: InsufficientDataBytes
The above exception was the direct cause of the following exception:
def test_historic_view_token_balance():
assert ETH_CHAIN == 'kovan', 'This test was designed for Kovan chain.'
address = '0xaAF3FFEE9d4C976aA8d0CB1bb84c3C90ee6E9118'
> assert view_token_balance(address, block_num=5876634) == 0
tests/core/test_eth.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/core/eth.py:101: in view_token_balance
return instance.functions.balanceOf(address).call(block_identifier=block_num)
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/web3/contract.py:1106: in call
**self.kwargs
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
web3 = <web3.main.Web3 object at 0x7f2029a5b860>
address = '0xFBCe7c17608eBd5640313eCf4d2ff09B6726bAB9', normalizers = ()
function_identifier = 'balanceOf'
transaction = {'to': '0xFBCe7c17608eBd5640313eCf4d2ff09B6726bAB9'}, block_id = 5876634
contract_abi = [{'constant': True, 'inputs': [], 'name': 'totalSupply', 'outputs': [{'name': '', 'type': 'uint256'}], ...}, {'constan...s'}, {'name': 'guy', 'type': 'address'}], 'name': 'allowance', 'outputs': [{'name': '', 'type': 'uint256'}], ...}, ...]
fn_abi = {'constant': True, 'inputs': [{'name': 'src', 'type': 'address'}], 'name': 'balanceOf', 'outputs': [{'name': '', 'type': 'uint256'}], ...}
args = ('0xaAF3FFEE9d4C976aA8d0CB1bb84c3C90ee6E9118',), kwargs = {}
call_transaction = {'data': '0x70a08231000000000000000000000000aaf3ffee9d4c976aa8d0cb1bb84c3c90ee6e9118', 'to': '0xFBCe7c17608eBd5640313eCf4d2ff09B6726bAB9'}
return_data = HexBytes('0x'), output_types = ['uint256'], is_missing_code_error = False
msg = "Could not decode contract function call balanceOf return data b'' for output_types ['uint256']"
def call_contract_function(
web3,
address,
normalizers,
function_identifier,
transaction,
block_id=None,
contract_abi=None,
fn_abi=None,
*args,
**kwargs):
"""
Helper function for interacting with a contract function using the
`eth_call` API.
"""
call_transaction = prepare_transaction(
address,
web3,
fn_identifier=function_identifier,
contract_abi=contract_abi,
fn_abi=fn_abi,
transaction=transaction,
fn_args=args,
fn_kwargs=kwargs,
)
if block_id is None:
return_data = web3.eth.call(call_transaction)
else:
return_data = web3.eth.call(call_transaction, block_identifier=block_id)
if fn_abi is None:
fn_abi = find_matching_fn_abi(contract_abi, function_identifier, args, kwargs)
output_types = get_abi_output_types(fn_abi)
try:
output_data = decode_abi(output_types, return_data)
except DecodingError as e:
# Provide a more helpful error message than the one provided by
# eth-abi-utils
is_missing_code_error = (
return_data in ACCEPTABLE_EMPTY_STRINGS and
web3.eth.getCode(address) in ACCEPTABLE_EMPTY_STRINGS
)
if is_missing_code_error:
msg = (
"Could not transact with/call contract function, is contract "
"deployed correctly and chain synced?"
)
else:
msg = (
"Could not decode contract function call {} return data {} for "
"output_types {}".format(
function_identifier,
return_data,
output_types
)
)
> raise BadFunctionCallOutput(msg) from e
E web3.exceptions.BadFunctionCallOutput: Could not decode contract function call balanceOf return data b'' for output_types ['uint256']
../../../.pyenv/versions/3.6.5/envs/general/lib/python3.6/site-packages/web3/contract.py:1385: BadFunctionCallOutput
If this is a bug report, please fill in the following sections.
If this is a feature request, delete and describe what you would like with examples.
The following code attempts to decode data using an ABI that is not quite correct. The correct ABI should be ['uint256', 'int128[7]', 'bytes', 'int128[3][3]', 'uint256', 'uint256']
. When executed, it results in an OverflowError
instead of a helpful exception message.
>>> data = HexBytes('0x000000000000000000000000000000000000000000000000000000000000000b00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000d00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000280000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005686f727365000000000000000000000000000000000000000000000000000000')
>>> decode_abi(['(uint256,int128[7],bytes,int128[3][3],uint256,uint256)'], data)
---------------------------------------------------------------------------
OverflowError Traceback (most recent call last)
<ipython-input-63-b16dfb7b0383> in <module>
----> 1 w3.codec.decode_abi(ot, data)
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/codec.py in decode_abi(self, types, data)
179 stream = self.stream_class(data)
180
--> 181 return decoder(stream)
182
183
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in __call__(self, stream)
125
126 def __call__(self, stream: ContextFramesBytesIO) -> Any:
--> 127 return self.decode(stream)
128
129
~/python-environments/web3/lib/python3.7/site-packages/eth_utils/functional.py in inner(*args, **kwargs)
43 @functools.wraps(fn)
44 def inner(*args, **kwargs) -> T: # type: ignore
---> 45 return callback(fn(*args, **kwargs))
46
47 return inner
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in decode(self, stream)
171 def decode(self, stream):
172 for decoder in self.decoders:
--> 173 yield decoder(stream)
174
175 @parse_tuple_type_str
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in __call__(self, stream)
125
126 def __call__(self, stream: ContextFramesBytesIO) -> Any:
--> 127 return self.decode(stream)
128
129
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in decode(self, stream)
143
144 stream.push_frame(start_pos)
--> 145 value = self.tail_decoder(stream)
146 stream.pop_frame()
147
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in __call__(self, stream)
125
126 def __call__(self, stream: ContextFramesBytesIO) -> Any:
--> 127 return self.decode(stream)
128
129
~/python-environments/web3/lib/python3.7/site-packages/eth_utils/functional.py in inner(*args, **kwargs)
43 @functools.wraps(fn)
44 def inner(*args, **kwargs) -> T: # type: ignore
---> 45 return callback(fn(*args, **kwargs))
46
47 return inner
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in decode(self, stream)
171 def decode(self, stream):
172 for decoder in self.decoders:
--> 173 yield decoder(stream)
174
175 @parse_tuple_type_str
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in __call__(self, stream)
125
126 def __call__(self, stream: ContextFramesBytesIO) -> Any:
--> 127 return self.decode(stream)
128
129
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in decode(self, stream)
142 start_pos = decode_uint_256(stream)
143
--> 144 stream.push_frame(start_pos)
145 value = self.tail_decoder(stream)
146 stream.pop_frame()
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in push_frame(self, offset)
93 self._total_offset += offset
94
---> 95 self.seek_in_frame(0)
96
97 def pop_frame(self):
~/python-environments/web3/lib/python3.7/site-packages/eth_abi/decoding.py in seek_in_frame(self, pos, *args, **kwargs)
82 Seeks relative to the total offset of the current contextual frames.
83 """
---> 84 self.seek(self._total_offset + pos, *args, **kwargs)
85
86 def push_frame(self, offset):
OverflowError: Python int too large to convert to C ssize_t
When we are about to push a new frame onto the decoding stream we should first sanity check the value, and if the value is beyond the end of the stream, we should raise a more intelligent error message.
What is the reason for padding with 32 zeros here for an empty bytes array?
Line 535 in decaadc
I think it's different from what happens here, namely no padding
https://github.com/ethereumjs/ethereumjs-abi/blob/ee3994657fa7a427238e6ba92a84d0b529bbcde0/lib/index.js#L146
I know the latter is deprecated but it seems this is how it is done by the opensea.io frontend.
Are both correct but according to different standards/versions?
Thank you.
The eth-abi decoder models the calldata of a transaction as a finite length datastream.
However, the evm models calldata as an infinite length bytestream.
Returning zero bytes when the transaction does not specify the bytes requested.
The tests in this repository do seem to indicate that the decoder is intentionally modelled this way.
I'm not sure why this is the case. It'd be cool if there is an additional option to model the bytestream as infinite.
I have a local branch where I adapted the ContextFramesBytesIO
object to mimic the behaviour of calldata in the evm.
I'm happy to submit a pr if this is something that could be added to the repository.
Coder classes currently use programmatic subclassing via the built-in type
function as an intermediate step toward creating a callable which performs the actual encoding/decoding of values:
https://github.com/ethereum/eth-abi/blob/master/eth_abi/encoding.py#L62
https://github.com/ethereum/eth-abi/blob/master/eth_abi/decoding.py#L41
Would it be simpler to just pass in coder settings as keyword arguments to an __init__
method on coder classes? So something like this:
class BaseEncoder(BaseCoder):
def __init__(self, **kwargs):
cls = type(self)
for key, value in kwargs.items():
if not hasattr(cls, key):
raise AttributeError(
'Property {key} not found on {cls_name} class. '
'`{cls_name}.__init__` only accepts keyword arguments which are '
'present on the {cls_name} class.'.format(
key=repr(key),
cls_name=cls.__name__,
)
)
setattr(self, key, value)
self.validate()
...
If there's a particular reason that this approach was chosen, then just ignore me ๐ .
The encode_abi
and decode_abi
functions (found here and here) do not really serve a useful purpose any more. For example, the following invocations are equivalent:
In [1]: from eth_abi import *
In [2]: encode_single('(int,int)', (1, 2))
Out[2]: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02'
In [3]: encode_abi(('int', 'int'), (1, 2))
Out[3]: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02'
Perhaps we should remove encode_abi
and decode_abi
and rename encode_single
and decode_single
to encode
and decode
respectively?
This could also match Solidity's own API better: https://solidity.readthedocs.io/en/v0.4.24/units-and-global-variables.html?highlight=abi#abi-encoding-functions
The initial version of eth-abi was designed to accomodate basic types (types of the form <base><sub><arrlist>
such as int256[2]
, fixed128x18
etc.
Later, support was added for tuples by extending eth-abi to handle type strings such as (int,int)
, (bool,(fixed,int))[]
etc.
However, the problem with this approach is that most interaction with the ABI encoding format is done via information obtained from a JSON ABI specification for a contract. The JSON ABI represents tuple types entirely differently like the following:
{
"type": "tuple",
...
"components": [
{"type": "int", ...}, {"type": "int", ...}
]
}
More information about the JSON ABI format can be found here: https://solidity.readthedocs.io/en/latest/abi-spec.html#json
Because of this, a number of utility functions were added to convert JSON ABI type information to type string representations. But this seems like a potentially unnecessary step. Ideally, we shouldn't have to use type strings as a kind of intermediate type format.
One possible fix for this could be to convert or extend eth-abi to detect appropriate coders for a type based on a JSON ABI type description (instead of a type string). So a type registry could receive a dict object representing a JSON ABI type and produce the correct coder.
I'm not sure how hard this would be or if there are any problems with that approach that I'm not considering. But I wanted to make a note of it here in case we choose to take a look at it later.
If the type
is given as capital letter, then the code is not working (some grammar
error). Please see the following code.
from eth_abi.grammar import parse
obj = parse("person[2]")
obj1 = parse("Person[2]")
---------------------------------------------------------------------------
ParseError Traceback (most recent call last)
~/ethereum/eth-account/venv/lib/python3.6/site-packages/eth_abi/grammar.py in parse(self, type_str)
122 try:
--> 123 return super().parse(type_str)
124 except parsimonious.ParseError as e:
~/ethereum/eth-account/venv/lib/python3.6/site-packages/parsimonious/nodes.py in parse(self, text, pos)
253 """
--> 254 return self._parse_or_match(text, pos, 'parse')
255
~/ethereum/eth-account/venv/lib/python3.6/site-packages/parsimonious/nodes.py in _parse_or_match(self, text, pos, method_name)
288 method=method_name))
--> 289 return self.visit(getattr(self.grammar, method_name)(text, pos=pos))
290
~/ethereum/eth-account/venv/lib/python3.6/site-packages/parsimonious/grammar.py in parse(self, text, pos)
114 self._check_default_rule()
--> 115 return self.default_rule.parse(text, pos=pos)
116
~/ethereum/eth-account/venv/lib/python3.6/site-packages/parsimonious/expressions.py in parse(self, text, pos)
119 """
--> 120 node = self.match(text, pos=pos)
121 if node.end < len(text):
~/ethereum/eth-account/venv/lib/python3.6/site-packages/parsimonious/expressions.py in match(self, text, pos)
136 if node is None:
--> 137 raise error
138 return node
ParseError: Rule 'type' didn't match at 'Person[]' (line 1, column 1).
During handling of the above exception, another exception occurred:
ParseError Traceback (most recent call last)
<ipython-input-6-ed74617bc5cf> in <module>
----> 1 obj = parse("Person[]")
~/ethereum/eth-account/venv/lib/python3.6/site-packages/eth_abi/grammar.py in parse(self, type_str)
123 return super().parse(type_str)
124 except parsimonious.ParseError as e:
--> 125 raise ParseError(e.text, e.pos, e.expr)
126
127
Fill this section in if you know how this could or should be fixed.
Currently, the https://github.com/gitcoinco/web repository Travis builds are failing due to a dependency conflict with other eth-
packages.
Explanation: gitcoinco/web#367
Validation is failing due to a conflict in dependencies:
eth-utils==0.7.*
This seems to cause the build to fail dependency validation.
eth-abi
to use >=1.0.0b1
of the eth-utils
package?It appears eth-tester
is resulting in the same error due to pinning <1.0.0
eth-abi
fails to properly parse/use the ABI generated by solc
for a library. One example of this is that for contracts the type
of an enum is reported as an integer type (uint8
if enum has 256 of less entries) while for libraries it's enum L.E
(where L
is a library name and E
is some enum).
An attempt to use such an enum
with eth-abi
results in an error:
ParseError: Parse error at '.E' (column 3) in type string '(L.E)'
Repro using Brownie.
Assuming you have it installed, just run these commands in shell in an empty directory:
brownie init
cat << EOF > contracts/L.sol
pragma solidity =0.7.1;
library L {
enum BOOL {NO, YES}
function f(BOOL b) public pure returns (BOOL) {
return b;
}
}
EOF
cat << EOF > scripts/trigger_bug.py
from brownie import L, accounts
l = L.deploy({'from': accounts[0]})
l.f(1)
EOF
brownie compile
brownie run trigger_bug
Brownie v1.11.10 - Python development framework for Ethereum
FProject is the active project.
Launching 'ganache-cli --accounts 10 --hardfork istanbul --gasLimit 12000000 --mnemonic brownie --port 8545'...
Transaction sent: 0x5723c43998c3eaad6eabeb2e840843db80b028a4e024a7ea7fe96f6ec0dad640
Gas price: 0.0 gwei Gas limit: 12000000
L.constructor confirmed - Block: 1 Gas used: 90940 (0.76%)
L deployed at: 0x3194cBDC3dbcd3E11a07892e7bA5c3394048Cc87
File "brownie/_cli/run.py", line 49, in main
return_value = run(args["<filename>"], method_name=args["<function>"] or "main")
File "brownie/project/scripts.py", line 52, in run
module = _import_from_path(script)
File "brownie/project/scripts.py", line 110, in _import_from_path
_import_cache[import_str] = importlib.import_module(import_str)
File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File "./scripts/trigger_bug.py", line 3, in <module>
l.f(1)
File "brownie/network/contract.py", line 1232, in __call__
return self.call(*args, block_identifier=block_identifier)
File "brownie/network/contract.py", line 1033, in call
tx.update({"to": self._address, "data": self.encode_input(*args)})
File "brownie/network/contract.py", line 1113, in encode_input
data = format_input(self.abi, args)
File "brownie/convert/normalize.py", line 16, in format_input
abi_types = _get_abi_types(abi["inputs"])
File "brownie/convert/normalize.py", line 112, in _get_abi_types
tuple_type = parse(type_str)
File "eth_abi/grammar.py", line 125, in parse
raise ParseError(e.text, e.pos, e.expr)
ParseError: Parse error at '.BOOL' (column 3) in type string '(L.BOOL)'
Terminating local RPC client...
Python version:
3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0]
Operating System: Linux-5.9.1-arch1-1-x86_64-with-glibc2.2.5
pip freeze result:
aiohttp==3.6.2
aiohttp-socks==0.4.2
aiorpcX==0.18.4
alabaster==0.7.12
anki==2.1.26
ankirspy==2.1.26
ansible==2.10.1
ansible-base==2.10.2
appdirs==1.4.4
aqt==2.1.26
argcomplete==1.11.1
argh==0.26.2
asn1crypto==1.4.0
async-timeout==3.0.1
attrs==20.2.0
Babel==2.8.0
backcall==0.2.0
bcrypt==3.2.0
beautifulsoup4==4.9.3
bitstring==3.1.7
btrfsutil==1.2.0
CacheControl==0.12.6
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==7.1.2
colorama==0.4.4
contextlib2==0.6.0.post1
cryptography==3.1.1
decorator==4.4.2
discid==1.2.0
distlib==0.3.1
distro==1.5.0
dnspython==2.0.0
docutils==0.16
ecdsa==0.16.0
Electrum==4.0.3
file-magic==0.4.0
filelock==3.0.12
fuse-python==1.0.0
git-cola==3.7
grpcio==1.34.0.dev0
gunicorn==20.0.4
html5lib==1.1
idna==2.10
imagesize==1.2.0
importlib-metadata==2.0.0
iotop==0.6
ipdb==0.13.4
ipython==7.18.1
ipython-genutils==0.1.0
isc==2.0
jedi==0.17.2
Jinja2==2.11.2
jmespath==0.10.0
jsonrpclib-pelix==0.4.1
jsonschema==3.2.0
lensfun==0.3.95
libmsym==0.2.4
louis==3.15.0
lxml==4.6.1
Markdown==3.3
MarkupSafe==1.1.1
meld==3.20.2
mps-youtube==0.2.8
msgpack==1.0.0
multidict==4.7.6
mutagen==1.45.1
namcap==3.2.10
numpy==1.19.2
opensnitch-ui==1.0.1
ordered-set==4.0.2
packaging==20.4
pafy==0.5.5
paramiko==2.7.2
parso==0.7.1
pathtools==0.1.2
pbkdf2==1.3
pep517==0.8.2
pexpect==4.8.0
picard==2.5
pickleshare==0.7.5
Pillow==7.2.0
pipenv==2020.8.13
pipx==0.15.6.0
ply==3.11
progress==1.5
prompt-toolkit==3.0.8
protobuf==3.12.4
ptyprocess==0.6.0
pwquality==1.4.4
pyaes==1.6.1
pyalpm==0.9.1
PyAudio==0.2.11
pycairo==1.20.0
pycparser==2.20
pycryptodomex==3.9.7
pyelftools==0.26
pyenchant==3.1.1
Pygments==2.7.2
PyGObject==3.38.0
pyinotify==0.9.6
PyNaCl==1.4.0
pyparsing==2.4.7
PyQt5==5.15.1
PyQt5-sip==12.8.1
PyQtWebEngine==5.15.1
pyrsistent==0.17.3
PySocks==1.7.1
pyspread==1.99.4
python-dateutil==2.8.1
python-slugify==4.0.1
pytz==2020.1
pyxattr==0.7.1
PyYAML==5.3.1
qrcode==6.1
ranger-fm==1.9.3
rednotebook==2.20
requests==2.24.0
resolvelib==0.4.0
retrying==1.3.3
s3cmd==2.1.0
Send2Trash==1.5.0
six==1.15.0
snowballstemmer==2.0.0
soupsieve==2.0.1
Sphinx==3.2.1
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
stevedore==3.2.2
team==1.0
text-unidecode==1.3
toml==0.10.1
traitlets==4.3.3
typing-extensions==3.7.4.3
urllib3==1.25.10
userpath==1.4.1
virtualenv==20.0.32
virtualenv-clone==0.5.4
virtualenvwrapper==4.8.4
watchdog==0.10.3
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==0.57.0
yarl==1.5.1
youtube-dl==2020.9.20
zipp==3.3.2
The inconsistent ABI for libraries is actually a problem in the compiler and is likely to be fixed in the near future (feedback welcome in ethereum/solidity#9278). It affects other tools too: dethcrypto/TypeChain#216, ethers-io/ethers.js#1126.
In the meantime (and for older versions when it's fixed), tools need to handle it differently than in contracts.
Though there is clear value in using property testing for certain kinds of tests (such as end-to-end encoding tests), I've recently had some doubts about some other ways we're using property testing.
For example:
https://github.com/ethereum/eth-abi/blob/master/tests/encoding/test_encoder_properties.py#L327-L395
To a large extent, that property test and others like it seem to be testing second implementations of functionality against the original implementations. This doesn't provide much value since the original implementation could have a flaw which wouldn't be revealed by testing it against a second implementation that uses the same logic.
It seems like we should replace these kinds of property tests with traditional tests that check for correct behavior given specific inputs.
We should add type hints to the code as well as a mypy
run to the CI environment.
Copy implementation from eth-keys
or py-evm
repo.
The encode_single
and decode_single
functions currently accept either a type string or a type tuple to identify the type that is being worked with. It seems preferable that a type string is all that should be accepted since type tuples are simple converted to type strings internally and an additional type check is performed on the input which could become costly if it ends up in a critical path.
We should consider changing encode_single
and decode_single
to only accept type strings and also remove the collapse_type
utility function. Since this function is extremely simple in nature, it can be manually copied to and used in whatever contexts it is still needed (such as in web3.py code).
decode_abi(types, data) fails with ValueError("Wrong data size for string/bytes object") when called with the payload from
Tx 0x070d2cf5c6ee0285e78b50aef397585865b41c2b56a8051370eaedac7d95d54c
To reproduce:
types = ['uint256', 'bytes']
data = '0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000'
decode_abi(types, data)
I request a release of 1.2.3 as #110 has been merged and it is needed for fixing ethereum/py-evm#1484
@davesque can you help?
It would be handy to be able to leverage this package to create type hints for other packages that describe restrictions in abi types for different purposes, e.g. https://github.com/ApeWorX/eip712 where we currently use is_encodeable_type
to parse a deferred type when describing an EIP-712 structured message like so:
class Order(EIP712Message):
a: "address" # implicit validation using `eth_abi.is_encodeable_type`
It would be much nicer if all of the registered types were made available in mypy-compliant form to use for type checking code as follows:
from eth_abi.types import address
class Order(EIP712Message):
a: address # explicit validation provided by eth_abi
Ideally, these types would work as aliases for int
, Decimal
, str
, bytes
, etc., doing the proper size and bounds checks as well (or at least allowing us to perform that validation easily) to make it easier to integrate this work into other libraries.
The process_type
utility function (found here) is left over from the previous style of parsing type strings. The parse
function in the grammar
module (found here) provides the same functionality and is more extensible.
We should remove the process_type
function in favor of the grammar.parse
function.
Here's the canonical way (per Solidity, at least) to encode (bytes[]("0xf8a8fd6d", "0xf8a8fd6d"))
:
0000000000000000000000000000000000000000000000000000000000000020
0000000000000000000000000000000000000000000000000000000000000002
0000000000000000000000000000000000000000000000000000000000000040
0000000000000000000000000000000000000000000000000000000000000080
0000000000000000000000000000000000000000000000000000000000000004
f8a8fd6d00000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000004
f8a8fd6d00000000000000000000000000000000000000000000000000000000
Broken down:
This is most easily thought of as a recursive encoding: The tail data for the first (and in this case, only) element of the ABI is the entire ABI encoding of that element.
In contrast, this library presently produces the current output:
0000000000000000000000000000000000000000000000000000000000000020
0000000000000000000000000000000000000000000000000000000000000002
0000000000000000000000000000000000000000000000000000000000000004
f8a8fd6d00000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000004
f8a8fd6d00000000000000000000000000000000000000000000000000000000
The head element, and the length of the array are correct, but then array elements are packed tightly, without the correct head/tail recursive encoding.
I tried to parse input of transaction and it gave me an error. Tx hash is 0xfc70289a7d33238c1d45fa62b7cd1f150e8951264025f782f87c5ad09b5838c9.
Source of this contract is publicly known, and both Etherscan and 4bytes show the same signature for that method_id (0x35adc0c5).
However, when I try to decode that particular transaction with that ABI (bytes16,address,address,uint256,uint16
), the eth-abi gives me an error. I believe that this is due to the fact that the first argument is bytes16 while the input looks like bytes32. Moreover, replacing last 16 bytes with zeroes solves the problem.
from eth_abi import decode_abi
seller_cancel_abi = ["bytes16","address","address","uint256","uint16"]
input_data = '0x35adc0c53039616566623164303162643464343438333039393266386631323132303631000000000000000000000000e144447b7cfc65a1f859a50df6e134548b5a084b00000000000000000000000014965de4057658ca30c4bc9d0411dd9d97b1392e000000000000000000000000000000000000000000000000712d5ff393d900000000000000000000000000000000000000000000000000000000000000000064'
input_method_data = input_data[10:]
input_data_bytes = bytes.fromhex(input_method_data)
decode_abi(seller_cancel_abi, input_data_bytes) # fails
fixed_input_data = '0x35adc0c53039616566623164303162643464343400000000000000000000000000000000000000000000000000000000e144447b7cfc65a1f859a50df6e134548b5a084b00000000000000000000000014965de4057658ca30c4bc9d0411dd9d97b1392e000000000000000000000000000000000000000000000000712d5ff393d900000000000000000000000000000000000000000000000000000000000000000064'
fixed_input_method_data = fixed_input_data[10:]
fixed_input_data_bytes = bytes.fromhex(fixed_input_method_data)
decode_abi(seller_cancel_abi, fixed_input_data_bytes) # works fine
NonEmptyPaddingBytes Traceback (most recent call last)
<ipython-input-7-657262f49cdb> in <module>()
5 input_method_data = input_data[10:]
6 input_data_bytes = bytes.fromhex(input_method_data)
----> 7 decode_abi(seller_cancel_abi, input_data_bytes)
~/Files/research/env/lib/python3.7/site-packages/eth_abi/codec.py in decode_abi(self, types, data)
163 stream = ContextFramesBytesIO(data)
164
--> 165 return decoder(stream)
166
167
~/Files/research/env/lib/python3.7/site-packages/eth_abi/decoding.py in __call__(self, stream)
125
126 def __call__(self, stream: ContextFramesBytesIO) -> Any:
--> 127 return self.decode(stream)
128
129
~/Files/research/env/lib/python3.7/site-packages/eth_utils/functional.py in inner(*args, **kwargs)
44 @functools.wraps(fn)
45 def inner(*args, **kwargs) -> T: # type: ignore
---> 46 return callback(fn(*args, **kwargs))
47
48 return inner
~/Files/research/env/lib/python3.7/site-packages/eth_abi/decoding.py in decode(self, stream)
171 def decode(self, stream):
172 for decoder in self.decoders:
--> 173 yield decoder(stream)
174
175 @parse_tuple_type_str
~/Files/research/env/lib/python3.7/site-packages/eth_abi/decoding.py in __call__(self, stream)
125
126 def __call__(self, stream: ContextFramesBytesIO) -> Any:
--> 127 return self.decode(stream)
128
129
~/Files/research/env/lib/python3.7/site-packages/eth_abi/decoding.py in decode(self, stream)
196 data, padding_bytes = self.split_data_and_padding(raw_data)
197 value = self.decoder_fn(data)
--> 198 self.validate_padding_bytes(value, padding_bytes)
199
200 return value
~/Files/research/env/lib/python3.7/site-packages/eth_abi/decoding.py in validate_padding_bytes(self, value, padding_bytes)
328 if padding_bytes != b'\x00' * padding_size:
329 raise NonEmptyPaddingBytes(
--> 330 "Padding bytes were not empty: {0}".format(repr(padding_bytes))
331 )
332
**NonEmptyPaddingBytes: Padding bytes were not empty: b'830992f8f1212061'**
Here's another transaction from the same contract that can't be parsed:
0x035b1e59b573ab7f4ec05f235c01b8ac0304c995e542d52ad321b4e2ea796315
Here's another transaction from similar contract that can't be parsed:
0x7977c27c9312ef4d98f5e42ff0e2774157af4fc3664e5211589721cacd4fbce0
All other transactions from these two contracts work fine.
Seems like eth-abi should accept that input even though it seems malformed.
There is no documentation in the README.md
Add documentation, including examples to the following methods. The format used in eth-utils
should be sufficient.
eth_abi.decode_single
eth_abi.decode_abi
eth_abi.encode_single
eth_abi.encode_abi
For arrays, encodePacked appears to fall back to standard encoding.
Line 666 in decaadc
Lines 615 to 631 in decaadc
However, solidity skips the length and pads dynamic array elements with zeros:
https://docs.soliditylang.org/en/v0.8.6/abi-spec.html#non-standard-packed-mode
The encoding of an array is the concatenation of the encoding of its elements with padding.
cf. ethereum/solidity#8441 for an example of the current padding scheme.
Additionally, and please correct me if I'm wrong here, eth-abi appears to encode bytestrings in tuples without padding as in the tests here
Lines 166 to 171 in decaadc
However my cursory reading of solc output says that they are in fact padded. (solidity disallows structs in abi.encodePacked, but it uses the packed encoding code path to calculate event indexes).
event.ir.txt
event.sol.txt
see:
Solidity has an experimental feature that will allow tuple types.
snipped below is from the linked solidity docs
pragma solidity ^0.4.19;
pragma experimental ABIEncoderV2;
contract Test {
struct S { uint a; uint[] b; T[] c; }
struct T { uint x; uint y; }
function f(S s, T t, uint a) public { }
function g() public returns (S s, T t, uint a) {}
}
The code above would result in the following ABI
[
{
"name": "f",
"type": "function",
"inputs": [
{
"name": "s",
"type": "tuple",
"components": [
{
"name": "a",
"type": "uint256"
},
{
"name": "b",
"type": "uint256[]"
},
{
"name": "c",
"type": "tuple[]",
"components": [
{
"name": "x",
"type": "uint256"
},
{
"name": "y",
"type": "uint256"
}
]
}
]
},
{
"name": "t",
"type": "tuple",
"components": [
{
"name": "x",
"type": "uint256"
},
{
"name": "y",
"type": "uint256"
}
]
},
{
"name": "a",
"type": "uint256"
}
],
"outputs": []
}
]
We should add a new TupleEncoder
and TupleDecoder
to handle these types.
TupleEncoder
should accept an iterable of values as it's input.TupleDecoder
should return a python tuple
of values as output.The README.md
should be updated to include at minimum, information about the accepted input and output formats for tuple ABI types.
Test cases should include declarative test cases for the following.
Property based test(s) should be created for:
encode -> decode
and decode -> encode
As mentioned in PR #118 , the eth_abi.grammar
module is starting to be used in web3.py. This means it's beginning to look more like public API.
We should write a section in the docs dedicated to use of the parsing and grammar facilities and add docstrings and maybe type annotations where needed.
Note: This issue ticket doesn't follow the usual format because I couldn't think of a way to fit it into that.
In the solidity ABI spec, it appears that zero-length tuple types are a valid type:
(T1,T2,...,Tn): tuple consisting of the types T1, โฆ, Tn, n >= 0
If that was intentional, then should it be assumed that this is some kind of unit type (e.g. like None
in Python or ()
in Haskell)?
Additionally, if it's correct to see zero-length tuples as unit types, how should we support that? I suppose an encoder for the unit type could just encode b'\x00'
in every case and just ignore whatever Python value it is given to encode. A decoder could also just return None
by default in every case for the decoded value but still consume and ignore 256 bits of data.
Solidity added the fixed
type back in version 0.3 ref
This library uses a vendored version prior to the patches that changed from real to fixed.
Update the vendored code
Cannot decode the first event of this transaction using decode_abi
but can decode using decode_single
: https://etherscan.io/tx/0xe331b086561bad5554503115728123d93b8cb64f89b71d94e90876127c773379#eventlog
Please include any of the following that are applicable:
>>> from eth_abi import decode_abi, decode_single
>>> from hexbytes import HexBytes
>>> variables = '0x000000000000000000000000f232cbab76d7e90911cd2573f1d8afe70d0c9f9b0000000000000000000000006ccc8faebd1dedd14831e80435c82930bec0db770000000000000000000000000000000000000000000000d3d6b463e32cc9400000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000'
>>> b_variables = HexBytes(variables)
>>> types = ['address', 'address', 'uint256', 'bytes']
decode_abi
>>> decode_abi(types, b_variables)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/abi.py", line 96, in decode_abi
return decoder(stream)
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
return self.decode(stream)
File "/home/ankit/venv/lib/python3.6/site-packages/eth_utils/functional.py", line 46, in inner
return callback(fn(*args, **kwargs))
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 164, in decode
yield decoder(stream)
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
return self.decode(stream)
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 136, in decode
value = self.tail_decoder(stream)
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
return self.decode(stream)
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 186, in decode
raw_data = self.read_data_from_stream(stream)
File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 583, in read_data_from_stream
data = stream.read(padded_length)
OverflowError: Python int too large to convert to C ssize_t
decode_single
>>> decode_single('address', b_variables[:32])
'0xf232cbab76d7e90911cd2573f1d8afe70d0c9f9b'
>>> decode_single('address', b_variables[32:64])
'0x6ccc8faebd1dedd14831e80435c82930bec0db77'
>>> decode_single('uint256', b_variables[64:96])
3907734100000000000000
>>> decode_single('bytes', b_variables[96:])
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
Don't know
>>> eth_abi.encode_single("address", b"a"*20)
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00aaaaaaaaaaaaaaaaaaaa'
>>> eth_abi.encode_single("address", b"a"*21)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/xuanji/gits/cf/plasma-cash/venv/lib/python3.7/site-packages/eth_abi/abi.py", line 43, in encode_single
return encoder(arg)
File "/Users/xuanji/gits/cf/plasma-cash/venv/lib/python3.7/site-packages/eth_abi/encoding.py", line 73, in __call__
return self.encode(value)
File "/Users/xuanji/gits/cf/plasma-cash/venv/lib/python3.7/site-packages/eth_abi/encoding.py", line 178, in encode
self.validate_value(value)
File "/Users/xuanji/gits/cf/plasma-cash/venv/lib/python3.7/site-packages/eth_abi/encoding.py", line 456, in validate_value
cls.__name__,
eth_abi.exceptions.EncodingTypeError: Value of type <class 'bytes'> cannot be encoded by AddressEncoder
contrary to what the error message says, bytes
can be encoded by AddressEncoder
I have a package which uses eth_abi
, and which lints its code with mypy
. Telling mypy
to ignore the eth_abi
import (import ... # type: ignore
) obviously works fine, but when I remove that ignore
, I get the following message:
test.py:13: error: Cannot find module named 'eth_abi'
test.py:13: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)
The line in question is simply:
from eth_abi import encode_abi
Per PEP 484, include a file py.typed
in the package. In my local environment, I was able to solve the problem by simply doing touch ~/.pythonz/lib/python3.7/site-packages/eth_abi-1.2.0-py3.7.egg/eth_abi/py.typed
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.