apeworx / py-solc-x

Python wrapper and version management tool for the solc Solidity compiler.

py-solc-x

Python wrapper and version management tool for the solc Solidity compiler.

Forked from py-solc.

Features

• Full support for Solidity >=0.4.11
• Install Solidity on Linux, OSX and Windows
• Compile Solidity from source on Linux and OSX

Dependencies

Py-solc-x allows the use of multiple versions of solc, and can install or compile them as needed. If you wish to compile from source you must first insall the required solc dependencies.

Installation

via pip

pip install py-solc-x

via setuptools

git clone https://github.com/iamdefinitelyahuman/py-solc-x.git
cd py-solc-x
python3 setup.py install

Documentation

Documentation is hosted at Read the Docs.

Testing

Py-solc-x is tested on Linux, OSX and Windows with solc versions >=0.4.11.

To run the test suite:

pytest tests/

By default, the test suite installs all available solc versions for your OS. If you only wish to test against already installed versions, include the --no-install flag.

Contributing

Help is always appreciated! Feel free to open an issue if you find a problem, or a pull request if you've solved an issue.

Please check out our Contribution Guide prior to opening a pull request, and join the Brownie Gitter channel if you have any questions.

License

This project is licensed under the MIT license.

py-solc-x's Issues

Add PEP484 Annotations

The Issue

The solcx codebase doesn't make use of PEP484 annotations. Adding them will make the code more readable and help prevent future bugs from being introduced.

Running MyPy with the following configuration should complete without errors.

[mypy]
disallow_untyped_defs = True
ignore_missing_imports = True
follow_imports = silent
files = solc/

Change location of solc binaries

Instead of keeping solc binaries at solcx/data, they could be located at Path.home().joinpath('.solcx'). This way it won't be required to reinstall solc every time a new virtual environment is created.

0.7.4 can't install on windows

Environment information

• py-solc-x Version: x.x.x
• solc Version: 0.6.12
• Python Version: 3.8.6
• OS: win

What was wrong?

0.7.4 cannot install. If downloaded, it will break saying "file is not a zip file." If pointing it directly at the exe file, it still shows the same error.

solcx.install_solc(version="latest", show_progress=True, solcx_binary_path="C:\solc_install\file.exe")

"raise BadZipFile("File is not a zip file")
BadZipFile: File is not a zip file"

How can it be fixed?

please make it install.

Source mappings have no match on the AST

Environment information

• py-solc-x Version: 0.8.2
• solc Version: 0.6.8+commit.0bbfe453.Linux.g++
• Python Version: 3.6.9
• OS: Ubuntu 18.04.4 LTS

What was wrong?

I compiled the Fomo3D contract using the 0.4.24+commit.e67f0147.Linux.g++ solc version. Afterwards, while observing the trace of the following transaction, I noticed that some source mappings of the bytecode that py-solc-x outputs, have no match on the AST that py-solc-x outputs.

Below are the source mappings that have no match:

Could not find ast node from source mapping: 49693 : 37 : 0
Could not find ast node from source mapping: 49693 : 65 : 0
Could not find ast node from source mapping: 49693 : 37 : 0
Could not find ast node from source mapping: 49693 : 65 : 0
Could not find ast node from source mapping: 49693 : 65 : 0
Could not find ast node from source mapping: 15759 : 18 : 0
Could not find ast node from source mapping: 15759 : 18 : 0
Could not find ast node from source mapping: 18654 : 9 : 0
Could not find ast node from source mapping: 46882 : 21 : 0
Could not find ast node from source mapping: 46882 : 30 : 0
Could not find ast node from source mapping: 46882 : 21 : 0
Could not find ast node from source mapping: 46882 : 30 : 0
Could not find ast node from source mapping: 46882 : 30 : 0
Could not find ast node from source mapping: 11308 : 10 : 0
Could not find ast node from source mapping: 11429 : 8 : 0
Could not find ast node from source mapping: 10295 : 42 : 0

Note: The above message is not an actual error message, it's simply me annotating what I perceive the mistake to be in the example above.

Coudn't install solidity 0.4.18 following the wiki instructions.

Environment information

• py-solc-x Version: 3.2.0
• solc Version: 0.4.18
• Python Version: 2.7.16
• OS: osx

What was wrong?

when running the install file for solc 0.4.18:

brew install https://github.com/ethereum/homebrew-ethereum/blob/2aea171d7d6901b97d5f1f71bd07dd88ed5dfb42/solidity.rb

I get this:

Updating Homebrew...
Warning: Calling Non-checksummed download of solidity formula file from an arbitrary URL is deprecated! Use 'brew extract' or 'brew create' and 'brew tap-new' to create a formula file in a tap on GitHub instead.
-#O#- # #
Warning: Calling Non-checksummed download of solidity formula file from an arbitrary URL is deprecated! Use 'brew extract' or 'brew create' and 'brew tap-new' to create a formula file in a tap on GitHub instead.
-#O#- # #
Error: solidity: /Users/iland/Library/Caches/Homebrew/Formula/solidity.rb:6: syntax error, unexpected '<'

^
/Users/iland/Library/Caches/Homebrew/Formula/solidity.rb:7: syntax error, unexpected '<'

^ /Users/iland/Library/Caches/Homebrew/Formula/solidity.rb:8: syntax error, unexpected '<' ^ /Users/iland/Library/Caches/Homebrew/Formula/solidity.rb:10: syntax error, unexpected '<'

install error

Environment information

• py-solc-x Version: 1.1.1
• solc Version: ?
• Python Version: 3.7.7
• OS: linux

What was wrong?

When running python3.7 -m solcx.install v0.8.9 (or similar) it complains about not solc present (which is why I ran this) and that it was loaded before being run.

RuntimeWarning: 'solcx.install' found in sys.modules after import of package 'solcx', but prior to execution of 'solcx.install'; this may result in unpredictable behaviour

Maybe this was because I already tried to run it before? Also after checking the output directory, it did install solc, but this was not visible from the output.

Full output (PATH somewhat redacted):

❯ python3.7 -m solcx.install v0.5.0
which: no solc in (/home/user/.local/bin:/home/user/bin:<system paths>)
/usr/lib64/python3.7/runpy.py:125: RuntimeWarning: 'solcx.install' found in sys.modules after import of package 'solcx', but prior to execution of 'solcx.install'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
which: no solc in (/home/user/.local/bin:/home/user/bin:<system paths>)

Improve error message for failed OSX installations

Environment information

• py-solc-x Version: 0.8.1
• solc Version: 0.4.x
• OS: OSX

What was wrong?

The error message when attempting to install a 0.4 version with OSX should specify the exact failing version, rather than only 0.4.x.

Related to iamdefinitelyahuman/brownie#487

How can it be fixed?

Modify the error message at solcx/install.py#L332

Compatibility with `solc-select`

Overview

py-solc-x should detect the presence of, and utilize, versions of solc installed via solc-select

Specification

import_installed_solc already handles a natively-installed solc version using which/where. The functionality here could be expanded to fit solc-select. I'm not sure exactly what this looks like as I haven't delved deeply into the package.

Unclear error message when OSX installation fails

This is an issue from an upstream tool: Consensys/mythx-cli#34, which I've been asked to report here:

• MythX CLI version: 0.2.1
• Python version: 3.7.5
• Operating System: 10.15.1 Catalina

Description

mythx analyze --mode quick contracts/**/*.sol
Unsupported macOS version.
We only support Mavericks, Yosemite, El Capitan, Sierra, High Sierra and Mojave.
Usage: mythx analyze [OPTIONS] [TARGET]...

Error: Error installing solc version v0.5.10: Command '['sh', '/var/folders/vk/01zl87497jx6bq66fhhhd9zm0000gn/T/py-solc-x-tmp/solidity_0.5.10/scripts/install_deps.sh']' returned non-zero exit status 1.

Setting version with `set_solc_version_pragma` will cause `get_executable` to fail

Environment information

• py-solc-x Version: 0.7.2
• solc Version: 0.6.4
• Python Version: 3.7.5
• OS: Linux

What was wrong?

When the version is set with set_solc_version_pragma, it strips the v from the version number. If the version is, for example, "v0.6.4", it will set the global solc_version variable to simply "0.6.4". However, install_solc will install an execuable such as solc-v0.6.4, which means that get_executable will expect solc_version to be of the form "v0.6.4". This means that calling set_solc_version_pragma before get_executable will cause the latter call to fail, even if the executable is installed.

How can it be fixed?

Calling _check_version in set_solc_version_pragma as in #49 should fix the issue.

methods to remove in 1.0.0 release

The following methods are planned for removal in the 1.0.0 release. This list is not final.

• main.is_solc_available
• main.solc_supports_standard_json_interface
• utils.filesystem.is_executable_available
• utils.types.is_boolean
• utils.types.is_null
• utils.types.is_number

solcx wrapper misinterpreting solc --help return code

Environment information

• py-solc-x Version: 1.1.1
• solc Version: 0.8.10
• Python Version: 3.9.7
• OS: osx

What was wrong?

Code (pretty much straight from https://web3py.readthedocs.io/en/stable/contracts.html):

    source = open('solidity/contracts/Greeter.sol').read()
    compiled_sol = solcx.compile_source(source)

Expected output: no errors
Actual output: solc returned 0, solc_wrapper expected 1 from help, so it throws an error

Abbreviated output:

solcx.exceptions.SolcError: An error occurred during execution
> command: `/Users/zzz/.solcx/solc-v0.8.10 --help -`
> return code: `0`
> stdout:
solc, the Solidity commandline compiler.

...

Test outside of solcx showing solc returns 0:

$ /Users/zzz/.solcx/solc-v0.8.10 --help -
solc, the Solidity commandline compiler.
...

$ echo $?
0

Full output attached:
eth1.txt

How can it be fixed?

Change solc_wrapper to always expect 0 at this line?

remove add_std from wrapper

See: ethereum/solidity#3695 (comment)

The --add-std flag does nothing since <0.4.11 and was removed in 0.4.21.

Move CI from Travis to Github Actions

Overview

Stop using Travis and start using Github Actions.

The beautiful thing about Actions is that anyone who forks the repo has access to CI. They don't need to make an account with a 3rd party site or enable anything... they don't even need to understand how CI works!

Specification

Translate .travis.yml to the Actions yaml format.

Add readthedocs documentation

Overview

Create proper documentation for this package.

Specification

The package is relatively small. Once #51 is implemented, it should be possible to autogenerate from the docstrings using Napoleon. It's also a good tester to get familiar with Napolean - it could be useful in larger projects.

Dependencies

#51

Allow installation of precompiled OSX binaries

The web3j team is serving precompiled solc binaries for OSX at https://github.com/web3j/solidity-darwin-binaries 👍

Not having to build from source could save a lot of time. Need to look into potential compatibility issues. Maybe attempt to use the precompile and if it doesn't work, fall back to building?

Ensure consistent output across solc versions

Overview

I should be able to call compile_sources or compile_files with:

• any number of sources
• any combination of expected outputs
• any active version of solc

and get a consistent output format.

I should be able to look this output format up in the documentation.

Specification

1. Create parametrized unit tests to verify that output is consistent across versions.
2. Where there are inconsistencies, solve using common sense.
3. Document everything

Dependencies

Step one requires #53 .
Step three requres #52 .

Automatically determine possible kwargs for `solc_wrapper`

Originally posted by @iamdefinitelyahuman in #78 (comment)

solc_wrapper kwargs should be generalized / future-proofed by regexing the solc --help output. This would reduce a lot of overhead around checking exactly which flags were added in which version. A few flags require specific logic but the vast majority are only setting a flag or setting a flag and adding a string after it.. shouldn't be that hard to do.

Here's a quick hacky experiment I did:

>>> help_output = solcx.wrapper.solc_wrapper(help=True)
>>> opts = [i.lstrip() for i in help_output.split('\n') if i.lstrip().startswith('-')]
>>> [next(x for x in i.split(' ') if x.startswith('--')) for i in opts]
['--help', '--version', '--license', '--evm-version', '--pretty-json', '--libraries', '--revert-strings', '--output-dir', '--overwrite', '--combined-json', '--gas', '--standard-json', '--import-ast', '--combined-json', '--assemble', '--machine,', '--yul-optimizations', '--yul', '--machine,', '--yul-optimizations', '--strict-assembly', '--yul-optimizations', '--yul-dialect', '--machine', '--link', '--libraries', '--metadata-hash', '--metadata-literal', '--allow-paths', '--base-path', '--color', '--no-color', '--old-reporter', '--error-recovery', '--ignore-missing', '--optimize', '--optimize-runs', '--optimize-yul', '--no-optimize-yul', '--yul-optimizations', '--ast-json', '--ast-compact-json', '--asm', '--asm-json', '--opcodes', '--bin', '--bin-runtime', '--abi', '--ir', '--ir-optimized', '--ewasm', '--hashes', '--userdoc', '--devdoc', '--metadata', '--storage-layout']

It will require fairly thorough testing and there's no guarantee it won't be broken in a future solc release - but I think this could be done in a way that's just as reliable as what we have today.

Use Version objects for solc versions

Overview

Use Version objects to represent solc versions, instead of strings.

Specification

1. solcx/install.py - store the solc_version variable as a Version and update related code.
2. Return Version object(s) anywhere a version is returned to a user, e.g. get_available_solc_versions and get_installed_solc_versions
3. All functions receiving a version in the user input should be able to understand both strings and Version objects, e.g. set_solc_version, install_solc
4. Remove the get_solc_version_string method (this information is still available from get_solc_version).
5. Update unit tests to ensure all new/modified code is adequately covered.

References

• semantic_version docs: the Version class

Dependencies

This will amount to a breaking change, so cannot happen until 1.0.0.

py.3.10.0 install error

Environment information

• py-solc-x Version: last
• solc Version: x.x.x
• Python Version: 3.10
• OS: osx/linux/win

What was wrong?

i can import the library /w python v.3.10.0 check the photo

Please include information like:

• what command you ran
• the code that caused the failure (see this link for help with formatting code)
• full output of the error you received

How can it be fixed?

Fill this in if you know how the bug could be fixed.

Support for solc path mappings

According to the solc documentation (https://solidity.readthedocs.io/en/v0.4.21/using-the-compiler.html#using-the-commandline-compiler), solc supports providing path mappings for includes, e.g.

solc github.com/ethereum/dapp-bin/=/usr/local/lib/dapp-bin/ =/usr/local/lib/fallback file.sol

I'm not sure if it's already support, however, if supported, I couldn't find the option.

If it is not supported: I would request/suggest to add an option to the compile methods which allows for specifying path mappings as described by the solc documentation.

I want to thank you.

I am using a Centos 7 host machine, and can't install and compile solc from the source properly.
Your tool saved me.
I am writing this to thank you.
Hope the best for you and your team.

Dynamic binary package referencing

Overview

Currently, install.py references Solidity v0.6.0 for installing binaries with a fixed variable. It would be great to have CURRENT_RELEASE dynamically fetch the most-recent binaries.

Specification

Create a function within install.py to fetch the current stable binary packages.

Failed to install on Mac

This is my first time to use this package, so I just write a simple code like the doc to have a try, like this:

from solcx import install_solc
install_solc('v0.5.12')

It is ok at the beginning, but later, it failed to install, the error message is:

[ 79%] Building CXX object test/CMakeFiles/soltest.dir/libsolidity/SolidityCompiler.cpp.o
[ 79%] Building CXX object test/CMakeFiles/soltest.dir/libsolidity/SolidityEndToEndTest.cpp.o
Traceback (most recent call last):
  File "/Users/skyge/Envs/test_solc/lib/python3.7/site-packages/solcx/install.py", line 359, in _install_solc_osx
    _check_subprocess_call(cmd, message="Running {}".format(cmd[0]))
  File "/Users/skyge/Envs/test_solc/lib/python3.7/site-packages/solcx/install.py", line 254, in _check_subprocess_call
    command, stderr=subprocess.STDOUT if verbose else subprocess.DEVNULL, **proc_kwargs
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test_solc_x.py", line 3, in <module>
    install_solc('v0.5.12')
  File "/Users/skyge/Envs/test_solc/lib/python3.7/site-packages/solcx/install.py", line 226, in install_solc
    _install_solc_osx(version, allow_osx, show_progress, solcx_binary_path)
  File "/Users/skyge/Envs/test_solc/lib/python3.7/site-packages/solcx/install.py", line 367, in _install_solc_osx
    "".format(cmd[0], e.returncode)
OSError: make returned non-zero exit status 2 while attempting to build solc from the source. This is likely due to a missing or incorrect version of an external dependency.

You may be able to solve this by installing the specific version using brew: https://solidity.readthedocs.io/en/v0.6.0/installing-solidity.html#binary-packages

I am not sure what is wrong with the build, so I follow the recommend: install by the binary-packages, and I run the following commands:

brew update
brew upgrade
brew tap ethereum/ethereum
brew install solidity

And solc --version can return a value:

solc, the solidity compiler commandline interface
Version: 0.6.4+commit.1dca32f3.Darwin.appleclang

So I run that test python file again, unfortunately, I got the same error, so what should I do next?

• py-solc-x Version: 0.8.0
• solc Version: 3.2.0
• Python Version: Python 3.7.2
• OS: macOS 10.14 Mojave.

Binaries for amd64 downloaded on armv7

Environment information

• py-solc-x Version: 1.1.0
• solc Version: all
• Python Version: 3.9.5
• OS: linux running on a raspberry pi 4

What was wrong?

When trying to compile my contracts, I get "Downloaded binary would not execute..."

https://github.com/iamdefinitelyahuman/py-solc-x/blob/7bed05bad9811d8ee36177b200c6cdce91e5ff70/solcx/install.py#L636

How can it be fixed?

Check the system architecture instead of assuming amd64.

https://github.com/iamdefinitelyahuman/py-solc-x/blob/7bed05bad9811d8ee36177b200c6cdce91e5ff70/solcx/install.py#L41

Does not detect solc built from source (even when solc is on path)

Version Info

• py-solc-x Version: 0.7.0
• solc Version: various
• Python Version: 3.7.5
• OS: osx

The Issue

I cloned and built 0.5.16 from source, then symlinked it to my path, because I've had trouble installing with Homebrew. However py-solc-x doesn't seem to find it.

% which solc
/usr/local/bin/solc
% solc --version
solc, the solidity compiler commandline interface
Version: 0.5.16+commit.9c3226ce.Darwin.appleclang
% python
Python 3.7.5 (default, Nov 25 2019, 17:09:49)
[Clang 11.0.0 (clang-1100.0.33.12)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from solcx import get_solc_version, set_solc_version
>>> get_solc_version()
Version('0.6.2+commit.bacdbe57.Darwin.appleclang')

I ran into this issue while trying to use mythx-cli.

Support for `Path` objects in `compile_files`

Overview

compile_files should accept Path objects as well as strings.

Specification

1. In main.compile_files, the input argument source_files should support a list of Path objects as well as a list of strings.
2. Add test cases. Consider the following:
   • PosixPath and WindowsPath
   • absolute and relative paths

Reference

• pathlib documentation

Add install.get_all_available_versions method

First of all, thanks for the awesome work.

What is the most efficient way to install all currently supported compiler versions?

So far we were installing all versions between 0.4.11 and "latest", but latest just became 0.4.26:
https://github.com/ethereum/solidity/releases/tag/v0.4.26

Do you have any suggestions?

Feature Request: Make solc install path configurable

Version Info

• py-solc-x Version: 0.7.2
• OS: linux

The Issue

Hey,

thank you for your great work!

Could you please provide an optional parameter and some settings to specify in which the solc binaries should be installed? Thanks!

Best regards
Matthias

Support --base-path

Overview

I am trying to leverage the --base-path solc feature introduced in version 0.6.9 to reduce complexity handling import remappings, however, there is no kwarg for solc_wrapper mapping to this flag.

Specification

Ideally this feature can be implemented like other CLI flags as a keyword argument:

solc_wrapper(base_path='/some/source/path/dir/', ...)

Add download progress bar

We should incorporate a progress bar into the install._download_solc so that users can see what is happening during the download. Motivation - I often have a poor internet connection, and it would be nice to see if progress is being made or it's hanging and needs to be killed.

tqdm does this well - is it OK to introduce another dependency? If not, whatever other solution is found has to provide cross-platform support.

Related stackoverflow post

compile_files does not accept absolute windows paths

https://ethereum.stackexchange.com/questions/71951/solcx-compile-files-key-error

I think the fix itself is simple - but it needs more testing and perhaps this is an opportunity to rebuild a portion of solcx using the pathlib package.

Add more structured data in `SolcError`

Overview

I'm trying to extract line ranges and individual errors from Solidity error messages but it seems like SolcError collapses all that information into one string. Would it perhaps be useful to preserve the compiler_output["errors"] variable for this purpose?

Specification

One easy way would be to add a new entry for compiler errors into SolcError like so:

raise SolcError(
  command,
  proc.returncode,
  json.dumps(input_data),
  stdoutdata,
  stderrdata,
  compiler_output["errors"],  # <-- new variable
  message=error_message,
)

This would still allow people to use existing arguments to the exception without change.

Lack of aarch64 support

Environment information

• py-solc-x Version: 1.1.1
• solc Version: 0.6.0
• Python Version: 3.9.5
• OS: linux/aarch64

What was wrong?

As per this PR #66 , aarch64 support was added, and no commits/changes were made to the relevant code. In the current master however these functions simply don't exist, see https://github.com/iamdefinitelyahuman/py-solc-x/blob/master/solcx/install.py

After installing py-solc-x attempting to run install_solcx('0.6.0'), the error
OSError: [Errno 8] Exec format error: '/home/ubuntu/.solcx/solc-v0.6.0' is shown.

How can it be fixed?

I suppose aarch64 support could be added back? I'm not aware of how, however.

New `solc` release, breaks imports of solcx

• py-solc-x Version: 0.2.1
• solc Version: none
• Python Version: 3.6
• OS: Ubuntu

What was wrong?

Because the current latest release 0.4.26 does not provide a solc-static-linux binary yet, all imports of solcx fail, because the __init__ function will try to install 0.4.26 if no version is installed.

Example:

  File "/root/.local/lib/python3.6/site-packages/solcx/__init__.py", line 27, in <module>
    install_solc()
  File "/root/.local/lib/python3.6/site-packages/solcx/install.py", line 83, in install_solc
    _install_solc_linux(version)
  File "/root/.local/lib/python3.6/site-packages/solcx/install.py", line 144, in _install_solc_linux
    _wget(download, binary_path)
  File "/root/.local/lib/python3.6/site-packages/solcx/install.py", line 130, in _wget
    "Downloading solc from {}".format(url)
  File "/root/.local/lib/python3.6/site-packages/solcx/install.py", line 118, in _check_subprocess_call
    **proc_kwargs
  File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['wget', 'https://github.com/ethereum/solidity/releases/download/v0.4.26/solc-static-linux', '-O', '/root/.local/lib/python3.6/site-packages/solcx/bin/solc-v0.4.26']' returned non-zero exit status 8.

Could there be some error handling for this as this seems to happen with every new release?
Thanks.

Replace format with f-strings

Overview

Replace .format with f-strings. This will drop support for <3.6 so needs to be part of v1.0.0.

Specification

Self explanatory. Be sure to also update setup.py to reflect the dropped support.

_download_solc throws TypeError for v0.4.26

Environment information

from eth-brownie/brownie#512

• brownie Version: 1.8.4
• ganache-cli Version: 6.9.1
• solc Version: solc installed with py-solc-x
• Python Version: 3.7.6
• OS: Fedora 31

What was wrong?

Using the following brownie-config.yaml:

compiler:
    solc:
        version: 0.4.26

and test.sol in contracts directory:

pragma solidity 0.4.26;

contract test {}

get following error from running brownie compile

Brownie v1.8.4 - Python development framework for Ethereum

Downloading solc v0.4.26 from https://github.com/ethereum/solidity/releases/download/v0.4.26/solc-static-linux
  0%|                                                                                                | 0.00/40.0 [00:00<?, ?iB/s]  File "brownie/_cli/__main__.py", line 58, in main
    importlib.import_module(f"brownie._cli.{cmd}").main()
  File "brownie/_cli/compile.py", line 32, in main
    project.load(project_path)
  File "brownie/project/main.py", line 543, in load
    return Project(name, project_path)
  File "brownie/project/main.py", line 154, in __init__
    self.load()
  File "brownie/project/main.py", line 204, in load
    self._compile(changed, self._compiler_config, False)
  File "brownie/project/main.py", line 93, in _compile
    optimizer=compiler_config["solc"].get("optimizer", None),
  File "brownie/project/compiler/__init__.py", line 112, in compile_and_format
    set_solc_version(version)
  File "brownie/project/compiler/solidity.py", line 82, in set_solc_version
    install_solc(version)
  File "brownie/project/compiler/solidity.py", line 90, in install_solc
    solcx.install_solc(str(version), show_progress=True)
  File "solcx/install.py", line 227, in install_solc
    _install_solc_linux(version, show_progress, solcx_binary_path)
  File "solcx/install.py", line 310, in _install_solc_linux
    content = _download_solc(download, show_progress)
  File "solcx/install.py", line 293, in _download_solc
    content += data
TypeError: can't concat str to bytes

Add support for solc's --include-path to compile_source

Overview

I am trying to compile a file that has multiple includes from a node module.

My entire working command line is:

/Users/zzz/.solcx/solc-v0.8.10 solidity/contracts/MyContract.sol --combined-json abi,asm,ast,bin,bin-runtime,devdoc,function-debug,function-debug-runtime,generated-sources,generated-sources-runtime,hashes,metadata,opcodes,srcmap,srcmap-runtime,storage-layout,userdoc --include-path solidity/node_modules --base-path .

I added the --include-path and --base-path on the end to get it to work.

When I look at compile_files it doesn't have include_path, or a way to add arguments to the compile line.

Or maybe it does, and I just don't understand.

Personally, I'd add a way to add to the command line in general, so people can work around missing parameters without source modifications.

However, my use case just requires include_path. That flag can be added multiple times, see https://docs.soliditylang.org/en/v0.8.9/path-resolution.html#base-path-and-include-paths.

Specification

Add include_path list.

Or, add extra_arguments list where I can add my own arguments on the end.

64 bit arm not supported

Environment information

• py-solc-x Version: 0.10.0
• solc Version: 0.6.1
• Python Version: 3.8.2
• OS: Ubuntu (Raspberry Pi, 64 Bit)

What was wrong?

According to release v0.10.0, arm architectures should be supported. However, it seems that this support is only limited to 32 bit arm architectures.

Code is checking for platform.machine() ^= "arm", but on my Raspberry Pi 3 B+ with Ubuntu/64 bit support, platform.machine() returns aarch64.

0.7.5 cannot install on windows. TypeError exception

Environment information

• py-solc-x Version: 1.0.1
• solc Version: 0.7.5
• Python Version: 3.7.6
• OS: win

What was wrong?

I ran the following commands:

solc_version = str(get_installable_solc_versions()[0])
install_solc_pragma(solc_version)

and I got:

Traceback (most recent call last):
  File "...\venv\lib\site-packages\solcx\install.py", line 632, in _validate_installation
    installed_version = wrapper._get_solc_version(binary_path)
  File "....\venv\lib\site-packages\solcx\wrapper.py", line 14, in _get_solc_version
    stdout_data = subprocess.check_output([solc_binary, "--version"], encoding="utf8")
  File "...\Python\Python37\lib\subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "....\Python\Python37\lib\subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "....\Python\Python37\lib\subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "....\Python\Python37\lib\subprocess.py", line 1148, in _execute_child
    args = list2cmdline(args)
  File "....\Python\Python37\lib\subprocess.py", line 555, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'WindowsPath' is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 143, in <module>
    install_solc_pragma(solc_version)
  File "....\venv\lib\site-packages\solcx\install.py", line 308, in install_solc_pragma
    install_solc(version, show_progress=show_progress, solcx_binary_path=solcx_binary_path)
  File "....\venv\lib\site-packages\solcx\install.py", line 465, in install_solc
    raise exc
  File ".....\venv\lib\site-packages\solcx\install.py", line 459, in install_solc
    _validate_installation(version, solcx_binary_path)
  File "....\venv\lib\site-packages\solcx\install.py", line 636, in _validate_installation
    "Downloaded binary would not execute, or returned unexpected output."
solcx.exceptions.SolcInstallationError: Downloaded binary would not execute, or returned unexpected output. If this issue persists, you can try to compile from source code using `solcx.compile_solc('0.7.5')`.

Process finished with exit code 1

arm support / check arch

Overview

It is easily possible to run most things on arm, or another architecture that is not x86. However solc-x doesn't check for it, doesn't complain and happily downloads the wrong binary, which then results in exec format error. When I copy a manually compile solc into $PATH, it works however.
At the very least, solc-x should check the arch and complain before downloading and running, or alternatively download the right binary accordingly.

Specification

Ideally, solc-x checks the architecture, and if a corresponding binary is available, downloads that.

Repo Housekeeping

Things to implement / include in CI prior to 1.0.0 release:

• linting - flake8 / black
• mypy (requires #33)
• isort
• tox
• bumpversion
• pre-commit
• Keep a Changelog

Provide a direct way to pass a custom solc binary path

Overview

In some scenarios where I am applying py-solc-x, a user might want to specify a solc version that is already set up on their machine. This means that the automatic installation should not be executed.
I have seen that there is the SOLCX_BINARY_PATH env var, which allows the user to specify which directory to look in for a solc binary. There is also a platform-specific check using which to look for the solc binary on the PATH.
When using the library (which is absolutely amazing btw!), I expected a more explicit way. Please consider the following examples.

Specification

An addition to the compilation parameter set, such as:

solcx.compile_standard(
   ...
   solc_path="/foo/bar/solc"
)

This would effectively short-circuit any internal checks looking for the solc binary and simply take over the parameter's location if it has been set. Alternatively, and following the paradigm of explicit module-level calls, an explicit setting outside of a function call can be considered as well:

import solcx
solcx.set_solc_installation("/foo/bar/solc")

This would result in a global short circuit of the checking logic and always use the path set with set_solc_installation.

At first glance both ways seem to not cause any breaking changes.

Add solcx entry point to allow commandline use

Something like solcx <version> <regular-args> to allow easy command line interaction with the multiple installed versions. If the given version isn't installed, return an error and suggest use of the --install flag.

Refactor `solc_wrapper` and the main compile functions

Overview

1. wrapper.solc_wrapper uses a long sequence of if statements to handle kwargs one by one. This feels inefficient.
2. compile_source, compile_files and compile_standard all use **kwargs to forward to solc_wrapper.
   • This makes the code less readable because you have to move between two files to see which kwargs are allowed.
   • Using the standard JSON input/output makes the compiler to silently ignore other flags, so most kwargs given to compile_standard have no effect. This could be confusing for some users.

Specification

1. Refactor out the long string of if clauses in solc_wrapper.
2. Move the specific kwargs out of solc_wrapper and into the compiler methods. Disallow invalid kwargs sooner.

Add docstrings

Overview

All public methods within the codebase should include docstrings.

Specification

Docstrings should use the NumPy style.

Dependencies

Not a hard dependency, but this should happen once #31 and other breaking changes are complete. Otherwise there could be some repeated work.

Tests are failing with solc 0.6.0

Solidity version 0.6.0 has just released and some of the tests are failing - https://travis-ci.com/iamdefinitelyahuman/py-solc-x/builds/141073240

Allow sequence of strings or `Path` objects for `allow_paths`

Overview

Currently, allow_paths must be given as a comma-separated string. It would be easier to use and more intuitive if this kwarg instead expected a sequence of either strings or Path objects.

Specification

1. In main.compile_source, main.compile_files, and main.compile_standard, expect allow_paths to be given as a sequence of strings or Path objects.
2. Add test cases. Consider the following:
   • PosixPath and WindowsPath
   • absolute and relative paths

Dependencies

#53

Script folder is stale code

The Issue

The scripts subfolder contains an old py-solc relic that's inaccessible and doesn't need to be a part of this package anymore. It should be removed.

Thanks @crawfordleeds for noticing this :)

Error Downloading on macOS

• py-solc Version: 0.5.0
• solc Version: 3.5.7 trying to download 3.5.5
• Python Version: 3.6.1
• OS: macOS

What was wrong?

When trying to install either through a terminal or through a python interpreter, an error occurs when trying to use wget to download the respective version of solc desired.

Using python3 -m solcx.install v0.5.5:

/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/runpy.py:125: RuntimeWarning: 'solcx.install' found in sys.modules after import of package 'solcx', but prior to execution of 'solcx.install'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Downloading solc from https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz
Executing: wget https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz -O /Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/site-packages/solcx/temp/solc-source.tar.gz
https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz: Unsupported scheme.
Traceback (most recent call last):
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/site-packages/solcx/install.py", line 337, in <module>
    install_solc(version)
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/site-packages/solcx/install.py", line 198, in install_solc
    _install_solc_osx(version, allow_osx)
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/site-packages/solcx/install.py", line 300, in _install_solc_osx
    _wget(download, temp_path)
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/site-packages/solcx/install.py", line 238, in _wget
    "Downloading solc from {}".format(url)
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/site-packages/solcx/install.py", line 226, in _check_subprocess_call
    **proc_kwargs
  File "/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['wget', 'https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz', '-O', '/Users/crawfordleeds/.pyenv/versions/3.6.1/lib/python3.6/site-packages/solcx/temp/solc-source.tar.gz']' returned non-zero exit status 1.

Through a Python 3.6.1 interpreter:

import solcx
solcx.install_solc('0.5.5')
Downloading solc from https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz
Executing: wget https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz -O /usr/local/lib/python3.7/site-packages/solcx/temp/solc-source.tar.gz
https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz: Unsupported scheme.
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/solcx/install.py", line 198, in install_solc
    _install_solc_osx(version, allow_osx)
  File "/usr/local/lib/python3.7/site-packages/solcx/install.py", line 300, in _install_solc_osx
    _wget(download, temp_path)
  File "/usr/local/lib/python3.7/site-packages/solcx/install.py", line 238, in _wget
    "Downloading solc from {}".format(url)
  File "/usr/local/lib/python3.7/site-packages/solcx/install.py", line 226, in _check_subprocess_call
    **proc_kwargs
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['wget', 'https://github.com/ethereum/solidity/releases/download/v0.5.5/solidity_0.5.5.tar.gz', '-O', '/usr/local/lib/python3.7/site-packages/solcx/temp/solc-source.tar.gz']' returned non-zero exit status 1.

It seems like the issue could be resolved from moving all wget commands to request or urllib2. For example, replace this with something more like this