failed to install termcolor about pyflow

[David-OConnor]

Released 0.2.6, which should fix this.

[sbdchd]

Working for me now, thanks!

❯ cd "$(mktemp -d)" && pyflow init && RUST_BACKTRACE=1 pyflow install black
Created `pyproject.toml`
🐍 Setting up Python...
Found lockfile
⬇ Installing black 19.10.0b0 ...
Added a console script: black
Added a console script: blackd
⬇ Installing appdirs 1.4.4 ...
⬇ Installing toml 0.10.1 ...
⬇ Installing click 7.1.2 ...
⬇ Installing regex 2020.5.14 ...
⬇ Installing pathspec 0.8.0 ...
⬇ Installing typed_ast 1.4.1 ...
⬇ Installing attrs 19.3.0 ...
Installation complete

OS: MacOS 10.15.3
python: Python 3.7.3
pyflow: pyflow 0.2.6

[David-OConnor]

Actually, Pyflow does attempt to install wheel on the downloaded Python, but running it rasies this error: ModuleNotFoundError: No module named '_ctypes'. Ctypes is missing from the compiled binary.

[David-OConnor]

Could you please post your OS (including distro if Linux), and pyproject.toml?

After some testing, it works on Windows for me, but not Linux. No wheel is avail, and something's going wrong when attempting to build one from the source distro. (As you've discovered)

[mmacedoeu]

Ubuntu bionic, for pyproject.toml you can use the created template after pyflow new ...

[David-OConnor]

Leaning towards this being an issue with wheel not being properly installed or used by pyflow, since running a system python setup.py bdist_wheel in the downloaded folder works. (This is the step failing)

[mmacedoeu]

I tried using a non-system python 3.8.2 (installed by pyflow) and python installed by penv and both failed.

[sbdchd]

I think I may have run into the same issue when trying to install black:

OS: MacOS 10.15.3
python: Python 3.7.3
pyflow: pyflow 0.2.4

❯ cd "$(mktemp -d)" && pyflow init && RUST_BACKTRACE=1 pyflow install black
Created `pyproject.toml`
🐍 Setting up Python...
Found lockfile
⬇ Installing typed_ast 1.4.1 ...
⬇ Installing black 19.10.0b0 ...
EX PAR: "__pypackages__/3.7/lib/black-19.10b0" bin: "__pypackages__/3.7/.venv/bin"
thread 'main' panicked at 'Problem running setup.py bdist_wheel: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/libcore/result.rs:1188:5
stack backtrace:
   0: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
   1: core::fmt::write
   2: std::io::Write::write_fmt
   3: std::panicking::default_hook::{{closure}}
   4: std::panicking::default_hook
   5: std::panicking::rust_panic_with_hook
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::result::unwrap_failed
   9: pyflow::install::download_and_install_package
  10: pyflow::sync
  11: pyflow::main
  12: std::rt::lang_start::{{closure}}
  13: std::panicking::try::do_call
  14: __rust_maybe_catch_panic
  15: std::rt::lang_start_internal
  16: main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

and similar errors on Ubuntu 18.04.2 LTS

$ cd "$(mktemp -d)" && pyflow init && RUST_BACKTRACE=1 pyflow install black
Created `pyproject.toml`
🐍 Setting up Python...
Found lockfile
⬇ Installing click 7.1.1 ...
⬇ Installing regex 2020.4.4 ...
thread 'main' panicked at 'Problem running setup.py bdist_wheel: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /home/server/.cargo/registry/src/github.com-1ecc6299db9ec823/pyflow-0.2.5/src/install.rs:344:13
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:77
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1052
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:204
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:224
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:472
  11: rust_begin_unwind
             at src/libstd/panicking.rs:380
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  13: core::option::expect_none_failed
             at src/libcore/option.rs:1199
  14: pyflow::install::download_and_install_package
  15: pyflow::sync
  16: pyflow::main
  17: std::rt::lang_start::{{closure}}
  18: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
  19: std::panicking::try::do_call
             at src/libstd/panicking.rs:305
  20: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:86
  21: std::panicking::try
             at src/libstd/panicking.rs:281
  22: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  23: std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  24: main
  25: __libc_start_main
  26: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[David-OConnor]

It looks like Black only has wheels for Python 3.6, so I'm suspicious it's the same. I haven't had issues with black on my system, but for now, guessing same bug.

[mmacedoeu]

Created new project with pyflow new pytest and set python version 3.8, different from system python:

❯ pyflow install termcolor
Please enter the number corresponding to your Linux distro:
1: 2016 or newer (Ubuntu≥16.04, Debian≥9, SUSE≥15, Arch, Kali, etc)
2: Older (Centos, Redhat, Fedora, older versions of distros listed in option 1)
1
Downloading Python 3.8.0...
Installing Python 3.8.0...
🐍 Setting up Python...
⬇ Installing termcolor 1.1.0 ...
Problem building termcolor from source. This may occur if a package that requires compiling has no wheels available for this OS and this system is missing dependencies required to compile it, or if on WSL and installing to a mounted directory.

[mmacedoeu]

Setup virtualenv with pyenv first:

pyenv install 3.8.0
pyenv virtualenv 3.8.0 pytest
pyenv local pytest

❯ pyflow install termcolor
⬇ Installing termcolor 1.1.0 ...
Problem building termcolor from source. This may occur if a package that requires compiling has no wheels available for this OS and this system is missing dependencies required to compile it, or if on WSL and installing to a mounted directory.

pip instal --upgrade pip
pip install --upgrade wheel
❯ pyflow install termcolor
⬇ Installing termcolor 1.1.0 ...
Found lockfile
Installation complete

So basically there is a problem when python is setup by pyflow as wheel is not included

[David-OConnor]

Oops - looks like I incorrectly assumed the Linux system python will have wheel installed by default. As a stopgap, latest commit clarifies that error to suggest installing wheel.

I think the true fix is to recompile the Python binaries Pyflow downloads to include wheel (And a few other things that may be missing), but I'm not experienced with this, and used the default instructions. These instructions exclude some extras. Python.org doesn't provide portable binaries, which is why I built them from source.

[Compro-Prasad]

@David-OConnor I am also facing this issue with python 3.8.

ModuleNotFoundError: No module named '_ctypes'

I was trying to import tensorflow. And now importing ctypes also fails. With python 3.9 installed on my system(Manjaro) this doesn't happen.

[MasonMcGill]

It looks like I'm facing the same issue with PyTorch on CentOS, unfortunately, and the only workaround I've been able to find requires root access. I really love the design of this tool, though, and I'd love to get it working for that project.